You'll only be able to play medium size universes with fewer AI Civs
This is a good point.
I only finished 1 game on immense with a single sector. It was a drag in the 'early late' game, but got a bit better as the number of civs dropped off.
Mostly I now either play large single maps, or tiny/small with a lot/max number of sectors.
I also tend to play less then 25 civs.
The default settings/setup works very well all game.