ALL buildings ought to be upgradeable. Upgrading an existing building costs:
1) More space. a lvl 1 building takes 1 tile, a lvl 2 building requires 1 free tile in either of the 4 cardinal directions, a lvl 3 building requires 2 free adjacent tiles, etc. This would require a certain amount of city planning.
2)Each upgrade would require a slightly higher resource cost than a new lvl 1 building would. ie a library costs 5 mats and 5 gold at lvl 1, the upgrade to lvl 2 should costs 6 / 6. Lvl 2 > lvl 3 costs 7 / 7 , etc.
3)Upkeep costs for upgraded buildings should increase at a much lower rate than building new buildings of the same type. ie each new library should cost 1 / 2 / 3 / 5 / 8 (first, second, third etc instance) , etc, while upgrading an existing building should cost 1 / 2 / 3 / 4 / 5 (lvl 1, lvl 2, lvl 3, etc.)
4)Citizen requirements for upgraded buildings should be much higher than building a new instance. ie 5 / 15 / 30 / 50 / 75.
5)Upgrading should have a slightly higher build time than building a new instance: 5 turns / 6 / 7 / 8 / 9.
However, upgraded buildings should have much better output than having many unique instances. ie 1 rp for each lvl 1 library, 3 rp for lvl 2, 6 rp for lvl 3, 10 / 15. Also, higher level buildings should grant unique features, like spawning champions, free spells, allow to recruit magi, etc. All library specific examples.
The numbers in these examples are used for demonstrative purposes only, and do not reflect actual game balance.