I noticed this in the version that had the prestige penalty in it with regard to number of cities. Basically your grand kiddies think you suck and leave for a more prestigious group. In this case the group with the least cities would likely be their choice given the messed up game mechanics. I haven't seen this happen again in v1.4, again unless your faction's prestige is very low in comparison to other's. One way around it is to reload the auto-save to just before and declare war on your buddies just before the kiddies get old enough to go out on their own. i.e. limit where they can go.