To delete Sov: just Left Click on the portrait on the Sov Selection Screen.
It is not possible to edit the Sov in-game...Just create a new one
To delete a Faction: go to (...)\My Documents\My Games\Fallen Enchantress\Races
And delete the file of the race.
You can delete the units you have created too (in Units folder), but it is better to leave them, because AI will use your created designs, generally they are far better than the standar models.