You have to open up Windows Explorer and go to "C:\Users\*USERNAME*\My Documents\My Games\Race" or "C:\Users\*USERNAME*\My Documents\My Games\Units" (for Sovereigns) and delete the ones you don't want anymore.  Replace *USERNAME* with the username you're using.  In Windows XP, also look for "Documents and Settings" instead of "Users".

There is an easier way to delete custom Sovereigns though (but not races or default Sovereigns).  Right-click on them on the "Who Am I?" screen and you'll either delete them or be given the option to delete them.