I had an interesting problem happen in my current game.  The Umber did a simultanous attack on several of my resources.  I defended a couple and lost one.  After dealing with the army, I rebuilt the mine.
A couple turns later the AI did the same attack on different resources.  He actually got me a little off guard and destroyed two of my three shards, and I mean destroyed the shards.  I could not rebuild the shrines.  The actual shard crystal itself disappeared from the entire game.
Are the shards themselves supposed to get destroyed?  If so, I will definitely be posting guards next to those from now on.