In guard mode you will never be bothered with an idle ship.  In Sentry mode you will be prompted that the ship is idle and forced to do something if another ship comes within sensor range.

That was how they worked in GCII, but in III ships on guard mode don't wake.

In my experience the sentry mode alerts you by becoming an idle ship that requires an order. I am assuming that you are referring to your ships stationed nearby the planet to be invaded. To wake up they must be quite close, warships usually don't have a lot of sensor range.

If tehy do wake up they will not do a thing, but you have an opportunity to give them an order to attack any transport or fleet that is threatening your planet. If you are talking about ships beyond range they will do nothing. It doesn't and shouldn't work that way. If you are at war the only reasonable defense is to station ships in orbit around your planets and bases.