That is, of course, assuming that they know you're there....... it drove me NUTS that those borders worked through FOW........ and that they made war, trade, and exploration miserable..... you want to lock down your borders? Do what us mortals do and actually defend them!
Was there FOW within your own cultural border? For some reason I remember anything within your border was always visible... But I agree - borders were too binary in Civ IV. It should be possible to infringe on others' borders without it being an automatic declaration of war. Really, it should be up to the person whose territory is being trespassed to declare war (but in such a situation, for any gameplay/diplomatic mechanics, the trespasser should be considered the aggressor).
