Yup to destroy enemy improvements I believe three conditions must be met:
1. having researched Fortification (I always do this now so not 100% its required for improvements but given you could not I think it is required because I can)
2. be at war with the opponent.
3. move onto the improvement so long as it is not part of the city.
If the cases it is part of the city, Razing the town/city/settlement will destroy the improvement. I saw this tonight in my 1.07 game.
Cheers,
Gene