Yes, this happens to me too - I was playing Magnar and the first hero I get was a trog-warrior. I try berserk but the first thing he do was to cast a spell. So I never used it again for the hole game.
Later in the course of the game I was in war against AI-controled trogs and in several battles they use berserk on their units only to run away from my units.
Anyway, I think that this type of ability isn`t very interesting - to loose control of your units to the AI. Maybe it would be better if the player retain control over berserked units, so the berserk ability provide stat boost instead. Something like that: flat +2 to attack (so for unit without any defense to have some use) + 2*hero level (only for heroes to compensate) + defense stat of the unit (hero) - so a berserked will loose all of defense but get a substantial attack boost. To make things more dynamic maybe it would be good to add additional effect - something like "rage": each time the berserked unit is receiving mundane (not magic) damage its attack will increase with 1.
So of a unit with 5 attack and 0 defense is berserked it will get 7 attack. If the same unit have 5 attack and 3 defense, it will be at 10 attack and 0 defense.
This will be very powerful ability but with some drawbacks - defense reduced to zero.
