I'd think giving them armor piercing, but more initiative penalties, would be a better gameplay distinction and more realistic.
(a trained archers can pulls arrows from quiver to fire again in 5-10 seconds, I'd guess. To re-load a cross bow, it must be put down, bow pulled back, locked, then the crossbow raised again and aimed. Easily adds another 5-10 seconds, I'd think)