It's mainly a physics thing. Energy doesn't just disappear. With some of the newer compound bows out there, the combined energy is around 500-600#'s, if not mistaken. When the energy is not transferred to an arrow, it gets transferred to the limbs, stressing them in ways that they were never meant to be stressed. Strings can also snap, cams can bend, basically everything can break.
Many people that aren't familiar to archery don't know that. It's quite a common thing to hear a friend picking up a bow, only to look surprised when it breaks after he dryfires it.
As a shameless plug to help the archery industry, if you like shooting guns, and want an alternative that is arguably more challenging and helps exercise your upper body, try out archery. Arrows can be used over and over again, making shooting it basically free after you have all the equipment. I plan on going bowhunting next season with my bow after practicing with it all summer. I'll be sure to keep you guys updated when I do that.