It's not as funny as you think.
When you're younger and in good health, your retinas can take a lot of stress and be fine. As you get older, though, the aqueous humor gets thicker, and your retina gets thinner and less elastic. If you already have eye disease or are at risk, recoil CAN be a serious consideration.
My Father was advised to stop shooting anything other than small bore rifles, handguns, and lightly recoiling shotguns (he just stopped skeet altogether) several years ago after several retina tears.
I SERIOUSLY doubt that handguns are going to be a problem.
Rifles and shotguns, however? Yeah, they can be a problem.