Avenger29 is right. Hearing damage is cumulative, and a major factor is "impulse" noise, things like gunshots, explosions, screaming, etc do immediate damage that never completely heals.
While occasionally shooting a .22 rifle without ear protection won't make you go deaf very fast, it is easily avoidable damage. Even the cheapest hearing protection would make a big difference, because it would still dampen the impulse your ears are being subjected to.
I work in high-noise industrial settings frequently, environments where earplugs or muffs aren't just a good idea, but you WANT them, because going without hurts. The best kind I've found are the re-usable silicone plugs
LIKE THESE. They are fairly cheap, re-usable for longer than I've ever had a pair before I lost them, and wash up with normal hand-soap while you wash your hands, or in the laundry. My favorite thing about them is that they just slip in your ear without manipulating the ear canal or having to compress the plug.
I wear them for hours at a stretch on the job, and then jam them in my ears for shooting, because they are easy, comfortable, and economical ... there's no reason you couldn't keep a pair with your range stuff or in your rifle cases.
