A lot has to do with aging and general fitness also. I will never be able to shoot or do many other things as well as I did 20-40 yrs ago. that's just a given with age. But if you stay in good shape, you should not degrade as quicklly at any one thing.
At 65 I do 1 hr, per day, 6 days per week of exersise, which includes 30-40 minutes on the elliptical trainer, and 20-30 minutes of weight training.
This has to help in everything else I do, as it builds endurance and strength. So my shooting skills, "although isolated more to certain muscles" still become used in the course of my daily exercise. My ability to be able to run without breathing hard, for extended periods of time, and upper and lower body strength, Biceps triceps, forearms shoulders, all play a part in the usage of a weapon, so my belief is that if you are athletic, you will recover much faster than if shooting is the only thing you do. The timing is probablly the thing that you probablly lose from not shooting. That and hand manipulation,which you can strengthen with daily exercise.Your eyesight is the first thing to noticeblly go, and we have spoken to various ways to approach that.
But I think that your skills errode less if you lead an active lifestyle with daily exercise. It should be easier to snap back, "like other sports", if you work out on a daily regimen. Many professional atheletes don't play "their sport" in the off season, but still stay in shape. Just like some professional Boxers fight once in a year or 18 months, they still do daily workouts in the gym.
That's like ring rust, they need to get their timing back first few rounds.