Yes and no.
Of course if you care enough about your personal safety to carry a gun around, then it should be worth it to put some effort into staying reasonably fit, wear shoes you can run and fight in, and learn at least a little bit about how to fight if you aren't able to use your gun.
However, the reason we carry guns in the first place is to gain a decisive advantage over an attacker. Not matter how much I exercise, I can't guarantee I will be tougher than the bad guys. (The truth is, I was never that tough, and as I'm getting older I'm losing a lot of the advantages I might have once had. I can pass all of my army fitness requirements, which puts me in front of the general population, but it's still not great.) Take for example, a petite female. She can exercise, study some flavor of martial art, establish a serious mindset, etc, but she will never give herself a real advantage over a 210 lb attacker. All any of us can do is work to even the odds. If I thought fitness was the definitive advantage, I would just work out and stop carrying a gun completely.