For me the answer is easy. For the last 50 years I have carried either a Ruger SBH .44 mag or a .357 Ruger or S&W. Which one depends on what I am doing. Fishing, the .44 is the choice, with a 30/30 attached to my pack. Hunting, the .357 gets the call as backup to the hunting gun. Never felt comfortable with a semi-auto because I like to carry various loads; light specials for small game, shot loads for snakes and hot loads for whatever. Can't do that with a semi.
I've always felt you should carry the biggest gun you can shoot well, and you have to be honest about that. If your limit or experience is a .22, so be it. Might not be the best in bear country, but depending on the bear, a handgun may not be the best choice at any time. I've always been more concerned about two legged critters than bears.