Nothing smaller than a .380ACP, as long as there are no exceptional circumstances.
However. When working with people who have significant physical challenges, that rule goes right out the window. A .22 works better than a sharp stick, and a sharp stick works better than bare hands. So if the only choice is between bare hands and a ballistically-challenged firearm, take the wimp-caliber firearm and don't look back.
Even if there's really no choice but to go with an underpowered round, you still choose the most powerful round you yourself can handle -- that's .380 ACP, then .32 NAA, then .32 ACP, then .22 Magnum. Forget .25 ACP and .22 LR -- if you can handle a .22 LR, you can handle a .22 Magnum. And if you can handle a .25 ACP, you can easily handle a .32.
Don't take this to mean that every situation is exceptional. Being a little uncomfortable is not an exceptional circumstance (though you could look for a better holster, better belt, or smaller gun in the same caliber). Having to dress around the gun is not an exceptional circumstance. Not wanting to "hassle" with a holster, or believing that carrying a larger caliber means you're paranoid, are not exceptional circumstances.
pax