The smallest thing that I will carry by choice, as a primary, is .45 acp in an auto, or .44 special in a revolver. The funny thing is that those two rounds are also my power ceiling for a handgun, as much more than that, such as .44 mag, or some such, and you start reaching a point of diminishing returns with controllability. I do however also have a P-32 that rides my hip pocket, most of the time, as a last ditch effort before resorting to throwing rocks. Now, if both .44 and .45 are in the shop, or some such (god forbid!) thereby not allowing me the choice of carrying them, I'll progress on down the list of other stuff I've got.
Conclusion, the biggest non-magnum I can lay hands on is what gets carried. No one ever got into a gunfight, whilst carrying a full size big bore, and thought: "Gee, I just realized how marginal a handgun can really be in a defensive situation, I sure wish I'd left this big belt anchor at home and packed my .25 today!"
~~~Mat