Most today will say 9mm Parabellum Uber Alles, because you can get a darn small, darn light, high capacity, light recoiling, or maybe a few of these attributes together, pistol, in a caliber that arguably performs notably better than .380, with more affordable practice ammo and more variety of SD ammo, and the guns will, in general, be only a little bit bigger than their .380 equivalents. I would have stuck to .380 for recoil/controllability, but I did shoot a friend’s Sig P365, and I must say... what a gun! I see why people really like the small 9mms now, for CCW.
That said...
.380 is great. Realistically probably good enough (if any handgun is) for self-defense, with good self-defense ammo. The pistols are smaller than 9mm or larger caliber guns. An LCP or P3AT is highly concealable and (IMO) not bad to shoot. It’s the sweet spot in compromises, but it’s neither as powerful as bigger rounds, nor as fun to shoot as bigger guns. It’s about as small as reasonably possible, though.
.32 is nice to shoot in most guns and arguably fine for self defense, too. The problem is, the guns chambered for it are mostly old-fashioned, not very high-cap for their size, and larger than they need to be, so they don’t bring a lot to the table. The KelTec P32 is the exception. The Beretta 3032 is pretty but has a bad reputation for cracking slides in normal use and is painful to shoot. The Colt 1903 is a dream to shoot, ok to carry, but it’s not drop safe, and definitely leaves you feeling like you could do a little better, with only 8+1, in a pistol that heavy. The Beretta 81 is great, just wish it were steel, but we’re getting into too chunky for ccw territory.
.32naa is a non-starter for me because the NAA pistol is (IMO) ugly. And there’s not much in the way of options or industry support.
.32 short/long/h&r mag/federal mag are good cartridges, and an LCRx in .327 would be an interesting option, carried with some type of .32 h&r mag defensive ammo. Might be better ballistics than .32acp, depending on the implementation. I have no practical experience with this though.
.25acp guns are tiny and you could do far worse. They suffer from the same issue though... antiquated designs that may or may not be safe with a round in the chamber, sometimes questionable reliability, sights too small to actually use, low capacity for size in most cases, and expensive ammo that’s pretty marginal for stopping.
.22 guns have cheap ammo, low recoil, and you could theoretically put a lot of reasonably aimed shots in a bad guy very quickly. They suffer from sometimes poor reliability and the fact that .22 isn’t a great stopper. IMO the NAA mini revolvers solve the reliability issue. Still not great speed or accuracy or capacity, but they allow you to have a gun when nothing else is concealable.