I have chosen to minimize the number of handgun cartridges in inventory, by trying to buy weapons chambered for certain cartridges. Reducing my weapons to all one cartridge might be an entertaining mental exercise, but nothing more. At minimum, I would want one centerfire cartridge, plus .22 LR handguns, and once I am retired from policin', and no longer mandated to use .40 at work, I could do quite well with just .357 Magnum and .22 LR. (I am required to buy my own duty weapons and ammo.)
Versatile as the .357 Mag might be, there are better choices for such things as home defense at night, such as .45 ACP and .45 Colt. Moreover, I have deep affection for these cartridges; I simply will NOT voluntarily part with my 1911 or my SA sixguns. Then, there is my Seecamp LWS-32, a niche pistol, but sometimes that niche is important. If/when I finally do move to rural acreage, I might well find practical applications for my .32 H&R-chambered SP101, or this Ruger might
someday be a carry gun, if I become too feeble for .357 Mag.
I certainly would not want to limit my long firearms to shooting mere handgun ammo, so I won't go there.
In actual practice, I have more .223 and .22 LR stockpiled than anything else. I have several hundred .40 Gold Dots, to match my personally-owned duty pistols, which means it is third in quantity.