It depends on the gun or level of anticipated use for it.
For example, my Beretta Model 20 has 2 magazines, which is plenty.
For serious guns, however, 12 is minimum. If the gun has a capacity over 10 rds, then 24 is better.
As mentioned, magazines can wear out, get damaged or get lost. Suddenly, a gun that made sense as a 17+1 capacity 9mm makes less sense as a 10+1
Rifles like AR's, should have 20 mags plus, especially now.
Keeping only enough ammo on hand to load your guns is not very wise.
Again, it's caliber or mission dependent. The .25 acp's one box of 50 is plenty (for me) as it's not a very serious gun. But a "serious" handgun should have as a bare, bare, bare minimum of 200 rds on hand at all times.
A serious rifle, 500 rds.
.