It's not the bullets that are particularly hard on the barrels.
It is, however, the amount of powder one burns through the barrel throat that leads to short barrel life.
Heavyweight VLD .223 Remington loads, say those 80gr Berger VLD loads folks like for NRA High Power, aren't so tough on barrels compared to that same bullet loaded into a .22-250 or .220 Swift, with a bunch more powder behind it. That extra powder's gotta go somewhere when it's lit off, and it does, continuing the burning process down the throat and the bore. Some would argue that unignited powder grains bead blast the throat before igniting further down the barrel. Result - eroded barrel throats, and lost accuracy after X amount of rounds fired.
Bottleneck rounds with small diameter necks, like the 6.5x284 and 7mm Remington Ultra Mag, concentrate their erosive effects compared to rounds like the .416 Rigby or .458 Winchester Magnum. P.O. Ackley described the former as "overbore", and noted a diminished return of velocity improvement with each extra grain of powder added once a certain "efficiency" was met. In other words, extra powder may mean more velocity in an overbore case, but the velocity increase won't be as dramatic as it would be with a more efficient, less overbore case. The difference between a 6.5-06 and 6.5-06 Ackley Improved or 6.5 Gibbs is an example of that. The improved versions do offer an increase in velocity, but not as much as the extra grains of powder would make one think.
Another way to look at barrel life is as a simple function of powder burned through it.
One would never think a .22 Long Rifle barrel would be burned out and have a chewed-up throat after a thousand or so rounds. There's maybe about 5-10 grains of powder per each round fired, right? Just getting to a pound of powder is going to take roughly 700 rounds at the higher figure.
Now take .30-378 Weatherby Magnum, with well over 100 grains of powder in each round. That beast is burning up to 122 grains of powder behind a .30 caliber 180gr bullet. That's 57 rounds per pound of powder, roughly 12 times as much powder per round as the little .22 Long Rifle.