I've been studying this matter for some time...
For years, I had a Rem 760 .30-06 with the plastic buttplate. I shot it quite a bit and could easily burn a box of 20 180 grain without any major discomfort. Then I was struck by the accuracy bug and dumped it. I had to have a bolt gun.
Sometime later, I ended up building my infamous Springfield Express Rifle .30-06 in a classic style stock of rock hard walnut with a decellerator pad. The gun weighs 10 lbs loaded. It also kicked me so painfully, I though I had something wrong with me. I thought the chamber was cut too tight. I thought a screw from the recoil pad was stabbing me. I shelved it with it's half box of ammo...
One night, I happened to glance in the mirror while hunting leopards in my den, and noticed the recoil pad of that rifle was sticking 2/3 of the way OFF my shoulder... Only the toe was in contact with my body- less than 2" really...
Art has it right. It's the stock fit. I am on a quest to have a properly fitted stock. Some guys are lucky, they have close to the average dimesions all stocks are being made. I ain't.
You may notice all the old time guns like Original Mausers, Rigbys, H&H, et al, had steel buttplates for calibers like .375 H&H and more but these rifles were fitted to the individual.
I think you can handle a .30-06 with a fitted stock easily...
P.S. I've tried two gunsmiths near me and that was a total waste of time- you start throwing around terms like pitch and drop and cast off and they look at you like you are crazy and suggest you get a boyds stock and measure your forearm bent at 90 degrees... The quest continues.