I've been hunting varmints and larger game, up to moose. My first shooting was with B-B guns and burned up 10 large Winchester tubes a week in my back yard when I was 9-14 yrs. old. We killed lots of red squirrels and a few birds before I was able/allowed to hunt with a .22LR rifle. As a teen with a buddy who was a couple of years older, we hunted woodchucks and crows with handloaded 30-06 rifles, as practice for deer hunting in Maine. We averaged about 220 yards the year I started keeping kill records, pacing every kill. I made a prone "A" rest out of copper tubing for long, prone shots. It fits in a back pocket (with legs sticking out). More recently, I killed a bull moose at 275 yards with my .270 Win, down a haul road in the North Maine Woods, while sitting on my folding stool, without a rest, using my 140 grain handloads...one shot, one kill as it crossed between the trees.
No, I don't think the average hunter is a great longer-distance killer, but have known lots of folks who are pretty darned good, out to about 250 yards. We don't get a lot of shots beyond that, but the true test is knowing when it's okay to shoot and when it isn't. It's all about knowing where the bullet will strike at the distance, being able to hold steady-enough for a high-percentage shot, and being knowledgeable as to where the bullet will be in it's trajectory, and compensate as necessary for that shot. Yeah, that's a "mouthful".
Nobody can hold perfectly steady for offhand shots, but the better shots know how to "quickly squeeze" one off as the crosshairs start to "cross the estimated kill zone" for that shot. It's okay to NOT shoot, if it doesn't look/feel right!