To the OP, you've just discovered that shooting groups is an ineffective way to find accurate loads. Not only do you have to shoot an absurd number of shots (far more than you suggest when seating depth is considered as a variable) but one or two flyers at the wrong time can completely obscure your results.
I follow this procedure for all hunting loads. Seems like it always works unless the barrel is hopeless:
1) Pick whatever bullet you want and measure OAL with the Hornady gauge or any reasonable substitute. Back .020" off the lands and set that as OAL.
2) Pick a very temp insensitive powder that gives a load ratio between 100% and 112% at max charge weight per QuickLoad and top velocity. I've tried lots of things, but over time concluded that AR-COMP, RL-16, RL-23 and Retumbo are almost always the right answers. The other Hodgdon Extreme powders are candidates as well.
3) Use QL to figure out what max charge weight and max velocity should be in my rifle. Sanity check with other data sources.
4) Move down 10% and load and shoot a chrono velocity ladder in 2%(ish) charge weight steps (2 rounds/step), stopping when I get close to or over max velocity. Use a linear regression to figure out what charge weight will give me QL's max velocity. That's my real max charge weight and replaces the one calculated with QL. This step compensates for case capacity, chamber dimensions, primer, powder lot, crimp etc. differences that make anyone else's max charge (and QLs) a guess at best. All those factors change max charge, but they don't change the relationship between pressure and velocity a meaningful amount, so when you get to max velocity you're also at max pressue Total rounds shot: typically 10 or less unless the gun has a very generous chamber and I have to go past QL's max.
5) Use the fired cases from 4) to get any dies or accessories I need made (premium dies, OAL gauge cases, whatever).
6) Load a velocity ladder down from the new max in 0.3gr increments down about 5%. The goal is to find three adjacent charge weights where the high and low weights give lower velocity than the middle one, or all three are nearly flat. The middle value becomes the charge charge weight for all subsequent work. This usually requires about 20 rounds, although you can usually reuse a couple of charge weights shot in step 4)
7) Prep fired cases from 4) and 6) to make accuracy loads with 0.002" shoulder bump using the charge weight from 6). Typically the first group fired here will be sub-MOA if the barrel is OK.
8) If need be, do a seating depth ladder from 0.010" back in 0.010" increments (5 shots per increment) to shrink group sizes.
Usually you've got a sub-MOA, temp-stable, charge weight insensitive load within 35 shots total.