I'm sorry if you mentioned it, but I didn't know if you stated what brass you're using?
I've played off and on with the Hornet for many years. I've had excellent groups, and frustratingly dismal groups for what was bizarre reasons.
I've settled on the following "patterns" and get consistent results now.
1. Hodgdon Lil'Gun for bullet weights 40gr and heavier. As you mentioned, a 120fps variation is not unusal with the Hornet with bullets lighter than 40gr wtih this powder. I've also found that larger powder charges than that given don't result in higher velocities uniformly, and may result in larger deviations. Don't ask me why! I've seen this with the .357mag as well. 17.8gr and 158gr bullet in the .357 is my "load". Tried as high as 19.0gr but results weren't as good.....
2. I use Small Pistol Primers. Considering the case volumn, pressures, thickness of brass, ect., ect. the Hornet is essentially a "Magnum Pistol Cartridge".
3. Lot# and of course mfg, of brass is a consideration. I "bench rest" uniform all my Hornet brass to include reaming the flash-holes, uniforming the primer pockets, and neck turning the brass. However, in the future I will simply sort the cases by weight and turn the necks for uniformity.
4. If possible, get some Norma, Lapua, or Sellier&Bellot, YES, S&B brass. The most accurate brass I've ever used is some S&B brass I got from some factory ammo I tried. I had to turn down my decapping pins to fit the tiny flash-holes on this brass, BUT, it gives much smaller deviations. After "lightly" turning the necks to remove some of the "high spots", and loading my favorite loads, I can expect sub-moa groups "on command". Very difficult with the Hornet, IME.
My favorite loads are as follows:
1. Sierra 40gr "Varminter" HP @ 1.780"OAL (fits my Ruger 77/22's magazine), 12.5gr Hod. Lil'Gun, Winchester SP primer. Either S&B or Winchester brass. Nets 3,000fps and Sub-MOA "usually". Have taken more deer with this load than anything else, Usually on depredation permits. Dropped 4, sequentially in 30 sec. one evening at dusk. Deer couldn't figure where I was shooting from and essentially ran back and forth across field like "a squirrel in the road". All were "head" shots, one shot, DRT. Works for me, YMMV.
2. Remington 33gr or Hornady 35gr V-Max. @ 1.755" (Touches lands on my rifle). #2400 @ 7.8gr. Small Pistol primer, Nets 2,300fps and essentially duplicates the .22wrm load. Quiet and exceptionally accurate.
3. Same as #2, except for 8.6gr of #2400. Duplicates factory V-max load in "MY" rifle. Velocity runs ~3,100fps.
4. Hornady 50gr SX, @ 1.855" 0AL (dosent' fit magazine). Acc#1680 @ 10.8gr. Winchester "MATCH PREP'ED" brass, WSP primer. I fired a 0.378" 5-shot group one evening, and had witnesses. Even with exact same 5 cases I have't been able to duplicate it. Velocity is only about 2,480fps. Have shot many 0.6-0.9" groups with this load, however.
5. Lyman #.224-415; 49.5gr SemiPt FNGC with air-cooled W/W alloy @0.225", and Hornady Gas-check. 6.7gr SR4759. Gives 2,300fps and near MOA groups at 100yds. Same bullet w/o gc, nor sized, and tumble lubed w/ Lee liquid alox over 2.2gr of Bulleye nearly duplicates .22lr. Not much louder than an air rifle. Not much more expensive either.
6. Crossman .22cal airgun pellets hand-seated skirt down, over a CCI small pistol or small rifle "MAGNUM" primer. Killed a "gazillion" pigeons while in college in the mid '70s from my Apartment balcony off a power-line with a T/C Contender w/2.5X scope. Duplicates a good "pump-up" air rifle.
Did I mention that the Hornet is one of my favorite rifles/cartridge!.....