I suspect a barrel problem.
Also, wait 1min between shots and 5min between strings.
You may be heating the barrel up and then when resuming firing, shooting a cooled barrel which often WILL cause a shift in zero in a light barrel.
You may also have a barrel with undersized threads/barrel shank. Under stress of firing and heating of barrel, it's casusing the zero to shift.
Try a Sierra 140gr flat-base over 46.0gr of IMR4350. If this won't shoot 1.5moa, IT WON'T SHOOT !!!
My gun doesn't like 120gr bullets nor H380. But, with 140-150gr Sierra's and either IMR,H4350, or RL17 will shoot <2moa for 3-shots.
It's a Rem. Mod-7 with 20"bbl. Not a tack-driver, but a pile-driver on game!
Try slowing down your strings of fire, and allow barrel to cool between strings and you may see it "calm" down...
One way to "treat" the barrel would be to shoot ~20rds through it as fast as you can. This will often heat up a barrel enough to "heat treat" it and relieve machining stress which will cause one to "walk" as it heats up.
If these suggestions don't help, send it to Weatherby for a possible re-barrel.
My Vanguard in .257wbymag is sub-moa for 3-shots (1/2moa with select loads), but will walk the 4-5 shots if fired quickly.
Good Luck !!!