What's between your ears is more important than what's in front of your eyes.
What you've learned about stance and hold, how you make recoil identically handled each time, your choice of what to do with the fore arm & the recoil characteristics of whatever surface it touches or what you hold it with,
The bullet only spends a Milli second or two in the barrel, so you can't even physically react fast enough, it's what you set the gun up and how you have it set to touch things that counts as to whether that bullet will be dragged off course. People who blame scopes for changes in point of impact usually don't understand these things.
If you are hand loading, it depends on how much you've learned how to control neck tension and seating tension, how you prepare your brass, how you set your primers etc.
If you're getting really fancy, it's whether not you've found accuracy nodes on your loading and are temperature adjusted. Whether you can pull your trigger and not have your aim point Vary by more than a 10th of an MOA with a dry fire.
If you're buying commercial, and it's junk commercial with rough & inconsistent cannelures, and wildly varying seating characteristics, it's not really going to matter what scope you buy.... Junk commercial stuff maybe three MOA accuracy where as quality hunting or handloads may be one MOA or considerably better.
The quality of your barrel means a lot, how you said you were jump or touch to the rifling, all of this makes a lot of difference. Knowing, controlling, adjusting all of these things is depending on what's between your ears, not what's in front of your eyes.