Exactly.Whatever die you use, whenever you have neck thickness variations, they always end up on the outside of the neck when you have a loaded round. In the case of the bushing die, the bushing transfers the variation to the inside of the neck. Then, when you seat a bullet, the bullet acts just like a mandrel, and the neck thickness variation is pushed to the outside of the neck. If you are a proponent of another method, fine.
Unless you are loading for a tight necked chamber, you can only go so far turning/reaming necks.
Buying the best brass is your best bet for accuracy. A sound house needs a sound foundation.
There are so many things that make such a much bigger difference in accuracy it is silly to worry about something that can make a few thousandths difference.
Invest in a good action, barrel, etc etc. Practice until you can shoot up to that potential, and then go from there.