The way I look at it, there is a huge difference between inexpensive and cheap.
One of the most accurate rifles I own was also one of the most inexpensive (Remington 788 in 6 mm Rem).
When I was a starving student, that rifle kept starvation at bay.
It also set some very high expectations on the performance I expect from a working hunting rifle. There are several aspects: Accuracy (consistently sub-MOA), "Point-ability", handling, reliability, and general lack of what I regard as unnecessary geegaws (like tactical turrets on the optics, fancy reticles, built in laser rangefinders, etc).
I guess, if I were to sum it up in one word, I like my working hunting rifles to be elegant. They should have the correct bits and pieces I think that they need and do what they are built to do well.
If you are fortunate, this can be done without spending a lot of money on the basic rifle. However,....
The problem comes if you are extremely fortunate and are able to leave your starving status far behind. Then you can spend a lot more money to get something better. The question is, how much better is it? When I was developing technology for a living, it was very easy to observe that you would spend about 90% of the project eking out the last 10% of performance. I have several good examples of this in my safe.
Often it comes down to a trade-off between time and money. But you have to start with a rifle that has good bones.
One other observation: You can buy a rifle that is very accurate, but you will not be accurate with it unless you invest your time in practice. While you are thinking about investments, you may want to add a thousand rounds of 22 lr to run through your Savage.