I want two guns, period. I'll take something short and handy in any centerfire caliber for big game and thugs, and a Browning Buckmark for small game.
The problem with thinking you're going to live on deer and elk is that after a couple of days you won't be able to eat that deer or elk, because it's spoiled. The reality is that it's pretty easy to pot rabbits and squirrels or shoot ducks on the water with a .22 on a day to day basis. It's also pretty easy to forage for food - river clams, cattail roots, berries, mushrooms, pine nuts, acorns (have to be cooked to be edible), fish, frogs, song birds - the reality is that there is food all around you and you don't need a heavy rifle for food.
You'd really only need the centerfire rifle for defense. You can shoot a deer if you want, but you're only going to eat about ten pounds before it spoils on you. It might make more sense in the winter when you can preserve the meat, but even then it would be easier to take small game than big game.