If you shoot just for the sport of shooting much of the time at all then a .22 is a nice gun to have and would be a good FIRST rifle. Note CAPS on that. It won't be the last you buy.
The thing is you can learn a lot about proper stance and trigger control and reading wind drift by shooting a lowly little .22 that will give you great grounding for shooting a long range center fire rifle.
From there consider the sort of distance and terrain in your area as well as any local hunting regs concerning calibers and seasons and pick out something that fits. Without knowing those particulars it's nearly impossible to make any sort of suitable recomendation to you.
If your area is mostly tight woods and the general game is smaller deer or similar then you certainly don't need some flat shooting 1000 yard Super Mag round. For tight areas where you'll almost never be able to take a shot longer than 100 yards you may find that a lowly iron sights lever rifle in .30-30 or .45-70 or something of this sort is fine.
If you want to plink mostly for fun then surplus ammo shot from surplus guns makes great sense. If you want to achieve consistently good accuracy then not so much. At that point you're better off even with something like a Savage Axis package. It's fairly cheap but still far more consistently accurate with factory ammo than most milsurp guns running milsurp ammo will ever be.
As for hunting are you looking at this as justification to get a rifle or three or are you serious about taking up that sport/pursuit? I ask because so many forget that there is more to hunting than the thrill of the stalk and the adrenaline of taking the shot. Once the game is down the HARD part is just starting.... not to mention that if you ask a bunch of HONEST hunters that they will comfirm that each pound of meat in the deep freeze cost easily upwards of $30 a lb when all the expenses are taken into account...
So sit down and honestly answer yourself about what it is you want to achieve or if you're simply looking for a fun way to kill some time.
While you're at it look around at the nearby clubs to see what they have for shooting competitions. If you shoot for fun then often it's nice to join such groups for the comradery and ability to see what others are using and how your own efforts stack up. Yeah, it'll be hard on the ego at first. But you'll also progress far quicker under the pressure. And generally the better shooters are very free with helpful information.