Obviously a .22LR of some sort, as others have posted. The Romy trainers are good. 60 bucks at SOG :
http://www.southernohiogun.com/surplus_rifles.html
Get a few .22s - a bolt-action, a semi, perhaps even a levergun. Do most of your shooting with those, since centerfire ammo is expensive.
Next, move up to an SKS/SAR-1/WASR-10 rifle. It's politically incorrect, a lot of fun to shoot, and VERY cheap to feed (1000 rounds for 90 bucks). 1000 rounds might seem like a lot to a newbie, but you can burn through that much ammo in an afternoon!
Then, I'd satiate the black rifle lust many inevitably have by getting an AR-15 of some sort. More expensive to buy, and more expensive to feed, but still cheaper than most other rifles (a box of .30-06 at Wally World is 11 bucks, whereas a box of .223 is 4 bucks - you simply get more bangs for your buck).
Then, get some other calibers. .270/.308/.30-06 is popular, a .243 is nice, and a .30-30 might fit the bill, too, depending on if/what you hunt. This should be in either a MBR like a FAL or a CETME, or a nice bolt gun w/scope.
All told, all this stuff will set you back around 2000+ dollars. If your wallet isn't screaming by now, you are a richer man than me.