Another vote for starting with a used .22
7.62Comm ammo is 12c a shot, .22 ammo is 2c a shot, do the math!
A good used Marlin 60 (or similar) .22 rifle will last for decades, and costs around $50-75 in Austin, TX.
There are some inferior or obsolete brands floating around out there, so stick to the Marlins. There's a good reason they've sold something like 10 million of them in the last 46 years.
The older ones hold 18 rounds in the tube-mag, the newer ones 14. Has a built-in groove to take a rimfire scope. Just enough recoil to feel like a real gun, but easy to handle.
If you go out and really practice (not just blast random rounds at the target) with a Marlin 22, you'll build up some major skills pretty quick.
After a while with that, you'll have a good feel for what you like in a rifle. You can buy something bigger, but I bet you'll still bring the Marlin every time you go to the range. I've fired everything from .177 to 155mm howitzer, and my $50 Marlin 22 gets the most use out of all my rifles. For handguns, its my Ruger MkII .22 pistol.
Good luck,
-MV