The .30'06.
The Real Hawkeye said it best, but in short:
1.) you can load it from low power levels to high power levels.
2.) While a jack-of-all-trades won't be perfect for everything, I don't believe there's anything (including pachyderms) that the '06 hasn't taken.
3.) It edges out the .308 because it handles the 200 and 220 grain projectiles well, the .308 doesn't. IMO, that's the only reason it edges out the .308 -- if you're limiting yourself to the lower 48, then the .308 would take top dibs in my book.
4.) Availability of bullets, you can find just about any kind of bullet from saboted .22s to the exotic premiums in .308.
5.) Commercial loads, more variety of commercial loads in .30'06 than any other, .308 inclusive.
For small game (if legal in your area to hunt with a centerfire), definitely load some light loads, and you should probably aim for the head.
Cartridges like the .243 aren't going to do any less damage than an '06 (in fact, the higher velocities involved with the varmint calibers, will probably destroy MORE meat on small game).
Realistically, other than pachyderms, the '06 can do just fine, and not be out of the line of reason. It's less than I'd want for brown bears or even moose, but witht he heavy 220-gr solids, it will (and has) do.
Now if the question was "which longarm, would be the best overall, do-it all?" then I'd agree with the 12 gauge guys. You just have to be a better hunter and get close to your game.
Since we've answered the "which rifle" question, why not throw a Ruger Single Six in your pack (still doesn't violate the one rifle rule), and you can pop 'em with .22 short, long, long rifle and magnums, and save the rifle for the bigger stuff!