You can't go wrong with either the .308 or '06. When I went looking for my first hunting rifle, I had a bunch of surplus boxer primed LC ammo I liberated from my supply sgt. in the Marines, I handload, so it made sense getting a .308. With lighter bullets to about 165gr, the 06 gives you no advantage at all. For example, I can sqeeze about 3025fps from my .308 Weatherby Vanguard with 150gr bullets, and haven't eclipsed 3075fps from my Winchester '06 with the same bullet. After that, however, the '06 really can push those heavier bullets.
My '06 just sits in the safe with my 7mm mag. The .308 is what I use for everything. I also like the fact that it's a short action cartridge. No magnum frenzy here.
I load 220gr Partitions for my .308, so you can go very heavy if you handload. These are fun to shoot, and 2300+fps ain't nothin' to sneeze at.
I've toyed around with getting a .325 WSM, or a .338 Federal. But I always come back to "what is the point?" My .308 does everything I need it to, it's killed elk, deer, bear, coyote. Plus it's a sub-MOA rifle. Brass is also plentiful and cheap.
Tom