It really depends upon the purpose. If you ask a sportsman, he'll tell you some combination of target guns and multi-thousand dollar shotguns. Ask a self-defense instructor and you'll get a different answer.
My take from a fighting perspective:
1) Pistol or revolver in the maximum caliber you can shoot quickly and accurately
2) Hunting style rifle accurized and with a scope in a common caliber such as 30-06, 308 Winchester, or whatever. It could be a semi-automatic Browning BAR, a traditional bolt action hunting rifle, or some lever gun. Just make sure it has iron sights as backup for the scope. Even a military surplus rifle, such as a Mauser or Swiss K-31 would work fine (assuming you can get a scope).
3) Rifle for fighting such as an M1A, Garand, FAL, AR-15, or AK-47.
If zombies are involved, you should probably go with something in the AR-15 or AK-47 class with large magazine capacity.
#2 can be used for hunting or fighting, as needed. It can also be used competitively. If you go to military rifle matches, many of the rifles listed can be used.