First, be aware that if you're going to do gunsmithing work on other peoples guns, the Federal BATF requires that you have a Federal license.
This especially if you keep someone's gun over night.
Working as a gunsmith, even as a hobby, without the license is a Federal crime.
As to the tooling.
For screwdrivers, if you just have to go cheap, the Wheeler sets from Midway USA are "okay". They're made in China, and the quality is usable.
If you break or damage a bit, you have to buy another one.
The best and the industry standard are the Brownell's Magna-Tip sets.
These are the finest quality, very hard and very smooth.
If you break a bit, simply email Brownell's and they'll send you a new one free.
You can buy the Magna-Tip in 3 or 4 bit sets with a handle for a specific gun, or you can buy master sets with various handles, and buy individual bits in other sizes as needed.
I recommend buying one of the Law Enforcement size handles. These are much more useful for most work, since they're just the right size.
Since you'll use a screwdriver far more than any other tool, this is where you splurge and buy the best, which is the Magna-Tip.
Gages depend on WHAT you need to gage. Unless you're going to be doing re-barreling of rifles, or checking head space on older guns, buying expensive sets of head space gages is rather a waste of money.
Buy only specific types of gages you really need for the type of work or guns you'll be working on.
Sanders suitable for recoil pad work can be bought at Sears or Harbor Freight.
Stock refinishing sand paper and sanding blocks can be bought at a local hardware store.
The stains and finishes can be bought from Brownell's, and if you use polyurethane or one of the oil finishes like Danish Oil or Tung oil, many hardware stores sell them.
There are dozens of types of finishes, so after picking one you want to use, you buy it where it's most available.
For stains, I recommend the various solvent and water-based stains from Brownell's.
These penetrate quickly even in old oil soaked wood and give true colors.
Buy most everything from Brownell's or Midway USA. Brownell's is THE gunsmith's supply house. Get online and look, but you're better to start by buying a hard copy catalog from them and using the online system for later. The price is refunded on the first order.
For other starting gear buy:
A small ball peen hammer.
A small brass or fiber hammer.
A good set of punches.
I recommend the Brownell's replaceable tip type. When you bend or break a punch all you have to do is replace the tip.
You can also buy or shorten tips to use as "starter punches" and special tips for roll pins.
An Opti-visor magnifier.
You can't work on small gun parts without one. Opti-Visor is the industry standard and you'll wear it almost constantly, so buy the best.
A good bench light.
Several vises.
You'll need one bigger shop vise and one smaller adjustable vise.
For the small vise I recommend one of the best I've seen in years, from Harbor Freight:
http://www.harborfreight.com/2-1-2-half-inch-table-swivel-vise-97160.html
Buy good files and a wire file "card" brush.
You'll need a small set of needle files and larger hand files WITH handles. Some from hardware stores are usable, but the types needed for gun work are bought from Brownell's or jeweler's supply houses.
Stones.
You'll need small, thin stones and at least one larger flat stone.
For the small stones I recommend the 6" square and triangular ceramic stones from Brownell's. These are better then other synthetic or even the hard Arkansas stones. The ceramics don't wear or dull the sharp edges like other stones, so they last a LONG time.
Books:
Books on gun disassembly are good. Gun Digest and the NRA sell excellent paperback books showing disassembly, and there are several paperbacks showing exploded views.
Depending on what guns you want to work on, I strongly recommend buying some of the Jerry Kuhnhausen Shop Manuals from Brownell's.
These were written as training aids for new gunsmiths, and cover one specific type of firearm in deep detail.
These show the factory method of repair, not the old jack leg methods or heating and bending or making parts.
You'll find these in almost any good gunsmiths shop.
As you get into it, you'll know what you need to buy and when. Buy tools only as you need them or you'll wind up wasting money on tools you won't use enough to pay for, or tools you won't use at all.
Just remember, having a FFL license to work on other peoples guns is not a joke. They can and will prosecute you.
Last, remember, that even doing free or really cheap work for friends is no protection from an unhappy customer from suing you.
I was in the business as a professional for 30 years and I saw some cases of the best of friends turned into bitter enemy's and hiring lawyers over a gun mistake.
I've heard from another gunsmith at least one case of a hobby gunsmith having a dispute with a buddy and the buddy calling the BATF and reporting the friend for working without a license.