I was heavily into Civil War reenacting in the 1986-89 period (the 125th anniversary events). I would say the first thing you need to do is hook up with a good unit. The hobby is built around local units (clubs), and there aren't any big national organizations worth mentioning. (This is as opposed to Civil War skirmishing, which is another hobby entirely, and is run by the North-South Skirmish Association.) The best way to find a unit is to attend a big reenactment as a spectator, and ask around.
As others have said, a unit that you join will probably lend you most of the equipment you need, to start.
You're going to end up having to buy at least a basic uniform (shoes, pants, jacket, hat), basic weapons (3-band rifle-musket and bayonet), and basic gear (cartridge box & sling, belt, cap box, canteen, haversack). Figure on $1,500 - $2,000 for everything.
The search for authenticity is a big thing in reenacting. Participants can be characterized as "farbs" ("far be it for me..."), mainstream types, and "hardcore campaigners." Many reenactors typically proceed up this ladder of categories until they reach the stage of "burnout" and drop out altogether.
Sometimes authenticity can be measured by what you don't wear. Wristwatches and modern eyeglasses are big no-nos here.
If you want to see the best of reenactors in action, watch the movie "Gettysburg" (working title was "The Killer Angels"). This is full of reenactors.
By the way, because of safety concerns, reenactors don't use live rounds. It's blanks only. In fact if you are found with a live round at an event, you will most likely be ejected. (Some events even ban ramrods.) If this is what you mean by "shooting," well and good. Otherwise, you're better off joining the "skirmishing" hobby, which is all about target shooting with CW weapons.