3pairs12, I'm a relatively new reloader. I finally feel like I mostly know what I'm doing (though I'm not wizened like some of these guys). All that confusion of what's going on and how all the tools work is still fresh on my mind, so here's my advice:
1. Get a reloading manual right away. Ideally, you should purchase this first before anything else. It's one of the most direct and reliable ways to get down the basics. I have a Lyman 49th ed. If you search around, it can be had for $20-25.
2. Use your friends who reload. They are an invaluable resource. I didn't have anyone to ask questions of, which made it much harder to get started.
3. There's a great temptation to buy RIGHT NOW to get loading right away. You'll save money by understanding what you need and why you need it before buying equipment, even if it does take longer. If you must get started, a kit like shaggy430 pointed out should cover the basics (but at least try to understand what everything in the kit does and why it's needed).
4. Google everything and try to understand it all. Just remember that the stuff online is no replacement for a good reloading manual. Before I got a manual, I had a hell of a time understanding what some articles were even talking about. Terminology alone can be confusing. Also keep in mind that not everything you read online is right or the only way.
5. Youtube! I can't tell you how many videso I watched and how helpful they were to understanding the process. Again, an experienced reloader can be an even better demonstrative resource, but I didn't have anyone to walk me through it all.
6. Get a bullet puller
These tend to not be included in kits and are often overlooked by new reloaders. Kinetic ones work well once you realize that you have to whack them really hard and that they won't break by doing so.
7. If you're absolutely sure you want to reload, ignore this, but if you're not, the Lee Loader is a low cost way to test the waters. Plus it's a good portable tool to take on trips with you. Keep in mind that you'll still have to buy other things like scales, calipers, etc. and that you can only use brass that's fire-formed in your chamber or fits within certain tolerances, but it's an inexpensive way to begin.
Good luck!