I started on an RCBS Rockchucker. Still use it, usually under a Piggyback II -- not the greatest of progressives, but it gets the job done. (Just put out about 1200 each .357 Mag and .45 ACP target loads last week.) I've got a real place to put the equipment now, but when living in an apartment, I improvised. Put a Black & Decker Workmate (one of those folding bench/sawhorse kind of set-ups) in the closet that was meant for a stacked washer/dryer. They're a little springy, but worked fine. Had the press already mounted to some wood blocks (used to clamp it to a thick, old wooden kitchen table) and just C-clamped the block to the bench. It was convenient and I could close the door on it whenever I wanted to put things out of sight. I loaded for just about everything I have on that set-up: .357/.38, .44 mag, .45 ACP, .223 Rem, 6.5x55, 7.62x39 (waste of time, for an SKS) .308 Win, .30-06. If you went with something like this, you could still have the press mounted on something, not worry about damaging any nicer furniture, and break it down after a session and toss it all in a closet somewhere. Just a warning though. You WILL want have a real place to load some time -- loading your own is too addictive not to have the equipment ready to go at all times.
Not sure about the various types of presses, as I've only got that set-up, a Redding turret, and a Ponsness-Warren shotshell loader. If you can get quality used equipment, it would probably be the best bet, since it'll do everything you want, do it well, and hold value if you decide to either quit loading or move on to other equipment.