All the above is correct except that you CAN get a "deprime only" die that does nothing usually done at the same time as depriming. There are occasions when you would want to deprime only. I have never had occasion to do that, but some like to.
Of the Lee single stage presses in current production, one (The Classic Cast) drops the primers down the center of the ram, where they can go into a tube or bucket. The other, (Challenger) drops ejected primers off to the side into a cavity which communicates with a tube where they can go into the bucket.
The bucket or "catch bottle" is not necessary. If you put a cap on the tube, it can hold a couple hundred primers before you have to empty it. Lee supplies a tube and cap with the press.
As far as how much the depriming pin should stick out the bottom of your cartridge goes, I have mine adjusted to just barely knock the primer out. If the primer is not fully removed, you will know it pretty quick because you will not be able to remove the casing from the shell holder (the primer sticking out hangs up on it). So, you screw the deprime pin in a little more until it never happens. Lee's depriming pins are installed in the dies with a collet and are not solidly mounted. That way, if the pin hits something immovable (like a Berdan-primed casing) it will slip up into the collet instead of bending or breaking. You can tell a Berdan-primed casing by looking inside and observing that there is no center flash hole, but two off-center flash holes. They are used mostly in Europe and in some American (non-reloadable) steel or aluminum cases.
As mentioned before, the "ABC's of Reloading" has a lot of this information in it and is a good read. Check one out at your local library. "ABC's" is compiled by editors and the authors change over time, so the older editions are just as useful as newer ones. Reading about reloading from many different (and authoritative) voices is a good thing.
Welcome to reloading and thanks for asking our advice,
Lost Sheep