Primers are pretty safe to keep in quantity, as long as the obvious safety precautions are observed. I inherited a portion of my uncle's estate (his reloading equipment) about 10 years ago, and still have some of the primers. They were the original bulk packed CCI's - cardboard packing only, stacked in neat rows of 10 right on top of each other (not the huge plastic packaging we have now!!!). They're upwards of 30-40 years old now, and they've been 100% reliable.
I load some up with those and shoot the ammo on the 4th of July every year, to remember him.
I do store powder and primers separately. Powder is stored on the opposite side of the room from where I reload. The bulk powder canisters I have for 50 BMG aren't stored at home; they're stored at my old gun shop (still have possession of the building). I haven't mounted my 50 BMG presses since I moved out here in 2008, so I haven't seen a need to bring any of the 50 cal stuff home.
The primers are stored in a metal cabinet - not a STRONG metal cabinet, I want the sides to blow out easily if it ever goes. It's merely designed to keep heat off of them as long as possible so I can get my family clear, because if those ever go, I don't want to be anywhere NEAR this building.
Powder should be stored openly (not in a metal cabinet!) as far away from potential fire-starting sources as possible. If it goes you want PLENTY of air volume around it so it doesn't accelerate and detonate. If there's room for the gas to expand the powder will burn quick, but not go KABOOM. For me, the best location is the opposite corner of the house from our fireplace, and as distant from the furnace as possible. The idea here is to provide maximum amount of evacuation time once the smoke detectors go off.
When I get a new rifle, and work up a new load, I'll get a pound of each powder and find out what works best through a couple of years of trial and error. Once I find A) I like the rifle, and B) I find what it likes to eat, I'll buy a "lifetime" supply so all of my components from that point are the same. By lifetime, I don't mean enough for ME to shoot forever, I mean the rough expected lifetime of that barrel. This way there's no guesswork. If two rifles use the same powder, but I buy the powder at different times, I mark the cans with a sharpie so I know which is for what rifle / caliber.
Just an anecdote... I've found I can't keep records for s***, so this method lets me know where I'm at at a glance. How many rounds have I shot out of {whatever} rifle? Go count the boxes that are left. I also eliminate any possibility of a new powder or primer lot throwing my loads off.
This is only for the scoped rifles, though.
On handguns, 223, and other "volume" stuff, I'm not nearly as anal. I just track generation of brass by which bucket it is in, and order in stuff whenever I find a good deal. I'll load up a thousand or two on the Dillon, shoot it until I run out, repeat.
That run on components after Obama was elected WAS a bit of an eye opener, wasn't it? It validated my hording, to a degree.
