They're a law enforcement branch of the federal government. No matter what kind of tax stamps you've got, they can't just come in your house without a warrant. The procedure, to the extent there is one, would be for them to call you and say they wanted to see your firearm and tax stamp. You'd have the option of bringing them to the local field office to show them. They never have the right to be inside your house without your permission unless a judge thinks you've done something wrong.
And there's no legal requirement that you store anything in a safe. It's not a bad idea to have a safe, but it's not legally required. What is legally required is that you don't make an illegal transfer to another person. If you have a safe that only you know the combination to, you can leave other people alone in your house. But that's just a practical consideration--not a legal requirement.
Aaron