Once.
A few months ago one Sunday afternoon one of my housemates and I see a fire engine pull up, no siren, at the end of our driveway (long driveway, over 1,000 feet, house is well set back in a wooded area) and just sit there. We go down to see what's going on and find that there are police units there as well. We are also told, bluntly, to return to the house, lock the doors, and stay inside.
About the time we get back into the house, one of our other housemates calls, asking if we know why the road is barricaded at both ends.
By this point, both I and the housemate there with me had decided it might be a good idea to have weapons on our person. We holster up and then continue going about some repairs that needed to be done to the house. About 15 minutes later there's a thump against the back door, a jiggle, and we hear it open (we were in the basement at the time).
We looked at each other and realized that we had made a very foolish error and had failed to remember that the back door was unlocked when we came back in. We both drew, muzzles down, fingers off the triggers, and cautiously made our way to the bottom of the stairs where we can hear someone walking around upstairs. (Of note, the basement is a daylight basement and the bottom of the stairs is set up in such a way that it offers both good concealment and a clear path to an exit.)
Not shouting, but loud and forceful I called out, "Hello? Who's there?"
"Hey guys, they lifted the barricade of the street!"
It was the other housemate. We were never given a reason, nor an all-clear from the police. Once our heart rates were back to normal the two of us who were home had a bit of a laugh at ourselves for worrying so much.
Then the next day I saw the story below on a local news website:
http://www.king5.com/localnews/stories/NW_030208WAB_bellevue_assault_SW.17099990.html
That house is across the street and two doors down from me. I've never felt foolish for carrying when at home since then.
The good part about all of this is that the friend who was out of the house is slightly anti, and the fact that my other housemate and I did not greet her at the door with guns aimed at her, even with us on higher alert, caused her to admit that she was impressed with how we handled it. She still thinks we're paranoid, but no longer has any fear of us shooting at random noises. So, progress is being made. We even got her to shoot a scoped pellet/BB gun, which she didn't want to put down.