Yes it sounds like a lot of fun, and yes it sounds like a great way to get someone shot.
Let's see, hypothetical situation... some kid is trying to "assassinate" your daughter. Your daughter is coming home from an evening of studying in the library. You happen to look out the window and see her being dropped off by friends. It's dark, but you also notice what appears to be someone lurking in the shadows. Everyday you hear about sex predators, kidnappers, home invaders, etc., so you immediately react to the threat by grabbing your gun and heading outside. You get outside as your daughter reaches the porch and you see someone step out and point what appears to be a gun at her. Your reaction is to aim and fire. You now have shot and probably killed some kid with a toy gun.
You may pick apart my scenario as extreme, unrealistic, pick an adjective, but I assure you, go back and read articles about police shooting people who only were "playing" around, my hypothetical situation is totally possible. Also, I don't call it mistaken or accidental shooting, because under the circumstances of those incidents, the threat appeared real at that moment, and the response was appropriate to the perceived threat. Only when the dust settles do we find out it was a squirt gun, dart gun, whatever. When someone you can't identify is pointing a "gun" at you or your child and you're in the dark are you really going to take the time to try to locate the orange safety cap?
In a more controlled environment, like summer camp, where there is generally no reasonable expectation someone might be out to do harm, then this is probably great fun. At home, school, work, whether "against the rules" or not, someone could easily end up getting hurt. "It hasn't happened yet" you say? Well, I don't want to be the first person to have to offer my condolances to another parent because their kid was killed due to a game.