They do to me, as do fanny packs on guys who don't look like rollerbladers, but that's because I'm a gun-type. Probably doesn't mean anything to sheeple.
This also means you are seeing people with guns when they don't carry guns because you think that because you know some gun folks wear vests and/or fanny packs to carry guns, everyone in vests or fanny packs must be carrying.
The true give aways are when vest and fanny pack wearers don't actually use their vests or fanny packs for anything normal. I laugh when I see a weight laden fanny pack being worn by a person such as a woman who is carrying a purse into which I have seen her access money, phone, keys, etc. or a guy I see with a wallet in the back pocket, cell phone on the belt, and keys in a front pocket. You have to ask, what is in the fanny pack?
Also, wearing the boxy brown or black leather fanny packs doesn't help. Roller bladers were mentioned - good example, but also tourists and hikers. When do you see them with these really ugly boxy fanny packs?
The same holds for vests. Why do folks wear vests with 400 pockets and then not use any? Camera folks, hunters, and fisherfolk actually carry stuff in their vests.
For city life, I carry stuff in my fanny pack and vest. I use them. They look like I use them. When my kids were babies, I had the coolest darned babybag any other male parent had ever seen. In my hunter's vest, I was able to put 3 or 4 diapers in the rear map/game/sandwich pocket. Wipes, cream, powder were in front cargo pockets. A bottle of milk in one waterbottle pocket and a bottle of juice in the other. Other pockets contained snacks, toys, etc. With things like diapers and wipes inside your vest, you can carry that minigun that is being discussed as for sale in another thread right now and nobody will know it. There are no pink or polkadot baby blue diaper bags for Double Naught Spy to carry.
Why don't we suspect women and their purses more? Simple. Women really do carry stuff in their purses. We see them use their purses. If you don't use a vest or fanny pack for things other than concealment or carrying of guns, others pick up on the fact that there is no reason for you to look silly, but a camera guy doesn't look silly with camera gear and a vest or a guy in a fanny pack pulling out his cell phone from the pack to answer it.