I think it's very possible to overthink this.
I think a lot of the trend in wearing things like 5.11 khakis comes from a healthy defense industry, and a lot of contractors and consultants wearing them. If you work for one of these outfits, they have very strict rules about carrying weapons on the job.
I wore BDU pants a lot when I was a teenager. But if I see them now, I usually see a skater or hipster, not a guy who can't wait to get a chance to pull the trigger on someone.
I don't much care anymore. I usually wear a ball cap from Magpul with a 3rd ID patch velcroed on the front, and a molon labe tab on the back. A lot of my clothing is t-shirts from gun industry vendors. I wear a Luminox watch, which a lot of people recognize as Navy gear. If the sun is shining, I will be wearing ESS shades with Rx inserts. The same ones I wore in Iraq.
I look at it this way. I lived in northern Europe for four years, All of that time I was in some kind of active or reserve military status. It never once occurred to me to try to hide that. But now, when I go overseas TDY, the guidance is to not stand out, don't tell anyone you're military, don't carry baggage that looks like you got it from the army. To be honest, I find it absurd. I would rather face any confrontation that comes my way than let bullies, cowards, and terrorists tell me how to dress. But I guess I'm just a different breed of cat. (And I don't want to go buy new baggage.) It's just a bit late in the game to start worrying what will happen if people realize I am in the profession of arms.