There is probably a very small subset of gun owners who are so lacking in spine, that the ownership and carry of a firearm serves as a "courage crutch."
The majority simply see their daily carry and home defense weapons as tools, an integral part of an overall plan that includes avoidance, conflict resolution, and finally, deadly force if need be.
The bad guys always have the initiative, and even if their strength does not exceed ours, their ruthlessness often does. All we can do is react.
Sure there some ultra-strong, tough and intimidating bad*** types that could probably browbeat a strong-arm mugger into fleeing, or turn the tables on a knife or gun wielding perp, but they are few and far between. And however large, skilled and intimidating they may be, those advantages can be negated with a bit of standoff distance, greater numbers, etc.
At the core of your reaction to what is really a bit of verbal bullying is perhaps a recognition that you need to build up your own self-confidence.