Since you asked ...
First of all, rights are rights. Without trying to define precisely, anything that you can't do just because other people don't like it, or you, is not a "right".
If you are lawfully carrying because you want to, for whatever reason, don't let it stop you, just be aware...