During my college years I routinely carried a katana around with me during my frequent evening walks. During the day I carried a bokken.
I got away with the katana because it was semi-sharp. The campus police couldn't find anything in Massachusetts law against a semi-sharp weapon so they said nothing. I was also a dispatcher for them so they knew I wasn't trouble. Sometimes they'd get calls from panicked students about me, especially when I emerged from the woods after a constitutional. They'd say, "Oh, that's just Dev, don't worry about him."
After three years I transferred to Suffolk University in Boston. Since it was against the law to carry a pistol on school grounds (MGL chapter 140) I carried my bokken instead. I rented a locker so that I could stow my bokken before class and pick it up after. A few weeks later the chief of campus police came up to me and asked me why I felt it was necessary to carry the bokken. I told him flat out that it was because I couldn't carry a firearm. The chief just gave me a weird look and that was that.
After I graduated I would often go out with both my bokken and my Beretta. This went fine until one night at a gas station, where a drunken idiot took issue with the bokken and tried to start a fight. I called the cops, he ran, but it was the last time I carried the bokken. I realized that it could get me into a situation where I could potentially lose my CCW if the cops thought I was acting imprudently or looking for trouble.
My sparring buddy from college lives in Connecticut, where he has a "dangerous weapons permit." He carried an expandable baton, but he could have carried whatever he wanted as long as it remained concealed. He gave up the baton after getting his CCW and a Glock 22, as like me he was worried about appearing to be "over-armed."