Just call the police and say you'd like to report what you believe to be a felon in possession of a firearm.
You can do this anonymously; I've called on suspicious things before, and they usually give you the option to remain anonymous. Just give them the story as it's been presented to you, and let them take care of it as they will.
The guy will never know you called the police, and the police won't hunt you down if it's false. Better safe than sorry, and it takes it off your mind.
I think "minding your own business" is a bad idea, as is worrying about it incessantly. Make the call, give your story as you understand it, and let happen what will.
Good luck.