No reason to question your gut, it's what saves lives. When a scruffy looking guy came to my door some weeks ago (no visitors here) I did not open the door to talk to him, but yelled thru the glass. Sure he was looking for a neighbor I think, but who can say? It wasn't normal. And around here, people are known for leaving doors open...all the time.
As far as any stereotyping, well, yes, I wouldn't trust me if I saw myself approaching, scruffy/hands in pocket/baggy pants/bad posture. But more than that, you probably wouldn't have trusted a guy in an Armani suit who was "trotting" or whatever you said towards you. You just don't do that at night. Don't for a second think you should be kinder and gentler next time. Like flipping a coin, each time is independent of the last.
As for the flanking...what do you if you wife isn't carrying, and someone lunges at her and you don't fire or you fire and miss, and she becomes in danger? Behind you, you know where said person is heading, and would be firing at something you have aim at. Not trying to disrespect a general rule or something, as I'm positive the people who posted it have more experience than me...but just begged the question in my mind.