It sounds like a tough call to me. Obviously the "off-duty cop" should have handled it better if he had a safety concern, and the OP could have asked for ID or stood up for himself a little (though I don't blame him a bit for doing exactly what he did, I'd have probably done the same thing).
Cop or not, it sounds like the guy was throwing his weight around, showing off his "authority" for his buddies. I have seen this before, with a young Forest Service authority of some sort (I forget). I was burning some brush (safely, right time of year, on 40 acres out of town). Long story short this guy saw smoke on the way back from some wildfire conference of some sort, in the company of some young women in the same vehicle, stopped and totally went off. I reacted politely, the same way the OP did, but it sure left a bad taste in my mouth, even now years later. My neighbor at the time was an old logger. He stopped by to look at it and laughed him head off, said he knew the guy and that he was just showing off for the ladies, at my expense.
One more story from many years ago, I guess it really doesn't apply here but I like telling stories
: A young, single female friend of mine was driving from her apartment to a health club in the evening and was going through a "lead foot" phase in her life. She was driving along a road near her house WAY too fast. A pickup pulled out in front of her and very deliberately slowed down. Like an idiot, she zoomed around him in a most-definitely no passing area. Suddenly the truck roared along side and passed her, skidding to a stop crossways in an intersection in front of her. A guy jumps out and comes at her screaming. She panics and goes up on the curb around him and gets away, telling all her friends at the health club about the maniac who tried to get her.
The next day, he shows up, in uniform, at the hospital where she worked as a nurse, found her and totally ripped her up one side and down the other. He told her that he came there with the intention of hauling her off to jail, but basically seeing that she was an attractive young nurse, he would let her off with a warning.
She was scared stiff and grateful to avoid a huge ticket for her admittedly reckless driving. I told her that while her driving was way out of line, his reaction was so terrible that had she pursued it, he could have lost his job and been in huge trouble, really should have. He let his anger at her driving cause himself to behave in a dangerous manner (can you say "Road Rage?).
Cops are people too, and make mistakes. I know it can be a tough job, and I really respect those that can do it professionally without letting their anger, ego, or emotions effect their judgment.