Open carry does not cause mass panic and chaos, but there are people that abuse it and try to either use it as an intimidation tool or because they are the special kind of person that tries to be mistaken for a cop. There are plenty of people who open carry regularly and are normal everyday people, but yes, to answer the OP's question, there are some bad apples.
That being said, OC is not for everyone, but it's not a bad thing either. Sometimes when I am wearing a holster with a jacket over it, I might want to give my jacket to my girlfriend because she is cold. In that kind of a situation, it's nice to know that I have the option. Usually I won't do something like that unless there are very few people around, only because I don't like worrying people, but I've OC'd before and for the most part it was never a big deal. You'd be surprised how many people never even notice.
I don't OC anymore. I used to, but I really don't like worrying people, and no matter how friendly and normal you are, some people are still worried. More power to the people that do OC, but it's not for me anymore.
ETA:
I was also getting tired of being mistaken for a cop. I'm not a cop and I don't want people to think that I am.