The first time I open carried was after a Friday night at the range when we went to a restaurant with a group of friends, and it was definitely easier to do in a group. We once went to a CHAMPPS restaurant in Reston and were "reported" by the management. The local police came, had a brief discussion with a few of us, and left after we explained the situation. We finished our meals and left. Ultimately the incident made it into the Washington Post, mainly because Reston is such an anti-gun area. CHAMPPS restaurants subsequently became pro-gun, though I'm not sure how that evolved. Fairly sure our little "project" had something to do with it.
That was the only time I was ever remotely "hassled." Since getting my CCW I open carry if I'm in a restaurant, and transfer my weapon because concealed carry is illegal in restaurants/bars, etc.. where liquor is served. As a rule no one even notices. I once even carried in a Trader Joe's, which is usually packed with PC types, and no one noticed... or if they did they didn't let on.
I can see being somewhat self-conscious about it at first, but you get over that more quickly than you'd think.