Some information on your location would be helpful to those offering advice.
Where I live: What's a gun free zone?
Carry at work: I do not carry at work, as it would be impractical, but I lock my firearm, in my vehicle and dis/re arm coming and going.
As I am unaware of where you are located, I can't be very helpful about what OC'ing in your particular area would be like. Therefore, I can only offer advice from my perspective.
I live near St. Louis Missouri. Unlike the rest of the nation, Missouri's concealed laws are pre-empted by the state, but open carry is not. This means that municipalities cannot regulate CCW, but ordinances against OC are allowed. What this does is create a patchwork of the state when it comes to open carrying a firearm.
You can be perfectly legal to open carry, but cross the street into another municipality, and you become a criminal.
The majority of Missouri is perfectly legal. Only a few municipalities have ordinances against it. These few are mostly in urban areas. As an open carrier, several friends and I have researched the local laws and discovered which areas have ordinances against OC. I have OC'ed in the urban areas of St. Louis with a close eye on my location and have never had a problem other than repetitively being asked, "Are you a police officer?" enough times for it to become a nuisance. I have even been standing in line at Starbucks in Olivette (near Ladue, a very liberal suburb of St. Louis) and overheard a Frontenac police officer standing behind me tell the man behind him, "Oh yeah. Carrying it like that is perfectly legal." I assume he was talking about my openly carried firearm. It is rare to encounter officers that are fully knowledgeable about OC law, but they do exist.
So, it depends on your location. I've read horror stories about folks getting harrassed in areas you would think OC would be easily accepted. I've often heard folks remark that they are surprised that I've never been seriously harrassed when I've carried openly in some of the places I do.
The best advice I can offer is to be 100% completely sure of the law where you choose to OC. The law is the law. Make sure you are 100% within it, and you should have no worries. Will you be harrassed? Depends on where you are at and how the local cops will react. As long as you are within the law, it would require them to break it in order to arrest you. Take all the crap folks say about "It may be legal, but that doesn't mean it's a good idea!" and throw it out the window. Since when has voluntarily participating in the expression of your God given civil rights not a good idea? Seems to me that volunteering to forfeit your rights because they can potentially make others uncomfortable is the idea that isn't very good.