The first sidearm I carried regularly was an M9, in USGI nylon holster. Sometimes I also wore it in a shoulder rig for armory duty. It was bulky, but not what I'd call heavy. Concealment wasn't an issue at the time.
In my LEO days, my trusty duty gun was a Sig SP2340, and I'd carry that off-duty (concealed) a lot. I also had a Sig P239 9mm with an IWB holster, that was about as small of a pistol I could still feel confident with.
When I got a TX CHL, after not carrying anything for a few years, I decided on my G30 and a Don Hume clip-on IWB as a primary. It felt heavy and bulky for a week or so, now I hardly even notice it's there. I'm 5' 10" and 165 lbs, not a big guy by any means, and can easily conceal all of the above in just about any attire with no problems. You don't need to spend a lot of money on designer rigs, just need to find what works best for each individual.