Yes, handgun barrels can absolutely be worn out by shooting. I suspect that most people don't shoot enough to do it, and I suspect that most people wouldn't notice until the issue is pretty severe.
In my experience, the absolute gilt edge of accuracy can be worn away with as little as 5000 rounds of hot jacketed bullets. Ten to fifteen thousand of those rounds normally will show significant wear in the bore as well as on the target - as an example, a four MOA gun will often become a six-to-eight MOA gun by the 10,000 round mark. My highest round count, with jacketed Magnums, was in a 4" .357 which, when new, would hold about six MOA. At 35,000 rounds with jacketed bullets and H110, AA#9, and 2400, the gun was no longer capable of keeping rounds reliably on a B-27 silhouette target, which is roughly 2'x4', at 100 yards.
Lead/cast bullets are a different story, but still can wear a bore. My primary competition revolver for many years was based on an S&W 686. Many hundreds of thousands of rounds went through it, but very lightly loaded with nearly pure lead bullets. It started out as about a four MOA gun and now is about a twelve MOA gun. The first inch or so of rifling is very noticeably worn (and it rattles like a G.I. 1911) but rapid fire at seven or fifteen yards it still hits about as well as the average out-of-the-box duty piece - which goes to my theory that most people don't notice wear in a handgun bore.