Does rebluing (or for that matter, any refinishing) lessen a gun's value?
Easy answer... Yes and no.
It depends on:
1. What is the firearm in question
2. What is its present condition
3. Exactly how will the gun be polished before the new finish is applied.
In answering I try to be specific in relation to a Colt's 1908 Vest Pocket Pistol.
This particular pistol has both a user and collector value. The latter one is usually higher. Collectors want all of the original finish they can get, and the more there is the more they will pay. Part of the reason is during the time it was made, Colt used two different methods of bluing the 1908, and neither is available today. They also Parkerized a handful they sold to the U.S. Army during World War Two. This, and nickel plating are the only regular "original finishes" that can be reproduced.
All of this means nothing to most users who would accept any blue on the basis of "blue is blue," and let it go at that.
From your description I would conclude that it has lost enough of the original finish to the point where most collectors (at least the big-bucks kind) wouldn't be interested in your pistol.
However a user won't be interested either if you have it reblued (or whatever) and it looks like it was polished on a bench grinder.
A careful polishing job will likely require a lot of hand work, to keep flats, flat and edges true. Too much machine buffing will wipe out lettering and numbers, and destroy the "crisp" look it had when it left the factory.
A top quality blue job can cost more then the gun itself is worth, and while it may add value to a potential buyer, the difference may not be enough so that if you sell the gun you can recover the cost of refinishing.
With all this said, I would advise you either pay what is necessary to get top quality refinishing, or leave the gun as it is now. The real question I can't answer is: "What is the little Colt worth to you?" Keep in mind they don't make them anymore.