I think a lot depends on what you do and how often you do it. Holster material is really irrelevant, as wear is going to occur, its just a matter as to how much and where.
If you live, work, and play in a clean, relatively artificial environment, and only carry occasionally, your results will likely be a lot different than those of someone who is more active, carrys every waking moment, and goes about their lives in a harsher environment.
Clean, dry, leather works fine with little noticeable wear with occasional use. Once you start changing any of those variables, that can quickly change. All holsters collect "crap", leather tends to allow it to be embedded into it, and is not readily cleaned, if you can get to it at all. This can accumulate over time and will tend to accelerate finish wear. Leather holsters usually contact more of the guns surface, and guns carried in a leather holster over time tend to have a more overall "washed out" look to their finish. Rust at varying levels is also a big issue, and there is little you can do about it, once its starts, and especially if the holster gets soaked and remains wet. Leather does not dry out quickly and the problem just tends to accelerate once it does.
Kydex is better suited as a hard use holster. It doesnt hold or embed dirt like leather, is easily cleaned under the tap in the sink, and its basically impervious to moisture. Its wear patterns are usually more concentrated, and focused at the contact points between the gun and holster. Other than these contact points, the rest of the finish generally shows no wear, but those points that do show wear, tend to be more dramatic than those you see with leather. Rust is pretty much a non issue with a kydex holster, or at least you normally wont see any where the gun is protected by the holster. Generally if you do see any, its in the exposed grip area not covered by the sweat shield.
Between the two, kydex will usually long outlast a leather holster, and they are generally cheaper than a leather holster, and are much more cost effective in that respect. Factor in the gun not rusting and that repair, and its even greater.
Regardless what you use, if you practice with your gun from the holster on a regular basis, youre going to see wear at a much more accelerated rate.