Kydex will wear the finish faster than "most" leather, but only in the areas where it actually makes contact with the gun.
Good Kydex holsters only grip the gun in key areas, like around the trigger guard and ejection port.
The plastic will abrade the finish rather fast, but the wear is self-limiting. It wears those small areas and no more.
Leather is "designed" to flex and stretch, and as a leather holster wears, it does stretch out of shape.
As it does so, the leather makes more and more contact with the gun.
Result is, the more areas the stretching leather touches, the wider and wider the gun finish is worn.
For this reason, you'll see a pistol carried in a good Kydex holster with finish wear on the contact areas and not much else.
A gun carried in a leather holster, especially a soft one or one that's old and stretched from use, will have the entire gun's finish worn.
The best leather holsters from the stand point of not wearing the finish are modern STIFF, HARD leather holsters .
The worst are those lined with suede, "elk", or pig skin.
The soft, sueded leather attracts and holds dirt and grit, and the soft surface makes more contact over a wider area than a smooth inside finish holster.
The combination of soft surface and grit abrades the finish.
GOOD Kydex holsters will wear the contact areas fast, making some people think it's going to really wear all the finish fast.
Since the inside of Kydex is slick and smooth, grit and dirt can simply be wiped or better, washed off.