If it were a choice between kydex and leather, I would go with leather, mainly for aesthetics, but also because it seems much more supple than kydex.
Consider the Uncle Mike's IWB neoprene holsters. They are comfortable, easy drawing, and can be worn cross-draw. It does take a little longer to reholster, but that is not the stage where speed is of the essence....