Shot a number of nines, they are nice, but when it came to buying my gun, I got a .40, then again it might have been because the only XD tactical they had was in .40, just felt right, and still does to this day. .45 are ok, haven't really shot a bunch, 10mm are a stretch for my hand, ie cant drop the mag one handed, and in the end it's about the ergos and accuracy, both nines and .40 do very well in those departments.
I've got a Kel-Tec p40, manufactured P40, not conversion kit, that I have yet to shoot, I understand that it's one of the guns that will teach you if limp wrist or not... Something to show that .40 can be snappier than 9mm, really got it because it was a great buy I'd be stupid to pass and in a caliber I already own (very big).