Zero to two inches depending on tightness of pants, holster design, your squishy factor, and width of gun.
An XD is pretty wide, probably close to two inches in shorts. Particularly if you'll be carrying your spare magazine(s) IWB.
Sounds like you're new to carry, has anyone told you that you need a real gunbelt to carry anything but a lightweight gun on your belt?
Trust me, don't skimp on the belt and make sure the holster is a slim design, too. Carrying my XD45c is fairly easy, but I'm a big guy and have a real gunbelt and a SuperTuck holster.