I would feel comfortable with any of the popular service calibers (9mm, .40, .45, 357 Sig) when properly loaded, ie: Ranger, Federal Tactical, Gold Dot, ect........
It shouldn't be a question of which caliber is capable of performing best, but rather, which caliber is one capable of performing their best with. Ammunition exists that performs well in any of the calibers mentioned. However, none of them will compensate for poor shot placement. If bullets are not well placed, caliber will quickly become a moot issue.
Service caliber handguns are relatively underpowered defensive tools, and being that rifles and shotguns have failed, expectations of what their capable of should be realistic instead of Hollywood.
A related story that may be of interest.
David over at ammolabforum, posted of an irate hostage taker who had a pistol to a woman's neck. A sniper was given the green light to shoot, but the shot was apparently complicated by the BG holding the woman very close into his body as well as other hostages being present in the room. The sniper's weapon was a .308 rifle utilizing a 168gr bullet, he opted for a face/head shot, the bullet struck the BG's lower jaw/face area literally taking it off, the damage done being described as horrific, the BG looking like something out of a horror movie. However, the BG had enough time to get shots off, striking the hostage in the neck, before stumbling to the ground and expiring. Fortunately, the hostage survived. Prehaps if the sniper was able to place his shot COM, or more in the center head area, it would have been lights out instantly. Though as was mentioned, the shot was complicated by how the woman was being held.
The BG did of course die, though before doing so, he was still able to seriously injury his hostage. If a .308 rifle with less than ideal shot placement failed to instantly incapacitate despite inflicting a "horrific" wound, what should are realistic expectations of a service caliber handgun be? Particularly, if our shots are not well placed.
Best, jnb01