The 180gr loads are going to have less recoil impulse than the 155gr and 165gr loads. All three pretty much have similiar performance in gel, but the 180gr bullet has better sectional density, which translates into better momentum in relation to bullet diameter giving you better penetration in other materials. First, I would get one box of each and determine if they both shoot accurately in your pistol......some pistols shoot one more accurately than another. Also, keep track of your felt recoil and see which one feels better to you.
Most of the time, hugging the middle ground in all things keeps you versitile. 165gr bullet will have better impact energy so that it will punch through hard and elastic materials better. However, it sacrifices some sectional density mass so it won't penetrate through soft barriers such as car seats, wood, or a sofa as well as the 180gr bullet will. Personally, I would go with the 180gr bullet as I shoot better overall with it. A heavier bullet also allows you to do things like skipping rounds off a brick wall or the pavement to hit someone as the heavier bullet retains more momentum to wound than a lighter and faster one.