There are three main factors that control POI due to differing loads. I'm going to ignore trajectory because at typical handgun ranges, this is usually not an issue
1. Gun design. Some semi-auto pistols are set up so the slide/barrel move straight backwards while the bullet is in the bore. In guns like that, there won't be much POI change from various loadings because during the time the bullet is in the bore the slide/barrel are pretty much only moving backwards--you get recoil, but the recoil isn't causing significant muzzle rise until the barrel & slide hit the frame and transfer momentum to the frame. In guns where the barrel is fixed to the frame (or some semi-autos with links where the barrel starts tilting almost immediately when the bullet starts moving) then recoil and muzzle rise happen together and you can see significant changes in POI due to different loads.
2. Muzzle momentum. More momentum means more recoil and in revolvers and other guns with the barrels fixed to the frame that means that muzzle rise starts when the bullet starts to move. The more momentum, the more it will rise. Momentum is the product of the velocity of the ejecta (everything that comes out of the muzzle) and the mass of the ejecta. This includes the gases generated by the burning powder, but for the purposes of comparison, you can often get away with just looking at the bullet momentum.
3. Barrel dwell time. In two loads with the same momentum, the one with less barrel dwell time will hit lower on the target.
So let's look at some loadings.
Let's take a 125gr bullet at 1450fps muzzle velocity and compare it to a 158gr bullet at 1147fps muzzle velocity. Both are shot from a 5" barrel gun where the barrel is fixed to the frame.
It turns out that the muzzle momentum is virtually identical for these two loads. So now we just need to see what the barrel dwell time is. We're going to estimate that by taking the average muzzle velocity while the bullet is in the barrel which is just the muzzle velocity divided by two. This estimate should be good enough to give us an idea of what is going on. We take the barrel length (converted to feet) and divide it by the average velocity in the barrel to get the estimated dwell time in seconds. These tend to be very small numbers.
It turns out that the 158gr bullet stays in the bore 26% longer than the 125gr bullet. So with the same amount of recoil (and therefore the same amount of muzzle rise in this gun design) we would expect the 158gr bullet to hit noticeably higher on the target.
What about comparing that 125gr bullet at 1450fps to a 142gr bullet at 1250fps? Again, both will be shot from a 5" barrel gun where the barrel is fixed to the frame.
This one is a little more complicated. The 142gr loading is about 2% lower in muzzle momentum but barrel dwell time is about 16% higher. I would guess that we would see a higher point of impact. Even though the muzzle momentum is a little lower, the bullet's in the barrel significantly longer.
What about a 125gr bullet at 1450fps and a 110gr bullet at 1100fps in our same test gun? Muzzle momentum is about a third lower in the 110gr loading but barrel dwell time is about a third higher. Not easy to tell if there will be a POI difference on this one. This one would give you a good excuse to go to the range and do some testing.