hunting experience
My hunting experience says no, caliber doesn't really matter. In fact, for me, smaller calibers tend to be more effective because I can shoot them better.
Pheasants have a reputation for being tough birds and hard to kill. Lots of people go after them with 12 gauge, #4 shot, magnums. A few months ago, I found myself going after them with ordinary, low-brass #7 1/2 shells in a 20 gauge. I found that if I put the pattern on the bird, it would die just as quick as if shot with a #4, 3-inch magnum.
On the other hand, the difference between a .22 solid slug and a hollow point can be huge. I've killed small game with both, and the hollow point is a LOT more effective than the solid bullet.
No experience with big game.