Grips make a lot of difference.
But I will readily admit my super black hawk in 44 mag sees as much 44 special as it sees 44 mag ammo. Actually I guess it sees more 44 special than 44 magnum.
I did not like my ruger redhawk in 44 mag, aftermarket rubber grips were too big overall and the stock wood grips just sucked for me.
I don't know that I will ever wind up with a 454 casull, I sort of want one but I don't want the cost of the ammo. But if I do get one it will see lots of 45colt ammo since those are available in cowboy loads which are pretty easy going.
Overall I recomend working your way up in calibers if possable. For instance, I was still messing with 357 magnum loads when I got into shooting 44 magnum stuff. Since 357 was not a settled in round I was used to the 44 mag did make me flinch.
A search on flinch will turn up ways to get rid of it.