I have a model 640-1, and I rarely carry it with full power 357mag loads. More often than not, I carry +p 38 rounds, though, because the first shot is not always the deciding shot, so I'd rather carry something I can shoot accurately, and quickly. Having fired a 357mag and a 44mag indoors without hearing protection, I now do a lot of lip-reading to compensate for my substantial hearing loss. Firing one inside an automobile would probably lead to complete hearing loss and concussion grenade-like sensory impact. And try firing one while in a grappling situation with the gun against your side, face, or any part of you person for that matter. You may do more damage to yourself than the assailant does to you.
My brother's favorite gun is his 41mag Mountain Gun. For me it's a toss-up between my 45colt Mountain Gun and my 22-4 Thunder Ranch. When we shoot together, I can get three well placed rounds off with either the 45colt or the 45acp to one of his from the 41mag. 357mag rounds thru a J-frame are similar to full power 41mag rounds thru a Mountain Gun. Not too painful to shoot in a controled situation, and in limited doses, but if multiple shots in varying positions and varying environments are a possibility I would opt for the lower recoil, lower blast +p 38's, or even just plain old 38special.
If I could guarantee that should the need to use a handgun in self defense ever arise (God forbid), I would be outdoors, have hearing protection on, and not need to fire the weapon with the muzzle in very close proximity to any part of your body, while shooting from a modified weaver stance, and not need a second shot, I might pick a 357mag, or 44mag for my carry piece. Since these variables are seldom totally controlable, I defer to a cartridge that is effective without significant muzzle blast and recoil. I usually carry a 45acp revolver, but feel confident with a j-frame with 38special if the clothing makes a larger gun impractical.
I know many guys who are comfortable and affective with a 357 (in a J-frame) or any smaller magnum caliber for that matter, but that doesn't rule out the indoor affects, or the impact the muzzle and cylinder gap blast has on exposed or even covered flesh when fired against your body. In my experience, self defense situations are seldom comfortable, convenient, or even expected. If you are going to carry a firearm in case of such an event, it is a wise idea to consider and prepare for scenarios other than a timed shooting event at the local range. Those skills are valuable, but not always applicable. More often, it is an awkward, ugly event involving distances and positions that many never practice for. If it were all about one-shot stopping power I'd carry a 500s&w magnum derringer.
I like shooting the 357mag in an N-frame smith, or a Ruger Blackhawk with a 4-5/8" barrel. Both guns are rugged enough to endure the beating, and pleasant enough to not cause pain or nerve damage. My J-frame is convenient for summer carry due to it's light weight and compact size, but shooting it with full-power 357mag loads seems oddly masochistic to me, and likely to loosen the gun prematurely. Why bother?