There's people way more knowledgeable than I on this topic, but I'll give my impressions:
First of all, you're right. 38's will fit a 357, but not vice versa.
The difference in recoil depends on the load of the round and on the particular revolver. From my experience with a S&W N-Frame (Models 27 &28), while there's a noticeable difference, the recoil of 357s still is pretty tame. With a S&W K-Frame (Models 13, 19, 65, 66), the difference is more noticeable, and the kick of the 357 is evident. However, I don't find it is at all unpleasant, and is certainly manageable.
I don't know about new guns, but it seems with used guns, 357's run about $50 more for similar condition revolvers. Ammo - you can find decent 38 Special ammo for about $6-7/box of 50 whereas 357 will be about $9-10/box of 50.