What can happen is over time, the metal sets up microscopic stress fractures.
The pin portion of the hammer doesn't likely have the inertia required to generate stresses high enough to cause a low cycle fatigue separation due to dry firing, assuming it's machined cleanly from appropriate material with an appropriate heat treat.
BTW, fracture means the part has separated, not to be confused with a crack.
#1
In the basic sense, loading and unloading a component in a manner that exceeds the mechanical properties of the design can cause fatigue cracks to originate. Typically they originate in a weak area, with a tight radius, deep machine groove, grain structure irregularity or similar anomaly or combination of anomalies. Each cycle advances the crack front. The crack keeps growing until the remaining intact cross section can't withstand the load, at which time the crack goes critical and you have a fracture (or separation).
Firearm component failures due to fatigue would generally be low cycle fatigue, due to the higher amplitude loading, versus high cycle fatigue which is more commonly associated with vibration and lower amplitude higher frequency loading.
I once owned a Charles Daly 12ga semi auto, which experienced a low cycle fatigue failure of the bolt. The bolt was made from some kind of pot metal (high zinc content) with a quicky surface treatment to aid against wear. There were coarse machine grooves in one of the radii which made a perfect recipe for a LCF separation. I sent Charles Daly the shotgun and half the fracture. They sent me a new shotgun. Because the failure was a due to a poor design, and would inevitably occur again, I exchanged that shotgun for store credit toward a Browning Gold. Web forums get purged, so this is the only remnants of proof I could find:
http://forums.officer.com/archive/index.php/t-116304.html 5th post down.
At any rate, I think I have a decent understanding of what I'm doing to my guns.
#2
My revolver is a Ruger Superblackhawk. Unlike the 130+ year old Colt design you mentioned, it's firing pin is not attached to the hammer. Additionally, it's metallurgy is superior to that of the 1870s. It looks to me like the hammer contacts the frame every time it drops. Plus, if it can handle loads generating 1200ft-lbs of energy, then I reckon it can handle some dry firing. Lastly, Ruger endorses dry firing.