I went through my GP100 doing some smoothing recently, and found that the two areas that made the most difference in smoothness were the trigger return spring hole and the mainspring rod. I did a first round without touching either of those, and when pulling back the hammer with the grip off could faintly hear the springs grinding on those surfaces. I rounded the edge of the rod and polished it with 1000 grit, and I polished the inside of the return spring hole with a piece of 1000 grit wrapped around a Q-tip, all by hand, slowly, checking progress regularly. Both helped immensely, and have no relation to the hammer, sear, or even hammer dog, so cannot adversely affect the safety of the trigger. I polished a number of other surfaces where contact was made in the trigger group, but did not touch any hammer/sear engagement surfaces. I also added shims to the hammer and trigger, both of which had drag marks, an 8 lb return spring, and a 12 pound hammer spring. I have not had any light strikes with a variety of commercial .38 and .357 ammunition.
I did all that in stages (polish, assemble, test, repeat), and can confidently say that the trigger return hole, which was a mess, and the mainspring bar, which is a stamped piece with a rough edge, made the biggest difference in smoothness. After the work was done, the trigger is great (although I have never handled a worked over S&W so I don't have anything to compare to). The biggest improvement was the DA trigger, it is really nice to shoot now. In fact, the last few times I have shot it, I have only shot it DA. Before the work, shooting DA was not particularly pleasant.