When this occurs on my pistols, almost always as a result of swapping safeties, I first make sure it is engaging the sear correctly, and fitting up into the slide notch all the way. I make certain the plungers are working as advertised.
Next, I make certain the safety has about 1mm between it, in the off position, and the slide flat when the slide reciprocates. If not, I file some clearance in the upper corner.
Finally, when I am satisfied with where the safety is and how it functions, I examine the recess in the safety that the plunger engages. This recess can usually be massaged a bit to make the safety more likely to stay engaged. Usually there either is no recess in the safety, or it is in the wrong spot.
I use a small round diamond Dremel bit about the size of the plunger itself to gently wallow the recess over, or to cut one altogether. The recess should not be exactly round, but rather teardrop shaped, with a ramp going up to help ease the plunger on it's path as the safety is snicked off.
I am betting your plunger is not engaging the recess in your thumb safety, or that you have no recess in the safety. It could be a host of other problems though, especially if it is the original thumb safety. Check the function of the plungers, remove them, polish them, oil them and reinstall them before you start cutting on parts. Remember it is easier to remove metal than it is to put it back.