Are you dropping the slide on an empty chamber? You shouldn't.
That is letting the trigger bounce the hammer off the sear, and far enough to let it fall to the stop. It will eventually batter up the engagement until it won't hold full cock at all. The grip safety blocks the trigger and by going through contortions to drop the slide without a normal grasp on the gun, keeps the hammer from following the slide down.
If it is doing it while chambering a round, it is already beat up and likely needs recutting or replacement of the hammer and or sear.
The very least it could be is insufficient sear spring tension, but don't bend the spring as a bandaid if the hammer or sear is battered.