Don't buy junky cheap lowers and you won't have that problem.
Admittedly I am an AR snob who only buys rifles that meet the TDP for the M4 (minus the full auto trigger pack and 14.5" bbl), or exceed it.
On that same note I'll give my vote to buying a Geissele Automatics trigger as your best bet for a reliable drop in trigger for an AR. Geissele is the only aftermarket trigger manufacturer to my knowledge to make a drop in trigger group that is safety certified by the Crane Naval Surface Warfare Center. Thus they make the only trigger groups that are authorized for the M4 SOPMOD program.
For a general purpose trigger on an M4 with a carbine length gas system I'd say it will be tough to beat the Geissele SSA, SD-C, or Super Tri-Con. These are all two stage trigger with a total pull weight of 4.5lbs. I hesitate to recommend a lighter trigger in a carbine length gas system because the carbines tend to be over gassed a bit resulting in faster bolt carrier speeds and more recoil forces to the rear, and then due to a higher buffer tube spring weight higher forward bolt carrier speeds on bolt closing. This results in more fore and aft movement, and can result in unintentional doubles with too light a trigger or one with too short a reset. Which is another reason the two stage trigger is preferred here as a more forgiving and safe trigger to use under stress.
The less bouncy gas systems you can probably run a Geissele SSA-E, SD-E, or even a High Speed national match trigger. Again all two stage triggers in the 3.5lb range, and all of them possessing the .mil certified sear geometry and parts quality that means a safe durable trigger group.
The last is the Geissele SD-3G, this is the Geissele single stage trigger. I had one in a Colt M4A1 SOCOM II (civilian version with the DD RIS II rail, and medium contour M4A1 barrel), and it was not safe. The reset is crazy short, and the trigger pull is 3.5lbs. Even maintaining a solid hold in the pocket of my shoulder the rifle could bounce back enough to reset the trigger and then trip it again from inertia as the bolt carrier group closed. I had to consciously smash the trigger and hold it down to the rear to prevent this. Not exactly good for deliberate shooting, but great if you want to hose down targets in a 3 gun match, which is what the trigger is designed for. I should have paid attention to the Geissele video on what this trigger is meant for. At any rate the super short reset triggers with low trigger pull weights have no place on a carbine length gas system weapon, and probably are totally inappropriate for a weapon that is anything but a range toy/race gun. I will note that my SD-3G trigger worked safely and perfectly when combined with a mid length gas system, and I acquired a Daniel Defense DDM4V11 PRO with an SD-3G in it that has also been perfect, but it is a very smooth rifle length gas system. I'll be changing out the SD-3G for an SD-C, or an SD-E here soon since I strongly prefer the more predictable feel of the two stage trigger.
For those who scoff at the notion of a two stage trigger being what you want in a fighting carbine/rifle I submit this from Pat McNamara, who has vastly more experience than I do or probably any of us at showing up at the appointed time and place to shoot savages in the face:
https://www.facebook.com/AliasTrain...472294.43053.197061490420046/355644741228386/
I do know that during my time in the USMC both stateside and in Iraq, I would have loved to have an issued service weapon with a Geissele two stage in it.