I'm going to offer you another idea on why your bullets are key-holing.
I've had this happen with several different Lee Tumble Lube moulds. It's the design of the particular bullet and gun you're shooting it in.
I've had this happen with the .40 175gr TL, and the 124gr RN and TC in 9mm.
The bullets just won't take the rifling and "strip" the grooves, and "shoot sideways".
Two possible solutions: {actually doing both will help most}
1. Use wheel wt. alloy with about 2% (by weight) of leadfree solder added (95%tin/5% Antimony).
2. Use 5gr of Bullseye. The 9.0gr of Bluedot, though a light load, is driving the bullet faster than it will stand.
I've had this problem with most any lead bullet in the Glock .40, and by using a harder alloy, conventionally lubing with 50/50 Alox, and using 3.9gr of Bullseye has given decent accuracy with the 180gr FN-TC Lee Bullet at .401"(not the TL-SWC!).
The Charter has shallow rifling and is not real compatible with the Lee Tumble lube design. Try the 240gr SWC-GC. It will give much better accuracy with any reasonable load.
Sorry for the bad news, but I've been down that road before with the Lee TL bullets. I've had good luck with the .38spl SWC, and RN moulds as well as the DEWC. I've gotten "so-so" accuracy from the 230gr RN .452.
Always, they perform best if driven to less than 800fps. Though the .401" 175 SWC's haven't shot well in anything I've tried them in. However, my brother reports good results with them in a Beretta 99 and the 3.9gr of Bullseye load. Cleaning every 40rds or so, helps too.
(Not a slam against the Lee's, I've won a lot of matches I've shot with them in .38spl w/148gr DEWC over 2.6gr of Bullseye. I shoot about 30,000 a year of them, and have worn out two moulds, and now on my third one in 15yrs. -6-cavity moulds!)