A bit of epoxy could be used. I have done that with 1911 grips to fill in the hole a bit and with S&W grips filling the gap between the frame and the grip cut out. Make sure there is release agent (comes with Acraglas) or automotive paste wax for other epoxys, such as Microbed, on the frame so the epoxy doesn't stick to the frame.
It takes just a little.