Spacing out the grips is going to move the "square edges" that fit against the upper frame out where they can eat into your hand. You may find that you are better off with new grips that are fatter up around the neck.
Also the stock grips are already a bit "fat" around the butt of the frame. So any spacers you get are going to want to be more of a wedge shape. As such I'd suggest that the optimum option if you are looking at gluing on a spacer is to shape the spacer such that it tapers from thicker at the top to thinner at the butt end. This will likely mean that you need to fiddle around with the screw and the insert in the other side to adjust for the angle. It's something you could do at the same time you're drilling the holes a touch deeper to let you stick to using the same screw and insert.