For good patterns, buckshot need to be loaded in flat layers. 20ga is a fairly small bore for using buckshot, at least the larger sizes. At least part of your problem is that the .32 buck shot are too large to sit in a flat layer, and they are being badly deformed on firing by being jammed...