This explains a lot and this information would have really helped if included in the OP as what you are trying to do is highly unusual...to say nothing of unduly over complicated.What I'm trying to prevent is dumping my rounds on the ground, I reload...
... I am using the now empty speed loader to grab all 6 at one time, and then drop it into my pocket.
People have been doing this for years without going through the gyrations that you are attempting. The more common way is to allow gravity to work for you. Aligning the speedloader with the rims, inverting the gun and ejecting the cases into the speedloader would be the easiest way to accomplish what you intend.
Please don't read this post as recommending, advocating or approving of this technique...it will build horrible habits