As mentioned the alloy temp will make a big difference in your weights as will the mold getting hotter as you go along.
If you do not have one already you might check and see if NOE has one of the lead thermometers in stock,
Lead Pot Thermometer
He has them priced well and is a great fellow to deal with.
As for the weights, your seeing the results of the above. As the aluminum heats up the cavity actually gets smaller. Depending on the alloy this will throw weights all over the place. There is a thermometer available to be inserted in some molds as well but it requires having a hole drilled in the end for the probe.When I first got started I picked one up but after about half a dozen times of using it I found that simply controlling the alloy worked just as good as long as I kept the cadence up with my pour rate.
With bigger bullets you don't have to run quite as fast to keep the mold hot. With one cavity molds using a pull pin for HP's or HB's you need to pour a bit faster to keep the pin hot or you will need to heat the pin in the alloy between pours to keep it hot.
My suggestion would be to pick up the thermometer, get your mold up to temp using either the casting pot to pre heat it or the alloy, get the alloy into the 675 - 700 degree range and start pouring. I would pour fast enough to get frosted looking bullets then alloy the mold to cool maybe 10 - 20 seconds and pour another. This should get you in the ball park for good bullets, Once you have the mold ht, and have a good pour you might set the old on a wet towel between pours just long enough to cool it a touch. Maybe a second or two at the most. This might help regulate the mold temp.
Casting is a balancing act where in some cases with some molds you can lean more to one side than with others. Once you hit a good middle ground you will find everything starting to balance out.
If your seriously into casting, and want to get as close as you can to repeatable temps with your alloy you should look into installing a PID controller (
Project PID ). This will allow you to set the temp you want to pour at, and the controller will hold it there within a few degrees one way or the other. Building it yourself might run you $100 for the parts and such depending on how thrifty you are at scrounging. I set up my Lee 4-20 with one after chasing the temp while pouring got to be a pain. Starting out with my alloy at say 685 and then once the level dropped it jumped up to 750 really quick. The PID will hold it where I want it down to the last drop.
When I started I found that my bullets varied in weight like yours, but after adding in the alloy thermometer to start with, and then the PID controller, I have found that my weights only vary by a few grains, and usually on a 10# batch less than 5-10 grains for the whole lot. I admit I might be a bit OCD about my bullets, but I use them for hunting and want each to be as close to the next. While it might not make a hill of beans to the end result on paper, in my mind while in the field I know I did everything I could to bring the best ammo I could assemble with me. With my shooting out to 50 - 100yds in some cases with my revolvers, I want all the added edge I can get.