I hear you on the money, time and kids things as well, I have a 12 year old son and 9 year old twin girls, I put in 55-60 hrs a week sometimes, and weekends are tied up with softball, dance, and baseball year round. (Desert off road is season is starting for us next weekend)
I find my reloading time when they go out with mom to various places and I have some alone minutes. In an hour you can load up a surprising amount of ammo on a progressive press. With single- stagers it takes longer to put loads together (of course).
For many years I have loaded for .38 spl, .38 wadcutter, .357, 41 mag (just got into this about two months ago), .44 spl, .44 mag and .45 Colt. I also recently ordered 100 .454 Casull and 500 .45 Super cases so I someday can start building loads for them. I occasionally load center fire rifle rounds but I don't shoot them as often, or in the volume, of my handguns.
A lot of it is forming the desire to start, and the ability to buy stuff in stages (on line, local sales, flea market/classifieds etc) makes the hurt of the initial equipment investment lot a less. Start saving your brass now, and you will be ahead of the game in that department when the time comes to get started...