You can indeed completely assemble a lower with no specialty tools. Just a light hammer and a few properly sized punches from the local harware store.
But...
Since you need the wrench to install the barrel (and a torque wrench to put on it) along with the upper vise block, you may as well buy the decent stuff right our of the gate.
I've built a few from scratch, as in nothing but a pile of parts, and have used the GunTech wrench I got from Wilderness Products when I ordered another Instructor belt and the ProMag action blocks that are on sale at Midway right now for $30 for the set. I paid about $40 for them back when I got mine.
For the torque wrench, your auto parts store may have a tool lending program. You leave a deposit and get it back when you bring the tool back.