The haz-mat fee is charged PER SHIPMENT so if they are sending you 1 brick at a time you will pay $22.50 haz-mat plus shipping for each brick!!
The H-M fee is charged by UPS or FedEx because primers are classified as flammable solid and have to be handled separately. H_M is NOT shipping; you will pay that in addition to the H-M fee, and insurance.
Your absolute best bet is to order as many as you can at one time in order to have the hazmat fee applied once; you can get 50K primers or 50# of powder or a combination of the two (a 1000-brick of primers is equivalent to a pound of powder, ROUGHLY) in one shipment.
Typically for a 50K order you will pay $20 shipping, $22.50 in Haz-Mat, and $5-$10 in insurance. Roughly about $50 in shipping costs to get 50K primers to your door, or $1 a brick. By the same token, it also costs the same $50 to ship 5K of primers to your door, or $10 a brick, so it pays to max out the hazmat by getting your friends, relatives, club associates, neighbors, etc to go in with you to get hazmat stuff.
Before ordering from Cabelas or Midway you need to check out Powder Valley and Wideners and Grafs. They are the ones who usually have primers or powder and Cabelas, Midway, etc buy often from them (or some other distributor)and mark it up to you.
That's why if you order primers/powder plus a set of dies from Midway, you get charged shipping twice- Midway doesn't keep the hazmat stuff in stock, it comes from a distributor.