If you want your neck tension to behave consistently from one firing to the next to ensure your primary ignition also remains the same from one firing to the next, yes, annealing every firing is a known Leading Practice.
Annealing every multiple firings - such as the every 4 firings you mentioned - is largely a misguided attempt at increasing brass life, under the premise preventing split necks increases brass life. This can be more easily solved by using appropriately sized neck bushings in the sizing die and a mandrel expander, effectively eliminating the value of intermittent, multi firing annealing to “add case life.” I started annealing that way ~20yrs ago, but have largely given up on the practice. I either anneal every firing, or not at all, dependent upon my application for the ammo.
Playing with 5.56 and making noise isn’t very dependent upon consistency. I don’t anneal my blasting ammo either.