As the other guys said, it's optional. Part of the fun for me is taking time with the reloading process and tools. So I do clean my primer pockets if I'm not in a hurry. Never made any difference in the shooting I do, though.
When I started reloading in the 80s, I didn't have money for much gear, especially a tumbler. I kept myself under a pretty strict budget. So I used a Lee Loader for my pistol and rifle ammo. This is almost as basic as reloading gets. While I always inspected the cases, they were all tarnished with uncleaned primer pockets; the ammo still worked just fine.