The way I clean my primer pockets, it's not (too much of) a pain. After I knock out the primer with a Lee universal decapping die, I hit the primer pocket with the appropriate RCBS primer pocket uniforming tool spinning in the RCBS Trim Mate case prep center before chucking the brass in the bowl. Since I've already uniformed all the primer pockets with this tool, all it takes is a second for the tool to clean out the primer residue, and then on to the vibratory case cleaner. Cases come out of the cleaner ready for priming and loading.
Do you have to do this? Not at all. I like doing it because it's just one more step in helping me make the absolutely best ammo I possibly can.