I have had occasion to need to polish a few dies. It depends on the brand and what you have to work with, what methods will be required. But on a basic level, if you spin the die and then use some very fine sandpaper wrapped around a dowel, it should alleviate your issue. I like 600 grit paper then move to 1000 grit, but anything similar will work. Then move to the finest scotchbrite pad you can get. Then flitz if you must. but I stop at scotchbrite.
For RCBS dies using 1/4x28 thread, its easy enough to use the threaded section as a mandrel. For dies with larger decapping rod threads, like a lyman. I make a mandrel out of a cap bolt that has a matching thread. Cut the head off and mount in a drill. But you need a 1/2" drill.
So what do you do if you don't have a suitable drill or drillpress? If you have any drill you can still make it work. First, get one of two things. Either double stick tape, or superglue. On your drill, turn your jaw teeth all the way in (open). Then either double stick tape, or use a drop of super glue to attach the die (top of die with guts removed) to the face of the chuck. Spin it and sand it, then remove it. Super glue actually works really well because a smack will remove it. It wont bond metal real strong. I've done it polish out H&I dies for my Lyman 450 sizer when a lathe wasn't available. It works like a charm.