Maybe just do what I've done on other things.
Take a drill bit of the appropriate size... and flip it around. Use the smooth portion as a gauge. This will tell you if there's a burr or anything out of spec that you need to deal with.
A lot of the time, just a bit of a push will knock it out. If you need more, I still suggest a reamer, but at least you'll have the right drill bit on hand if you need it.
What are the appropriate sizes? Specifically, what size would you use for the pivot and takedown spring channels?