The answer to that whether it will hurt your ammo in some way really comes down to how long you plan to have it in a magazine. I've seen claims that oil will harm ammo, but have never seen or heard any absolute proof of that claim. I'd say it's bogus, most modern ammo (well, factory ammo anyway) would have the primers sealed in some manner. Cosmoline is only an oil based preservative, kind of like a runny grease compound.
As to how to clean the interior of your magazines, no matter what kind you have there has to be some method to disassemble them. Do that, soak them in mineral spirits and swab them out until they're dry.
Truthfully, unless they're like totally caked with the stuff, and you load your magazines and leave them for extended periods, I doubt you have a problem.