joneb,
I've got an SR40C... which I have removed the magazine safety from.
I just looked at the mag safety that I removed from the pistol and looked it over... I've dropped the striker onto the safety numerous times not thinking about it being there... Anyway, there is not a mark on the safety itself anywhere... it is pristine. I last had my slide apart when I removed the mag safety, and looked over the striker pretty close... there were no marks or wear or dings on it as well.
I wouldn't worry too much about it. As was said before, a lot of people remove it then you don't have to worry about it. It literally only takes 20 seconds to remove... Well, maybe a minute if you've never done it before. Very easy to do.