I am from NYC and have been mugged at least 50 times
It serves to reason since you're still here to tell us about it, however you handled each these >50 muggings turned out as good as they could have.
Any landing I walk away from is a good one, and any violent encounter I end unperforated I consider a success.
Thug: Gimme your wallet.
Me: No.
It is fantasy to believe a mugging starts out with a simple request. Many begin long before the victim is aware of it when the team interviews the victim and then signals to each other to initiate the action. Often a victim is unaware of the encounter until the attack begins, and the weapon used to coerce his valuables determines the level of violence in the encounter.
The more deadly, or perceived deadliness, of the weapon, the less violence is usually employed in the encounter. Guns might be used to bludgeon, but are rarely fired to prove the attacker's resolve. As we move down the spectrum of weapons through knives, to clubs and improvised weapons, to bare hands, the more likely the victim is to be hurt in the encounter. In a robbery the attacker is using violence as a tool for compliance, and not expressively, meaning to satisfy a vendatta or personal desire to use violence for its own sake.
Knowing this, we can make decisions about how we choose to respond in the interaction, and keep the attackers from moving out of using violence as a tool into expressive violence - meaning he now has personal motivation to hurt us. Likely the attack upon you is not their first. This is their "job"; it is how they make a living. By the time you meet them, they are professionals who learned how to commit robbery from others and have refined their skills by having done it repeately. Insulting them by not respecting their skills and experience is foolish and will move them out of instrumental into expressive violence.
Choosing to respond with "No" probably isn't realistic. And neither is drawing a gun once the encounter has progressed to the point that the robbers have actually began requesting valuables.
I carry my cash and credit cards in a money clip. My wallet contains my other items such as ID. I feel a lot less attachment to my money clip, and have no reservations parting with it, knowing that nothing in it can't be replaced nor does it contain anything personal like my ID and photos.