It really depends on state law. For example, in some states it is illegal to conceal a weapon (pistol) period. The permit then is a defense to the illegal act. In those states, if a police officer catches sight of your "concealed" pistol/weapon, the officer has reasonable and articulable suspicion to perform a Terry Stop (detain you) and demand to see a permit.
In Washington, it is only illegal to conceal a pistol without a license and open carry is legal without any license. It is also only illegal to have a loaded handgun in a vehicle without a license. So, let's say a police officer sees you exit a vehicle and pull your jacket over your gun. Since it is only illegal to conceal a pistol without a license, the officer must have reasonable and articulable suspicion that you both have a concealed pistol AND that you do not possess a license in order to lawfully detain you because of the gun only.
I don't think the difference between the words "permit" or "license" on the piece of paper really matter, by itself. Although some people might be making the distinction between the two different ways the law is written above: permit allows you to have an affirmative defense to breaking the law, license means it is only illegal to do so without it.