My advice is simple but difficult to describe, so here goes for what it's worth. Last I heard about $.02.
Please read "In The Gravest Extreme" a couple times cover to cover. Then go to Pax's website
http://www.corneredcat.com
If you can't figure it out, then reread it all over again.
The laws are mostly simple, to protect you and the BG both.
The gray areas are there so that a jury can decide how to color it in.
Good, bad, or indifferent, there's no absolute answer to your question on this forum, or in most courts of law. It's up to a grand jury or a jury to decide.
We hate to keep repeating the phrase "What would a reasonable person do in that situation? But it will be critical in your defense.
Unholstering a handguns, but not aiming it at a person in some jurisdictions is considered an action of defense. Others may consider it an act of a deadly threat.
Nobody can truly answer your question accurately because from one city, county or state the laws are continually changing, and the presumption of action is undefinable in most jurisdictions.
You
must feel that your life, or another persons is in iminent danger of great physical harm or death.
Pulling out your firearm to simply warn somebody off with your handgun is usually (but not always) considered brandishing, and is a misdeamenor in most jurisdictions. In some, it's a felony and will cause permanent revocation of your license and capability to own or have possession of any other firearm.
If in doubt, don't pull it out!
IMO, De-escalate, retreat or escape should always be considered your first option (within your home or "castle" may have different standards, unless it's required by your state or local laws.) If you can't, then defend yourself with whatever means you have at your disposal.
Good luck trying to answer the question all of us has dealt within ourselves.
You must make the decision, we can't do it for you, and I pray I won't be with you if you ever need to decide. I'll be home safe with my family if all goes as planned.