New York City's restrictions on rifle and shotgun ownership are based on the NYC Administrative Code and are of no force or effect outside of the five boroughs. Period. You are still a New York State resident and as such may purchase a rifle or shotgun in NYS, outside of NYC, without a license. No crime is committed as long as the weapon never enters the five boroughs.
No store owner can be charged with a crime for selling an adult who is a NYS resident and otherwise qualified a rifle or shotgun simply based on the fact that the person maintains their residence in NYC. NYS Penal Law does not prohibit the purchase or possession of rifles or shotguns by a NYC resident. It is a local NYC ordinance.
My advice is that you explain this to your gun shop owner when you purchase the weapon should he or she ask. Tell them that you live in Orange County as well as New York City. My other advice to you would be to change your primary residence to Orange County since you can actually obtain a NYS Pistol License with full carry endorsement provided you get the right judge.
What New York City sometimes does is send Detectives to surrounding area gun shops to look at their records and see if a New York City resident has purchased a rifle or shotgun. The NYPD will then show up to your city residence en masse wearing bullet proof vests and raid jackets and demand that you allow them into your residence to look around. They never have a search warrant or an arrest warrant when they do this. It's all scare tactics; you have to agree to let them in. If they try this with you don't even open the door. If they tell you that they're going to get a warrant, call their bluff. They can't get one. Why? Because for them to get a warrant, search or arrest, they must have probable cause to believe that you have the weapon within the five boroughs of NYC where a license is required. Unless they have a witness who observed you in possession of the weapon within the five boroughs or a witness who is willing to swear that you stated your intention of bringing the weapon into the five boroughs they cannot meet their burden of proof and no judge will issue a warrant. Merely purchasing a weapon in Orange County is insufficient grounds for the issuance of a warrant absent concrete evidence of the transportation of said weapon into the five boroughs of New York City. Should the NYPD violate your rights, let them. You will be a wealthy man.
As a life long New Yorker I've been dealing with this crap for years and years. You've got nothing to worry about. Enjoy your purchase.
And change your home of record.