Rifles and Shotguns, provided they aren't NFA or covered by the NYS Assault Weapons Ban, are easy to purchase outside of NYC. You need a current picure ID (i.e. driver's license) and you need to sign the same transfer form as everywhere else. No special permits, waiting periods or registration required.
Handguns are illegal to possess and/or purchase unless you have a pistol license of which there are many different types. Do not touch a handgun without a pistol license; doing so is categorized as a class D violent felony and carries a sentence of 2 1/3 to 7 years in a state pennitentiary. There is an NRA instruction exception to this rule. Consult a lawyer and the Penal law before handling any handguns in NYS. Your pistol license application is submitted either to a Judge (most of the state) or to a Police Department or Sheriff's Department (NYC, Nassau County, Suffolk County). The expense, wait period and frustration level vary from county to county. In some areas of Upstate NY unrestricted CCW is possible for the ordinary citizen. In NYC you'd better be an A-list movie actor, billionaire, second cousin to God himself or have incriminating photos of the Mayor. Otherwise, all you'll get is a premises permit.