I use a P O Box for all my mailing, and have bought and sold guns through it. The key is to send any funds via Post Office money order, and send the MO and FFL via reply-receipt certified mail. That way, whoever picks up the money has to sign for it, and can't get the cash without signing the back of the MO as well. This means that if there's anything fraudulent going on, you now have a felony misuse of the Federal mail system, which the Postal Police take very seriously indeed. (I've trained alongside them at the Federal Law Enforcement Training Center in Georgia, so I know something whereof I speak.)
Any seller who won't accept funds via USPS Money Order, or sign for their receipt, is to be avoided, IMHO - whether or not he/she uses a P O Box or a physical address is irrelevant.