When I updated my license to move to a new city, I informed my CLEO, and sent in the paperwork. I received the amended license a couple of weeks later. THEN, I was contacted by my local PD regarding an inspection. The detective was polite, and so was I. I indicated that it was only a collector's license, and that I was not selling guns out of my house. He said he'd be in touch after a week or two, as he'd been busy with "real" cases.
As I was agonizing over the need and the authority for them to inspect a C&R Licensee's home (BTW, I could never come up with such authority from the multitude of sources I consulted), I began to wonder if it was worth the license (I only got it to save myself the cost of some ammo and supplies, and have only ever purchase three rifles with it). The detective called back after a couple of weeks and left me a message, saying that he wasn't going to bother with the inspection after all, since I had told him I had a safe and that I lived on the third floor and all. I didn't bother to follow up, but I suspect that he realized he was confusing the requirements for FFL01 and C&R licensee.
Bottom line is to be polite, and assert your rights at all times, even if there's nothing to hide.