In shooting free pistol and 10m air pistol (the most demanding handgun sports insofar as pure accuracy goes) the coaches and training manuals are fairly uniform in their suggestion that the first 2-5 seconds are required to get the gun to "settle". This period is then followed by another 5 or so seconds during which you experience your maximum stability, which is in turn followed by decreasing stability caused by eye and arm fatigue along with diminishing oxygen levels. This all assumes a trained shooter, of course.
And of course, free and air pistol don't have much to do with "reality", or practical shooting as it is otherwise known.
So the short answer, I think, is that one should look at the sights for as long as is necessary to achieve the desired result, but no longer.