How much do you train, how often do you shoot, and how familiar are you with each gun you shoot? I don't really want an answer, but it takes a lot more investment to know the gun well enough to know where it is pointing without verifying that visually.
Front sight is simple to train and works for people that invest only limited time in training, enough to be competent in shooting the gun accurately enough at most self defense ranges, understanding some of when they should and should not shoot, and learning other things as well in a limited time span that give a useful skill set of self defense. This is going to be most citizens that carry.
The front sight allows rapid and quick orientation of the gun in the right direction without being so comfortable with the gun you can point shoot at things from different angles at different directions and reliably hit things with different levels of stress. It also applies to many guns including those someone is new to or when they change over to a new model.
Good point shooting really relies on having repeatable grip, comfort, and experience with a model gun, and knowing where the barrel is pointing with that grip. When you switch to a new gun that will need to be relearned and take a time investment. With a shotgun it is much simpler because the point of contact that is repeatable is the stock and shoulder, as well as other dimensions relating to the fit of the gun which are all well defined and easily repeatable, rather than just the grip of the pistol.
This means most shotguns of similar dimensions or that fit you similarly will point similarly. The grip of handguns varies much more and the portions of the hand different grip dimensions contact and at what angle relative to the bore varies. Width varies, trigger pull, reset, length, hand size relative to the gun, it all changes each time you change pistols. Some guns I find point shoot accurately, other are slightly to the side with a similar feeling grip and require readjustment or aiming to overcome, often related to the trigger length of pull and width and how much it fills the hand.
Most people that use guns are not going to shoot large amounts of rounds recreationally, and a lot of good point shooting is more conveniently practiced outdoors informally with space to do so safely at different targets from different directions, while most people have cramped indoor ranges to train in.
It takes a lot of practice to really be confident and comfortable point shooting, and to be skilled enough to retain enough accuracy while being shot at or dealing with injury and not just the comforts of the range. What is in front of your sight is always what is in front of your sight whether you are seeing blurry and dazed or whether you are calm and collected.
Suggesting that most people become confident and capable of point shooting for self defense is like suggesting most people get race car training to commute to work. It is not something most will continue to practice and keep a sharp skill, and people that would be better served by simplier training that covers more instead get very focused training they probably won't do as well with in practical situations.
You also might encourage people with large egos to think they are always that good even when they are not, and put others in danger. Hitting the bad guy is important, and not hitting other people is also important.