Personally, in my 9mm HD weapon, I keep a full magazine of Federal 147 grain +P+ JHP, the 2 backup magazines are loaded with Remington 124 grain +P JHP. Truly, it's because I have a very limited supply of the 80's vintage +P+ ammo that I got from an FFL friend. If I can not grab a spare mag, that's what I want. I have practiced with this loading extensively. If I had more, all 3 mags would be loaded likewise. My backup mags are loaded with what I believe are the next best option, which I have also practiced with, at length. (and expense)
My main go-to revolver, a beauty of a S&W 66, has the heaviest loads that it can bear. For bear. Hornady 158 grain XTP, or Buffalo Bore 180grain Hard cast LFN.
EDIT: To answer your main question - can it affect your shooting? Yes, it can. If you practice with different loadings than those that you might be called upon to use in HD/SD. It might, yes. Practically, not much really beyond real-world SD situations. In self defense, you are not bulls-eye shooting at 100 yards prone. Your adrenaline level, breathing and lung function, general health and cardio, and more specifically your special circumstances will have more of an affect on your shooting than the loading of your first magazine. If you truly need your HD/SD weapon, you will either be successful or be dead before you need to reload. Real world gunfights are not Hollywood. I do not mean this to be negative, but reality is brutal, fast, and unforgiving. I am sorry, but the real world moves much faster than you think that is does. Suddenly, everything changes. Forever.