Everything is relative. If you shoot the gun a lot then yes, or if you carry the gun yes. But if you have a dozen Tupperware guns, and just shoot them once a year, then no. It's a lot of money to invest in a gun on a percentage wise basis, unless you plan on keeping it, and really love it. You aren't likely to get it back, if you trade it off after a few months of a year.
Customizing anything is a risk, some people just want their stuff factory stock, and never fooled with, also some can't shoot well enough to make a difference, or just keep it as a house gun, in which case the stock action is adequate.
If you are a good shot, then you should get it done, especially if it's your carry. If not I would pass.