Most slings are attached to the forearm. If the forearm is touching the barrel, there will be a different force on the barrel between using the sling and not using the sling. This will affect the point of impact.
If you sight in without the sling, and then shoot with the sling, you won't hit where you expect. Generally.
A rifle with a free-floated stock won't have this problem.
Art