does yours have open sights? if so, use them at 25 yds, to eliminate the possiblility of the scope being the problem. next thing to do, make sure your front action takedown screw is torqued down at about 2o to 25 lbs. next thing to do, is get some aluminum tape, and make some finger bandaid sized slices. lay about 2 or 3 slices on top of each other, then stick them inside the bottom of the stock where the action fits down into; on the bottom, the front , the rear, the sides, of the action area. this will make it nice and snug, and keep from moving the rifle around inside the stock, between shots. next thing to do, is the sand out the stock channel, and make absolutely sure, the bbl is free floated. after all this, if it still does this, then you have a bad bbl/chamber.
Also, it proly doesn't have the best of triggers, go over to rimfirecentral.com, and go to the remington thread, and look at the 597 sub thread, and read the sticky at the top, to improve your trigger.