Current 870 rifle type rear sights have three main components, two of which are secured by screws. First is a dovetailed ramp, which is silver soldered to the barrel. Attached to that by a cross screw is a small block which fits onto the inclined top surface of the ramp and clamps into place by the action of the cross screw. This allows for elevation adjustment of the rear sight.
Attached to this small elevation block with a small vertical screw is the rear sight blade, which is adjustable for windage.
Sounds as if your elevation block cross screw is either missing, stripped or damaged... or for some reason not doing its job.
lpl