I've had the same problem. The little spring back there can be quite stiff. Try more pressure--if you have to, push in the little metal button with a screw driver or awl to give you less of a hard time.
You gotta slam it pretty good with the heel of your hand or a rubber mallet.