Yes, it was the buffer, no idea what gave me the thought that it's a "retainer". I've never actually fired the rifle yet, as you can guess it's because the bolt never went forward.
When it first happened, I just scratched my head, dissembled the rifle, looked it over, reassembled it, and tried...