I have the same rifle, and like his, the grain in the forestock does not match, or even come close, to the grain in the buttstock. Simply removing the spacer and joining the two stocks looks weird. Plus, if you just take out the spacer you have a problem at the end of the rifle fitting the nose cap - it screws into the barrel so you can't move it back to take up the space left by the spacer.
Best to put a piece of wood in place of the spacer, stained like the stock. Not perfect, but not as funky as the brass spacer. With the two different grains there's no hope of getting it to look like one piece of wood anyway.