I’d take a bolt action .30-06. Fighting wasn’t in the cards for the Corps but that was hardly a given when they set out for the unknown, and a bolt action’s rate of fire would be devastating, if not downright magic, to enemies equipped primarily with spears and bows. Also plenty good for any big game hunting between St. Louis and the Pacific. I’d take a couple crates of ammo because they had good sized boats and pack animals, so it’s not as if that would be an undue burden.
For the pot I’d take a Winchester model 12, with a dial-a-choke. Ugly, but very versatile and nigh on unbreakable, according to lore.
In terms of cutlery I’d probably take a birchbark-handled Finnish puukko. I wouldn’t bother with my own axe because I’m sure a few were packed already by my comrades, and an axe of that time would be every bit as good, or better, than what’s readily commercially available today.