drink your coffee and go hunt. in fact, take your coffee w/ you if you need it.
i've hunted a few deer, and i've found that scent alone doesn't mean a deer is gone. but scent and noise, he's gone. scent and movement, he's gone...
you would be surprised at what you can get away with. i was helping my brother fill his tag one morning after i had tagged out (a very nice whitetail buck, if i say so myself). we were sitting in a good ambush spot. after a couple of hours, he wanted to move to another spot. before we moved, though, he wanted to have a smoke. so he lit up a cigarette, and stood there, smoking. halfway through the smoke, 3 does came down the trail, walking right in front of us (not more than 10 yards out). they got downwind, and they went to full alert. we did not move, and the deer just stood there, looking for the source. after a few minutes, they slowly moved off. as soon as he finished his smoke, a fair 4-pointer (4 on the left, 4 on the right) came down the trail. my brother decided he wanted this deer, but wanted to see first what would happen when he got downwind... so, the buck gets downwind, and basically same reaction... we stood stone still, the buck was on full alert looking for the source. he couldn't see us, so he started to walk away when the 25-06 slammed him (around 30 yards)...
i think deer are more scared of movement or noise than scent. scent will certainly put them on alert, but scent alone isn't a hunt-killer... personally, i don't take any chances that might tip the odds in the deer's favor, but coffee? i drink it all the time before and during the hunt.