chutestrate,
Here's the easiest way to think about the situation.
When you're shooting at a target that is perfectly level with you, your scope is aimed level but the bore of the gun is aimed slightly upwards. That slight upward angle is to "loft" the bullet up enough to compensate for gravity's constant downward pull. If the bore and scope were both aimed level, the bullet would begin falling as soon as it left the bore, making it impossible to hit anything unless it is lower than the barrel.
So there's a slight upward angle to the bore compared to the scope. This is critical to understanding what's going on.
Now imagine what happens when you shoot STRAIGHT upwards as if you're trying to hit a bullseye 50 yards directly above you. Your scope is pointing straight up, but remember that the bore is angled "upwards" with respect to the scope. So the bullet will hit "above" the aiming point in the scope. It will hit a little behind your position on the target above you, "above" the crosshairs. So if you really want the bullet to go STRAIGHT up, you can't aim your scope straight up, you'll have to aim it a little "below" where you want the bullet to hit to compensate for the slight angle of the bore with respect to the scope.
Now imagine what happens when you shoot STRAIGHT downwards as if you're trying to hit a bullseye 50 yards directly below you. Your scope is pointing straight down, but remember that the bore is angled "upwards" with respect to the scope. So the bullet will hit "above" the aiming point in the scope. It won't go straight down, it will impact a little in front of your position on the target below you, slightly "above" the crosshairs. So if you really want the bullet to go STRAIGHT down, you can't aim your scope straight down, you'll have to aim it a little "below" where you want the bullet to hit to compensate for the slight angle of the bore with respect to the scope.
In other words, whether you aim at a significant angle up or down, the effect is the same. The bullet will impact a little higher than you would otherwise expect it to so you need to aim a little lower than you would otherwise.
Those are obviously extreme cases since we almost never have occasion to shoot straight up or down, but the general principle applies even to more normal shots when the angles are signficantly upwards or downwards from the shooter.
If you want to think about this in terms of gravity, this is what is going on.
That upwards angle of the bore with respect to the scope imparts a little bit of loft to the bullet to compensate for gravity. When you aim at something that's higher than you are, now the bore is angled up even more (due to your aiming upwards) and the effect is that you are now lofting the bullet MORE than you need to. So the bullet will go higher than you expect and therefore you need to aim low to compensate.
When you aim at something that's lower than you are, now it's ok for the bullet to fall a little bit because the target is below the bore. That means that the upwards angle of the bore with respect to the scope is providing too much compensation--a little too much loft than it should. So the bullet will go higher than it needs to and therefore you need to aim low to compensate.