When it's way overhead - it's the moon. You've seen it all your life. It's just there.
When it's on the horizon - your automatic distance-judging machinery (we're talking more than just parallax and depth perception here - I mean the whole process) says "hey that thing is really far away, and way, way past the horizon, way further away than those mountains, so it's obviously really, really big by comparison (which is completely true).
Humans perceive the world more or less on a horizontal plane - so we interpret things in that plane differently than we do things above/below us.
"The distance from the surface to 300 mb averages 9,300 gpm. When the sun is directly overhead, radiation passes through 9,300 meters of atmosphere between the surface and 300 mb. When the sun is at an angle, it must pass through a greater length of the atmosphere before reaching the surface. When the sun is passing through the atmosphere at a 45 degree angle, the radiation must pass through 13,000 meters of atmosphere between the surface and 300 millibars. When the sun is passing through the atmosphere at a 5 degree angle, the radiation must pass through 107,000 meters of atmosphere between the surface and 300 millibars.
"The amount of scattering and refraction differential depends on how far the sun's radiation must pass through the atmosphere. As the length of passage of light through the atmosphere increases, the amount of scattering and the refraction differential increase. When the sun or moon are within about 10 degrees of the horizon, they have the illusion of looking much bigger than they do higher in the sky and the coloring looks different. This increase in scattering is also the cause of sun sets and sun rises giving a red color on the horizon. Although the size looks different when close to the horizon, it is mostly an illusion."
Taken from http://www.theweatherprediction.com/habyhints/215/
The harvest moon is one of his favorite illusions, and there just tons of little nuggets like this. It reads like an optical Wittgenstein: a series of short, numbered observations about the way light and vision work that you'd never have thought of, but once you read, you can't stop noticing.
E.g.: you can always predict exactly where a rainbow will be, you can tell leaded glass at a glance, wire meshes only cast vertical shadows, the green flash is very real, sunsets look exactly like sunrises, you can navigate by a polarized lens, hills look steeper from above for the same reason that steeples look shorter than the shadows they cast on a cloud (related to the harvest moon), and the line of the sun on the ocean always points right to you. Have you ever noticed atmospheric perspective, or how drastically colors change with distance? You will after reading this book.
Minnaert makes some interesting claims about the harvest moon in particular. For example, lie on your back and look at the sky for a while: the flattened dome effect tends to wash out rather quickly, and the moon looks roughly the same size in any direction. He also claims that binocular vision is closely related to the perceptual flattening of the celestial dome, and one-eyed people do not perceive the harvest moon. I'm not sure if he's totally correct, but it's interesting to think about, isn't it?
It changed the way I look at the world. I believe scientific consensus has moved on in certain areas in the years since it was originally composed, but Minnaert's careful thought has aged superbly. Definitely worth it.
The psychological effect depends on the horizon. Tilting your head to the side alters the horizon's alignment, so the moon next to "some straight, vertical line" doesn't appear as big.