This question makes sense initially. After all, a fire feels warmer as you get closer to it. Why doesn't moving from a lower elevation to a higher elevation not have this effect? The distance between the earth and the sun is 92,960,000 miles (149,600,00 km). So even if you climb to the top of a large mountain, relatively you are not really that much closer to the sun. The "relative" minuscule difference does not significantly affect the temperature.
Pressure goes in the opposite direction of altitude. As the altitude increases, the pressure decreases. As the altitude decreases, the pressure increases.
As you sit at sea level, there is a large amount of air above you. All that air has weight and that weight is pressing downwards upon you. Air, unlike a liquid, can be compressed. As a result, air at sea level is compressed more than the air at 8000 feet (2483 meters). For example, at sea level the air pressure is 14.7 psi (pounds per square inch) while at 8000 feet the air pressure decreases to 10.9 psi.
That difference in pressure is one of the causes for a change in temperature.
When we talk about air, we are talking about gas. "Air" is really just a mixture of various gases.
As discussed above, gasses do not have definite volumes. Gases have the ability to be compressed or expand. When air is compressed it takes up less space and has a lower density. Think of a tank of compressed gas. The tank, aside from the weight of the metal container, is heavy. When you release the gas it spreads out.
This is exactly what happens at different elevations. At higher altitudes the air pressure is reduced, due to less compression, and the air expands. At lower altitudes the air pressure is increased, due to more compression, and the density is increased. This can be seen as the density of a cubic foot (meter) of air decreases as the elevation increases. When you get to the edge of the atmosphere, the density is nearly 0.
Gases are composed of molecules that are in a constant motion and collide into each other. When a gas expands, such as it does at a higher elevation, the molecules have more room to move around and do not run into each other as much. Without bumping into each other so much, their kinetic energy is reduced, and the average temperature is lowered. During this process, expanding kinetic energy is transformed into potential energy. Temperature is a measure of kinetic energy so the temperature decreases.
In general terms, when you pressurize air the temperature increases. When using a hand pump, the temperature increases on the sides of the pump. When you decrease pressure, the temperature drops. Think of using those compressed air canisters for cleaning a computer. As the air is released, the can gets cooler.
Simply put; air with less density holds less heat.
The result is a Standard Lapse Rate
On average the air temperature decreases by 6.5° C for every 1000 meters (0.62 miles) you increase your elevation.
The earth surface absorbs radiation and emits radiation. The emitted radiation is then absorbed by the greenhouse gases and clouds in the atmosphere or makes its way back to space. The less heat that makes it to space, the warmer the atmosphere is. Going back to the previous discussion regarding density, the denser the gasses the more heat is absorbed. At higher elevation, the density of the gases is lower. The result of the lower density is more heat escaping to space and less to warm the atmosphere at higher elevations.