In the current ice age, the cycle between glacials and inter-glacials is primarily driven by variations in the Earth's orbit, and therefore the distribution of incoming solar radiation over the Earth's surface by season and latitude. Obviously, as you mention, this then interacts with global climate through atmospheric and ocean circulation. They're called Milankovitch cycles after Milutin Milanković, who calculated them all (by hand!) at the beginning of the 20th century.
Obviously orbital forcing would exist in "greenhouse" times, to answer the parent comment.
If you are interested, there are three variations:
- eccentricity, change in the ellipticity of the orbit around the Sun, ie how close to a circular orbit it is. So if it's close to circular, temperature variations by season will be low. If it's more eccentric, you will get greater variation by season.
- tilt of the axis of rotation, which varies around 22-24 degrees. So when the Earth is tilted over more, at the poles it will be hotter when facing the Sun, colder when facing away.
- axial precession, which is the wobble of the axis of rotation, a bit like if you put a toy gyroscope on a table. This has a latitudinal and seasonal effect, particularly at the poles.
These cycles are approximately 100, 41 an 25 thousand years respectively. Put them together and the periods very closely follow the current ice age climate variations.