The temperature sensitive part in a VCO is actually the exponential converter inside, which has a strong temperature dependence. This is because the current through a transistor depends on the ratio V_BE / V_T, and V_T is proportional to temperature.
It's a neat trick that lets you build good temperature sensors out of transistors, which is very convenient, but in these VCOs you have to add temperature compensation to the exponential converter, and the temperature compensation is far from perfect.
If you take linear VCOs (instead of 1V/octave exponential), they'll be much more temperature stable.