The idea, I think, isn't that this grey goo would get energy from dirt and rock and literally consume the Earth. Rather, it would use dirt and rock plus solar energy to grow and if its efficient enough out compete existing plants. The other bad possibility is that some nanite comes along that can digest typical organic matter but that can't be digested in turn. Then it proceeds to much on the biosphere while leaving the rest of the planet alone and just eat all the organic life (that hasn't hermetically sealed itself away).
Either of these scenarios would, of course, be Bad Things.
I'm not sure either is actually credible, but I'm not aware of any reason to declare them impossible.
