You might be right. In my testing, mdworker and friends behave like you'd want them to, I don't see them polluting the cache on my Lion machine. Haven't tried time machine yet.

EDIT: I can't get either spotlight or time machine to show any cache polluting behavior at all, at least not in the way that I run them. I used "mdutil -E /" to force a re-index of my disk, and I kicked off an initial time machine backup on a secondary drive I had lying around. I see both backupd and mdworker doing a lot of disk reads using iotop, but top shows my inactive memory not really changing as drastically as I'd expect, like if I were to cat a giant file to /dev/null.