The latency from the distance light travels does not vary that much. The rest of the ~600 ms latency is caused by the same thing as latency on ground systems (routers, etc.). And the processors on spacecraft are not particularly fast.