Doesn't fit with my experience. Moving from using ssh -X to just pointing remote X applications at my local X server without tunnelling it over SSH consistently improves performance for me. Not just in terms of experience, but also no process maxes out a CPU, so the machine has spare compute capacity for other tasks. It's really not an issue of ssh not being multithreaded. That shouldn't happen even on a single core. If sshd divided that workload over my n cores, it would still be very bad.