You're also massively overstating the complexity of an X server. Writing X servers that are advanced enough to be usable is something a lot of individuals have done. It's nothing like matching a modern browser. Incidentally, if I were to try to do this today, I'd gut a proxy like Xephyr and make it rootless to support older clients, and make the main server itself only support the subset of the X protocol modern clients speak.