The usability improvements on the Linux desktop happened in spite of X11. The conversation is about font rendering—and why should font rendering be a part of your windowing system? For most apps, it’s not—it’s in Pango, and Pango dropped support for X fonts. All of these changes which already happened have been eroding whatever advantages X11 offered in the first place.
So it’s time to decentralize all the random functionality in X11, and just move it into client-side libraries.