Inertia and general lack of demand is a much better explanation. Most people either haven't experienced high-DPI monitors or just don't care that much about their PCs.
But it affects non gaming scenarios too. Static text is fine. But try scrolling that sharp text, or move something on the screen. Low refresh rate - more artifacts (motion blur, ghosting etc.). You'd see very clear difference with high refresh rate ones.
So resolution is not everything when it comes to monitors. For anything dynamic, refresh rate is more important.
And video content generally works fine at 30, let alone 60.
For comparison, people said the same thing about IPS panels (and it was true for a long time), but these days you can get a decent IPS panel for barely more than the equivalent TN.
idk whether the the GPU is the limiting factor, but browsing gif heavy subreddits on a 4K display brings my computer to its knees. I have a haswell i5 @ 4.3GHz and a 1080ti so I don't think my machine is underpowered.
Going to nitpick on this - 4K phones don't actually render at 4K. They only displayed video at 4K, and rendered at half resolution. But you're otherwise correct that you don't really need a power GPU to do basic UI work at 4K. Or rather, that even low-end GPUs these days are fairly powerful.