The most popular stable diffusion UI uses gradio, which is the same paradigm of defining a web frontend in the backend, and the performance just falls apart after the code reaches a certain size. The need for most interactivity to roundtrip to the server and back opens up tons of places where a bad connection could leave the whole app in a half-broken state. And beyond that, it's just slow. So much so that mobile use on cellular is out of the question. Other SD user interfaces handle parameter adjustment in the frontend, but something as simple as swapping two integer inputs requires an seconds-long API call unless you bypass the whole framework with custom JavaScript.

Then there's the fact that your backend framework has to define the subset of all frontend features it supports in Python. The moment you hit on a usecase that isn't supported in Python, you have to learn JS anyway and write code in the frontend. Except it's the worst kind of frontend code: hacks around broken or missing frontend features that depend on implementation details specific to the framework that are subject to change at any time.

I feel that these kinds of frameworks are better suited towards researchers or backend devs who really do not want to have to set up an NPM project just to have a simple (and I really mean simple) frontend UI to interact with. But don't dare try to write the next gigantic ML app on top of it or you'll have to deal with the performance and maintenance issues that follow.