I think there are three reasons why non-textual programming is constricted to a few domains, and is not widespread as general-purpose:

- As you point out, all the tooling around programming like editors, exchange formats, version control, debuggers etc. are only available for text, rendering everything else generally unusable.

- There are many failed attempts, with many still being textual in the end, only with over-the-top syntax highlighting, such as Scratch. These attempts have paved the way of non-textual programming with gravestones, making the field look like a graveyard, so nobody wants to get into it anymore. Scratch might still have its place in education as an gentle introduction to the concepts of programming, but no software developer is seriously going to use it for any big project.

- People are used to text and they will reject other approaches instinctively. You can read many statements in this discussion here like: "Graphical programming would make your head explode", which I think is similar to how people where afraid of using trains in the early days: "Driving to fast will kill you" and "Nobody can survive such speeds". It might take a whole new generation of programmers to accept anything non-textual as intuitive (which is why it could become popular with artists and designers I think).