Now an OS without application compatibility is kind of DoA unless there is a very compelling reason to switch. Add in hardware compatibility and it gets even worse.
Much bigger hill to climb then incorporating an existing browser engine into a custom spin of a browser. Even a browser engine from scratch would be smaller than a new bare metal OS.
The OS game is over. Desktop computing is becoming a professionals-only thing. We can talk about pros and cons of Windows, MacOS, and Linux, but it's a shrinking market without room for a fourth player.
Now let's make that OS talk to a graphics card--whoops, no Nvidia for you, peon!
An OS isn't a problem. Hardware support on an OS--that's a huge problem.
With how mature the personal computer market is, this is a very big hurdle.
And that’s not even covering the numerous hobby OSs out there like Haiku, SerenityOS, ReactOS, TempleOS, SkyOS.
Then you have experimental OSs like Singularity too. There’s numerous examples of them alone but I think you get my point. :)
If you're looking for binary stability and to ship your app as a file, ELF is extremely stable. If your app accesses files, accesses the network through sockets, and use stable libraries like SDL or GTK it will work fine as a regular binary and be easy to ship. People just don't want to write their apps in C, when the operating system is designed for that.
Many native apps like Blender, Firefox, etc ship portable Linux x64 and arm64 binaries as tar gz files. This works fine. You can also use flatpak if you want automatic cross platform updates but yes, the format is unfortunately bloated.
It's not that easy to ship a JavaScript app on other OSes either and electron apps abound there too.