And this then causes the "Red Queen's race"[0], similarly to animal evolution, where if you don't adapt to the ecosystem, you go extinct.
Some projects put in that work (e.g., PostgreSQL), but it comes at a significant tax on productivity that could otherwise be put into new features.
Backward compatibility requires some care, but it's not exactly hard. Simply don't change interfaces or exposed behavior. You can always add new APIs, just don't break the current ones.
The only time it is hard is when there is a security vulnerability that is difficult to fix without breaking compatibility. But that's not very often.
I came of age in the culture of never, ever breaking compatibility. You'd get raked over the coals in design meetings for ever voicing such forbidden thoughts. It's a wonderful approach because everything just keeps working. As it should be. The software industry needs to grow up to what it used to be.