But I'm sure it is one of many in a line of "reasons the software crisis didn't have to happen" - riiight. The "mythical man month"? It's 'cause they didn't do it right in 1960/1970/1980/1990/2001/... If only "they" would learn...
Take two systems which do the same things. One does it as expected, the other surprises you in subtle ways. One does it as specified, the other sometimes does not. One does it quickly, the other makes you wait a bit. One fits in a few pages of code, the other takes a whole book.
I agree that you can't tell what's right in advance. But it's not a matter of preference, it's a matter of ignorance. In hindsight, when you see the results, you can most of the time point out what could have produced better results, if only you knew. You can even go meta, wondering why you didn't knew, then try and change that in the future.
On the subject of "doing it right" - every software engineer or maybe most passionate software engineers, have an ability to look at a spec or a piece of code and see how it is "done right". I certainly have my conception, my agenda, of what the right way to code and good software engineering is.
The thing is that this comes after fifty years of efforts to "do it right" failing in the sense that we don't have a single language we're satisfied with, we don't have a single operating system people unambiguously call good etc. The edifice of modern computing seems to lack a sound foundation.
The grizzled software engineer often knows this as a fact without caring about why and indeed the why is obscure.
Oddly enough, I think can illustrate my explanation for "why" by noting the common complain that programmers are "constantly reinventing the wheel". Now, if we look at the automotive engineers who build cars, we will note they too are "constantly reinventing the wheel" (literally now). Yet no one complains that the automotive engineer must create a new wheel for a new car with different mechanical properties than the old car which had the old wheel. Here we can see problem and the solution ("reinventing the wheel") does not seem intuitively to we-humans as a problematic state of affairs.
Looked at this way, initial criticism of a software engineer "reinventing the wheel" is a bit ridiculous. It seem logic that an engineer needs to change the components whenever they are putting together a different system for a different purpose. Moreover, the variety of distinct circumstances a software system needs to be engineered for is vast - it faces far more meaningful-context-changes than a car. In this perspective, it is absurd to expect the
Yet, this intuitively, with our human intuition, just doesn't seem right. I would claim that the intuition we have involve a natural conflating of something like "the idea of a wheel" with "the software implement of a wheel". A software implementation of a wheel or anything is more nebulous than a physical wheel but it is still not "the idea of a wheel". But as we "naturally" conflate these two item is easy for us to believe that we only need to think up the concept of an entity and we will have captured the thing. And this natural conflation of the ideas is perhaps where things go wrong... Where we get off expecting "general purpose operating system" to satisfy the gazillion purposes assigned to it, etc.