Lets see:
* 35 million dollar budget a year.
* developing software that only has "one job", regardless of how hard that job is.
* In other words, no evolving software to meet the demands of an evolving world. Fixed requirements 'forever' (as much as forever can be in software terms). No one at NASA is going "hey, I know we developed this for earth orbit but lets alter the design and try use 90% of the same code to go to the moon as well earth orbit. We'll save time and money." (Unlike SpaceX to some extent)
* irreversible decisions. safety critical etc.
* Due to lavish budget and safety critical risks, reduced schedule pressure. No one wants to be on the other end of a independent report concluding that long hours resulted in sloppy code that got astronauts killed.
The above often even doesn't apply to most relatively expensive (in the millions) non-safety critical military and government software. Watch the Pentagon Wars for a hilarious illustration that comes closer to the reality that software engineers face in those environments.
Its a scandal that space shuttle software is being used to push pseudo-scientific process ideology. Its often claimed that SEI developed CMMI process using the space shutte software process as a blueprint. CGI Federal was CMMI Level 5 on paper and QSSI was CMMI Level 3. Meaning that the companies responsible for the most public software scandal in recent US history that is healthcare.gov wasted something like $300 mil at t0 writing the "wrong stuff" while claiming to do it "the right way".
Enforcing abstract processes instead of dealing with case-to-case facts reminds me of this quote:
"We are all capable of believing things which we know to be untrue, and then, when we are finally proved wrong, impudently twisting the facts so as to show that we were right. Intellectually, it is possible to carry on this process for an indefinite time: the only check on it is that sooner or later a false belief bumps up against solid reality, usually on a battlefield." --George Orwell
Contrast all this with Elon Musk's thoughts on Process:
"Now I have to tell you something, and I mean this in the best and most inoffensive way possible: I don’t believe in process. In fact, when I interview a potential employee and he or she says that “it’s all about the process,” I see that as a bad sign.The problem is that at a lot of big companies, process becomes a substitute for thinking. You’re encouraged to behave like a little gear in a complex machine. Frankly, it allows you to keep people who aren’t that smart, who aren’t that creative."
>when agile developers/hackers/programmers grow up they end up doing design up front.
er, what? The agile push came people who were reacting against creating pages of useless design up front documentation that was Dead On Arrival as soon as it was written. People in general become more capable in design up front as their knowledge of the domain grows, but that too depends on context. I'm not sure how valuable design up front would be to even an experienced software engineer who is faced with a project in a programming language he hasn't worked with and a customer domain and platform he doesn't know....
related, SpaceX Lessons Learned: http://lwn.net/Articles/540368/
er, what? The agile push came from people who saw an opportunity to make money [1]. To understand that you need to understand both the advantages of waterfall, it's disadvantages, and how those disadvantages were mitigated. Documentation is but a single (and pretty minor) aspect of it.
The biggest single issue with waterfall is also is biggest advantage - segmentation (what Barry Boehm called segments or stages of a project). Stages follow each other sequentially. Requirements analysis follows inception, design follows requirements analysis, development follows design, test development, and so forth.
The advantage is that it is predictable (ironically, agile's biggest disadvantage). It's disadvantage is that it's crazy expensive to reverse the sequence, i.e. go from test back to dev, from dev back to design, and so on.
The mitigations back in the early 80's were to prototype (fail fast) and to turn a single protracted release into many smaller releases (what became known as the spiral model) - aka fail often. In amongst all of that TDD happened, and someone decided to turf documentation because everyone hates doing it anyway.
eXtreme programming came along and gave protyping and iterative releases a shiny new badge. Pair programming wasn't for everyone, so someone removed that, added daily standups and gave it a shiny old name (scrum was first applied to product development in the early 80's and formally documented in '86[2] - it was nothing like what it is today).
Oh, and modern agile processes contract release cycles even more.
Note that there are other intrinsic (requirements almost always missing/incomplete, stages produce communication gaps, and as you mention, documentation) and extrinsic issues (it's a pm methodology, not a dev methodology) with waterfall.
[1] http://www.infoq.com/articles/agile-teenage-crisis
[2] https://www.wittenburg.co.uk/Entry.aspx?id=dce9dde8-d770-47c...
er, what? The agile push came from people who saw an opportunity to make money [1].
There are only four items in the Agile Manifesto. The second item specifically highlights documentation cost overhead of traditional methods as a major grievance: "Working software over comprehensive documentation" (see also [1]). I don't think money is a differentiating factor, we're talking about the software business in the end after all. If you're referring to process snakeoil marketing, I'm against that no matter which side is doing it. I was responding to your assertion that software developers go from "agile" to "design up front" as they "grow up". My view is that the people that began popularizing the agile term were in fact "grown up" programmers. Are we talking about different things when we talk about design? What is the output of "design up front" in the traditional sense if not documentation and diagrams? (for example, see the military standards and ISO standards for what's required for detailed design)
I don't see anything objectionable in the descriptions you give, although I don't view the waterfall process as being predictable in any meaningful sense. Its only predictable in so far as its underlying assumptions hold. And in the software world at large such assumptions (fixed unchanging requirements for example) rarely hold. My claim is that circumstances for the space shuttle software is the exception, not the rule.
No, they write software this good because they have enough time (e.g. money) to write quality software.
Right up there with Steven Covey's "Begin with the end in mind". I've witnessed it again and again - when agile developers/hackers/programmers grow up they end up doing design up front. Fail fast and often has it's place, but that place is finite.
For anyone wanting to dig a bit deeper, check out the NASA Software Safety Guidebook [http://www.hq.nasa.gov/office/codeq/doctree/871913.pdf]
