I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
To folks out there: do the important work, not the glamorous work, and you'll not only sleep well, but you might actually matter as well.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
It is something to do with that being the engineers actual job, to find and understand the problems with the product. so when talking to a customer, that is what tends to come across, all the problematic stuff. The good stuff that works, not important to them.
But using "uncertain" language seems unconvincing to people outside of these types of cultures.
Also of course the power dynamics.
In my mind, I'm thinking "so long as a meteor doesn't cause an extinction event," but the manager graciously pushes the target date back a week.
- bad communication possible in any medium
- pptx in NASA even today!
- issue is managers/SMEs communication differences
- issues with technical papers
- long
- boring
- vs word, excel, pdf...
Manager/SME Differences
- context vs conclusion
- tell a compelling story
- but give away the ending FIRST
- inherent personality differences
- motivations/incentives/mindsets
Learning from disasters
- medium guides message and messenger
- blame tool - binary choice?
- presentation aide vs distributed technical artifact
Questions?
I’ve found that most folks have no intention of improving their communication effectiveness. Everyone is much happier, blaming the audience.
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
Working on a few possible solutions.
Details (for those interested):
System x is connected to ...blah blah blah...
I question your premise. :J
I'm just kidding, that's interesting I'll have to think about applying that. I don't suppose that would translate over to blogging? The fear of course is that one makes a statement and the commentariat thinks the speaker is full of it for not having provided backup instead of questioning. Maybe it's dependent on what type of forum it is.
It would have been nicer if that had been the first sentence of that (interesting) comment.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
[0]: https://www.edwardtufte.com/notebook/new-edition-of-the-cogn... [1]: https://www.edwardtufte.com/notebook/the-columbia-evidence/
https://www.nasa.gov/history/rogersrep/v2appf.htm
The words "a safety factor of three" will live with me for every day of my life.
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
https://www.researchgate.net/publication/208575160_The_Cogni...
<https://www.edwardtufte.com/notebook/powerpoint-does-rocket-...>
> “The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA.”
I don't think PowerPoint is the problem in and of itself, but rather its use as a crutch to compensate for poor communication. Of course, even among scientists, few can count themselves at Feynman's level in terms of communication skills. Maybe this is a skill that NASA scientists need to brush up on, perhaps with Pluralsight courses or something? lol
White boards are... ok... better than powerpoint but still fail to sell it like a chalkboard does. I think it is the noise.
I teach in a classroom that had blackboard that had stood the test of time for decades. When it was replaced with a whiteboard, things went downhill. The markers dry out quickly, without much notice, so that students often have trouble reading the material. And the whiteboards get harder to erase year after year.
I guess the advantage of whiteboards is that a variety of colours can be used. But some students have deficiencies of colour recognition, so that's not really helpful. (I never used coloured chalk, for the same reason. Maximal contrast is the key.)
And the noise. That click drag click of chalk. Students after the transition to whiteboards told me that they really missed that. It enlivened the lectures. And when students were writing down notes, they knew to look up when they heard the sound.
Back to the point about the "visual show" and doing slides in real time. Yes, yes, yes. Once in a while I need to show something on the projector. The moment I turn it one, I see students start to disengage.
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
"We checked the test data: possible to damage tiles significantly" "Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
I would say the disaster occurred despite PowerPoint, not because of it. It's not clear to me at all why the slide author thought all that text was needed, when it seems to communicate almost nothing. If anything I would blame it on the culture around "real documents", where having more information is treated as better (probably because they serve multiple functions - to educate, but also as a record of activities), even if it makes it bloated and hard to read.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
He has a legendary hatred of PowerPoint.
https://www.edwardtufte.com/product/the-cognitive-style-of-p...
I took his course, a number of years ago, and he mentioned it.
"The imagery request was soon rescinded by NASA Mission Management Team Chair Linda Ham after she investigated the origin of it. She had consulted with Flight Director Phil Engelauf and members of the Mission Management Team, who stated that they did not have a requirement for imagery of Columbia. Ham did not consult with the Debris Assessment Team, and cancelled the imagery request on the basis that it had not been made through official channels." https://en.m.wikipedia.org/wiki/Space_Shuttle_Columbia_disas...
Found the chapter here: https://williamwolff.org/wp-content/uploads/2013/01/tufte-ch...
https://justin.searls.co/posts/sprinkling-self-doubt-on-chat...
Some previous discussions:
The first time I came across this blog post a few years ago, I found it enraging, and I still do. As I said I said in my personal notes at the time, "The choice to use this fabricated material for this article and the false claims in it—claims not actually confirmed by the sources that it cites—amounts to what would be considered serious misconduct elsewhere." But there's no real accountability here, of course, because at the end of the day it's just some jerk with a blog serving up distorted pop science insights to aspiring entrepreneurs.
Death by PowerPoint: the slide that killed seven people (2019) - https://news.ycombinator.com/item?id=30615114 - March 2022 (197 comments)
Death by PowerPoint: The slide that killed seven people - https://news.ycombinator.com/item?id=19668161 - April 2019 (127 comments)
What was the alternative?
Columbia could not have made it to ISS.
Columbia could not have repaired the damage in orbit.
Columbia could not have lasted, after two weeks in space, long enough to launch a rescue mission.
I know the "In Flight Options Assessment" said they could launch at an accelerated pace but the assessment assumes that it's ok to launch another vehicle with the same problem, no fix, and no completed analysis of the cause.
Yeah, they suspected the external tank bipod foam, but WHY did the foam come off? Was it a fluke? Had some unknown factor not present in previous external tank bipod foam applications but now present in all external tank bipod foam applications manifested?
>Two major assumptions, apart from the already stated assumption that the damage had to be visible, have to be recognized – the first is that there were no problems during the preparation and rollout of Atlantis, and the second is the question of whether NASA and the government would have deemed it acceptable to launch Atlantis with exposure to the same events that had damaged Columbia. At this point, at least two of the last three flights (STS-112 and STS-107) had bipod ramp foam problems, and the flight in-between these two, STS-113, was a night launch without adequate imaging of the External Tank during ascent.
https://govinfo.library.unt.edu/caib/news/report/pdf/vol2/pa... (page 397)
That's not a valid assumption.
This is from a pre-flight safety report for STS-113
>“More than 100 External Tanks have flown with only 3 documented instances of significant foam loss on a bipod ramp”
STS-1 through STS-111, April 1981 - June 2002: three "significant" bipod foam losses
STS-112, October 2002: significant foam loss
STS-113, November 2002: night time, but they saw 112 and went "oh shit" and wrote a report
STS-107, January 2003: yet another, fatal, significant foam loss
If two of the last three flights had foam problems and the one that didn't only didn't because you couldn't see if it did, and over 100 of the preceding flights only had three, you don't risk four more lives.
You start designing a memorial at Arlington.
Apollo 13 probably seemed pretty hopeless. They didn't just say "sorry fellas, maybe you'll get lucky and make it, good luck."
The late great Bob Hoover said, "If you’re faced with a forced landing, fly the thing as far into the crash as possible." You keep trying until you can't anymore.
Saying "well, maybe they'll survive," and not even telling them, is not the move when you have nearly a month to figure something out.
If they did a spacewalk and found the damage, what were their options?
> A Columbia rescue mission would have been the most monumentally difficult and epic space mission in history, and it would have required absolutely everything going right to bring the crew home safely. But NASA has shown time and again its ability to rise to the occasion and bring its formidable engineering and piloting expertise to bear. Instead, the worst instincts of the agency - to micromanage and engage in wishful thinking instead of clear-eyed analysis - doomed the crew.
https://www.quora.com/If-NASA-had-known-ahead-of-time-Columb...
A much more deeply researched article here:
https://arstechnica.com/science/2016/02/the-audacious-rescue...
I can’t imagine it will ever happen, it will be such a bad look for NASA
> Think about your message. Don’t let that message be lost
That was the important bit of the article. Regardless the medium, ensuring you deliver the message you want should always be the driving point.
Also: Yes to less slides and more Position Papers (but keep em brief please :-D)
You want the most important information in the right places, communicated with as few words as possible, using the most accurate words possible.
You want the key takeaways to be the things that people are most likely to remember from each slide.
You want to minimize distractions and try not to pollute slides with a bunch of vaguely related stuff. A crowded slide risks communicating nothing.
It's a real dramatic change compared to how I am used to using powerpoint for technical audiences or when I had to make presentations during school.
So you’re better off either opening with the conclusion or putting it on the next slide, but accidentally jumping two slides forward can still ruin your audience’s attention span. So sometimes it’s better for it not to be in the deck at all.
If they've largely grown up in the social media era and click away from reels/shorts that don't have animated captions, you'd design a very different deck.
This problem seems mostly solved now.
The shuttle was about as reusable as the Falcon 9 is now. Though at vastly higher expense.
Both Columbia and Challenger were quite preventable, but the problems were basically ignored because they clearly hadn't destroyed the orbiter. Never mind that both showed random behavior outside the design spec, sooner or later the orbiter was going to roll a 1. They had seen the blow-by that killed Challenger, they had seen orbiters scoured by the foam before.
However, it didn't really have a solution. The blow-by problem that destroyed Challenger was fixable, the foam problem was not. You could reduce the chance of a problem but nothing could be done about the fundamental problem of having parts of the heat shield aerodynamically behind other parts of the spacecraft, especially cryogenic parts of the spacecraft. (Ice could also do damage.)
If a rescue mission could have been mounted, it’d be something straight out of science fiction - tying ships together with cables and shuttling astronauts in their orange pressure suits through the side doors of two disconnected shuttles.
Then you shouldn't be in charge of communicating highly technical subject matter to decision makers, especially if lives are at stake.
This slide was presented with a verbal talk track, and anyone who can't handle focusing on the topic because the slide is boring shouldn't be in a position of responsibility.
So on the remote chance you're not just trolling: If you're doing anything safety critical, please quit your job before you kill someone. You vastly overestimate human's (including yours) ability to process information. I am being 100% sincere.
But you are right, most engineers would consider that reasonable, while complaining about the "muggles" that just don't get it.
As a Software Architect, one of my main responsibilities has been to take information presented like above and turn it into something that non-technical people can digest.
Being able to express a complex concept in simple terms is an invaluable skill.
> Everything is fine.
> Stuff is good.
> There's no problem.
> It's all going great.
> Actually, everyone on board is likely to die.
The principles of that slide apply to a lot of other circumstances.
While it does not wash the responsibility of the executives, the engineers have also the responsibility to be clear in their communication
What makes you think this, given the subsequent events?
You can’t force someone to think but you can force a lot of people not to, or you can make it difficult to avoid. It’s worth investing the energy into stacking the deck the right way.
It also shows why software engineers are superior engineers. The history of software engineering has some single-digit kills, at most 20 in total. Meanwhile, shuttle engineers are supposed to be the best in the aerospace business and they've lost some 13 or so? All aerospace would be put in the thousands.
An old saying is "Any fool can build a bridge. It takes an engineer to build a bridge that barely stands". But these are fools who built bridges that didn't stand.
One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
BTW: Here's is the original content with Tufte review https://www.edwardtufte.com/wp-content/uploads/bboard/images...
Buddy, if you think that is "real engineering", you've got a lot of things to learn.
Even in software engineering, rails and react aren't really engineering. Real engineering includes calculations -- data structures, algorithms, data structures, and user interfaces designed to work with people who are deaf or blind. You wanna tell me that rails is performant? Have you done calculations? You wanna tell me react's algorithms are well-defined in memory and time? Have you ensured that your "software" is usable by people with disabilities?
