But it’s not a real defense for the Max because the problem with the Max is that Boeing has shifted a lot of the work from hardware to software.
This defense only punts the ball a few yards because the question now is why Boeing chose to shift a lot of the reliability from hardware, which this defense admits is better engineered, to software, which it admits is almost intrinsically worse.
If anything, this defense leads to the conclusion that the Max is an intrinsically unsafer plane, because it has shifted far more of the burden to software which is an intrinsically worse engineering discipline.
If it was just a matter of the software QA being rushed then the additional time gained by the grounding of the planes could solve the problem. But if it is software engineering itself which is the problem then no additional time will help unless it leads to shifting some of the burden of safely flying the plane from software to hardware, which doesn’t even appear to be an option Boeing has considered.
This is so not true.
Processes? Like T-CMM/TSM, FAA-iCMM, ISO 90003, TCSEC, TSP, etc., etc.? Languages? Like Ada, MISRA, Modula-2/IEC1131, Cyclone, etc., etc.? Tools? Like FRAMA-C, MALPAS, SPADE, SPARK, TLA+, etc., etc.?
The simple fact of the matter is that we've had all the processes, languages & tools to write high-reliability, secure code for decades. Go check out the CMU SEI for how long that one institution has been trying to get people to do the right thing.
The joke is that the software industry by and large wants to pretend we don't, and claim that there's some magical new tech that needs to get invented and when it arrives everyone will jump on it and suddenly software bugs will be a thing of the past. And every new tool, tech or whatever that shows up re-proves some unpleasant basic facts: writing secure, provable, reliable code is time consuming, usually difficult, which translates to relatively expensive. And the software industry doesn't want to hear that, or invest in it, and excuses it's miserable security record with "it's not us, the tech doesn't exist to do the job right".
As a sidenote, this is why I have a visceral distaste for the Rust people (the language seems fine). If they'd spent those (presumedly) thousands of man years building on and improving Ada instead of being all NIH averse we'd be much further down the road. They could have participated in the process and gotten their wish list included in the Ada-2012 standard and built on 35+ years of experience. But, hey, it's more fun to start from scratch every decade or so.
Gotta fundamentally disagree with you right there. We absolutely can make high-quality software with our current tooling. The issue is that doing that is expensive and time consuming, and the Market optimizes on good enough to be sold and not dropped.
This expense and difficulty isn't an inherent fault of the tools, but rather the monstrous other side of the coin in proving what your system isn't.
There are many implementations that are composable to generate an end result, the trick is to expend the energy to ensure you've made the specific one that also doesn't run into undefined behavior, domain specific or otherwise.
You will never escape from the tyranny of having to clearly communicate to a perfectly obedient machine exactly what it is you want it to do; part of which is being able to identify when you don't have all your requirements right.
My dishwasher stopped working today. Last week I had to replace a wheel bearing in my van ($900). I just got a refund for a pond aerator that stopped working after a few weeks. And the head came off my plastic toy.
Hardware fails all the time: does that mean I can generically say all manufacturing has shitty QC?
See MCAS programmers at $9/h.
So they did not do enough testing before, just the happy case and never considered sensor failures. I am now wondering if they tested the rest of the software or FAA will just look into MCAS and ignore all the rest.
Why didn’t they find these bugs during the plane’s development and certification? It appears the testing then was less rigorous. They should extend the same rigorous testing to their other aircraft models also.
You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time.
I get that planes cost more money than I can fathom, and that making a whole fleet of impossible amounts of money costs a gazillion dollars. Still, this one seems spent. Nobody is going to knowingly fly on a 737 Max.
They ought to have retired the plane last year. They can design a new plane (that because of economics, will probably be very similar to this plane), release it when it's been properly vetted, swap out Maxes for it, retrofit those Maxes, etc.
I realize this is naive armchair quarterbacking from someone who has never worked in aviation, but there's a reason that Philip Morris is called Altria now and that Weinstein Co was merged into Spyglass. If the public doesn't trust your brand, no amount of "but we fixed it with this patch we rushed out the door" is going to change that.
I will respectfully disagree with you. Most airlines won't tell you your aircraft when you book a ticket and even if they do, they seem allowed to change it at the last minute. If you have spent $500,$1000, $XXXX on a ticket, will you not board if you discover the plane has been switched? Will you avoid airlines that don't guarantee you a certain plane?
There are not many great options to travel long distances quickly. If this plane becomes commonplace, I'm afraid consumers will be forced to use it. If there are other ideas on how this could play out, I'm happy to consider them, but I fear this is nearly a given.
Because the first ticketing site that did that would gain a definite edge. And even if the airline doesn’t tell you what plane is flying, we know what routes and airlines are flying 737 Max’s, which is enough to raise an indicator. If that happens, flying a 737 Max may potentially cause airlines to lose money even if the particular flight isn’t a 737 Max, because it might be one.
I don't think this is remotely true. I just checked Delta and British Airways; both show the aircraft type in "details". (Pick the 747 flights from BA while you still can!)
> even if they do, they seem allowed to change it at the last minute
Airlines shuffle aircraft occasionally, but usually the aircraft type for a particular flight is predictable. If airline A flies the 737 Max and airline B flies the A320 on the same route, people will flock to airline B.
[1] https://en.wikipedia.org/wiki/List_of_Boeing_737_MAX_orders_...
This would bankrupt the company.
(Not necessarily uncalled for. But not something it will do on its own.)
> If the public doesn't trust your brand
The public has shown one preference, above all else, when it comes to flying: pricing. The 737 MAX will be renamed, re-certified and nobody but an obsessive minority will avoid flying on it.
(This is why, btw, we need strong airline regulators. Market pressure is ceteris paribus insufficient.)
Is that even possible? From what I understand, it's a strategic company for the United States, so they have to keep it alive no matter what (I recall reading that it's actually a law) - or at least its military branch.
This is all about operator logistics and lock-in by costs of retraining and infrastructure. (I guess, this may be the real point to be addressed, since, as this is such a crucial factor, procedures are prone to be repeated in other configurations in the future. Failures are highly likely to be repeated, regardless of the actor.)
Edit: To emphasize the last thought, the 737 Max may be more of a systemic failure of the entire business, its regulations and how they are conducted, than a failure by a single actor on a single instance.
I assure you that outside of people interested in tech, few I know have any idea what the MAX8 is, how to tell one from the NG or an A320. Once the plane flies it’s going to be business as usual.
Most people are not well represented by the fine folks on HN.
After a few years, no one will care anymore.
- buggy
- the one where bugs were not yet found
I only remember hearing about "mathematically proven software" as an undergrad and just googled to find this name. I've always been interested in learning more of what it's about but never jumped in.
Formal verification is a good and useful tool, but it provably cannot cover the entire system, and practical limitations will limit it even further.
Formal verification of source code is still subject to compiler bugs. Formally proven compilers are subject to bugs in the larger system (IIRC Csmith was able to find an incorrectness in code generated by CompCert because of a bug in a system header file).
How Amazon Web Services Uses Formal Methods
I would imagine the MCAS belongs to this class. But even if your software is correct, the design it implements might be flawed, say by assuming the input you get from a single fallible sensor is to be trusted.
So far only one "aircraft" has had perfect software, and that was the Space Shuttle, every single other aircraft out there has had software issues that are worked out over the life of the aircraft, just like every piece of software, even that which has very strict testing regimes, has had defects in it.
Actually it had 3+ known bugs.
With all of these aircraft that are so computer reliant it becomes this magic box that is nearly impossible to diagnose and fix quickly. You do the circuit breaker reset, then reset the whole jet, then check the connectors of the components of the system, then change some of the computers/controllers, all the while checking for any fault code that might lead you down the right path.
This process often takes 30-60 minutes by which time you’re boarded and ready to go and if it’s not fixed by then it turns into getting everyone off the aircraft and finding a different aircraft so the broken ship can be taken to the shop and a through investigation of the issue can be done.
Meanwhile the customers riding in the MD88 already had their mechanical part replaced and they’re on their way, none the wiser because the mechanic got it diagnosed and replaced before boarding was even done.
Bugs in software happen because situations where they arise are sometimes hard to predict. You can test your software all you want but it's not until it's in the field that you start discovering new issues because people tend to do things in ways developers didn't consider.
Tesla's software has over a billion miles of data on it and it still has issues in some basic functionality. And let's not talk about Iowa which in itself was a major failure in software release management.
MCAS used only the one sensor, this decision made so as to avoid recertification.
http://www.b737.org.uk/mcas.htm
“Are we vulnerable to single AOA sensor failures with the MCAS implementation or is there some checking that occurs?”
https://www.aviationtoday.com/2019/11/02/boeing-ceo-outlines...
That's not really true. The airframe is fine, except it doesn't handle like a 737. MCAS was meant to make the MAX handle like a 737.
Mentour Pilot, a 737 instructor with a youtube channel, has covered this fairly extensively: https://youtu.be/TlinocVHpzk?t=951
Assuming that's not hyperbole and just to be pedantic:
mov ax,cs
mov ds,ax
mov ah,9
mov dx, offset Hello
int 21h
xor ax,ax
int 21h
Hello:
db "Hello World!",13,10,"$"Expected: "Hello, World!"
Actual: "Hello World!" (missing comma)
See spec: https://en.wikipedia.org/wiki/%22Hello,_World!%22_program
Any fuel-air explosive will do.
Thus the obvious solution to quality problems is to switch missile software engineers and aircraft software engineers, and encourage them not to care about quality.
Yes, electronics fail in the most weirdest ways due to connector failures, RF interference, software error, sensor failure.
When my systems start failing or acting up due to improper stabilization PID gains, etc. I have a big switch for MANUAL mode. I am able to fly this thing as long as the servos, radio, and camera get power. All sensors could be sheared off. I have no idea what my airspeed is ever because I don't use pitot tubes so I use a known engine throttle % whose stall characteristic I understand for level flight in various wind conditions and I don't make sudden maneuvers at throttle below this point.
Fixed wing planes have remarkable aerodynamic stability and I don't understand why 737 MAX cannot be piloted in a fly by wire manner with all computer aids disabled, giving the pilots direct control of the servos with a big red switch that mechanically disconnects the flight computers. This requires almost no code to implement.
The pilots do have direct control over the stabilizer trim, and have always had the ability to disable the electronic system in case of stabilizer trim runaway. This was not new to the MAX, and would have effectively disabled MCAS.
It can be. MCAS can be disabled by disabling the electric stability trim system. The problem is that if you do that in a situation where MCAS has already adjusted the trim far enough from where it should be, it can be mechanically impossible to put the trim back where it belongs without using the electric trim system. So you have to first use the electric trim system to put the trim back where it belongs, then disable it so MCAS can't mess it up again.
https://en.wikipedia.org/wiki/Fault_tree_analysis
Basically, you should be designing every system to gracefully handle the failure of every other system on which it is dependent.
So the MCAS routines, if they had been done correctly, and properly classified as to the hazard level, should have taken into account failures of the Flight Computer they were running on, anomaly detection via cross-check with the second AoA vane, etc. That quite clearly did not happen.
The same approach applies with any other hardware/software integration. Your sensors will break. You therefore need to determine what you need to do when that happens.
It's a bad barrel: a company that has, on a cultural level, put its business motive above its responsibility to deliver a safe and high-quality product. We have seen documented evidence that employees knew there were dangers and problems, and discussed these issues, but nobody cared enough to slow things down and get the product right.
I suspect that a thorough review of some of the more complex Airbus airframes currently in operation would result in some similarly scary findings, tbh.
Part of me feels like many of these companies don’t keep code secret to protect IP, instead they do it because they know it’s a burning train wreck and don’t want people to find out.
https://www.safetyresearch.net/blog/articles/toyota-unintend... https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_...
I do see an opportunity for software that ensures you are only booking journeys on the aircraft you feel are safe.
So, they can't even name the mcas system anymore?
Ahh, "we'll ship it when it's ready, not on some arbitrary deadline." Music to any engineer/builder's ears.