While I agree that AI in current forms can be very useful, I believe that the problem of e.g. driverless taxis requires understanding of other humans and empathizing with their intentions to be truly viable. Driving is a social activity, and the current self-driving tech is about as convincing as trying to carry on a conversation with Alexa at a party. I do believe that we need AGI before self-driving will be more than a better cruise control.

I think the better cruise control is very useful and I love to see it, but Tesla’s marketing of it as “full self-driving” is disingenuous as best, and industry-chilling + deadly (as we’ve seen) at worst.

> Even if this is the case, why does it matter?

Because people are using Deep-learning over single-lens cameras to replace depth-perception... and then wondering why the cars that do this run into stationary objects with flashing lights. https://static.nhtsa.gov/odi/inv/2021/INOA-PE21020-1893.PDF

No one really cares about where deep learning works. People are complaining about all the areas where deep learning is failing, with dramatic and deadly results.

Entirely agree. The problem is that most people don't understand that and easily fall prey to thinking that AI is a magical black box that can solve any problem you throw at it. In no small part because of all the hype by salespeople and the media. The reality is that NNs are great but only for some types of problems AND where there is a high tolerance to false positives and negatives. Clearly, this does not include problems where safety is critical (unless you can really demonstrate that AI does better at safeguarding than humans AND you can convince the public to not be scared).

All true. But due to the hype we often see people trying to use overly complex ML models in problem domains where simpler deterministic statistical techniques such as linear regression analysis would be just as accurate with lower costs and better testability.

The issue is that people think that neural net technology is AI when, in fact, there is 0 Intelligence there. Which is fine, except when you have people like Elon Musk, Google and others thinking that there is I there, just need to throw more processors and data at it, and I will magically emerge. We now face a growing number of non-I cars on the road starting to kill people, so it's a real urgent problem that all who use neural nets need to, at minimum, understand that that tech will never give I and to communicate that fact to their customers and their goals with using it.

the problem is the unboundedness of the failure modes of (the current generation of) AI, not with how well it can approximate hard functions within regions of interest (which is essentially always a small subset of the range of said hard functions).

The main problem is human overconfidence in the resulting functions, which are often measuring things humans didn't realize was embedded in the data (eg with tumor recognition), or are utterly hopeless at representing the infinite edge cases that the real world can present (eg self-driving cars).

It's one thing to have hoped that these methods could solve these problems when the improvements were coming rapidly, but there will always be a limit to how well these systems can perform. And the problem is that they fail in entirely non-intuitive ways, making human oversight to correct for errors very difficult or impossible as well.