Yeah, to be clear, Watson failing is good news. This is marketing people losing their minds because they didn't understand the products they were selling, and hospitals failing to do due diligence.

I've talked to literally dozens of people / companies / academics running at problems in healthcare using deep learning and large data sets, and the recurrent theme is cowboys who pretty consistently fail to follow basic practices of experimental rigor. ML is frankly an area where more rigorous and clear regulation is desperately needed, because the potential to cut corners / cheat is so easy and the challenges of vetting these systems are specialized enough that most organizations won't successfully be able to catch fairly basic experimental mistakes.