undefined | Better HN

0 pointsdeepnotderp9y ago0 comments

I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either.

On the other hand, we're nowhere near human level intelligence in most tasks (Go, Poker and image classificaiton non withholding) so I can understand the argument for explainability from a practical perspective. I think we're making some good progress in that direction though, and I'll list them below:

1) Attention maps in CNNs can tell us what the net is usually looking at.

2) "Attentive Explanations" use attention mechanisms to point to the object of interest to generate an explanation for VQA tasks, check the paper (warning PDF): https://arxiv.org/pdf/1612.04757

3) A recent project used a similar explanation mechanism that forced the network to output "what it was thinking" while playing an Atari game.

4) NTMs (neural turing machines) allow weighted memory access, which alleviates the back box issue to some extent.

0 comments

4 comments · 4 top-level

dmreedy9y ago

>> since our goal is human level intelligence and human intelligence isn't easily explained either

I think this might be where the real divide lies. To present a false dichotomy, there are two schools. One that is interested in building systems that attain human-level intelligence, and another that is interested in understanding the nature of human intelligence by building models using our closes understood analog (computation).

We're definitely making progress on Deep Learning models that tell us more about how they think they work, but philosophically, I'm not sure we're making progress on understanding the nature of what they experience. I don't mean this in any polemical sense, just trying to explain the particular dissatisfaction (and it -is- dissatisfaction, not contempt or disdain. The stuff is amazing) I feel towards these advances.

romaniv9y ago

>I'm not sure I fully buy the "explainability" argument against deep learning since our goal is human level intelligence and human intelligence isn't easily explained either.

There are two types of explainability. One is explaining why the system converges on correct solutions in general. Another is explaining why a particular decision was made the way it was made. Human intelligence is usually very much explainable in the second sense. This is not always true, but such cases are called "intuition" and we tends not to trust such judgements for mission-critical tasks.

ylem9y ago

The following article has an interesting treatment of interpretability: https://arxiv.org/pdf/1606.03490.pdf

sgt1019y ago

Explainability is important - understanding why and how, but when it comes to the what the issue is accountability. If you are dragged up in court and accused of manslaughter (hello 5 years in an open prison) then you really want to be able to say, hand on heart, that what you did was professional and that you could reasonably have expected "not to kill". "I A/B tested it and everything checked out" isn't going to cut it.

Also; the nights in the prison cell contemplating that you are held responsible for offing some poor sod. Not so good.

j / k navigate · click thread line to collapse