> [...] the "black magic" part comes mostly from their mathematical nature and very little from them being "inteligent computers". A brain is a graph, in which a subset of neurons are "inputs", some are outputs, and others are "hidden". The nodes are interconnected between each other in a fashion, which is called the "topology" or sometimes "architecture" of the net.
The deep question about deep learning is "Why is it so bloody effective?"
The effectiveness comes from their non-linear nature and their ability to "learn" (store knowledge in the weights, that is derived from the training process). And black magic, of course!
As a side note, I was playing a board game last night (Terra Mystica I believe) and wondering if you could get 5 different neural networks to play the game and then train them against each other (and once they are good enough, against players). I wonder how quickly one could train a network that is unbeatable by humans? Maybe even scale it up to training it to play multiple board games til it is really good at all of them before setting it lose on a brand new one (with a similar genre). Maybe Google could use this to make a Go bot.
But what happens if this is used for evil instead? Say a neural network that reads a person's body language and determines how easily they can be intimidated by either a criminal or the government. Or one that is used to hunt down political dissidents. Imagine the first warrant to be signed by a judge for no reason other than a neural network saying the target is probably committing a crime...
This is likely due to the way Go works , random playout provides a rough estimate of who controls what territory ( this is how Go is scored ).
Recently two deep-learning papers showed very impressive results.
http://arxiv.org/abs/1412.3409
http://arxiv.org/abs/1412.6564
The neural networks were tasked with predicting what move an expert would make given a position.
The MCTS takes a long time 100,000 playouts are typical - once trained the neural nets are orders of magnitude faster.
The neural nets output a probability for each move ( that an expert would make that move ) - all positions are evauluated in a single forward pass.
Current work centers around combining the two approaches, MCTS evaluates the best suggestions from the neural net.
Expert Human players are still unbeatable by computer Go.
There was in fact a group within Google that worked on this: http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf
Move Evaluation in Go Using Deep Convolutional Neural Networks Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver
- They use more parameters (and fewer computations per parameter.)
- They are hierarchical (convolutions are apparently useful at different levels of abstraction of data).
- They are distributed (word2vec, thought-vectors). Not restricted to a small set of artificial classes such as parts-of-speech or parts of visual objects.
- They are recurrent (RNN).
etc.
Seemed like a great way to highlight the limitations of patterns.
