"Even though an amplifier and a computer are both made of transistors, they have almost nothing else in common. In the same way, a real brain and a three-row neural network are built with neurons, but have almost nothing else in common." -- Jeff Hawkins (On Intelligence)
One example is that most NNs neglect the time domain.
A balanced approach recognizes the importance of learning from data, but does not _rely_ on big data. A holistic approach entails a close examination of biological learning systems.
