the tenuous connection DL once had to neuroscience (perceptrons) is a distant memory
It seems the whole point is to bring in additional details of how brains work, that the think may be relevant to artificial NNs.
Lots of graph nodes, with weighted connections, performing distributed computation (mainly hierarchical pattern matching), learning from data by gradually updating weights, using selective attention (and/or recurrence, and/or convolutional filters).
Which of the above is not happening in our brains? Which of the above is not biologically inspired?
In fact this description equally applies to both a brain and GPT4.
The model can be the closest working model but that doesn't mean it is complete. It's very likely that cells can store memories/information independent from weights.
Do you see the difference?
That's rather a bold claim given that artificial neural networks are universal function approximators.
Neurons are not connected by a simple graph, there are plenty of neurons which affect all the neurons physically close to them. There are also many components in the body which demonstrably affect brain activity but are not neurons (hormone glands being among the most obvious).
> with weighted connections
Probably, though we don't fully understand how synapses work
> performing distributed computation (mainly hierarchical pattern matching)
This is a description of purpose, not form, so it's irrelevant.
> learning from data by gradually updating weights
We have exactly 0 idea how biological neural nets learn at the moment. What we do know for sure is that a single neuron when alone can adjust its behavior based on previous inputs, so the only thing that is really clear is that individual neurons learn as well, it's not just the synapses with their weights which modifies behavior. Even more, non-neuron cells also learn, as is obvious from the complex behaviors of many single-cell organisms, but also some non-neuron cells in multicellular organisms. So potentially, learning in a human is not completely limited to the brain's neural net, but it could include certain other parts of the body (again, glands come to mind).
> using selective attention (and/or recurrence, and/or convolutional filters).
This is completely unknown.
So no, overall, there is almost no similarity between (artificial) neural nets and brains, at least none profound enough that they wouldn't share with a GPU.
