You can only hold around 7 to 10 numbers in your mind well, in your working memory. Let me give you a few: 6398 5385 3854 8577

You have 1 second, close your eyes and add them together. Write down the result.

I’m pretty sure that GPT-4 at its 4k setting would outperform you.

[The point being, we have not seen what even GPT-4 can do in its optimal environment. Humans use paper, computers, google, etc. to organize their thoughts and work efficiently. They don’t just sit in empty space and then put everything into the working memory and magically produce the results. So imagine now that you do have a similar level of tooling and sophistication around GPT-4, like there is present around humans. I’m considering that and it is difficult to extrapolate what even GPT-4 can do, in its optimal environment.]

>And if only scaling that context length weren't quadratic...

There are transformers approximations that are not quadratic (available out of the box since more than a year) :

Two schools of thoughts here :

- People that approximate the neighbor search with something like "Reformer" and O(L log(L) ) time and memory complexity.

- People that use a low-rank approximation of the attention product with something like "Linformer" with O(L) complexity but with more sensibility to transformer rank collapse

So how many of those 100 trillion synapses are actually in the part of the brain that does the thinking? Because the brain has different regions (subsystems) responsible for different things.

> But these systems need to grow a ton of new helper functionality and subsystems to hope to achieve that kind of performance. And, I'll venture that kind of thing is a lower bound on the abilitites of any being who would be able to savage the world with it's intellect. It will have to be able to link up so, so many disparate threads to do it. It boggles our minds, which are only squaring a measly 100T dimension every tick.

Agreed: LLM are just one of many necessary modules. But amazing nonetheless. The quadratic scaling problem needs an attentional-conceptual extractor layer with working memory. Hofstadter points out that this needs to be structured as a recursive “strange loop” (p 709 of GEB). Thalamo-cortico-thalamic circuitry is a strange loop and attentional self-control may happens by phase- or time-shifting activity of different circuits to achieve flexible “binding” for attention and compute.

I’m actually optimistic that this is not a heavy computational lift but a clever deep extension of recursive self-modulating algorithms across modules. The recursion is key. And the embodiment is also probably crucial to bootstrap self-consciousness. Watching infants bootstrap is an inspiration.