> Well I do actually believe this! To me it's the only logical thing. The laws of physics apply equally to a brain and a computer, one is just a lot more fancy than the other one.

I agree 100% with this statement taken in isolation. I don't believe there's more to a human brain than physics.

There is an interesting theory about the brain using quantum mechanics (https://en.wikipedia.org/wiki/Quantum_mind), but that still puts it firmly in the realm of physics even it is true.

My point about "it's all just atoms" was about the fact that we need mental models to discuss things. The models will never be perfectly accurate. Just like software frameworks, they're leaky abstractions. Sure, some models are just plain wrong and should be discarded, but in general we can't reason without them.

And it looks like you agree with that (?):

> I do see the difference and understand what you are getting at. I agree that it's useful to distinguish them in general.

Thanks!

So, on to the core of the discussion:

> It's also useful to realize that it is just a regular program at the end of the day too, just a really complicated one that does some neat stuff.

Sure. If you look at GPT-4 as a whole, it's just a regular program that executes like any other program. It has instructions that use internal data to process inputs from the user and responds with an output to the user.

Nothing new here. Any Turing machine can do this, given enough time and memory. Heck, I saw a video of an 8-bit AVR boot Linux using a simple ARM instruction simulator. Only took 3.5 hours to get the login prompt :)

> Believing that AI is "magic" is pretty dangerous I think.

Not sure what you mean exactly with the "magic" part? Is this a point about something other people think that is inaccurate? Or did I write something that you don't agree with?

To restate my position: I currently believe the neural networks inside the LLMs used to be "stochastic parrots", but that we saw a step-change in performance 1-2 years ago.

We reached a new level of model size (>100B parameters), training data (trillions of tokens), and training time (>1M GPU hours). Somehow the backpropagation training of the neural networks changed the network parameters so that algorithmic processing capabilities emerged.

This isn't fundamentally different from neural networks evolving algorithms to perform OCR, FFTs, balancing a inverted pendulum, playing Go, etc.

Here, the LLMs evolved language processing algorithms. Not only that, they started evolving algorithms for reasoning, abstraction, logic, planning, and problem-solving. Together with that they also formed models about the world to help with the reasoning.

This was driven by the training which seeks to optimize the accuracy of the next word prediction. Lookup tables only get you so far here. At some point you need to understand the context to accurately predict the next word.

For example, in French and German, there are multiple variants of the word "it". To translate the English phrases: "The box wouldn't fit in the suitcase because it was too large" and "The box wouldn't fit in the suitcase because it was too small", you need to understand if "it" refers to the box or the suitcase.

There are 10^80 atoms in the universe. Even if you assume a tiny vocabulary of 100 words, you get more than 10^80 possible combinations after stringing 40 words together. And even if you have unlimited storage, there's not 10^80 tokens to train with. And even with unlimited storage and examples, we don't have unlimited CPU cycles for the training.

So it's clear to me that a "stochastic parrot" (or Chinese room) will be very simplistic even in 1000 years, no matter how much computers progress in that time. And therefore, the latest LLMs must have evolved algorithms for reasoning, abstraction, logic, planning, and problem-solving.

I don't know if that is what you mean by "magic"?

To me, that's not magic. It's just an algorithm (neural network training) creating algorithms and data structures. Impressive as heck for sure, but not magic. I could be wrong, and am more than happy to consider alternatives if you have any?

We have recent examples of similar emergent behavior from big neural networks where they evolve algorithms far beyond what a human programmer can create. For example, AlphaGo which beat the human Go champion. The AlphaGo programmers could never beat him, but they managed to evolve a program that were "smarter" than them (no, Go-playing is not general intelligence).

Now, I could be wrong about the level of intelligence with the latest LLMs like GPT-4. Maybe they're a lot dumber than they appear. But in that case I'm in good company. From what I can tell, the major AI researchers agree with me in that GPT-4 possesses some form of intelligence. It's not a stochastic parrot.

And to end with something I agree with: You wrote that whatever happens because of LLMs in the near future, it's because of human actions. I agree. The LLMs have no agency in themselves. It's humans that use and misuse them.