Hold up. Any AI that exists is an IO function (algorithm) perhaps with state. Including our brains. Being an “x completion” algorithm doesn’t say much about whether it is AI.

Your comment sounds like a rhetoric way to say that GPT is in the same class as autocomplete and that what autocomplete does sets some kind of ceiling to what IO functions that work a couple of bytes at a time can do.

It is not evident to me that that is true.

LLMs predict language, and language is a representation of human concepts about the world. Thus, these models are constructing, piece by piece, conceptual chains about the world.

As they learn to construct better and more coherent conceptual chains, something interesting must be happening internally.

It's not wild. "Predict the next word" does not imply a bar on intelligence; a more intelligent prediction that incorporates more detail from the descriptions of the world that were in the training data will be a better prediction. People are drawing a straight line because the main advance to get to GPT-4 was throwing more compute at "predict the next word", and they conclude that adding another order of magnitude of compute might be all it takes to get to superhuman level. It's not "but what if we had a better algorithm", because the algorithm didn't change in the first place. Only the size of the model did.

If the text completion algorithm is sufficiently advanced enough then we wouldn't be able to tell it's not AGI, especially if it has access to state-of-the-art research and can modify its own code/weights. I don't think we are there yet but it's plausible to an extent.

I agree, it's an extremely non-obvious assumption and ignores centuries-old debates (empiricism vs. rationalism) about the nature of reason and intelligence. I am sympathetic to Chomsky's position.[1]

https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...

ChatGPT is not AGI, but it is AI. The thing that makes AI lose all meaning is the constantly moving goal posts. There's been tons of very successful AI research over the past decades. None of it is AGI, but it's still very successful AI.

An algorithm that completes "A quantum theory of gravity is ..." into a coherent theory is of course just a text completion algorithm.

There has been debate for centuries regarding determinism and free will in humans.

> You think his existential worries are less important than being liked by his coworkers??

Yes, actually. This is overwhelmingly true for most people. At the end of the day, we all fear being alone. I imagine that fear is, at least in part, what drives these kinds of long-term "existential worries," the fear of a universe without other people in it, but now Ilya is facing the much more immediate threat of social ostracism with significantly higher certainty and decidedly within his own lifetime. Emotionally, that must take precedence.

He may have wanted Sam out, but not to destroy OpenAI.

His existential worries are less important than OpenAI existing, and him having something to work on and worry about.

In fact, Ilya may have worried more about the continued existence of OpenAI than Sam after he was fired, which looked instantly like a: "I am taking my ball and going home to Microsoft.". If Sam cared so much about OpenAI, he could have quietly accepted his resignation and help find a replacement.

Also, Anna Brockman had a meeting with Ilya where she cried and pleaded. Even though he stands by his decision, he may ultimately still regret it, and the hurt and damage it caused.

I think his existential worries about humanity were overruled by his existential worries about his co-founder shares and the obscene amount of wealth he might miss out on

Because, if they announced it and it seemed plausible or even possible that they were correct, then every media outlet, regulatory body, intelligence agency, and Fortune 500 C-suite would blanket OpenAI in the thickest veil of scrutiny to have ever existed in the modern era. Progress would grind to a halt and eventually, through some combination of legal, corporate, and legislative maneuvers, all decision making around the future of AGI would be pried away from Ilya and OpenAI in general - for better or worse.

But if there's one thing that seems very easy to discern about Ilya, it's that he fully believes that when it comes to AI safety and alignment, the buck must stop with him. Giving that control over to government bureaucracy/gerontocracy would be unacceptable. And who knows, maybe he's right.