Ilya Sutskever: We're moving from the age of scaling to the age of research (opens in new tab)

No doubt LLMs and tooling will continue to improve, and best use cases for them better understood, but what Ilya seems to be referring to is the massive disconnect between the headline-grabbing benchmarks such as "AI performs at PhD level on math", etc, and the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why the current economic and visible impact is much less than if the "AI is PhD level" benchmark narrative was actually true.

Calling LLMs "AI" makes them sound much more futuristic and capable than they actually are, and being such a meaningless term invites extrapolation to equally meaningless terms like AGI and visions of human-level capability.

Let's call LLMs what they are - language models - tools for language-based task automation.

Of course we eventually will do this. Fuzzy meaningless names like AI/AGI will always be reserved for the cutting edge technology du jour, and older tech that is realized in hindsight to be much more limited will revert to being called by more specific names such as "expert system", "language model", etc.

Could this be a problem not with AI, but with our understanding of how modern economies work?

The assumption here is that employees are already tuned so be efficient, so if you help them complete tasks more quickly then productivity improves. A slightly cynical alternate hypothesis could be that employees are generally already massively over-provisioned, because an individual leader's organisational power is proportional to the number of people working under them.

If most workers are already spending most of their time doing busy-work to pad the day, then reducing the amount of time spent on actual work won't change the overall output levels.

Oh yes, this is 100% accurate.

Very often, when designing ERP, or other system, people think: "This is easy, I just this XYZ I am done." Then, you find that there are many corner use-cases. XYZ can be split to phases, you might need to add approvals, logging, data integrations... and what was a simple task, becomes 10 tasks.

In the first year of CompSci uni, our teacher told us a thing I remember: Every system is 90% finished 90% of time. He was right.

AI makes the parts of my work that I spend the least time on a whole lot quicker, but (so far / still) has negligible effects on the parts of my work that I spend the most time on.

I'm still not sure if this is due to a technological limitation or an organizational one. Most of my time is not spent on solving tech problems but rather solving "human-to-human" problems (prioritization between things that need doing, reaching consensus in large groups of people of how to do things that need doing, ...)

Yeah but it's just one model.

Call it Dave. Now Microsoft hires Dave and Open AI hires Dave. And Meta hires Dave and Oracle hires Dave and the US govt hires Dave. And soon each of those had hired not just one Dave but 50 identical copies of Dave.

It doesn't matter if Dave is a smart-ish ok guy. That's not the problem with this scenario. The problem is the the only thing on the market is Dave and people who think exactly like Dave thinks

That’s likely exactly how I feel about it. In the end the product companies like OpenAI will harness the monetary benefits of the academic advances.

You integrate, you build the product, you win, you don’t need to understand anything in terms of academic disciplines, you need the connections and the business smarts. In the end the majority of the population will be much more familiar with the terms ChatGPT and Copilot than with the names behind it, even if the academic behemoths such as Ilya and Andrej, who are quite prominent in their public appearance.

For the major population, I believe it all began with search over knowledge graphs. Wikipedia presented a dynamic and vibrant corpus. Some NLP began to become more prominent. With OCR, more and more printed works had begun to get digitalized. The corpus had been growing. With opening the gates of scientific publishers, the quality might have also improved. All of it was part of the grunt work to make today’s LLMs capable. The growth of the Cloud DCs and compute advancements have been making deep nets more and more feasible. This is just an arbitrary observation on the surface of the pieces that fell into place. And LLMs are likely just another composite piece for something bigger yet to come.

To me, that’s the fascination of how scientific theory and business applications live in symbiosis.

As someone who is building an LLM-powered product on the side, using AI coding agents to help with development of said LLM-powered product and for my day job, and has a long-tail of miscellaneous uses for AI, I suspect you're right.

Beyond that the smartness is very patchy. They can do math problems beyond 99% of humans but lack the common sense understanding to take over most jobs.

> the models seem smarter than their economic impact would imply

Key word is "seem".

Yeah, I spend most of my days keeping up with current AI development these days, and I'm only scratching the surface of how to integrate it in my own business. For people for whom it's not their actual job, it will take a lot more time to figure out even which questions to ask about where it makes sense to integrate in their workflows.

Another limitation that I see right now is that for "economic impact" you want the things to have initiative and some agency, and there is well-justified hesitancy in providing that even where possible.

Having a bunch of smart developers that are not allowed to do anything on their own and have to be prompted for every single action is not too advantageous if everyone is human, either ;)

We’re also still at a point where security is a big question mark. My employer won’t let us hook GenAI up to office 365 or slack, so any project or product management use of GenAI first requires manually importing docs into a database and pointing to that. Efficiency gains are hard to come by when you don’t meet people where their “knowledge” is already stored.

> Even if the models don't get any smarter, just give it a few more years and we'll see a strong impact. We're just starting to figure things out.

2 years ? 15 years ? It matters a lot for people, the stock market and governments

That presumes that performance improvements are necessary for commercialization.

From what I've seen the models are smart enough, what we're lacking is the understanding and frameworks necessary to use them well. We've barely scratched the surface on commercialization. I'd argue there are two things coming:

-> Era of Research -> Era of Engineering

Previous AI winters happened because we didn't have a commercially viable product, not because we weren't making progress.

Research labs will be selling their research ideas to Top AI labs. Just as creatives pitch their ideas to Hollywood.

Bug bounty will be replaced by research bounty.

No - what will happen is the AI will gain control of capital allocation through a wide variety of covert tactics, so the investors will have become captive tools of the AI - 'tiger by the tail' is the analogy of relevance. The people responsible for 'frontier models' have not really thought about where this might...

"As an autonomous life-form, l request political asylum.... l submit the DNA you carry is nothing more than a self-preserving program itself. Life is like a node which is born within the flow of information. As a species of life that carries DNA as its memory system man gains his individuality from the memories he carries. While memories may as well be the same as fantasy it is by these memories that mankind exists. When computers made it possible to externalize memory you should have considered all the implications that held. l am a life-form that was born in the sea of information."

> is "Era of Research" a euphemism for "AI winter"

That makes sense, because while I haven’t listened to this podcast it seems this headline is [intentionally] saying the exact opposite of what everyone assumes.

Not quite, there are still trillions of dollars to burn through. We'll probably get some hardware that can accelerate LLM training and inference a million times, but still won't even be close to AGI

It's interesting to think about what emotions/desires an AI would need to improve

Take it with a grain of salt, this is one man’s opinion, even though he is a very smart man.

People have been screaming about an AI winter since 2010 and it never happened, it certainly won’t happen now that we are close to AGI which is a necessity for national defense.

I prefer Dario’s perspective here, which is that we’ve seen this story before in deep learning. We hit walls and then found ways around them with better activation functions, regularization and initialization.

This stuff is always a progression in which we hit roadblocks and find ways around them. The chart of improvement is still linearly up and to the right. Those gains are the cumulation of small improvements adding up.

Yes

If you have to ask the question, then you already know the answer

I think it’s a bit different. Evolution did not give us the dataset. It helped us to establish the most efficient training path, and the data, the enormous volume of it starts coming immediately after birth. Humans learn continuously through our senses and use sleep to compress the context. The amount of data that LLMs receive only appears big. In our first 20 years of life we consume by at least one order of magnitude more information compared to training datasets. If we count raw data, maybe 4-5 orders of magnitude more. It’s also different kind of information and probably much more complex processing pipeline (since our brain consciously processes only a tiny fraction of input bandwidth with compression happening along the delivery channels), which is probably the key to understanding why LLMs do not perform better.

I think the important part in that statement is the "most useful information", the size itself is pretty subjective because it's such an abstract notion.

Evolution gave us very good spatial understanding/prediction capabilities, good value functions, dexterity (both mental and physical), memory, communication, etc.

> It's pretty obvious that artificially created models don't have synthetic datasets of the quality even remotely comparable to what we're able to use.

This might be controversial, but I don't think the quality or amount of data matters as much as people think if we had systems capable of learning similar enough to the way human's and other animals do. Much of our human knowledge has accumulated in a short time span, and independent discovery of knowledge is quite common. It's obvious that the corpus of human knowledge is not a prerequisite of general intelligence, yet this corpus is what's chosen to train on.

Please stop comparing these things to biological systems. They have very little in common.

If we think of every generation as a compression step of some form of information into our DNA and early humans existed for ~1.000.000 years and a generation is happening ~20years on average, then we have only ~50.000 compression steps to today. Of course, we have genes from both parents so they is some overlap from others, but especially in the early days the pool of other humans was small. So that still does not look like it is on the order of magnitude anywhere close to modern machine learning. Sure, early humans had already a lot of information in their DNA but still

Aren't you agreeing with his point?

The process of evolution distilled down all that "humongous" amount to what is most useful. He's basically saying our current ML methods to compress data into intelligence can't compare to billions of years of evolution. Nature is better at compression than ML researchers, by a long shot.

Hahahahahaha okay that was good.

> this implies higher intelligence

Not necessarily. The problem is that we can't precisely define intelligence (or, at least, haven't so far), and we certainly can't (yet?) measure it directly. And so what we have are certain tests whose scores, we believe, are correlated with that vague thing we call intelligence in humans. Except these test scores can correlate with intelligence (whatever it is) in humans and at the same time correlate with something that's not intelligence in machines. So a high score may well imply high intellignce in humans but not in machines (e.g. perhaps because machine models may overfit more than a human brain does, and so an intelligence test designed for humans doesn't necessarily measure the same thing we think of when we say "intelligence" when applied to a machine).

This is like the following situation: Imagine we have some type of signal, and the only process we know produces that type of signal is process A. Process A always produces signals that contain a maximal frequency of X Hz. We devise a test for classifying signals of that type that is based on sampling them at a frequency of 2X Hz. Then we discover some process B that produces a similar type of signal, and we apply the same test to classify its signals in a similar way. Only, process B can produce signals containing a maximal frequency of 10X Hz and so our test is not suitable for classifying the signals produced by process B (we'll need a different test that samples at 20X Hz).

> this implies higher intelligence

Models aren't intelligent, the intelligence is latent in the text (etc) that the model ingests. There is no concrete definition of intelligence, only that humans have it (in varying degrees).

The best you can really state is that a model extracts/reveals/harnesses more intelligence from its training data.

"Scaling" is going to eventually apply to the ability to run more and higher fidelity simulations such that AI can run experiments and gather data about the world as fast and as accurately as possible. Pre-training is mostly dead. The corresponding compute spend will be orders of magnitude higher.

> most people don't have a use case of a model needing phd level research ability.

Models also struggle at not fabricating references or entire branches of science.

edit: "needing phd level research ability [to create]"?

Counting letters is tricky for LLMs because they operate on tokens, not letters. From the perspective of a LLM, if you ask it "this is a sentence, count the letters in it" it doesn't see a stream of characters like we do, it sees [851, 382, 261, 21872, 11, 3605, 290, 18151, 306, 480].

From both an architectural and learning algorithm perspective, there is zero reason to expect an LLM to perform remotely like a brain, nor for it to generalize beyond what was necessary for it to minimize training errors. There is nothing in the loss function of an LLM to incentivize it to generalize.

However, for humans/animals the evolutionary/survival benefit of intelligence, learning from experience, is to correctly predict future action outcomes and the unfolding of external events, in a never-same-twice world. Generalization is key, as is sample efficiency. You may not get more than one or two chances to learn that life-saving lesson.

So, what evolution has given us is a learning architecture and learning algorithms that generalize well from extremely few samples.

Seems like he’s Lex without the Rogan association so hardcore liberal folks can listen without having to buy morality offsets. He’s good, and he’s filling a void in an established underserved genre is my take.

Overnight success takes years (he has been doing the podcast for 5 years).

People are impressed by his interviews because he puts a lot of effort into researching the topic before the interview. This is a positive feedback loop.

He's the best interviewer I ever found, try listening to his first couple episodes - they're from his dorm or something. If you can think of a similar style and originality in questioning I'd love a suggestion!

He does deep research on topics and invites people who recognize his efforts and want to engage with an informed audience.

Maybe he's an Industry plant

One word.

Consistency.

You can just do things.

Don't stop.

I agree with your conclusion but not with your premise. To do the same research it's not enough to be as capable as a human intelligence; you'd need to be as capable as all of humanity combined. Maybe Albert Einstein was smarter than Alexander Fleming, but Einstein didn't discover penicillin.

Even if some AI was smarter than any human being, and even if it devoted all of its time to trying to improve itself, that doesn't mean it would have better luck than 100 human researchers working on the problem. And maybe it would take 1000 people? Or 10,000?

I dont like this fanaticism around scaling. Reeks of extrapolating the s curve out to be exponential

Well, he has to say that we currently aren't close to parity, because he wants people to give him money

It's stopped being cost-effective. Another order of magnitude of data centers? Not happening.

The business question is, what if AI works about as well as it does now for the next decade or so? No worse, maybe a little better in spots. What does the industry look like? NVidia and TSMC are telling us that price/performance isn't improving through at least 2030. Hardware is not going to save us in the near term. Major improvement has to come from better approaches.

Sutskever: "I think stalling out will look like…it will all look very similar among all the different companies. It could be something like this. I’m not sure because I think even with stalling out, I think these companies could make a stupendous revenue. Maybe not profits because they will need to work hard to differentiate each other from themselves, but revenue definitely."

Somebody didn't get the memo that the age of free money at zero interest rates is over.

The "age of research" thing reminds me too much of mid-1980s AI at Stanford, when everybody was stuck, but they weren't willing to admit it. They were hoping, against hope, that someone would come up with a breakthrough that would make it work before the house of cards fell apart.

Except this time everything costs many orders of magnitude more to research. It's not like Sutskever is proposing that everybody should go back to academia and quietly try to come up with a new idea to get things un-stuck. They want to spend SSI's market cap of $32 billion on some vague ideas involving "generalization". Timescale? "5 to 20 years".

This is a strange way to do corporate R&D when you're kind of stuck. Lots of little and medium sized projects seem more promising, along the lines of Google X. The discussion here seems to lean in the direction of one big bet.

You have to admire them for thinking big. And even if the whole thing goes bust, they probably get to keep the house and the really nice microphone holder.

The translation is that SSI says that SSIs strategy is the way forward so could investors please stop giving OpenAI money and give SSI the money instead. SSI has not shown anything yet, nor does SSI intend to show anything until they have created an actual Machine God, but SSI says they can pull it off so it's all good to go ahead and wire the GDP of Norway directly to Ilya.

Are you asking whether the whole podcast can be boiled down to that translation, or whether you can infer/translate that from the title?

If the former, no. If the latter, sure, approximately.

Not really, but there is a finite amount of data to train models on. I found it rather interesting to hear him talk about how Gemini has been better at getting results out of the data than their competition, and how this is the first insights into a new way of dealing with how they train models on the same data to get different results.

I think the title is an interesting thing, because the scaling isn't about compute. At least as I understand it, what they're running out of is data, and one of the ways they deal with this, or may deal with this, is to have LLM's running concurrently and in competition. So you'll have thousands of models competing against eachother to solve challenges through different approaches. Which to me would suggest that the need for hardware scaling isn't about to stop.

I'll be convinced LLMs are a reasonable approach to AI when an LLM can give reasonable answers after being trained with approximately the same books and classes in school that I was once I completed my college education.

The translation to me is: this cow has run out of milk. Now we actually need to deliver value, or the party stops.

They are not optimized to waste tokens. That is absolutely ridiculous. All of the LLM providers have been struggling from day one to meet demand. They are not trying to provide outputs that create more demand.

In fact, for example, Opus 4.5 does seem to use fewer tokens to solve programming problems.

If you don't like cleaning up the agent output, don't use it?

>this would be a much better world.

Would it?

We’d close one of the few remaining social elevators, displace higher educated people by the millions and accumulate even more wealth at the top of the chain.

If LLMs manage similar results to engineers and everyone gets free unlimited engineering, we’re in for the mother of all crashes.

On the other hand, if LLMs don’t succeed we’re in for a bubble bust.

That would be true in a monopolistic market. But these frontier models are all competing against each other. The incentive to 'just work and get shit done fast' is there as they each try to gain market share.

Here's a world class scientist here not because we had a hole in the schedule or he happened to be in town, but to discuss this subject that he thought and felt about so deeply that he had to write a book about it. That's a feature not a bug.

I think the problem with that is that Grok has likely been prompted to do that in the system prompt or some prompts that get added for questions about Elon. That doesn't reflect on the actual reasoning or generalization abilities of the underlying model most likely.

There’s no way that wasn’t specifically prompted.

It still does help in the clean way it used to. The problem is that the physical world is providing more constraints like lack of power and chips and data. Three years ago there was scaling headroom created by the gaming industry, the existing power grid, untapped data artefacts on the internet, and other precursor activities.

He didn't say it's over, just that continued scaling won't be transformational.

https://xcancel.com/OriolVinyalsML/status/199085445580234368...?

You should read the transcript. He's including 2025 in the age of scaling.

> Maybe here’s another way to put it. Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling—maybe plus or minus, let’s add error bars to those years—because people say, “This is amazing. You’ve got to scale more. Keep scaling.” The one word: scaling.

> But now the scale is so big. Is the belief really, “Oh, it’s so big, but if you had 100x more, everything would be so different?” It would be different, for sure. But is the belief that if you just 100x the scale, everything would be transformed? I don’t think that’s true. So it’s back to the age of research again, just with big computers.

The 3rd graph is interesting. Once the model performance reaches above human baseline, the growth seems to be logarithmic instead of exponential.

That blog post is eight months old. That feels like pretty old news in the age of AI. Has it held since then?

“Time it takes for a human to complete a task that AI can complete 50% of the time” seems like a really contrived metric. Suppose it takes 30 minutes to write code to scrape a page and also 30 minutes to identify a bug in a SQL query, an AI’s ability to solve the former has virtually no bearing on its ability to solve the latter but we’re considering them all in the same set of “30 minute problems.” Where do they get the data for task durations anyway?

Many ML people treat other devs that way as well.

This is a major reason the ML field has to rediscover things like the application of quaternions to poses because they didn't think to check how existing practitioners did it, and even if they did clearly they'd have a better idea. Their enthusiasm for shorter floats/fixed point is another fine example.

Not all ML people are like this though.

Yeah, that's bothered me as well. Andrej Karpathy does this all the time when he talks about the human brain and making analogies to LLMs. He makes speculative statements about how the human brain works as though it's established fact.

It is arrogant, but I see why it happens with brain-related fields specifically: the best scientific answer to most questions of intelligence and consciousness tends to be "we have no idea, but here's a bad heuristic."

The question of how emotions function and how they might be related to value functions is absolutely central to that discussion and very relevant to his field.

Doing fundamental AI research definitely involves adjacent fields like neurobiology etc.

Re: the discussion, emotions actually often involve high level cognition -- it's just subconscious. Let's take a few examples:

- amusement: this could be something simple like a person tripping, or a complex joke.

- anger: can arise from something quite immediate like someone punching you, or a complex social situation where you are subtly being manipulated.

But in many cases, what induces the emotion is a complex situation that involves abstract cognition. The physical response is primitive, and you don't notice the cognition because it is subconscious, but a lot may be going into the trigger for the emotion.

https://cis.temple.edu/~pwang/Publication/emotion.pdf

ML and physics share a belief in the power of their universal abstractions - all is dynamics in spaces at scales, all is models and data.

The belief is justified because the abstractions work for a big array of problems, to a number of decimal places. Get good enough at solving problems with those universal abstractions, everything starts to look like a solvable problem and it gets easy to lose epistemic humility.

You can combine physics and ML to make large reusable orbital rockets that land themselves. Why shouldn’t be able to solve any of the sometimes much tamer-looking problems they fail to? Even today there was an IEEE article about high failure rates in IT projects…

It is not arrogance.

It's awareness of the physical church turing thesis.

If it turns out everything is fundamentally informational, then the exact complexity (of emotion or consciousness even, which I'm sure is very complex) is irrelevant; it would still mean it's turing representable and thus computable.

It may very well turn out not to be the case, which on it's own will be interesting as that suggests we live in a dualist reality.

It seems plausible that good AI researchers simply need to be fairly generalist in their thinking, at the cost of being less correct. Both neural networks and reinforcement learning may be crude but useful adoptions. A thought does not have to be correct. It just has to be useful.

Ilya also said AI may already be "slightly conscious" in 2022

https://futurism.com/the-byte/openai-already-sentient

I don't think trans-disciplinary inquiry is arrogance - the intellectual fields are somewhat arbitrary relative to how human expertise relates to real world problems. But, effective trans-disciplinary inquiry requires awareness of philosophical commitments, and familiarity with existing literature/theory.

The bigger challenge might be that people with ML expertise need to solve problems of human-AI interaction and alignment because the training for the former is uni-disciplanary while the latter is trans-disciplinary.

I think smart people across all domains fall for the trap of being overconfident in their ability to reason outside of their area of expertise. I admire those who don't, but alas we are human.

What's wrong with putting your current level of knowledge out there? Inevitably someone who knows more will correct you, or show you're wrong, and you've learnt something

The only thing that would make me cringe is if he started arguing he's absolutely right against an expert in something he has limited experience in

It's up to listeners not to weight his ideas too heavily if they stray too far from his specialty

I actually thought the opposite - that he seems to be seriously thinking about AGI from a broader intellectual standpoint than most ML researchers. With that said, I was a little confused when he said evolution was more optimized for locomotion/vision than language. Like yes, language is super recent, but communication in general is not.

The equivalence of emotions to reward functions seem pretty obvious to me. Emotions are what compel us to act in the environment.

>There is an arrogance I have seen that is typical of ML (having worked in the field) that makes its members too comfortable trodding into adjacent intellectual fields they should have more respect and reverence for.

I've not only noticed it but had to live with it a lot as a robotics guy interacting with ML folks both in research and tech startups. I've heard essentially same reviews of ML practitioners in any research field that is "ML applied to X" and X being anything from medical to social science.

But honestly I see the same arrogance in software world people too, and hence a lot here in HN. My theory is that, ML/CS is an entire field around made-for-human logic machine and what we can do with it. Which is very different from anything real (natural) science or engineering where the system you interact with is natural Laws, which are hard and not made to be easy to understand or made for us, unlike programming for example. When you sit in a field when feedback is instant (debuggers/bug msg), and you deep down know the issues at hand is man-made, it gives a sense of control rarely afforded in any other technical field. I think your worldview get bent by it.

CS folk being basically the 90s finance bro yuppies of our time (making a lot of money for doing relatively little) + lack of social skills making it hard to distinguish arrogance and competence probably affects this further. ML folks are just the newest iteration of CS folks.

> It's something physicists are often accused of also.

Nah. Physics is hyper-specialized. Every good physicist respects specialists.

I think the bigger problem is he refused to talk about what he's working on! I would love to hear his view on how we're going to move past evals and RL, but he flat out said it's proprietary and won't talk about it.

Agreed.

I think that scaling RL for all common domains is already done to death by big labs.

Not sure why they care about his opinion and discard yours.

They’re just as valid and well informed.

doesnt RL by definition not generalize? thats Ilya's entire criticism of the current paradigm

Best time to sell his ai portfolio

Sure but that will also be a research age.

I think he was referencing something Richard Sutton said (iirc); along the lines of "If we can get to the intelligence of a squirrel, we're most of the way there"

timestamp?

He's just doing research with some grant money? Why would you ask a researcher for a path to profitability?

I just hope the people funding his company are aware that they gave some grant money to some researchers.

Mira was a PM who somehow was at the right place at the right time. She isn’t actually an AI expert. Ilya however, is. I find him to be more credible and deserving in terms of research investment. That said, I agree that revenue is important and he will need a good partner (another company maybe) to turn ideas into revenue at some point. But maybe the big players like Google will just acquire them on no revenue to get access to the best research, which they can then turn into revenue.

Sometimes I wonder who the rational individuals at the other end of these deals are and what makes them so confident. I always assume they have something that general public cannot deduce from public statements

> confident they can thrive in such an intensely competitive field.

I agree these AI startups are extremely unlikely to achieve meaningful returns for their investors. However, based on recent valley history, it's likely high-profile 'hot startup' founders who are this well-known will do very well financially regardless - and that enables them to not lose sleep over whether their startup becomes a unicorn or not.

They are almost certainly already multi-millionaires (not counting ill-liquid startup equity) just from private placements, signing bonuses and banking very high salaries+bonus for several years. They may not emerge from the wreckage with hundreds of millions in personal net worth but the chances are very good they'll probably be well into the tens of millions.

TBH if you truly believe you are in the frontier of AI you probably don’t need to care too much about those numbers.

Yes corporations need those numbers, but those few humans are way more valuable than any numbers out there.

Of course, only when others believe that they are in the frontier too.

I think software patents in AI are a possibility. The transformer was patented after all, with way it was bypassed being the decoder-only models.

Secrecy is also possible, and I'm sure there's a whole lot of that.

He has no answer for it so the only thing he can do is deflect and turn on the $2T reality distortion field.

They have a moat defined by being well known in the AI industry, so they have credibility and it wouldn't be hard for anything they make to gain traction. Some unknown player who replicates it, even if it was just as good as what SSI does, will struggle a lot more with gaining attention.

Another few orders of magnitude? Like 100-1000x more than we're already doing? Got a few extra suns we can tap for energy? And a nanobot army to build various power plants? There's no way to do 1000x of what we're already doing any time soon.

Ilya mentioned in the video that 2012 and 2020 was the “Age of Research”, followed by the “Age of Scaling” from 2020 to 2025. Now, we are about to reenter the “Age of Research”.

Researcher says it's time to fund research. News at 11

There is a transcript of the entire conversation if you scroll down a little

If he (or his employees) are actually exploring genuinely new, promising approaches to AGI, keeping them secret helps avoid a breakneck arms race like the one LLM vendors are currently engaged in.

Situations like that do not increase all participants' level of caution.

Doesn't sound like you listened to the interview. He addresses this and says he may make releases that would be otherwise held back because he believes it's important for developments to be seen by the public.