undefined | Better HN

0 pointsnorir1y ago0 comments

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human: design a Turing complete programming language that is both human and machine readable and implement a self hosted compiler in this language that self compiles on existing hardware faster than any known language implementation that also self compiles. Additionally, for any syntactically or semantically invalid program, the compiler must provide an error message that points exactly to the source location of the first error that occurs in the program.

I will get excited for/scared of LLMs when they can tackle this kind of problem. But I don't believe they can because of the fundamental nature of their design, which is both backward looking (thus not better than the human state of the art) and lacks human intuition and self awareness. Or perhaps rather I believe that the prompt that would be required to get an LLM to produce such a program is a problem of at least equivalent complexity to implementing the program without an LLM.

0 comments

Xenoamorphous1y ago

> Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human

That’s possible for a highly intelligent, extensively trained, very small subset of humans.

hatefulmoron1y ago

If you took the intersection of every human's abilities you'd be left with a very unimpressive set.

That also ignores the fact that the small set of humans capable of building programming languages and compilers is a consequence of specialization and lack of interest. There are plenty of humans that are capable of learning how to do it. LLMs, on the other hand, are both specialized for the task and aren't lazy or uninterested.

luckydata1y ago

does it mean people that can build languages and compilers are not humans? What is the point you're trying to make?

fragmede1y ago

It means that's a really high bar for intelligence, human or otherwise. If AGI is "as good as a human, and the test is a trick task that most humans would fail at (especially considering the weasel requirement that it additionally has to be faster), why is that considered a reasonable bar for human-grade intelligence.

bob10291y ago

This sounds like something more up the alley of linear genetic programming. There are some very interesting experiments out there that utilize UTMs (BrainFuck, Forth, et. al.) [0,1,2].

I've personally had some mild success getting these UTM variants to output their own children in a meta programming arrangement. The base program only has access to the valid instruction set of ~12 instructions per byte, while the task program has access to the full range of instructions and data per byte (256). By only training the base program, we reduce the search space by a very substantial factor. I think this would be similar to the idea of a self-hosted compiler, etc. I don't think there would be too much of a stretch to give it access to x86 instructions and a full VM once a certain amount of bootstrapping has been achieved.

[0]: https://arxiv.org/abs/2406.19108

[1]: https://github.com/kurtjd/brainfuck-evolved

[2]: https://news.ycombinator.com/item?id=36120286

jedberg1y ago

I will get excited when an LLM (or whatever technology is next) can solve tasks that 80%+ of adult humans can solve. Heck let's even say 80% of college graduates to make it harder.

Things like drive a car, fold laundry, run an errand, do some basic math.

You'll notice that two of those require some form of robot or mobility. I think that is key -- you can't have AGI without the ability to interact with the world in a way similar to most humans.

ata_aman1y ago

So embodied cognition right?

Vampiero1y ago

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for an average human: designing a functional trivia app.

There, you don't need to invoke Turing or compiler bootstrapping. You just need one example of a use case where the accuracy of responses is mission critical

alainx2771y ago

o1-preview managed to complete this in one attempt:

https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2...

I think the parent comment's challenge is more appropriate.

Vampiero1y ago

Have you personally verified that the answers are not hallucinations and that they are indeed factually true?

Oh, you just asked it to make a trivia app that feeds on JSON. Cute, but that's not what I meant. The web is full of tutorials for basic stuff like that.

To be clear I meant that LLMs can't write trivia questions and answers, thus proving that they can't produce trustworthy outputs.

And a trivia app is a toy (one might even say... a trivial example), but it's a useful demonstration of why you wouldn't put an LLM into a system on which lives depend on, let alone invest billions on it.

If you don't trust my words just go back to fiddling with your models and ask them to write a trivia quiz about a topic that you know very well. Like a TV show.

j / k navigate · click thread line to collapse

0 comments

Xenoamorphous1y ago

> Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human

That’s possible for a highly intelligent, extensively trained, very small subset of humans.

hatefulmoron1y ago

If you took the intersection of every human's abilities you'd be left with a very unimpressive set.

luckydata1y ago

does it mean people that can build languages and compilers are not humans? What is the point you're trying to make?

fragmede1y ago

bob10291y ago

This sounds like something more up the alley of linear genetic programming. There are some very interesting experiments out there that utilize UTMs (BrainFuck, Forth, et. al.) [0,1,2].

[0]: https://arxiv.org/abs/2406.19108

[1]: https://github.com/kurtjd/brainfuck-evolved

[2]: https://news.ycombinator.com/item?id=36120286

jedberg1y ago

I will get excited when an LLM (or whatever technology is next) can solve tasks that 80%+ of adult humans can solve. Heck let's even say 80% of college graduates to make it harder.

Things like drive a car, fold laundry, run an errand, do some basic math.

You'll notice that two of those require some form of robot or mobility. I think that is key -- you can't have AGI without the ability to interact with the world in a way similar to most humans.

ata_aman1y ago

So embodied cognition right?

Vampiero1y ago

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for an average human: designing a functional trivia app.

There, you don't need to invoke Turing or compiler bootstrapping. You just need one example of a use case where the accuracy of responses is mission critical

alainx2771y ago

o1-preview managed to complete this in one attempt:

https://chatgpt.com/share/67373737-04a8-800d-bc57-de74a415e2...

I think the parent comment's challenge is more appropriate.

Vampiero1y ago

Have you personally verified that the answers are not hallucinations and that they are indeed factually true?

Oh, you just asked it to make a trivia app that feeds on JSON. Cute, but that's not what I meant. The web is full of tutorials for basic stuff like that.

To be clear I meant that LLMs can't write trivia questions and answers, thus proving that they can't produce trustworthy outputs.

If you don't trust my words just go back to fiddling with your models and ask them to write a trivia quiz about a topic that you know very well. Like a TV show.

j / k navigate · click thread line to collapse