undefined | Better HN

0 pointsbbor1mo ago0 comments

To keep my usual rant short: I think you’re assuming a categorical distinction between those two types of innovations that just doesn’t exist. Calculus certainly required some fundamental paradigm shifts, but there’s a reason that they didn’t have to make up many words wholesale to explain it!

Also we shouldn’t be thinking about what LLMs are good at, but rather what any computer ever might be good at. LLMs are already only one (essential!) part of the system that produced this result, and we’ve only had them for 3 years.

Also also this is a tiny nitpick but: the fields medal is every 4 years, AFAIR. For that exact reason, probably!

0 comments

13 comments · 3 top-level

symfrog1mo ago· 9 in thread

We have had LLMs for much longer than 3 years.

Nevermark1mo ago

I took humans thousands of years, then hundreds of years, to come to terms with very basic concepts about numbers.

Its amazing to me when people talk about recombining things, or following up on things as somehow lesser work.

People can't separate the perspective they were given when they learned the concepts, that those who developed the concepts didn't have because they didn't exist.

Simple things are hard, or everything simple would have been done hundreds of years ago, and that is certainly not the case. Seeing something others have not noticed is very hard, when we don't have the concepts that the "invisible" things right in front of us will teach us.

adi_kurian1mo ago

Anyone in the arts is aware that creativity is not the new, it is the repackaging of what already exists into something that is itself new.

1 more reply

godelski1mo ago

It's why the invention of teaching has been so important. Took a long time for humans to develop calculus. A long time to then refine it and make it much more useful. But then in a year or two an average person can learn what took hundreds of years to invent. It's crazy to equate these tasks as being the same. Even incremental innovation is difficult. You have to see something billions of people haven't. But there's also paradigm shifts and well... if you're not considered crazy at first then did you really shift a paradigm?

1 more reply

danielmarkbruce1mo ago

No, we haven't, for any reasonable definition of L.

wavemode1mo ago

OpenAI themselves must not have a "reasonable definition of L", then. Their own papers and press releases refer to GPT-2 (from 2019) as a "large language model".

https://openai.com/index/better-language-models/

1 more reply

Yizahi1mo ago

Sure we do, since Fei-Fei Li and team created that annotated dataset, which allowed to train first LLMs. So LLMs are here for more than a decade already.

1 more reply

asdfasgasdgasdg1mo ago

When people say this what they mean is that we've had plausibly useful LLMs for around three years, and I would say that is basically true. The stuff before 2023 could barely be classified above the level of an interesting toy.

asdfasgasdgasdg1mo ago

When people say this what they mean is that we've had plausibly useful LLMs for around three years, and I would say that is basically true.

nextaccountic1mo ago

Fine, 8 years? That's not a long time

pegasus1mo ago· 1 in thread

The fundamental paradigm shift is the categorical distinction. And what would constitute many new words for you? It introduced a bunch of concepts and terms which we take for granted today, including "derivative", "integral", "infinitesimal", "limit" and even "function", the latter two not a new words, but what does it matter? – the associated meanings were new.

azakai1mo ago

There was a lot new in calculus, but it also didn't come out of nowhere.

That Newton and Leibniz came up with similar ideas in parallel, independently, around the same time (what are the odds?), supports that.

https://en.wikipedia.org/wiki/Leibniz%E2%80%93Newton_calculu...

m4x1mo ago

I think your comment about inventing new words is an interesting one. One of the things that I believe limits our ability to discover new ideas is our ability to describe related concepts. For example, the reason we still can't have clear discussions on consciousness is probably partly due to the fact that the necessary concepts haven't been cemented in language. We need new language before we can describe consciousness.

I would guess LLMs are limited in their ability to be genuinely novel because they are trained on a fixed language. It makes research into the internal languages developed by LLMs during training all the more interesting.

j / k navigate · click thread line to collapse

0 comments

13 comments · 3 top-level

symfrog1mo ago· 9 in thread

We have had LLMs for much longer than 3 years.

Nevermark1mo ago

I took humans thousands of years, then hundreds of years, to come to terms with very basic concepts about numbers.

Its amazing to me when people talk about recombining things, or following up on things as somehow lesser work.

People can't separate the perspective they were given when they learned the concepts, that those who developed the concepts didn't have because they didn't exist.

adi_kurian1mo ago

Anyone in the arts is aware that creativity is not the new, it is the repackaging of what already exists into something that is itself new.

1 more reply

godelski1mo ago

1 more reply

danielmarkbruce1mo ago

No, we haven't, for any reasonable definition of L.

wavemode1mo ago

OpenAI themselves must not have a "reasonable definition of L", then. Their own papers and press releases refer to GPT-2 (from 2019) as a "large language model".

https://openai.com/index/better-language-models/

1 more reply

Yizahi1mo ago

Sure we do, since Fei-Fei Li and team created that annotated dataset, which allowed to train first LLMs. So LLMs are here for more than a decade already.

1 more reply

asdfasgasdgasdg1mo ago

When people say this what they mean is that we've had plausibly useful LLMs for around three years, and I would say that is basically true.

nextaccountic1mo ago

Fine, 8 years? That's not a long time

pegasus1mo ago· 1 in thread

azakai1mo ago

There was a lot new in calculus, but it also didn't come out of nowhere.

That Newton and Leibniz came up with similar ideas in parallel, independently, around the same time (what are the odds?), supports that.

https://en.wikipedia.org/wiki/Leibniz%E2%80%93Newton_calculu...

m4x1mo ago

j / k navigate · click thread line to collapse