Ask HN: Will transformer based LLMs hit an improvement ceiling?

1 pointsjaguar758mo ago2 comments

A fundamental question I have is whether the current transformer based architectures are powerful enough to allow scaling towards AGI or are there ceilings here that are becoming more evident?

Though folks from the leading LLM companies are in the forefront of this, it’s hard to take their word due to an obvious conflict of interest. And it’s unclear if academia has more theoretical knowledge here as it seems these commercial companies are also leading the research work here.

2 comments

eimrine8mo ago

What do you expect from AGI? What aspects of LLM have impressed you the most?

jaguar75OP8mo ago

AGI is a vague term but a naive definition is smarter than any human in any fields/domain.

Clearly current LLMs are impressive in document/code generation but I’m having a hard time extrapolating that to AGI (assuming that it’s still reliant on current transformers arch).

j / k navigate · click thread line to collapse