undefined | Better HN

0 pointsgregschlom1y ago0 comments

> A human brain doesn't need anywhere close to this volume of data, in order to be able to produce good output.

Maybe not directly, but consider that our brains are the product of million of years of evolution and aren't a blank slate when we're born. Even though babies can't speak a language at birth, they already have all the neural connections in place in order to acquire and manipulate language, and require just a few years of "supervised fine tuning" to learn the actual language.

LLMs, on the other hand, start with their weights at random values and need to catch up with those million years of evolution first.

0 comments

skeledrew1y ago

Add to this, the brain is constantly processing raw sensory data from the moment it became viable, even when the body is "sleeping". It's using orders of magnitude more data than any model in existence every moment, but isn't generally deemed "intelligent" enough until it's around 18 years old.

emp173441y ago

It’s unlikely that sensory data contributes to cognitive ability in humans. People with sensory impairments, such as blind people, are not less cognitively capable than people without sensory impairments. Think of Helen Keller, who, despite taking in far less sensory information than the average person, was still more intelligent than average.

skeledrew1y ago

Without sensory data there cannot be actual cognitive ability, though there may be potential for it. The data doesn't have to be visual; bear in mind we have 5 senses. When vision is impaired, hearing becomes far more sensitive to compensate. And theoretically, if someone were to only have use of a single sense, they may still be able to use the data from it to actualize their cognition, but it would take a lot more effort and there would be large gaps in capability. Just as, technically, preprocessed vision* is the primary "sense" of LLMs.

* Preprocessed since the data is actually of 1D streams of characters, and not 2D colour points (as with vision models).

nitwit0051y ago

There can't be that much pre-built into the brain. There isn't that much dna, and only a portion of it can be going to the design of the brain.

A lot of what we're able to do has to be from some sort of generic capability.

johnnyanmac1y ago

sadly, those weights will not be inherited like they would to a baby. They'll be cooped up until the company dies, and that data probably dies with them. No wonder LLM has allegedly hit some stalls already.

j / k navigate · click thread line to collapse

0 pointsgregschlom1y ago0 comments

> A human brain doesn't need anywhere close to this volume of data, in order to be able to produce good output.

LLMs, on the other hand, start with their weights at random values and need to catch up with those million years of evolution first.

0 comments

skeledrew1y ago

emp173441y ago

skeledrew1y ago

* Preprocessed since the data is actually of 1D streams of characters, and not 2D colour points (as with vision models).

nitwit0051y ago

There can't be that much pre-built into the brain. There isn't that much dna, and only a portion of it can be going to the design of the brain.

A lot of what we're able to do has to be from some sort of generic capability.

johnnyanmac1y ago

j / k navigate · click thread line to collapse