undefined | Better HN

0 pointslxgr9mo ago0 comments

> it's hard to imagine the world can be fully described with 200 terabytes of data

Why? Intelligence and compression might just be two sides of the same coin, and given that, I'd actually be very surprised if a future ASI couldn't make due with a fraction of that.

Just because current LLMs need tons of data doesn't mean that that's somehow an inherent requirement. Biological lifeforms seem to be able to train/develop general intelligence from much, much less.

0 comments

3 comments · 1 top-level

williamtrask9mo ago· 2 in thread

Well, we're opining about a statement about the world. Is the universe only 200 terabytes of information?

"Biological lifeforms seem to be able to train/develop general intelligence from much, much less."

This statement is hard to defend. The brain takes in 125 MB / second, and lives for 80 years, taking in about 300+ petabytes over our lifetime.

But that's not the real kicker. It's pretty unfair to say that humans learn everything they know from birth -> death. A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.

lxgrOP9mo ago

> A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.

That also seems several orders of magnitude off. Would you suspect that a human that only experiences life through H.264-compressing glasses, MP3-recompressing headphones etc. does not develop a coherent world model?

What about a human only experiencing a high fidelity 3D rendering of the world based on an accurate physics simulation?

The claim that humans need petabytes of data to develop their mind seems completely indefensible to me.

> A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.

Isn't that like saying that you only need the right data? In which case I'd completely agree :)

williamtrask9mo ago

"The claim that humans need petabytes of data to develop their mind seems completely indefensible to me."

And yet every human you know is using petabytes of data to develop their mind. :)

j / k navigate · click thread line to collapse

0 comments

3 comments · 1 top-level

williamtrask9mo ago· 2 in thread

Well, we're opining about a statement about the world. Is the universe only 200 terabytes of information?

"Biological lifeforms seem to be able to train/develop general intelligence from much, much less."

This statement is hard to defend. The brain takes in 125 MB / second, and lives for 80 years, taking in about 300+ petabytes over our lifetime.

lxgrOP9mo ago

> A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.

What about a human only experiencing a high fidelity 3D rendering of the world based on an accurate physics simulation?

The claim that humans need petabytes of data to develop their mind seems completely indefensible to me.

> A lot of that learning bias was worked out through evolution... which takes that 300+ petabytes and multiplies it by... many lifetimes.

Isn't that like saying that you only need the right data? In which case I'd completely agree :)

williamtrask9mo ago

"The claim that humans need petabytes of data to develop their mind seems completely indefensible to me."

And yet every human you know is using petabytes of data to develop their mind. :)

j / k navigate · click thread line to collapse