undefined | Better HN

0 pointsplutonorm4y ago0 comments

It's recently been shown that even though the numbers are represented with different tokens, the network learns to form an internal representation that understands the progression from one token to the next.

0 comments

nikolayasdf1234y ago

The idea that each number has to be inside ones Brain or Neural Network or Token is plainly wrong.

Network has to grasp the "abstract" number, but it clearly did not grasp that concept.

Isinlor4y ago

How would you test if it grasped the concept?

plutonormOP4y ago

https://arxiv.org/pdf/2201.02177.pdf

This paper shows fairly conclusively that the network 'groks' modular addition.

1 more reply

catach4y ago

Being able to extrapolate to numbers that were not in the training set, perhaps? At least that'd be a basic part of the requirement.

1 more reply

j / k navigate · click thread line to collapse

0 pointsplutonorm4y ago0 comments

0 comments

nikolayasdf1234y ago

The idea that each number has to be inside ones Brain or Neural Network or Token is plainly wrong.

Network has to grasp the "abstract" number, but it clearly did not grasp that concept.

Isinlor4y ago

How would you test if it grasped the concept?

plutonormOP4y ago

https://arxiv.org/pdf/2201.02177.pdf

This paper shows fairly conclusively that the network 'groks' modular addition.

1 more reply

catach4y ago

Being able to extrapolate to numbers that were not in the training set, perhaps? At least that'd be a basic part of the requirement.

1 more reply

j / k navigate · click thread line to collapse