undefined | Better HN

0 pointsdmvaldman5y ago0 comments

i think you are assuming that what is happening under the hood is that a human-inputted sentence is being parsed into a grammar. it is not.

0 comments

Barrin925y ago

I know that it isn't. That's part of the problem. There is no attempt to generate some sort of structure that can be interpreted semantically and reasoned about by the model. The model just operates on the input superficially and statistically. That's why there has been virtually no progress on trivial tasks such as answering:

"I took the water bottle out of the backpack so that it would be [lighter/handy]"

What is lighter and what is handy? No amount of stochastic language manipulation gets you the answer, you need to understand some rudimentary physics to answer the question, and as a precondition, you need a grammar or ontology.

FeepingCreature5y ago

Have you tried feeding this to GPT and seeing if it continues it in a way that reveals understanding?

It sounds like you're saying "It doesn't work because it can't work", but you haven't actually shown that it doesn't work.

Barrin925y ago

yes, I have. You can paste these into the website of the Allen Institute for AI, yourself here. (https://demo.allennlp.org/reading-comprehension/MjE1MzE1Mg==)

In the example above it guesses wrongly, but again this is not surprising because it can't possibly get the right answer (other than by chance). The solution here cannot be found by correlating syntax, you can only answer the question if you understand the meaning of the sentence. That's what these schemas are constructed for.

1 more reply

jklm5y ago

FWIW, I fed this into AIDungeon (running on OpenAI) and got this back: “The bottle is definitely lighter than the pack because you can throw it much further away than you can carry the pack. You continue on into the night and come to an intersection.”

j / k navigate · click thread line to collapse

0 comments

Barrin925y ago

"I took the water bottle out of the backpack so that it would be [lighter/handy]"

FeepingCreature5y ago

Have you tried feeding this to GPT and seeing if it continues it in a way that reveals understanding?

It sounds like you're saying "It doesn't work because it can't work", but you haven't actually shown that it doesn't work.

Barrin925y ago

yes, I have. You can paste these into the website of the Allen Institute for AI, yourself here. (https://demo.allennlp.org/reading-comprehension/MjE1MzE1Mg==)

1 more reply

jklm5y ago

j / k navigate · click thread line to collapse