undefined | Better HN

0 pointsskybrian3y ago0 comments

Fundamentally unsuited because of how they train it using "fill in the blank."

Training a large model to guess when it doesn't know the answer results in fiction. They need to do something else to get nonfiction.

By contrast, for Go the model was trained not to make illegal moves, because checking for that as part of the training is easy and cheap.

0 comments

halpmeh3y ago

We have models that accurate classify things, e.g. whether or not an email is spam. There isn’t a fundamental limitation into building something like a truth classifier into a generative model so that it optimized for outputting “true” statements. The hardest part is probably identifying what is truth and what is falsehood. That’s a fundamental problem with humanity, not neural networks.

skybrianOP3y ago

Well, we could quibble about what "fundamental" means but my point is that the way they train large language models doesn't work for this. Something different needs to happen.

imtringued3y ago

Truth has nothing to do with humanity unless you mean the specific way humans construct belief systems.

Anyway I already told you the answer. The AI will need a series of trainable belief systems to verify whether statements are internally consistent. The strange part about this is that the AI would need to have a way to obtain validation and each prompt would have to derive a new belief system which you must use in the next prompt.

In other words, the model must be able to learn continuously. That is something that these single shot AI models are not capable of.

nextaccountic3y ago

> There isn’t a fundamental limitation into building something like a truth classifier into a generative model so that it optimized for outputting “true” statements.

Problem is, they didn't do that

j / k navigate · click thread line to collapse