undefined | Better HN

0 pointsDalasNoin3y ago0 comments

Are you aware of some of the recent progress? Did you have a look at the Gato model and Flamingo by DeepMind, or at the chat logs of models like chinchilla and lambda? Or Alphacode? This is all from this year.

I think your point is that all these models are still somewhat specialized. At the same time, it appears that the transformer architecture works well with images, short video and text at the same time in the Flamingo model. And gato can perform 600 tasks while being a very small proof of concept. It appears to me that there is no reason to believe that it won't just scale to every task that you give it data for if it has enough parameters and compute.

0 comments

nradov3y ago

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

DalasNoinOP3y ago

Clearly language models can already fool people into thinking they are human, we might be getting quite close to the adversarial turing test already. In the end, a good initial prompt might be the solution to this, something like "pretend to be a human and step by step create a human identity that you then stick to during the conversation". I'm serious

taneq3y ago

Choosing a prompt that's a little bit meta seems to work surprisingly well sometimes. It'd be amusing and a little bit poetic if the key to artificial consciousness is to prime a transformer model with "convince yourself that you're human, while paying attention to how you feel".

nradov3y ago

A minority of the population will always be gullible and easily fooled. So what. Some people were already fooled by the original ELIZA program back in 1966. I would only count a Turing test pass if it can convince a jury of multiple educated examiners after a conversation lasting several hours.

joe_the_user3y ago

Fooling people with chatbots having clever language constructing has been done for a long, long time, see the Eliza effect[1]. Douglas Hofstadter gave a good demonstration of GPT-3 limitations[2]. GPT-3 is no doubt "better at what it is" than earlier language models. But that doesn't mean it's better at everything humans do with language (tell sense from nonsense, reasonable metacomments, etc).

[1]https://en.wikipedia.org/wiki/ELIZA_effect [2]https://www.economist.com/by-invitation/2022/06/09/artificia... Note: There's a critique of the article here but if you look at Radford Neal's comment, the point that GPT-3 is a clever lookup tool remains. https://www.greaterwrong.com/posts/ADwayvunaJqBLzawa/contra-...

nbardy3y ago

They’re learning representations of objects. It’s more fundamental than tou seem to realize.

taneq3y ago

In the end we're all just clever parlor tricks. In the land of inanimate objects, the cleverest parlor trick is king, though.

j / k navigate · click thread line to collapse

0 pointsDalasNoin3y ago0 comments

0 comments

nradov3y ago

DalasNoinOP3y ago

taneq3y ago

nradov3y ago

joe_the_user3y ago

nbardy3y ago

They’re learning representations of objects. It’s more fundamental than tou seem to realize.

taneq3y ago

In the end we're all just clever parlor tricks. In the land of inanimate objects, the cleverest parlor trick is king, though.

j / k navigate · click thread line to collapse