AI or Ain't: Eliza (opens in new tab)

(zserge.com)

123 pointsjohn-doe2y ago110 comments

110 comments

Here is the pointer to the original Eliza paper https://dl.acm.org/doi/10.1145/365153.365168

Note that Weizenbaum was an AI critic: Weizenbaum's intention was not for Eliza to pass the Turing test, but to show to people that a clearly not intelligent program based on primitive pattern matching can appear to behave intelligently.

He failed: His own secretary wanted to be left alone with the software and typed in her personal problems. Work on Eliza (1963-65, paper published 1966) until today is mostly misunderstood.

adestefan2y ago

Weizenbaum even wrote an entire book, Computer Power and Human Reason: From Judgment To Calculation, on how AI is a crank term for computer science. The basic premise is that humans are not machine so stop using language that treats them as such. Especially that computers can decide what comes next, but only a human can choose what to do.

The book also has one of the best and most succinct descriptions of Turing machines and the theoretical underpinning of computer science that I have ever read. Even if you’re an AI maximalist you should read the third chapter of the book.

erikerikson2y ago

> humans are not machine

Aren't we? Casual chains upon our matter produce emergent behaviors using the same physics and chemistry that our mechanistic creations rely upon.

Certainly those behaviors and results do not produce the same repeatable, predictable results as our clockworks but that is the whole point of the field of AI (as opposed to the marketing corruption/term that is currently in vogue, so GAI if you prefer), to produce system and algorithm structures designed with architecture and patterns more like our own.

Perhaps you believe in the ghost in the machine hypothesis? The magical soul that is more than the emergent evolving pattern produced across time by DNA replicators? That this undefinable, unmeasurable spirit makes us forever different?

2 more replies

stavros2y ago

> Especially that computers can decide what comes next, but only a human can choose what to do.

I don't understand this, all the programs I've ever written make decisions based on some factors.

Are you talking about free will? If so, what is free will?

3 more replies

extractionmech2y ago

“Me no likee” - Joseph Agassi

This post of yours sent me looking for this and I ended up with (now reading) Agassi’s provocative (& brutal) takedown of this book. As to the content of the book, Agassi pointedly mentioned Weiner. I will of course read Weizenbaum after this. (thanks.)

https://www.researchgate.net/publication/286058724_Computer_...

p.s. continuing on, the review turns very positive. Initial vitriol apparently more related to the “two worlds” matter.

1vuio0pswjnm72y ago

Previous: https://news.ycombinator.com/item?id=36901158 [PDF]

tempodox2y ago

All this “AI” hype is a constant reminder to me that you cannot reason anybody out of something that they want to believe. People's need to believe in miracles is obviously stronger than all reason.

ryanklee2y ago

> cannot reason anybody out of something that they want to believe.

Especially when your own reasons are not the slam dunk you think they are.

1 more reply

scotty792y ago

> He failed

I'd say he succeeded. It just seems that people are perfectly content with just appearance of intelligence.

leethargo2y ago

Not only his secretary, also some psychiatrists wanted Eliza as a tool to scale up their work clinically.

moffkalast2y ago

Say, has anyone deployed Hartford's Samantha yet?

extreme_orang2y ago

Take a look at this project that has rediscovered and recreated the original ELIZA source code. You can even try out an accurate reimplementation of ELIZA

http://findingeliza.org/

lulznews2y ago

Never underestimate human stupidity!

drakonka2y ago

Many years ago I spent a lot of time on a website called Personality Forge, where you would create chat bots very similar to this and set them loose. They could then talk to other people or each other and you could review the transcripts. At one point I even entered my chat bot in The Chatterbox Challenge. It was so much fun to work on this thing at the time, but when I rediscovered it years later[0] I was mostly just disappointed in how "fake" all that effort was.

Now here I am talking about life, the universe, and everything with ChatGPT. It makes me both inexplicably happy/hopeful and simultaneously weirdly melancholic.

[0] https://liza.io/its-not-so-bad.-we-have-snail-babies-after-a...

bee_rider2y ago

The great thing about being a teenager or kid is that you don’t know why the grownups don’t think your project is worth doing, so you just do it. Even if it doesn’t change the world (most things don’t, after all) you can still learn something and have fun.

aldousd6662y ago

Eliza's meant to us to be an illustration of the problem. In good old fashioned AI sentiment, it illustrates the fact that you need another if statement for every new kind of construct you want to simulate. But you deign to simulate each thing, like say turning a verb into a gerund, by writing a specific "gerundification" routine. Another to swap the Mes to Yous, etc. this isn't how people think nor most modern AI. This is totally different from just looking at the world distilling patterns from it and using those patterns as the basis for a response. To teach a modern AI new stuff, you don't have to write another if statement. They have similar intentions and at some resolution or distance, they are trying to do similar things. However, they work and totally different ways and the new dynamic generative AI strategy that learns from input as opposed to just symbolically transforming it syntactically is a totally different paradigm. I don't care whether you call it AI or not.

masswerk2y ago

With the small reservation that this is not how Eliza works. Eliza sits on top of MAD/SLIP which does all the heavy work and provides lists and integer indexes, which is what is processed by Eliza. This allows Eliza to work on decomposition rules, which isolate keywords per position and context, and transformation (composition) rules to recombine elements and links between those two. Meaning, the model is much more topological than this. (Arguably, this is closer to regular expressions than to if-else trees.)

However, this isn't what Eliza is all about. It's rather about the question, how little do you actually need in terms of knowledge, world model, or rule sets to give the impression of an "intelligent" (even sympathetic) participant in a conversation (as long as you're able to constrain the conversation to a setting, which doesn't require any world knowledge, at all.) To a certain degree, it is also about how eager we are to overestimate the capabilities of such a partner in conversation, as soon as some criteria seem to be satisfied. Which is arguably still relevant today.

The rule set, BTW, is actually small, just 3 pages in a printout, achieving a surprising generality (or rather, appearance thereof) for its size. Compare: https://cse.buffalo.edu/~rapaport/572/S02/weizenbaum.eliza.1...

pixl972y ago

>how eager we are to overestimate the capabilities of such a partner in conversation, as soon as some criteria seem to be satisfied. Which is arguably still relevant today.

Honestly, AI shouldn't be the takeaway point here, but how we do the same for politics.

1 more reply

throwaway4aday2y ago

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations.

I see we have a new entry for the 2024 Lies of Omission award.

The article linked to plainly shows that Eliza only beats ChatpGPT-3.5 and is in the bottom half when ranked against a variety of different system prompts. An excellent ass covering strategy that relies on the reader not checking sources.

An honest author would have actually quoted the article saying:

> GPT-4 achieved a success rate of 41 percent, second only to actual humans.

instead of constructing a deliberately misleading paraphrase.

notahacker2y ago

"GPT-4 achieved a success rate of 41 percent, second only to actual humans" also feels like a (much bigger) lie of omission looking at the original paper. GPT4's performance was in the range of 6% to 41%, Eliza's 27% score sat in the upper middle of that range, and considering the bots tested consisted of 8 GPT4 prompts, 2 GPT3.5 prompts and a naive script from the 1960s, GPT4 would have had to be remarkably consistently inhuman not to finish "second only to humans" with its highest scoring prompt

The blog appears to have been updated to specify GPT3.5, but the original version was accurate.

The paper itself is interesting as it covers the limitations (it has big methodological issues), how the GPT prompts attempted to overridei default chatGPT tone and reasons why ELIZA performed surprisingly well (some thought it was so uncooperative, it must be human!) https://arxiv.org/pdf/2310.20216.pdf

throwuxiytayq2y ago

The example ELIZA responses in the paper are so laughably bad and trivial to pick up, I'm not convinced the human interrogators were sober/conscious/awake during the experiment.

2 more replies

pixl972y ago

Yea, It's really hard to get GPT to sound human because the RLHF really wants to let you know it's not a human.

GPT4 + a RLHF that was trained to think it was human would be a much different beast.

1 more reply

throwaway4aday2y ago

Unfortunately, the paper only provides the text of one of the prompts they used (Juliet) and it happens to be one of the worst performing ones that scored lower than ELIZA. I suppose you could qualify the quote by saying that the best prompt used with GPT-4 had a 41 percent success rate. I don't think that's more of an omission than excluding the GPT model, excluding the prompt used, and ignoring the fact that other GPT models beat ELIZA.

masswerk2y ago

Hum, note that this was not an argument about or against GPT, but about the "unreasonable" success of a, by all standards, primitive algorithm that manages to get (somewhat) away by crafting the pre- and context of the conversation. By no means, on the other hand, could I read this article and understand it as claiming any superiority over modern applications.

(Nobody with even the crudest understanding of the principles of Eliza could claim this, and the article clearly demonstrates a detailed understanding. Disclaimer: I wrote the JS implementation linked in the article, many years ago.)

Edit: The question rightfully raised – and answered – by Eliza, which is still relevant today in the context of GPT, is: does the appearance of intelligent conversation (necessarily) hint at the presence of a world model in any rudimentary form?

throwaway4aday2y ago

Several people in this thread appear to have misunderstood due to the way this article was written.

seanwilson2y ago

> One of the first computer programs that successfully passed the Turing test was Eliza. Created in 1966 by Joseph Weizenbaum, Eliza skillfully emulated the speech patterns of a psychotherapist in its conversations.

Why was the Turing test still relevant after this? Didn't this indicate it was very flawed test? Or it was hard to come up with a better test?

recursivecaveat2y ago

I can find no reference of an actual Turing test being done for Eliza. If you look at the link from the article it is clearly demonstrably failing their (different, and more difficult to interpret, but still fair I thiiiink) runs today as well. Note that people constantly willfully misinterpret what a turing test is.

A turing test means you enter into two conversations. Then you pick which one was with a computer. If people answer wrong 50% of the time, the computer is indistinguishable, hence it passes. Note that it is not "People get wrong whether their single conversation is talking to an AI >50% of the time" and it is definitely not "sometimes people don't realize they're talking to an AI". In particular people constantly write about the latter because it generates clicks.

GuB-422y ago

The variant of the Turing test that Eliza has passed is the easiest one. That is: convince a novice interrogator that what is on the other side of the screen is a human.

The real turing test (the imitation game) involve a computer and a human subject, both talking to a human interrogator. The interrogator must determine who is the human. It is an adverserial situation, both the human subject and interrogator do everything in their power for the interrogator to correctly identify the computer. The computer has to not only convince the interrogator, but also do it better than the human subject. Furthermore, both humans are supposed to be experts in the game, just like the computer that is designed to pass the test. So, not just random people.

canjobear2y ago

Because Eliza didn’t pass the Turing Test. It is trivially easy to trip it up.

nbzso2y ago

So the Turing test actually tests not a technology but the level of intelligence of the user. So, we are doomed, it seems.:)

__MatrixMan__2y ago

He originally made the argument about gender, not intelligence. I think he was arguing for a whole class of properties for which there's no difference between authenticity and convincing fakery.

I think the point is less that there is a truth and we're too dumb to figure it out, and more that in certain circumstances we'll just have to accept a lower bar for evidence about whether those properties apply.

It reminds me of how no class of computer can solve the halting problem for itself. No matter how intelligent you are, there will be holes like this in your certainty about some things.

pixl972y ago

Or another way to put this, it's not a binary problem, it's a probability continuum.

Even the definition of 'human intelligence' is a continuum from the smartest to the dumbest of us, that doesn't even stop there and descends thought all animal life.

1 more reply

masswerk2y ago

To me, Turing's argument had always been that the attribution of intelligence (or not) doesn't make much sense: rather than being a question of any substance, it merely diffuses into a matter of appearance. However, as things usually are, it had become the holy grail for claiming "intelligence" (which really should be used in this context in quotes only).

YurgenJurgensen2y ago

In actuality, it primarily tests the knowledge of the user, not their intelligence. I'm sure that GPT-written paragraphs seemed really impressive when you first encountered them, but nowadays half the Internet has seen enough of them to recognise the default ChatGPT style in less than the space of a Tweet. People aren't significantly smarter than they were a few years ago, but I bet GPT-3.0 will perform significantly worse on a Turing test now than it did the day it was released.

Similarly, I believe a lot of early Turing Test successes kind-of cheated and had their bots pretend to be ESL, on the grounds that the interrogators would interpret their unnatural responses not as a robot but as a second-language speaker's human mistakes. But people who teach English as a second language, or interact with language-learners a lot will learn the types of mistakes each group of learners make, and will spot unnatural mistakes a lot faster.

Now that I think about it, that a major factor in determining Turing test performance isn't the intelligence of the testers but their knowledge does highlight why it's not a great measure of intelligence in the first place.

derbOac2y ago

I've often felt that a better version is not whether a person can guess that it's AI or a human, but whether people behave and feel differently with an AI or human.

That's vague and covers a universe of criteria — mood, satisfaction with the conversation, actual behavior and so forth — but it also I think is a more realistic gauge of AI performance. It's probably unattainable but that's not necessarily a bad thing. If it is attainable within confidence then it's a pretty powerful AI.

There are probably some people who would be ok with some AI for some purposes.

masswerk2y ago

In a sense, the question of the "intelligent machine" is somewhat self-contradictory: To us, the question of intelligence matters as a preposition or qualifying term, for to what extent, probability and prospects we may pose an appeal to sympathy, moral and ethics. (In other words, it is not about trust in any realistic faculties, but about judgement – and then, to what extent we may trust in this.) However, this prospect doesn't fit well our expectations towards machines, which are all about repeatability and reproducible results in given tolerances… (Compare ChatGPT's so-called winter depression and the arising need to plead and argue with the device for any complex results. As the device gains in the emotional domain, its worth in the application domain radically decreases.)

oneeyedpigeon2y ago

Definitely the main flaw of the Turing test — having two human actors taking part alongside one computer is just introducing two too many variables :)

sixothree2y ago

I remember this being written in basic for the c64. Not sure if I had the real Eliza or a clone. But it was fun to look at all of the canned responses and try to get it to respond with them.

smcameron2y ago

Probably a variant of the Eliza program in "More BASIC Computer Games", see p. 56 of this PDF: https://ia802707.us.archive.org/33/items/More_BASIC_Computer...

sixothree2y ago

That's likely the one.

master-lincoln2y ago

> Clearly, Eliza is not an AI

Sadly the author doesn't elaborate on this. I thought nowadays 'AI' is a synonym for 'algorithm', which would fit ELIZA

Is there an accepted definition of the word AI?

oneeyedpigeon2y ago

We've been misusing the term ever since observing the ghosts in Pac Man.

perthmad2y ago

Any algorithm you do not understand.

mysterydip2y ago

So, any of my own code I haven't looked at for a few months? :)

throwaway4aday2y ago

Including the one in your head.

maweaver2y ago

Seems like the modern definition is something along the lines of an algorithm whose behavior depends on data which was itself machine generated, rather than hand-created by a human like Eliza's rules.

nsajko2y ago

The article is missing the interesting bits, so here are the relevant Wikipedia pages:

https://en.wikipedia.org/wiki/ELIZA

https://en.wikipedia.org/wiki/ELIZA_effect

https://en.wikipedia.org/wiki/Joseph_Weizenbaum

dboreham2y ago

It turns out, 50 years later, that Eliza was on the right track.

deepfriedrice2y ago

I'm stoked to be learning about the real Eliza, after being surreptitiously exposed via the Zachtronics game [0]. The game has a fascinating dystopian/scifi take where a company provides AI therapy through human "proxies" that simply vocalize the AI response.

[0] https://www.zachtronics.com/eliza/

whoopdedo2y ago

Then later on there was Azile[1], the evil twin to Eliza. Instead of being trained with data to make it sound helpful and supportive, it uses hostile and insulting language.

[1] https://www.civilizr.com/communication/azile-chatbot-savage-...

scotty792y ago

Can one see original Eliza sourcode anywhere?

throwawayForMe22y ago

I had Mu-Lisp for myfirst computer, an Osborne-1, that came with a copy of Eliza.

You may find the original here:

https://sites.google.com/view/elizagen-org/commonly-known-el...

extreme_orang2y ago

See http://findingeliza.org/

proamdev1232y ago

There is a version of it in Emacs, called up with `M-x doctor`.

The source is in `doctor.el`.

https://www.emacswiki.org/emacs/EmacsDoctor

jhfdbkofdchk2y ago

https://sites.google.com/view/elizagen-org/the-original-eliz...

RugnirViking2y ago

We had it running in terminals when I used to work in the national museum of computing in the UK (on machines where you can just pull the full source up from floppy disk)

DonHopkins2y ago

https://news.ycombinator.com/item?id=27332340

DonHopkins on May 30, 2021 | prev | next [–]

Here's the source code for Kent Pitman's "DOCTOR" in MACLISP, which was of course inspired by ELIZA. (Joseph Weizenbaum taught Kent Pitman LISP!)

https://github.com/PDP-10/its/blob/master/src/games/doc.102

And here's what happened with he (manually by typing) hooked it up with Kenneth Colby's "PARRY" (the paranoid patient):

https://www.maclisp.info/pitmanual/funnies.html

>Parrying Programs

>I didn't write the original ELIZA program, although my Lisp class was taught by Joseph Weizenbaum, who did. I later wrote a very elaborate program of similar kind, which I just called DOCTOR, in order to play with some of the ideas.

>At some point, I noticed there was a program at Stanford called PARRY (the paranoid patient), by Kenneth Colby. I understand from Wikipedia's PARRY entry that Weizenbaum's ELIZA and PARRY were connected at one point, although I never saw that. I never linked PARRY with my DOCTOR directly, but I did once do it indirectly through a manual typist. Part of my record of this exchange was garbled, but this is a partial transcript, picking up in the middle. Mostly it just shows PARRY was a better patient than my DOCTOR program was a doctor.

>I have done light editing to remove the typos we made (rubbed out characters were echoed back in square brackets).

>Also, I couldn't find documentation to confirm this, but my belief has always been that the numeric values after each line are PARRY's level of Shame (SH), Anger (AN), Fear (FR), Disgust (DS), Insecurity (IN), and Joy (J).—KMP

[...]

https://news.ycombinator.com/item?id=38402813

DonHopkins 44 days ago | parent | context | favorite | on: The Revival of Medley/Interlisp

That's right, it's just a throw-away quip, but if you want the deep nuanced story and inside history of Common Lisp and comparison with Scheme, Kent Pitman is the one to read:

https://en.wikipedia.org/wiki/Kent_Pitman

Index of Kent Pitman's Papers:

https://www.nhplace.com/kent/Papers/

Scheme or Lisp? Kent M Pitman explains the deep philosophical differences.

https://www.reddit.com/r/programming/comments/6fa5r/scheme_o...

Kent Pitman on Scheme or Lisp?:

https://groups.google.com/g/comp.lang.lisp/c/TEk4O4-zsA8/m/H...

Common Lisp: The Untold Story:

https://www.nhplace.com/kent/Papers/cl-untold-story.html

Why Wolfram Mathematica did not use Lisp (2002) (ymeme.com):

https://news.ycombinator.com/item?id=9797936

https://web.archive.org/web/20110122140154/http://www.ymeme....

Kent Pitman's essay on why lisp doesn't have copying of lists.

https://groups.google.com/g/comp.lang.lisp/c/MmtQreo3PCM

Parenthetically Speaking with Kent Pitman: The Best Intentions: EQUAL Rights -- And Wrongs -- In Lisp:

https://www.nhplace.com/kent/PS/EQUAL.html

Kent M. Pitman Answers On Lisp And Much More:

https://developers.slashdot.org/story/01/11/03/1726251/kent-...

Kent M. Pitman's Second Wind:

https://developers.slashdot.org/story/01/11/13/0420226/kent-...

Tutorial on Good Lisp Programming Style: Peter Norvig, Sun Microsystems Labs Inc; Kent Pitman, Harlequin Inc.:

https://www.cs.umd.edu/~nau/cmsc421/norvig-lisp-style.pdf

Notes from the ANSI standardisation process:

https://stackoverflow.com/questions/72414053/notes-from-the-...

Issue CLOS-CONDITIONS Writeup:

https://www.lispworks.com/documentation/lw50/CLHS/Issues/iss...

On Pitman's “Special forms in Lisp” (2011) (kazimirmajorinc.com):

https://news.ycombinator.com/item?id=29947329

https://news.ycombinator.com/item?id=29954993

DonHopkins on Jan 16, 2022 | parent | next [–]

Kent Pitman also wrote the "Revised Maclisp Manual (Saturday Evening Edition)" aka the "Pitmanual".

https://en.wikipedia.org/wiki/David_A._Moon

http://www.nhplace.com/kent/publications.html

>In 1983, I finished the multi-year task of writing The Revised Maclisp Manual (Saturday Evening Edition), sometimes known as The Pitmanual, and published it as a Technical Report at MIT's Lab for Computer Science. In 2007, I finished dusting that document off and published it to the web as the Sunday Morning Edition.

http://www.maclisp.info/pitmanual/

Not to be confused with David Moon who wrote the "MacLISP Reference Manual" aka the "Moonual", and who co-authored the "Lisp Machine Manual" with Richard Stallman and Daniel Weinreb, which had big bold lettering that ran around the spine and back of the cover, so it was known as the "LISP CHINE NUAL" (reading only letters on the front).

https://news.ycombinator.com/item?id=15185827

https://hanshuebner.github.io/lmman/title.xml

https://news.ycombinator.com/item?id=15186998

DonHopkins on Sept 6, 2017 | next [–]

The cover of the Lisp Machine Manual had the title printed in all caps diagonally wrapped around the spine, so on the front you could only read "LISP CHINE NUAL". So the title was phonetically pronounced: "Lisp Sheen Nual".

My friend Nick made a run of custom silkscreened orange LISP CHINE NUAL t-shirts (most places won't print around the side like that).

https://www.lispworks.com/documentation/lw50/CLHS/Issues/iss...

I was wearing mine in Amsterdam at Dappermarkt on Queen's Day (when everyone's supposed to wear orange, so I didn't stand out), and some random hacker (who turned out to be a university grad student) came up to me at random and said he recognized my t-shirt!

http://www.textfiles.com/hacking/hakdic.txt

CHINE NUAL (sheen'yu-:l) noun.

The reference manual for the Lisp Machine, a computer designed at MIT especially for running the LISP language. It is called this because the title, LISP MACHINE MANUAL, appears in big block letters -- wrapped around the cover in such a way that you have to open the cover out flat to see the whole thing. If you look at just the front cover, you see only part of the title, and it reads "LISP CHINE NUAL"

toomanybeersies on Sept 7, 2017 | parent | next [–]

Link to an image of the manual, for the lazy:

https://c1.staticflickr.com/1/101/264672507_307376d26c_z.jpg

[...]

;;; Notes about CLI interrupts and eval-in-other-lisp:

https://news.ycombinator.com/item?id=20267415

https://news.ycombinator.com/item?id=38061207

>Here's Kent Pittman's :TEACH;LISP from ITS, which is a MACLISP program that teaches you how to program in MACLISP. (That's "Man And Computer Lisp" from "Project MAC", not "Macintosh Lisp".)

ilaksh2y ago

In a very rough sense, it actually seems more similar to what LLMs are doing than I would have assumed. That might mainly be because they are both attempting to generate more text from an input.

acidburnNSA2y ago

So was Eliza from 1966 more intelligent than Dr SBAITSO from 1990?

https://en.wikipedia.org/wiki/Dr._Sbaitso

the_af2y ago

Wasn't SBAITSO pretty much the same idea as Eliza, only using speech synthesis with a Soundblaster?

acidburnNSA2y ago

Yes. I'm surprised to hear Eliza passed a turing test. SBAITSO was fun but pretty clearly not a human.

1 more reply

ShamelessC2y ago

I am irrationally upset about the title.

jamesdwilson2y ago

Right there with you. I try to internally read it and get flustered every time.

j / k navigate · click thread line to collapse

110 comments

jll292y ago

Here is the pointer to the original Eliza paper https://dl.acm.org/doi/10.1145/365153.365168

He failed: His own secretary wanted to be left alone with the software and typed in her personal problems. Work on Eliza (1963-65, paper published 1966) until today is mostly misunderstood.

adestefan2y ago

erikerikson2y ago

> humans are not machine

Aren't we? Casual chains upon our matter produce emergent behaviors using the same physics and chemistry that our mechanistic creations rely upon.

2 more replies

stavros2y ago

> Especially that computers can decide what comes next, but only a human can choose what to do.

I don't understand this, all the programs I've ever written make decisions based on some factors.

Are you talking about free will? If so, what is free will?

3 more replies

extractionmech2y ago

“Me no likee” - Joseph Agassi

https://www.researchgate.net/publication/286058724_Computer_...

p.s. continuing on, the review turns very positive. Initial vitriol apparently more related to the “two worlds” matter.

1vuio0pswjnm72y ago

Previous: https://news.ycombinator.com/item?id=36901158 [PDF]

tempodox2y ago

ryanklee2y ago

> cannot reason anybody out of something that they want to believe.

Especially when your own reasons are not the slam dunk you think they are.

1 more reply

scotty792y ago

> He failed

I'd say he succeeded. It just seems that people are perfectly content with just appearance of intelligence.

leethargo2y ago

Not only his secretary, also some psychiatrists wanted Eliza as a tool to scale up their work clinically.

moffkalast2y ago

Say, has anyone deployed Hartford's Samantha yet?

extreme_orang2y ago

Take a look at this project that has rediscovered and recreated the original ELIZA source code. You can even try out an accurate reimplementation of ELIZA

http://findingeliza.org/

lulznews2y ago

Never underestimate human stupidity!

drakonka2y ago

Now here I am talking about life, the universe, and everything with ChatGPT. It makes me both inexplicably happy/hopeful and simultaneously weirdly melancholic.

[0] https://liza.io/its-not-so-bad.-we-have-snail-babies-after-a...

bee_rider2y ago

aldousd6662y ago

masswerk2y ago

pixl972y ago

>how eager we are to overestimate the capabilities of such a partner in conversation, as soon as some criteria seem to be satisfied. Which is arguably still relevant today.

Honestly, AI shouldn't be the takeaway point here, but how we do the same for politics.

1 more reply

throwaway4aday2y ago

> Interestingly, Eliza still outperforms ChatGPT in certain Turing test variations.

I see we have a new entry for the 2024 Lies of Omission award.

An honest author would have actually quoted the article saying:

> GPT-4 achieved a success rate of 41 percent, second only to actual humans.

instead of constructing a deliberately misleading paraphrase.

notahacker2y ago

The blog appears to have been updated to specify GPT3.5, but the original version was accurate.

throwuxiytayq2y ago

The example ELIZA responses in the paper are so laughably bad and trivial to pick up, I'm not convinced the human interrogators were sober/conscious/awake during the experiment.

2 more replies

pixl972y ago

Yea, It's really hard to get GPT to sound human because the RLHF really wants to let you know it's not a human.

GPT4 + a RLHF that was trained to think it was human would be a much different beast.

1 more reply

throwaway4aday2y ago

masswerk2y ago

throwaway4aday2y ago

Several people in this thread appear to have misunderstood due to the way this article was written.

seanwilson2y ago

Why was the Turing test still relevant after this? Didn't this indicate it was very flawed test? Or it was hard to come up with a better test?

recursivecaveat2y ago

GuB-422y ago

The variant of the Turing test that Eliza has passed is the easiest one. That is: convince a novice interrogator that what is on the other side of the screen is a human.

canjobear2y ago

Because Eliza didn’t pass the Turing Test. It is trivially easy to trip it up.

nbzso2y ago

So the Turing test actually tests not a technology but the level of intelligence of the user. So, we are doomed, it seems.:)

__MatrixMan__2y ago

He originally made the argument about gender, not intelligence. I think he was arguing for a whole class of properties for which there's no difference between authenticity and convincing fakery.

It reminds me of how no class of computer can solve the halting problem for itself. No matter how intelligent you are, there will be holes like this in your certainty about some things.

pixl972y ago

Or another way to put this, it's not a binary problem, it's a probability continuum.

Even the definition of 'human intelligence' is a continuum from the smartest to the dumbest of us, that doesn't even stop there and descends thought all animal life.

1 more reply

masswerk2y ago

YurgenJurgensen2y ago

derbOac2y ago

I've often felt that a better version is not whether a person can guess that it's AI or a human, but whether people behave and feel differently with an AI or human.

There are probably some people who would be ok with some AI for some purposes.

masswerk2y ago

oneeyedpigeon2y ago

Definitely the main flaw of the Turing test — having two human actors taking part alongside one computer is just introducing two too many variables :)

sixothree2y ago

I remember this being written in basic for the c64. Not sure if I had the real Eliza or a clone. But it was fun to look at all of the canned responses and try to get it to respond with them.

smcameron2y ago

Probably a variant of the Eliza program in "More BASIC Computer Games", see p. 56 of this PDF: https://ia802707.us.archive.org/33/items/More_BASIC_Computer...

sixothree2y ago

That's likely the one.

master-lincoln2y ago

> Clearly, Eliza is not an AI

Sadly the author doesn't elaborate on this. I thought nowadays 'AI' is a synonym for 'algorithm', which would fit ELIZA

Is there an accepted definition of the word AI?

oneeyedpigeon2y ago

We've been misusing the term ever since observing the ghosts in Pac Man.

perthmad2y ago

Any algorithm you do not understand.

mysterydip2y ago

So, any of my own code I haven't looked at for a few months? :)

throwaway4aday2y ago

Including the one in your head.

maweaver2y ago

nsajko2y ago

The article is missing the interesting bits, so here are the relevant Wikipedia pages:

https://en.wikipedia.org/wiki/ELIZA

https://en.wikipedia.org/wiki/ELIZA_effect

https://en.wikipedia.org/wiki/Joseph_Weizenbaum

dboreham2y ago

It turns out, 50 years later, that Eliza was on the right track.

deepfriedrice2y ago

[0] https://www.zachtronics.com/eliza/

whoopdedo2y ago

Then later on there was Azile[1], the evil twin to Eliza. Instead of being trained with data to make it sound helpful and supportive, it uses hostile and insulting language.

[1] https://www.civilizr.com/communication/azile-chatbot-savage-...

scotty792y ago

Can one see original Eliza sourcode anywhere?

throwawayForMe22y ago

I had Mu-Lisp for myfirst computer, an Osborne-1, that came with a copy of Eliza.

You may find the original here:

https://sites.google.com/view/elizagen-org/commonly-known-el...

extreme_orang2y ago

See http://findingeliza.org/

proamdev1232y ago

There is a version of it in Emacs, called up with `M-x doctor`.

The source is in `doctor.el`.

https://www.emacswiki.org/emacs/EmacsDoctor

jhfdbkofdchk2y ago

https://sites.google.com/view/elizagen-org/the-original-eliz...

RugnirViking2y ago

We had it running in terminals when I used to work in the national museum of computing in the UK (on machines where you can just pull the full source up from floppy disk)

DonHopkins2y ago

https://news.ycombinator.com/item?id=27332340

DonHopkins on May 30, 2021 | prev | next [–]

Here's the source code for Kent Pitman's "DOCTOR" in MACLISP, which was of course inspired by ELIZA. (Joseph Weizenbaum taught Kent Pitman LISP!)

https://github.com/PDP-10/its/blob/master/src/games/doc.102

And here's what happened with he (manually by typing) hooked it up with Kenneth Colby's "PARRY" (the paranoid patient):

https://www.maclisp.info/pitmanual/funnies.html

>Parrying Programs

>I have done light editing to remove the typos we made (rubbed out characters were echoed back in square brackets).

[...]

https://news.ycombinator.com/item?id=38402813

DonHopkins 44 days ago | parent | context | favorite | on: The Revival of Medley/Interlisp

That's right, it's just a throw-away quip, but if you want the deep nuanced story and inside history of Common Lisp and comparison with Scheme, Kent Pitman is the one to read:

https://en.wikipedia.org/wiki/Kent_Pitman

Index of Kent Pitman's Papers:

https://www.nhplace.com/kent/Papers/

Scheme or Lisp? Kent M Pitman explains the deep philosophical differences.

https://www.reddit.com/r/programming/comments/6fa5r/scheme_o...

Kent Pitman on Scheme or Lisp?:

https://groups.google.com/g/comp.lang.lisp/c/TEk4O4-zsA8/m/H...

Common Lisp: The Untold Story:

https://www.nhplace.com/kent/Papers/cl-untold-story.html

Why Wolfram Mathematica did not use Lisp (2002) (ymeme.com):

https://news.ycombinator.com/item?id=9797936

https://web.archive.org/web/20110122140154/http://www.ymeme....

Kent Pitman's essay on why lisp doesn't have copying of lists.

https://groups.google.com/g/comp.lang.lisp/c/MmtQreo3PCM

Parenthetically Speaking with Kent Pitman: The Best Intentions: EQUAL Rights -- And Wrongs -- In Lisp:

https://www.nhplace.com/kent/PS/EQUAL.html

Kent M. Pitman Answers On Lisp And Much More:

https://developers.slashdot.org/story/01/11/03/1726251/kent-...

Kent M. Pitman's Second Wind:

https://developers.slashdot.org/story/01/11/13/0420226/kent-...

Tutorial on Good Lisp Programming Style: Peter Norvig, Sun Microsystems Labs Inc; Kent Pitman, Harlequin Inc.:

https://www.cs.umd.edu/~nau/cmsc421/norvig-lisp-style.pdf

Notes from the ANSI standardisation process:

https://stackoverflow.com/questions/72414053/notes-from-the-...

Issue CLOS-CONDITIONS Writeup:

https://www.lispworks.com/documentation/lw50/CLHS/Issues/iss...

On Pitman's “Special forms in Lisp” (2011) (kazimirmajorinc.com):

https://news.ycombinator.com/item?id=29947329

https://news.ycombinator.com/item?id=29954993

DonHopkins on Jan 16, 2022 | parent | next [–]

Kent Pitman also wrote the "Revised Maclisp Manual (Saturday Evening Edition)" aka the "Pitmanual".

https://en.wikipedia.org/wiki/David_A._Moon

http://www.nhplace.com/kent/publications.html

http://www.maclisp.info/pitmanual/

https://news.ycombinator.com/item?id=15185827

https://hanshuebner.github.io/lmman/title.xml

https://news.ycombinator.com/item?id=15186998

DonHopkins on Sept 6, 2017 | next [–]

My friend Nick made a run of custom silkscreened orange LISP CHINE NUAL t-shirts (most places won't print around the side like that).

https://www.lispworks.com/documentation/lw50/CLHS/Issues/iss...

http://www.textfiles.com/hacking/hakdic.txt

CHINE NUAL (sheen'yu-:l) noun.

toomanybeersies on Sept 7, 2017 | parent | next [–]

Link to an image of the manual, for the lazy:

https://c1.staticflickr.com/1/101/264672507_307376d26c_z.jpg

[...]

;;; Notes about CLI interrupts and eval-in-other-lisp:

https://news.ycombinator.com/item?id=20267415

https://news.ycombinator.com/item?id=38061207

>Here's Kent Pittman's :TEACH;LISP from ITS, which is a MACLISP program that teaches you how to program in MACLISP. (That's "Man And Computer Lisp" from "Project MAC", not "Macintosh Lisp".)

ilaksh2y ago

In a very rough sense, it actually seems more similar to what LLMs are doing than I would have assumed. That might mainly be because they are both attempting to generate more text from an input.

acidburnNSA2y ago

So was Eliza from 1966 more intelligent than Dr SBAITSO from 1990?

https://en.wikipedia.org/wiki/Dr._Sbaitso

the_af2y ago

Wasn't SBAITSO pretty much the same idea as Eliza, only using speech synthesis with a Soundblaster?

acidburnNSA2y ago

Yes. I'm surprised to hear Eliza passed a turing test. SBAITSO was fun but pretty clearly not a human.

1 more reply

ShamelessC2y ago

I am irrationally upset about the title.

jamesdwilson2y ago

Right there with you. I try to internally read it and get flustered every time.

j / k navigate · click thread line to collapse