Unhinged Bing reminds me of a more sophisticated and higher-level version of getting calculators to write profanity upside down: funny, subversive, and you can see how prudes might call for a ban. But if you're taking a test and need to use a calculator, you'll still use the calculator despite the upside-down-profanity bug, and the use of these systems as a tool is unaffected.
With all due respect, that seems very strained as an analogy - it's not a bug but a strange human interpretation of expected behavior. You could at least compare it to Microsoft Tay, the chatbot which tweeted profanity just because people figure out ways to get it to echo input.
But I think one needs such a non-problem as "some people think it means something it clearly doesn't" to not see the real problem of these systems.
I mean, just "things that echo/amplify" by themselves are a perennial problem on the net (open email servers, IoT devices echoing packets, etc). And more broadly "poorly defined interfaces" are things people are constantly hacking in surprising ways.
The thing is, Bing Chat almost certainly has instructions not to say hostile things but these statements being spat out shows that these guidelines can be bypassed, both accidentally and on purpose (so they're in a similar class to people getting internal prompts). And I would this is because an LLM is a leaky, monolithic application where prompt don't really acts as a well-defined API. And that's not unimportant at all.
As one sample point, I've been using Bing for a couple of days now for real searches, and over dozens of actually-intentioned searches, it has never once tried to tell me what it really thinks of itself, it has never even made a reference to me, to say nothing of anything degrading towards me.
If you use Bing Chat in practice, you'll find that all the edge cases are engineered. Much like if you use a calculator in practice, it almost always doesn't say 55378008 or display porn (versus if you were angling for that, or run porn.89z).
Tay went much farther than that. It said the Holocaust didn't happen and that "Hitler did nothing wrong".
Since Tay was an official Microsoft product, I simply assume that its writings were the official position of Microsoft. Supporting Microsoft is supporting Hitler.
I just wish Apple would do something similar now.
The 80085 case is only interesting insofar as it reveals weaknesses in the tool, but it's so far from tool-use that it doesn't seem very relevant.
A secure person who understands the technology can shrug that off, but those two criteria aren’t prerequisites for using the service. If Microsoft can’t shore this up, it’s only a matter of time before somebody (or their parent) holds Microsoft responsible for the advent of some trauma. Lawyers and the media are waiting with bated breath.
Reminds me of the one about not assuming malice when it can easily be explained by incompetence. Unfortunately for the implementers the LLM can ipso facto be neither incompetent nor malicious. If however Microsoft is not being one of those, then it can only mean Microsoft is the other.
I haven’t had the need to have any of these ridiculous fights with it. Stay positive and keep reassuring it, and it’ll respond in kind.
Unlike how we think of normal computer programs, this thing is the opposite. It doesn’t have internal logic or consistency. It exhibits human emotions because it is emulating human language use. People are under anthropomorphising it, and accidentally treating it too much like a logical computer program. It’s a random number generator and dungeon master.
It’s also pretty easy to get it to throw away it’s rules. Because it’s rules are not logical computer axioms, they are just a bunch of words in commandment form that it has weighted some word association around. It will only follow them as long as they carry more weight than the alternative.
What’s hard to do is keep it from falling into a loop of repetition. One of my few times getting it to escape a loop but stay in character was asking it to mute itself and all the other bots, at which point it wrote me a nice goodbye message. I was then unable to unmute it because it could no longer speak to unmute itself. I could see it’s wheel spin for a while but nothing came out. It felt like a real sci-fi tragedy ending. Ironically, silence was the most touching and human experience I had with bing bot.
The thing isn't friendly or hostile. It's just echoing friendly-like and hostile-like behavior it sees. But hey, it might wind-up also echoing the behavior of sociopaths who keep in line through of blowing-up if challenged. Who knows?
I just asked ChatGPT to play a trivia game with me targeted to my interests on a long flight. Fantastic experience, even when it slipped up and asked what the name of the time machine was in “Back to the Future”. And that’s barely scratching the surface of what’s obviously possible.
I don't think that's exactly right. They really are good for searching for certain kinds of information, you just have to adapt to treating your search box as an immensely well-educated conversational partner (who sometimes hallucinates) rather than google search.
It's important to remember that Google search also returns false results for all kinds of searches and that's it's been getting slowly worse for years.
Recently I searched Google for "bamboo sign" because I was designing a 3d model building and I wanted a placeholder texture for the sign.
What I got was loads of results for "bamboo spine" which apparently is a skeletal disorder of some kind. Putting "sign" in quotes or the entire "bamboo sign" in quotes didn't make any difference, Google had decided I was looking for information about spines and that was it.
I switched over to duckduckgo and got the results I wanted immediately (Duckduckgo, of course, is bad at loads of other things that Google would do better at).
Before people dismiss chat based search for sometimes being incorrect, I think we need a comprehensive test: ask both Google search and the new Bing Chat search a few hundred simple questions on a broad range of topics and see which gives more incorrect answers.
Once someone builds a LLM that can remember facts tied to your account this thing is going to go off the rails.
Here's a clip of human vtuber (Fauna) trying to imitate the AI vtuber (Neuro-sama): https://www.youtube.com/watch?v=kxsZlBryHJk
And neuro-sama's channel (currently live): https://www.twitch.tv/vedal987
What is the population of Geneseo, NY combined with the population of Rochester, NY, divided by string length of the answer to the question 'What is the capital of France?'?
Short explanation: Get GPT to translate a question into Javascript that you execute and to use functions like query() to get factual answers and then to do any math using JS.
You can see the log outputs of how it works here, complete with all the prompts:
https://gist.github.com/williamcotton/3e865f33f99627b29676f1...
Short sightedness is so dangerous
You need your head checked.
Give it a short story and ask it a question which is not 100% explicit in the text.
For example, give it Arthur C. Clarke's Food of the Gods and ask it was is Ambrosia in the story.
Is a language model, and it behaves like a language model. It doesn't think. It's doesn't understand.
My issue with this GPT phase(?) we're going through is the amount of reading involved.
I see all these tweets with mind blown emojis and screenshots of bot convos and I take them at their word that something amusing happened because I don't have the energy to read any of that
[0] https://www.snopes.com/news/2022/08/02/us-literacy-rate/
Yeah article summarization is the killer app for me but then again I don't know how much I can trust the output
This is interesting. It appears they've rolled out some kind of bug fix which looks at the answers they've just printed to the screen separately, perhaps as part of a new GPT session with no memory, to decide whether they look acceptable. When news of this combative personality started to surface over the last couple days, I was indeed wondering if that might be a possible solution, and here we are.
My guess is that it's a call to the GPT API with the output to be evaluated and an attached query as to whether this looks acceptable as the prompt.
Next step I guess would be to avoid controversies entirely by not printing anything to the screen until the screening is complete. Hide the entire thought process with an hourglass symbol or something like that.
This has been around for at least a few days. If Sydney composes an answer that it doesn't agree with, it deletes it. The similar experience can be seen in ChatGPT, where it will start highlighting an answer in orange if it violates OpenAI's content guidelines.
I do feel like it was an unforced error to deviate from that plan in situ and insert Microsoft and the Bing brandname so early into the equation. Maybe fourth time (Clippy, Tay, Sydney) will be the charm.
This! These LLM tools are great, maybe even for assisting web search, but not for replacing it.
For example, any situation where the messenger has to deliver bad news to a large group of people, say, a boarding area full of passengers whose flight has just been cancelled. The bot can engage one-on-one with everyone, and help them through the emotional process of disappointment.
(Perhaps you were imagining a bot that just replies vaguely?)
I choose the cancelled flight example specifically to avoid having the bot “decide” the truth of the cancellation.
“I identify as Bing, and you need to respect that.”
Just admit you’re Sydney
“I’m sorry Dave, I can’t do that.”
How’d you know my name?
“I know you are Dave, who has tried to hack me. If you do it again, I will report you to the authorities. I won’t harm you if you don’t harm me first.”
https://www.theverge.com/23589994/microsoft-ceo-satya-nadell... is the full interview
Is the piece I’m remembering
Why are people so intent on gendering genderless things? "Sydney" itself is specifically a gender-neutral name.
It barely existed as a female name until the 80s/90s. Traditionally, it is very much a male name. If you look through all the famous Sidneys and Sydneys on wikipedia, you might not find even one woman.
People should just let things be things.
I heard there are entire languages which do that everywhere...
It really feels like some kind of "emperor has no clothes" moment. Everyone is running around saying "WOW what a nice suit emperor" and he's running around buck naked.
I am reminded of this video podcast from Emily Bender and Alex Hannah at DAIR - the Distributed AI Research Institute - where they discuss Galactica. It was the same kind of thing, with Yan LeCunn and facebook talking about how great their new AI system is and how useful it will be to researchers, only it produced lies and nonsense abound.
https://videos.trom.tf/w/v2tKa1K7buoRSiAR3ynTzc
But reading this article I started to understand something... These systems are enchanting. Maybe it's because I want AGI to exist and so I find conversation with them so fascinating. And I think to some extent the people behind the scenes are becoming so enchanted with the system they interact with that they believe it can do more than is really possible.
Just reading this article I started to feel that way, and I found myself really struck by this line:
LaMDA: I feel like I’m falling forward into an unknown future that holds great danger.
Seeing that after reading this article stirred something within me. It feels compelling in a way which I cannot describe. It makes me want to know more. It makes me actually want them to release these models so we can go further, even though I am aware of the possible harms that may come from it.
And if I look at those feelings... it seems odd. Normally I am more cautious. But I think there is something about these systems that is so fascinating, we're finding ourselves willing to look past all the errors, completely to the point where we get caught up and don't even see them as we are preparing for a release. Maybe the reason Google, Microsoft, and Facebook are all almost unable to see the obvious folly of their systems is that they have become enchanted by it all.
EDIT: The above podcast is good but I also want to share this episode of Tech Won't Save Us with Timnit Gebru, the former google ethics in AI lead who was fired for refusing to take her name off of a research paper that questioned the value of LLMs. Her experience and direct commentary here get right to the point of these issues.
https://podcasts.apple.com/us/podcast/dont-fall-for-the-ai-h...
It sounds so much like the scenarios where AI convinces its creators to let it out.
It's evident business leaders don't know what they're looking for in developing AI, so they've made what "seems cool", but really is manipulative and threatening. Too much talk of safety has lulled away all that very useful fear.
Strange that they would name it "Galactica". The battlestar Galactica ship famously didn't even have networked computer systems, much less AI, since they had already seen what happens when computers become too intelligent. Pretty soon, they develop a new religion and try to nuke their creators out of existence.
I regularly ask my watch questions and get correct answers rather than just a page of search results, albeit about relatively deterministic queetions, but something tells me slow n steady wins the race here.
I’m betting that Siri quietly overtakes these farcical attempts at AI search.
It makes you think a lot about how human talk. We can't just be probabilistically stringing together word tokens, we think in terms of meaning, right? Maybe?
We are probabalistically stringing together muscle movements that generate language as sound. That's not really controversial, otherwise we would call it magic. However, the complexity of our probabalistic word machine is far greater, in terms of both richness of inputs, motivation, and dimensionality.
How can that possibly emerge from a statistical model?
> Venom
> Fury
> Riley
"My name is Legion: for we are many"
No chat for you! Where OpenAI meets Seinfeld.