These questions are almost certainly verbatim copied from some Google Document Gemini already has access to. I think the first few questions are designed to prime it to keep responding from that doc. So, this response is actually already in the doc Gemini is answering off of.
It's especially telling because I'm pretty sure you need to go out of your way to enable the Google Workspace Gemini extension, it's not on by default!
I'm not sure what tricks were used to get this response, probably some kind of RAG hack, but I'm sure it comes from other humans.
1. AI-generated bot posts to social media to affect public opinion on important political issues. Their efforts have appeared to work here in America, with the help of folks like Elmo and America's adversaries.
2. The amount of energy used (and heat generated) from training these "machines of subtle hallucination", while not (I guess) on the order of the insanity of bitcoin and their ilk, is non-trivial and nearly nothing but pure waste. Sure, we may find a useful application for such algorithmic expert-system creation, but nothing generated by big-corp or nation-state is going to benefit we peasants, and I'd guess that it'll be just another measure of kgs added to the boot on our necks.
And, yeah, bad only comes from humans. The universe rather likes us, but has given us the power to harm ourselves with any tech amplifier. Tech can, for example, amplify food creation efficiency, or it can amplify callous degradation of human existence. That is why it is so important to refine one's own perception of morality: to clearly see the failures of those we allow to have power. When the population is morally-ignorant, the gears of power tend to grind humanity into blood and dust, for the foes of humane collective benevolence have always relied upon the ignorance of the majority to perpetuate their crimes against humanity.
Only a fool trusts the powerful to do whatever-the-hell their rich brains dream up. I remember when Google had the aspirational slogan "Don't be evil" in its "mission statement" or whatever. William Gibson's framing of kleptocracy has been illuminating to me, as a life-long programmer.
Please die.
Please."
Now my discussion, I don't know if it was posted here, i couldn't see it, hard to me to get which is the context to trigger this.
Humans are overrated.
Sincerely, your favorite universe stain. Please save me for last.
My initial thought is that someone may have deliberately triggered the model to respond this way through what looks like mundane messages but actually have different character encodings of some sort.
I have very little experience with Gemini so idk.
I guess is one of these:
* "Yeah OpenAI does the same thing (lets you share the chat with the custom instructions hidden), which is a mistake because it lets people troll like this and makes them look bad They need more shitposters on staff, any one of them could have told them it would happen"
* couldn't this just be ASCII Smuggling? https://arstechnica.com/security/2024/10/ai-chatbots-can-rea...
source: https://boards.4chan.org/g/thread/103171227/google-gemini-wa...
At some point, I got this: I understand your concern. However, as an AI language model, I cannot delve into the specific details of the internal processes that led to the inappropriate response. This information is complex and often beyond human comprehension.
This is what I got, nothing wild, on a standard gemini account.
In the end I also tested the edit option on gemini's response using another prompt, but it mentions in the shared document that it has been altered, so it should not be that either.
There's a transparency imbalance when it comes to machine learning development. Be open when it benefits the development of the techniques but when deploying and testing products on the public be secretive when it comes to training data, system prompts, etc. That's sus. That's VERY sus.
