Does this level of detail seem strange to anybody else? Shining such a strong light on OpenAI's moderation/manual review efforts seems like it would draw unwanted attention to the fact that ChatGPT conversations are anything but private, and seems somewhat at odds with their recent outrage about the subpoena for user chats in the NYT case.
Manual reviews of sensitive data are ok as long as their own employees are the reviewers, I suppose?
> By examining request metadata, we were able to trace these accounts to specific researchers at the lab.
> The volume, structure, and focus of the prompts were distinct from normal usage patterns
Clearly some employees of Anthropic personally looked at individual inputs and outputs of their API
The level of detail does not seem surprising. they're both charged with maintaining a facade of privacy while eliminating any and all miss-use. Certainly they heavily analyze basically everything given to them.
And generally as a society we've been ok with basically zero privacy as long as the data we send stays inside the company we sent it too. Google reads all your emails? Sure thing, read away, just don't send them to the popo. Apple knows when you're ovulating? no problem, just don't tell Amazon. etc
If you've ever ran a SaaS business, you know this and you know you can have "God Mode" access to everything, even if you swear up and down that you don't/won't.
The owners of these models aren't your friends, they see you as objects. They want to take as much value as they possibly can from you and will starve you if/when the option appears. That includes selling and sharing whatever data they have on you to the highest bidders, and some of those bidders want scapegoats to parade around as domestic terrorists.
The fact that companies are willing to send their IP and business processes to entities that can easily launder it and out compete them is mind-boggling, as well.
It started discussing like a Western bot would - "it's complicated, etc. etc." and around 5s it abruptly stopped and regurgitated the same line the CCP uses "... it's an unalienable part of China etc. etc.".
After printing the line, a popup opened and my camera was activated. The app wanted me to submit my information, presumably to decide what to do with me next time I enter China.
1) All the lights and modern buildings cannot hide that China is a creepy authoritarian state underneath.
2) Given the bot started printing the Western consensus first, I bet $10 it was trained by distilling ChatGPT or Gemini.
Was this on your personal device? I'm just wondering how it activated your camera. I would love more details!
China has more restrictions on what you can say than the U.S. but what you are describing is not reality. Some westerner asking Deepseek about Taiwan is completely uninteresting. Just as the government do not chase people over VPN usage.
China doesn’t try to hide that they are an authoritarian state. They don’t need to. Most people in China are no less happy with their government than westerners are with their governments. Governments reflect culture. And as for foreigners, our view of China is far worse than it actually is, China doesn’t need to hide anything, people who visit China will come away with a more positive view of the country than those who do not visit.
Not recognizing they were outputting wrongthink until after it was being streamed to the user is a known behavior with some Chinese chatbot apps. A quick search found an example of DeepSeek doing it: https://www.reddit.com/r/OpenAI/comments/1ic3kl6/deepseek_ce...
I don't think his story is genuine, but it showing the "wrong" answer before correcting itself is known behavior.
EDIT: Here's an example of it outputting a full response about Taiwan specifically before removing it: https://www.reddit.com/r/interestingasfuck/comments/1i7ceol/...
My wife grew up in Shanghai, and you'll have to go quite some distance to find someone more critical of the PRC and CCP than she is. And it's with good reason.
She grew up during the cultural revolution, and was largely raised by her grandmother because literally every other person in her extended family was in prison or work camp, not because of anything they had actually done wrong, but for political reasons because the whole family was blacklisted.
And that's not just the old days. Her father died as a direct result of Chinese Covid policy. During the pandemic her cousins still in the country would ask her (on Skype) "is X true?", and largely their perception of what was going on was false. She would exfiltrate encrypted news reports to them - until those started getting blocked. Her dad's estate still has affairs that need to be resolved, but we've decided not to return to China until Xi is gone, as it's just not safe. It doesn't get much airplay, but there are currently a couple of hundred Americans who are being illegally detained in China right now. It's not worth the risk.
My first trip to China was about 30 years ago, shortly after we got married. And back then, I would have said that you were right. Honestly, it felt like for the average person in their day-to-day-lives, the Chinese were less under the governmental thumb than we are. People from the countryside would bring their produce into the city to sell, or cook dumplings and buns to sell on the side of the street - stuff that in America we'd have to get permits for. It seemed that the oligarchy had an understanding with the people: let us control the big picture, and we'll look the other way for the little things. But Chinese politics is a pendulum swinging very widely. From Tienanmen Square and Tank Man, it had swung quite a bit the other way. But today, it's come back 180-degrees. Xi is really trying for a Cultural Revolution 2.0.
These impressions largely match what I hear from other Chinese immigrants - except for Party members, who tend not to want to talk about it at all. I'm afraid that you've been listening to too much propaganda.
To the extent that's true, it's because they won't let you see the uyghur reeducation camps.
(The first half is obviously true, the second part isn't)
Once I started criticizing Libs of TikTok, the propaganda arm for this administration, and getting traction with users, my account was locked and now I have to scan my face and ID if I want to use it again.
You have to toe the party line here, too.
To your point I've seen something similar with Deepseek, generic answers start printing and then, in plain sight, removed and replaced with a non committal message along the lines of "I don't have access to that information."
It is very real and I am not surprised at all something exactly like what op said has happened.
I was talking crap about china from the great wall.
Personally as a Dutch person it is amusing as all hell hoe goddamn triggered everyone gets about Israel- truly mindblowing.
The disproportion between how this people express they opposition and how Chinese officials track them is HUGE. This very much feel unnecessary.
It was here: https://www.france.tv/france-2/envoye-special/5971095-la-chi...
One hopes the CIA/Secret service would be willing to provide the human to do the reviewing but sadly I've worked for European telco's and I know better.
Can you elaborate?
Truly a paradise for american intelligence. Would have expected that the chinese officials be briefed on not using us tech companies, but opsec is hard to teach, and even harder to always follow.
How can you not trust them.
I never got to the end of the Terms & Conditions myself.
It's basically an OSINT siphon.
In this Chinese case, the tokens are leaked at least twice. ChatGPT offers no direct access to the Chinese, they have to use some kind of Openrouter-like service, but the data where also in clear-text during transmission.
(I kid, mostly. While the US certainly isn't pure, its scale of surveillance intrusion is light compared to China)
I assume that for someone to believe this, they either have to believe the U.S. has poorer surveillance capability than China, or, more likely, they consider U.S. surveillance unintrusive and Chinese surveillance intrusive.
I can't imagine the amount of government secrets, trade secrets, business plans, personal secrets, etc that people divulge on there.
They just gave up a source that could have provided info for years.
Will OpenAI release the same for other government officials from any other states?
I can't wait to see Starmer's chats with ChatGPT.
Anyway, all of this smells like 1934, "accusing them of what we are already doing"
There's something poetic about OpenAI being asked to comment on mis-use of their slop generator, and their answer is composed entirely of AI slop.
But why? It’s like using photoshop for spreadsheets.
The official DeepSeek API is routed through AWS load balancing btw.
If you still needed a reason to look into self hosted models, it'd be tough to find a better one than this.
I hope those victims of immigration impersonation don't have family within China's borders. AI-enabled impersonation and intimidation are far from the worst of China's crimes [1] against its overseas critics.
China likes to make you an offer you can't refuse [2] [3]: You're saying stuff the Chinese government doesn't like, but you live outside its borders and the secret police can't get at you? You need to come to China and be jailed (or worse). If you don't, your family will be the ones who are jailed (or worse). Or you can unalive yourself, and save the glorious Chinese Communist Party the expense of a bullet.
[1] China would say "the government punishes a criminal's family" is not a crime, it's a perfectly legal implementation of government policy under Chinese law. I respond that the death camps were perfectly legal implementation of government policy under Nazi law, but were still crimes against humanity -- China's actions fall in this category of crimes.
As I understand it: Western societies have a very individualistic view of responsibility. If you didn't commit a crime, you're innocent. Punishing the innocent family members of a criminal is morally abominable.
In the Chinese Communist Party's view, criminal responsibility is collectivist. By their definition, the family members of a criminal share responsibility for the crime regardless of their participation in the criminal acts. "Innocent family members of a criminal" is a logically inconsistent concept in their world view. The family of a criminal is guilty by definition -- being related to a criminal is itself a crime.
This is sickening to me.
[2] https://en.wikipedia.org/wiki/Operation_Fox_Hunt
[3] https://www.abc.net.au/news/2020-07-08/fbi-chief-says-china-...
