- ML is getting more powerful and will continue to do so as time goes by. While this point of view is not unanimously held by the AI community, it is also not particularly controversial.
- If you accept the above, then the current AI norm of "publish everything always" will have to change
- The _whole point_ is that our model is not special and that other people can reproduce and improve upon what we did. We hope that when they do so, they too will reflect about the consequences of releasing their very powerful text generation models.
- I suggest going over some of the samples generated by the model. Many people react quite strongly, e.g., https://twitter.com/justkelly_ok/status/1096111155469180928.
- It is true that some media headlines presented our nonpublishing of the model as "OpenAI's model is too dangerous to be published out of world-taking-over concerns". We don't endorse this framing, and if you read our blog post (or even in most cases the actual content of the news stories), you'll see that we don't claim this at all -- we say instead that this is just an early test case, we're concerned about language models more generally, and we're running an experiment.
Finally, despite the way the news cycle has played out, and despite the degree of polarized response (and the huge range of arguments for and against our decision), we feel we made the right call, even if it wasn't an easy one to make.
If this is your whole point, then I think you are missing something fundamental. Implementing these models doesn't require reflection, or introspection, or any sort of ethical or moral character whatsoever; and even if it did, all that will happen eventually is someone (without the technical background) will simply throw a lot of money at someone else (with the technical background, but who needs to, you know, eat, and pay rent, and so on) to implement it. You are fooling yourself if you think your stance makes a single mote of difference in this arms race.
In fairness, if that's true, then no one has any need of her model.
More seriously speaking, why does anyone need, say, "training set x", or "model y", to make their implementation work? You don't. So I don't really understand why everyone is so worked up about not releasing this stuff? If you want to do it, do it. If not, don't. But there's no need to say, "I demand everyone do it, and I'll have a meltdown if they don't."
I’m also confused by the threat models earnestly put forth in your blog post. Are we really concerned about deep faking someone’s writing? The plain word already demands attribution by default: we look for an avatar, a handle, a domain name to prove the person actually said this.
It seems more like the "nukes are safer when multiple rational state level actors have them", rather than anyone able to pull a git repo.
Mostly it's scary not because it's good - as writing goes, it's quite bad. It forms coherent sentences, but otherwise it's nonsense. I've seen similar nonsense producers in early 90s on basis of Markov chains and what not.
No, the scary part is how much it reminds me of what I am reading in the media all the time. My current pet concern is that AIs will start passing the Turing test not because AIs are getting so good but because humans are getting so bad. A bunch of nonsensical drivel can easily be passed as a thoughtful analysis or a deep critical think-piece - and that's not my conjecture, have been repeatedly proven by submitting such drivel to various academic journals and it being accepted and published. I'm not saying people are losing critical thinking skills - but they are definitely losing (or maybe never even had?) the habit of consistently applying them.
Exactly. When it comes to generating a large volume of apparently-good sentences, non-AI (or classical) approaches are still better than good. Those will be equally disruptive, since the defending side is yet to develop a proper countermeasure based on the "sensible"-ness of content. Plus, they will be much easier to customize and adapt to the situation, while ML-based solutions often need remodeling and retraining when repurposed.
> My current pet concern is that AIs will start passing the Turing test not because AIs are getting so good but because humans are getting so bad
AI will start deceiving the public even before it pass Turing test. It's much harder to spot bots amidst people than in a 1vs1 chatroom.
Only people with a large amount of money and a lot of expertise. What you are doing is the opposite of democratizing AI.
Yet from Google we heard nothing. Which is the optimal decision for them - they only lose by blowing the whistle.
>Recycling is NOT good for the world.
>It is bad for the environment,
>it is bad for our health,
>and it is bad for our economy.
>Recycling is not good for the environment.
>Recycling is not good for our health.
>Recycling is bad for our economy.
>Recycling is not good for our nation.
The first paragraph keeps repeating the <X> is <bad | not good> for the <Y> pattern 8 times.
>And THAT is why we need to |get back to basics| and |get back to basics| in our recycling efforts.
"get back to the basics" is repeated twice in the same sentence.
>Everything from the raw materials (wood, cardboard, paper, etc.),
>to the reagents (dyes, solvents, etc.)
>to the printing equipment (chemicals, glue, paper, ink, etc.),
>to the packaging,
>to the packaging materials (mercury, chemicals, etc.)
>to the processing equipment (heating, cooling, etc.),
>to the packaging materials,
>to the packaging materials that are shipped overseas and
>to the packaging materials that are used in the United States.
It literally repeated packaging 5 times in the same sentence and the overall structure was repeated 9 times. Also what type of packaging is based on mercury?
(This of course doesn't make it an amazing feat of computer engineering.)
The overarching narrative is great, but that's probably driven by the great antithesis supplied by the experimenter.
It'd be interesting to know how this works, what happens if less or more is given as thesis/antithesis/assignment, and after how much output it turns into gibberish (or repeats).
Heck, maybe having to compete with this will raise human discourse (Joking).
Have you done a plagiarism search on that text to see how similar it is to the input corpus? I'm by no means an ML expert, but I've played around with models for random name generation and one thing I've noticed is that as the models become more accurate, they also become much more likely to just regurgitate existing names verbatim. So if you search the list of names and notice something that seems particularly realistic, it could be because it's literally taken in whole or in part from the training data set!
(The talking unicorn example on their page is also meant to demonstrate that, no, it's not just memorizing, but I think it's a bit more compelling to check from the raw samples)
How is that open?
How is that not centralization of power?
Here are a few that comes to mind.
-Secrecy? but how will you continue to exist on the PR scene if you don't release anything?
-Are you willing to pay every developer who is able to replicate your paper, more than what the black market would pay?
-How are you working on incentive alignment to make sure that all people who can replicate your results have more incentive to do good than bad, specially in the current environment where users and valuable data are silo-ed by a few companies?
-Misdirection to keep an edge, i.e. planting bugs/ Not fixing bugs for public ; spreading false results; only working on problems that need high resources to limit the number of actor who will be able to replicate ?
-Tracking the people who have the competence to replicate and take preemptive measures.
-Restrictions on GPU/CPU/silicone wafer.
Who can regulate? How can we regulate? What are the negative consequence of regulation? What happens if we don't, at what odds and time horizon?
That said, withholding the pretrained models probably won't make much difference, because bad actors with resources (e.g., certain governments) will be able to produce similar or better results relatively quickly.
All it will take is (1) one or two knowledgeable people with the willingness to tinker, (2) a budget in the hundreds of thousands to a few millions of dollars at most, and (3) a few months to a year. Nowadays a lot of people are familiar with Transformers and constructing and training models across multiple GPUs.
Ok, accepting that premise, what people/organisations would you share the research with and based on what criteria?
One of the reason Elon distanced himself because of what OpenAI team wanted to do. I am wondering if this new paper has anything to do with that? Or what it is in general that Elon doesn't agree with what OpenAI is doing?
Thanks!
This seems to be a particularly weak argument to make. How is their model going to impersonate someone in a way that a human can not?
https://d4mucfpksywv.cloudfront.net/better-language-models/l...
Reading that piece gives me the same weird feeling as watching AlphaStar playing through a StarCraft game.
"All models still underfit WebText and held-out perplexity has as of yet improved given more training time."
This is the worst headlines in this matter. This is one of the leading media in India. A language model being touted as Fake news AI tool. This is like calling a car, A run over machine by Ford.
https://www.hindustantimes.com/tech/elon-musk-distances-hims...
That's a great dysphemism. Gonna start using that.
So for the Ford analogy to be apt, Ford would have to have designed a car nobody has ever seen, and released a video which is basically just hundreds of hours of the car running people over.
I mean, a car has lots of well understood non-running-people-over capabilities. But have they demonstrated that this model is useful for anything other than generating fake news-sounding spam text?
Secondly, have you seen the results? I was dumbfounded and fascinated. I spent hours reading the samples.
Maybe I'm just out of the loop and this truly isn't anything significant, but then that only proves that OpenAI was successful: Now I am aware of the latest advances in NLP and hopefully so too are many more.
Yes, I've seen the result. They're nice but, as the article points out, not extraordinary compared to state of the art, open NLP research.
OpenAI's behaviour here smells of Gibsonesque 'anti-marketing', using the misunderstanding of AI and its capabilities in the general population as a means to stir up publicity for their organisation.
This is unethical, misrepresents progress in the field, and produces confusion in the press.
Like many, I was viscerally shocked that the results were possible, the potential to further wreck the Internet seemed obvious, and an extra six months for security actors to prepare a response seemed like normal good disclosure practice. OpenAI warned everyone of an “exploit” in which text humans can trust to be human-generated, and then announced they would hold off on publishing the exploit code for 6 months. This is normal in computer security and I’m taken aback at how little the analogy seems to be appreciated.
Why? There were news about bots writing news ~5 years ago. Given a few simple facts the AI generated the regular info-scarce but fluffy news-piece.
Now OpenAI added better everything (better language models, more data, better "long-term memory" for overall text coherence), and we got better fluff.
It seems like a GAN and a simple Markov chain generator. (Even if it's not that simple of course.)
And maybe it's the equivalent of the "modern art meme" style transferred to AI/ML research. ( https://i.pinimg.com/236x/71/e1/21/71e12151f4b59d8433d32c126... )
What I'm trying to convey is that wrecking the net with auto-trolls was already possible, but for some reason Mechanical Turk was cheaper.
> OpenAI warned everyone of an “exploit” in which text humans can trust to be human-generated
Sokal already did that, and so did http://thatsmathematics.com/mathgen/ ... but of course this might be qualitatively different, because it can be targeted. (Weaponized, if you will.) But the defense/antidote is the same, but it takes a lot more than 6 months to make people better at critical thinking, but maybe you already heard about the difficulties of that :)
Tesla does not even offer their full self driving package anymore. No coast to coast drive yet. Hard to say that's an amazing job.
OpenAI abandons their open source GitHub repos after a year, is now not releasing code, and is always in DeepMind's shadow. Alive, yes. Successful, no.
At the same time Andrej dropped out the idea of a fully learned end-to-end model (that's just impossible with the current deep learning technology), and started replacing the somewhat working heuristics with machine learning methodically one-by-one. Also he ramped up the data gathering pipeline.
He needs to build the full simulation, agent systems that can simulate other drivers/humans, implement reverse reinforcement learning...there's so much to do where Waymo is far ahead (but Tesla is ahead in data gathering).
and the previous paper
https://s3-us-west-2.amazonaws.com/openai-assets/research-co...
It's a transformer, not LSTM, and it's very large but not structured in a particularly unusual way.
The strongest counterargument I've seen to OpenAI's decision is that the decision won't end up mattering, because someone else will eventually replicate the work and publish a similar model. But it still seems like a reasonable choice on OpenAI's part–they're warning us that some language model will soon be good enough for malicious use (e.g. large-scale astroturfing/spam), but they're deciding it won't be theirs (and giving the public a chance to prepare).
The lead policy analyst at OpenAI has already tried to engage the community in discussing the malicious use of AI, on many occasions, including this extremely well-researched piece with input from many experts: https://maliciousaireport.com/ . But until OpenAI actually published examples, the conversation didn't really start.
In the end, there's no right answer - both releasing the model, and not releasing the model, have downsides. But we need a respectful and informed discussion about AI research norms. I've written more detailed thoughts here: https://www.fast.ai/2019/02/15/openai-gp2/
Only when a human wants to fool a human, it impersonates whatever possible but a human, then suddenly charges a shitload of ape shit, and then behaves like it never happened.
Without a decent natural language translation or automatic reasoning, which they have not, looks like N-gram where N equals to number of words in language corpus.
Pretty dumb and disrespectful to politicize a blog post about OpenAI.
