OpenAI are extremely sensible to draw attention to the fact that AI is approaching a boundary that has practical implications. It is good that everyone is being alerted that that boundary might be crossed at any time in the foreseeable future.
Now the novelty is that this can be better targeted. But even simple Markov-chain based text generators were good enough to fool people for a bit.
And there was always people that had too much free time to write. A lot. (See for example the crackpots and conspiracy theorists that bombard physics forums. See the 9/11, Zeitgeists, etc. movies. See how much has been written about anti-vaxx, about quantum woo, etc.)
Reputation systems work pretty well for countering spammers.
And against APTs (advanced persistent threats, spearfishing attacks, etc) there's no real "universal" protection anyways. (You need a competent security team to out think and out resource the attackers in every possible dimension.)
This AI is the same as the paid Russian trolls and the unpaid scammers, and so on.
I agree with your last point though - it falls into the same category as paid Russian trolls. I think that's exactly why they were hesitant to release the pre-trained models - they didn't want to make it easier/cheaper for a bad actor to replicate the 2016 election.
It remains to be seen whether their decision will make an iota of a difference. But I understand their motivation.
I work in this field, and yes, this is very novel (at least in terms of the quality).
It's the biggest improvement in quality I've ever seen. The long term coherence is so much better than anything else that has ever been built.
Conjecture: GPT-2 trained on reddit comments could pass a "comment turing test", where the average person couldn't distinguish whether a comment is bot or human with better than, say, 60% accuracy.
There is no real profit to be made by generating realistic looking text. Spammers don't work that way, spammers haven't cared about realistic looking text for years. Nor have spam filters cared much about text for a long time, exactly because it's so easy to randomise. Anti-spam is not a good reason to hold back on language generation models, in my view.
As for HN, if bots can write posts as good as humans, great, why hold back?
It seem that as far as information warfare goes “less is more” works quite well and they rely on targeted people to spread the news for them.
When you want to drive an agenda you don’t need unique 100,000 comments you need a good copy pasta.
Overall I’m sick of this dramatization of the AI catastrophe until there will be a proven path with agency for it to actually operate in the real world.
A chat bot isn’t a threat to anyone even if it turns homicidle.
Regarding an agenda, sure, good pasta is fine and all, and regular ol people are fine, but it is not cost effective. This is a million times cheaper, which means you can use it everywhere, not just the obvious places, you can be everywhere, and you can do more than just push a couple big items, you could push tens of thousands of them, micro targeted all the way down to the individual. Don’t dismiss it so easy—the potential scale is far, far larger than anything existing to date.
And I would note that the reason 100,000 comments aren’t effective now is precisely because they are too formulaic, too obviously fake when used on such a large scale. This has the potential to create real, live, seemingly active and believable online communities of millions of people, all at fractions and fractions and fractions of a penny compared to current methods. People read news, then comments (or reviews or whatever), because they use them to determine the validity of the content they just read; if it’s no longer possible to tell from the comments what’s a scam and what isn’t... well, you could do a lot of things with that.
