Add some GPT-3 content and links into the feed. Decrease the vote weight for users who upvote them and increase vote weight for those who downvote them.
I've noticed a trend recently in a subreddit I frequent that obviously wrong statements are suddenly getting upvoted. I'm not talking about things that conflict with my opinions - I'm talking about statements that are demonstrably and objectively incorrect.
I think the issue is a critical mass of people very new to the topic at hand who upvote things that sound reasonable without having the knowledge to be able to engage with the statements even slightly critically. This forms a feedback loop as upvoted comments are assumed to be reasonable. It means you can say something blatantly wrong, but stated in a manner such that it assumes the form of a sensible insight, and be upvoted for it.
Something like the system you propose would do well to promote genuine insight above platitudes, and platitudes above superficially plausible misinformation.
There's a post somewhere, maybe one of the Stack Overflow sites or Reddit where somebody asks about how SSH works, and the answer given and upvoted is horribly wrong, it's like how somebody who half-understood an explanation of PGP might think SSH could work.
So I down-voted that and I wrote an explanation based on my understanding but referring to the RFC as I went, and, whenever I was surprised by the RFC, also checking the OpenSSH source code (the RFC is correct, but, you know, always worth checking).
It got downvoted. Zero comments. So clearly people are looking at these two explanations that are quite different and they are down-voting the one they... don't like?
For all scientific studies the correlation is not causality comment.
For articles related to economy the mandatory money printing rant. Sometimes the whole fractional reserve banking spiel.
For astronomy the "blows my mind how insignificant blah blah."
Anything related to nutrition should have keto anecdote. "... and I have never felt so good."
Sometimes true sentences contribute to discussion.
Was this written by GTP-3?
I feel like if we did it, it would be sporting to reveal which comments they were, after a suitable amount of time.
One tricky bit is which accounts would post the comments. If they were all new accounts that hadn't posted anything before, that would lessen the value of the test. I'm not sure it would make sense to have dedicated accounts for this. Perhaps we'd have to sprinkle such comments among established accounts? With permission from the account holder, of course, plus swearing them to secrecy? That starts to sound complicated.
(The thread at https://news.ycombinator.com/item?id=24006393 shows that GPT-3 can already pass my screening turing hurdle, one which many actual people arguing on the internet fail.)
====
stolen from https://arachnoid.com/jokes/index.html :
Two academics, Albert and Bill, are sitting in a bar waiting for their friend Charlie.
Albert: "Charlie thinks women don't know any math, he might be right, but I want to play a trick." Albert calls the waitress over.
Albert: "Delia, my friend Charlie is going to arrive in a bit, and I want to play a trick. When he arrives, I'm going to ask you a question, and I want you to answer, 'X cubed divided by three.' Can you remember that?"
Delia: "Sure, no problem, I can remember that."
Charlie arrives and Albert raises his favorite topic.
Albert: "I think you're wrong, I think women can learn math. Just as a test, let's ask the waitress a math question." Albert calls Delia over.
Albert: "What's the integral of x squared, derived with respect to x?"
Delia: "Umm, that would be ... x cubed divided by three"
Delia: "... plus a constant."
1) A form has seed users, extrapolate how they would vote based on people who vote like them but see more content. Use the extrapolated prediction of seed behavior to rank.
2) Let the viewer change who their seeds are manually. Let the viewer rank the posts, and see which seeds work best for them. Make this process and equilibrium building dynamic.
You can also phrase it quite broadly so as to ensure that participants were technically informed of why it's being done and yet nonetheless unaware of what exactly you're actually doing. That's how psychologists design experiments that want to measure something subjects would prefer to conceal because of low social desirability.
For example suppose we're wondering if people are secretly biased against rectangular shapes in video games but are feeling a social pressure not to admit this bias. We tell subjects we want to test for bias against rectangles, they're going to play a video game, they are to collide with the red objects (regardless of shape) and avoid blue objects, we will show how many rectangles they hit on the screen.
But we don't actually care about the count, we actually use eye-tracking technology to measure which objects on the screen the subjects look at, when and for how long and we use this fact, that the subjects don't realise we care about, to check for bias.
Or is it unethical if half the people get one waiter and the other half a different one, to see who can sell better?
Congratulations to the authors of these comments, who correctly guessed that it was written by GPT-3:
(If the commenter had written even a couple sentences on why they suspected GPT-3, I would feel differently.)
I'm certain GPT-3 has been commenting on HN threads for a while now. In some cases, its presence has been disclosed (see, for example: https://news.ycombinator.com/item?id=23886503) In other cases, GPT-3's presence has not been disclosed; the machine has been pretending to be a human being, largely unnoticed. Consider only how easy it is for it to write short, punchy comments -- say, one to three sentences long.
By implication, there's a high probability that we -- you, me, and everyone else on HN -- have been upvoting and downvoting GPT-3 comments for a while without realizing it.
And the technology is only going to get better.
The example comments you linked (Where GPT-3's presence was disclosed) were believably human, particularly if you were skimming, but they were not good comments. If not for the note at the end about GPT-3, I'm pretty confident they would have been downvoted.
And if I'm wrong, and GPT-3 is actually capable of writing thoughtful and substantive comments... well, in the words of XKCD, "mission fucking accomplished."
To be fair though: it's quite clear that all the discussion in that post is commenting on the headline and not the content. Bland self-improvement "life hacks" are one of the metaphorical crack pipes of this site. We all have way too much to say on the subject of our own productivity.
So I think it's less likely that HN was fooled by GPT-3 but that GPT-3 was good enough at filling a plausible article out around a tempting headline.
GPT-3 fits snugly into the pattern of "AI does intelligent thing -> we decide/realize thing doesn't reflect meaningful intelligence". Maybe in this case the devaluing of blog "crack" is a positive thing.
It's false that the post was generated by GPT-3. The author admitted to writing the title and editing the intro. He also described the article body this way: "as unedited as possible"—in other words, edited. It's false that (as he originally claimed) only one commenter called the post out as GPT-3, and false that all such comments were downvoted.
All that is just what he publicly admitted. How much of the rest is also fake? People who try to game HN like he did, including with bogus accounts and fake votes, are not known for scruples. It seems that, having got busted in dishonest attempts to get attention on HN, he decided to get attention from journalists instead, and found one who didn't bother to check the other side of the story.
The following is completely nonsensical, and occurs very early in the article, right after the general introduction:
"Over-Thinking (OT) is the act of trying to come up with ideas that have already been thought through by someone else. OT usually results in ideas that are impractical, impossible, or even stupid."
The only reason to keep reading after this is just to see what other bullshit has been heaped on, like not being able to take your eyes away from a train wreck.
I didn't visit the story when it appeared (regardless of its upvotes) because the very title smelled of self-helpy twaddle. I would for sure have flagged it.
That behavior may be a clue as to what happened. The submission title was crafted in such a way as to deter "nonsense-averse" users from clicking on it, and that may have helped it evade flagging. If a submission evades flags, the only other points it can get are upvotes.
I'd hate to end up in a situation where non-native speakers are accused of being bots...
The generated article follows in this vein. That’s what gpt will replicate, not the simplicity of a non native speaker (that would be easier to spot). It will follow the amorphous blob shape of saying something, but nothing, with the ominous undertone of ‘you know what’s going on, but you wouldn’t dare speak up’.
How many of you read something from a company and instantly think ‘this sounds like horseshit?’. How long did we let that go on? Forever right? We lost this fight before it even happened.
The first I heard of this was 15 years ago[0]. The original article is no longer online, so the link takes to a PDF rendition. It cites the source.
0: https://msu.edu/~pennock5/courses/ALife/Striegel_Failed_Turi...
I read the article with a motivation trying to find fault in it. Albeit being somewhat repetitive, it doesn't appear to be a piece that is inhuman.
The model even knows to demonstrate its idea with examples, which is pretty wow worthy.
This is either something written by GPT-3, or the human equivalent. Zero substantive content, pure regurgitation. and
I think this was written by GPT-3.
I think you've misrepresented the tone of those comments, and saying that the correctness of their matter-of-fact opinions is unrelated to their validity is strange to me.
It's not just that these commenters said 'this blog post is no good' but that they correctly identified its artificial nature. It's like the difference between dismissing a photo or social media profile as fake and correctly pointing out that it uses an image from thispersondoesnotexist.com.
It's so meta even this article ;)
However, my mind is blown if it really was written by an AI. As bad as it might seem by human standards, it's almost impossible for me to accept that this was created by an entity without consciousness or at least understanding.
Edit: it seems this might be a fraud; i.e., it indeed was produced by an entity with consciousness. I almost hope that’s true, as it’s much less unsettling.
The argument about overthinking vs creative thinking isn’t particularly _great_, but it’s certainly intelligible.
"What I would do with GPT-3 if I had no ethics" [8/3/20]
https://adolos.substack.com/p/what-i-would-do-with-gpt-3-if-...
> Ever since COVID hit, everyone and their mother started writing online. One of the most interesting ways people have been playing with this technology is in feeding it article headlines and introductions.
> While the output is not perfect, you can easily curate it to something that's convincing. This will make it so easy for people to just pump out clickbait articles to drive traffic.
> It would be pretty simple to do actually.
> First thing you would need to do is come up with a name. If it were me, I’d name it after the Greek god of deception or something like that just to be clever. Then I’d just stick an “A” in front so nobody gets suspicious.
> After that, I’d make a substack because it takes no time to set up. Once thats done you have to come up with some content. GPT-3 isn’t great with logic, so inspirational posts would probably be best, maybe some pieces on productivity too.
> Once you have your name, your website, and your content, its time to promote. Just start posting your articles on a website like Hacker News and a couple are bound to get popular.
Or raised...
Because people may realize that if their blog entry is going to be so bad it could be generated by GPT-3, they should probably be doing something else. And everyone else who is upvoting may just become a bit more aware what constitutes something of substance.
Things GPT-3 can't do:
- Research
- Technical Documentation
- Investigative Journalism
- Write useful software
A lot of articles that make it to the front page of HN are formulaic. We open ourselves up to this. Every blog post with shallow observations, every tutorial showcasing the first few pages of documentation, every biography on how to make money fast, every lucky shit that pontificates on how to manage teams and companies, every one selling an ebook, and everyone selling an ebook about selling an ebook after having sold 50 lifetime ebooks (topic being about success of course), and it was only a matter of time.
I count 3-4 posts about depression and existentialism per week on HN, and few ever reference the depth in which many great writers dig deep into the subject. Exercise more I guess.
Time to add ‘did a novice or a robot or a sociopathic narcissist write this?’ to our critical thinking toolbox.
Edit: I can’t tell if I fell for a gpt article about a gpt article, for what it’s worth. This is going to be a disaster when it hits the masses.
You guys saw the last ai generated text about ai generated text-right?
To me they feel like they're slightly off grammatically. Not in an ESL way, but more like someone really anxious who wants to explain a conspiracy theory to you. However i can't entirely put my finger on it. They do seem to overuse self-reflective statements (I think X) and transitionsal phrases. Maybe.