Detect ChatGPT Generated Content (opens in new tab)

At the university level, the problem of cheating is more a problem with our idea of a university. In an ideal world, you go to a university to learn. If someone wants to cheat, ultimately they're just cheating themselves (and, if at a private university or even most public universities, throwing a lot of money away). "Cheating" isn't really the university's problem in that view.

Ofc, in reality universities aren't (or not only, and in most cases not primarily) about learning, but about credentialism. Employers and various social systems outsource to universities the role of verifying that people actually know something, or are the "right" sort of person, or other signals.

I don't know how one goes about fixing that, or if it's even possible, but I'd like to see more acknowledgement of it. Fixing "cheating" feels like the equivalent of looking to clever programming to fix product bugs.

This seems to be a reductive response to a confidence based policy approach. There are many systems out there today that provide a confidence score of bots vs humans. I happen to work with a product that uses this exact approach and it's highly effective. So with education scenarios I think that the teacher/professor/administration will use these systems to inspect the entirety of the submissions. From there a baseline value will be derived and from there outliers will appear that may require deeper analysis/interview. Schools are in the position that they can't get it wrong more often than not (they need tuition dollars after all), but a severe detractor will need to be hanging over the heads of students to avoid cheating with it.

I think the other thing we'll see is a nanny state approach by some educational institutions. Falling in a trap of being sold on software to "block" or "monitor" students using these tools. It would be easy to implement on a campus network to a certain extent (correlating student network login to URL accessed and potentially MitM) but the reality is smart students will know better and will use phone hotspots and VPN. The other dark side to consider is that the owners of ChatGPT could provide logs of user accounts and queries to higher education as a service.

At the end of the day my guess is all approaches are going to be tested at some level. But the cat is out of the bag and this is going to generate some very interesting countermeasure solutions/approaches along the way.

That’s a very naïve outlook in reality like with every system the burden of proof would end up falling on the accused not the accuser…

> The problem with such things is that you would need 100% accuracy in most cases for it to be useful.

Why? If there is 75% confidence a students report was generated using ChatGPT, then that’s enough to sit down with the student and discuss the content in person and see if they actually know it. A tool such as this could help the teacher having to avoid doing this with every student, and also reinforce to students that if they don’t actually know the material there’s still a chance they get caught.

> the consequences for honest students would be too severe to actually act

Only if acting is immediately accusing them of plagiarism rather than working with the student to ensure it’s really their work.

Agreed. This is useless. I asked GPT-3 to write a HN post decrying GPTZero:

> GPTZero is absolute trash! It can't even detect plagarism correctly. It's a complete joke and waste of time! Don't bother with it! You'd be better off just manually checking for plagiarism. It's so unreliable it's not even worth mentioning. Just save yourself the time and money and don't bother with GPTZero. Worst AI tool ever!

GPTZero classified it as fully human written. This is a cat and mouse game where the mouse is always going to lose.

The GNU preamble is entirely AI written. According to this tool. To be honest I always suspected Stallman. :)

it's a toy project built by a CS undergrad student over their winter break that went viral because journalists desperately want something that can expose AI writing. It's built on streamlit and a few glued together libraries, not really surprising the results aren't good. It's not some state of the art custom tool but they are marketing themselves as the solution to solving academic plagiarism

how mad would you be if someone accused you of using AI to write something but you actually did write it

Same, ran a few samples and it was wrong every time. My hope is that teachers will run some tests before just willy-nilly trusting the scores it gives and start accusing students, but sadly I know a lot of teachers and I bet within that group few will run their own tests and will just accept it as authority.

The only way I could get it to say something was completely written by a human, was by adding spelling mistakes

Disagree, this is very much needed. The problem with AI-generated content is that it looks superficially worthwhile and plausible, but in fact it often says things that are not correct.

> it doesn’t matter if content was written by a human or by a machine

It matters a lot instead. On the top of my mind I can think about few reasons:

- if you've to train a NLP model you'd rather do with data that is not autogenerated as AI generated content is rarely used to train new models (like generating a bounce of dogs pic with dall-e and use as input for image detection may not create a precise model)

- if you pay content creators to generate content and they use ChatGPT that's definitely a breach on contract and also a problem

- many search engines (e.g. Google) already heavily penalize auto generated content

- avoid cheating at exams / officials test/certs

> Even though I disagree with a need for such tools

“Need” may be arguable, but why disagree?

Yeah what we really need is a language model that can take some text and remove everything that is void of substance, and run both human and machine writing through it.

If OpenAI implements such an endpoint, then a big chunk of potential customers would just not use ChatGPT… I mean, if I’m a student planning to use ChatGPT to enhance my uni essays, but it turns out OpenAI actually can say to my uni if my texts were AI generated, well, I’m not gonna use ChatGPT at all.

This works right up until the model is open sourced or otherwise replicable.

Put another way, it only works in today's environment.

When ML hardware is as widely distributed as classical compute, and all the models are on HuggingFace, you will be back to 0

This is a pretty good idea, at least for commercial models.

I'll just put this xkcd here - https://xkcd.com/810/ (Constructive)

Nothing, this is how adversarial training works, but it also works both ways.

If your story was ChatGPT assisted one can argue that it was GPT0 which misclassified it.

I ran your comment through this Detector, and received the following message.

Your text is likely to be written entirely by AI

False positives and all.

Disclosure: The tool is built by me. It uses 6 distinct methods to classify text with a 93% accuracy rate, based on testing a dataset of 100k+ samples.

This is an everlasting battle. Schools are already using these kind of tools to detect cheating, often with false positives that may have devastating outcome.

AI is so good that even human written text is indistinguishable from AI.

Trivial to break by making tiny changes.

Conversely, I wrote a few sentences in simple English off the top of my head (about The Sound of Silence, because they were talking about it on the radio this morning) and it said it was "likely to be written entirely by AI".

False positives in a plagiarism tool are pretty bad IMO. It should definitely skew towards "we can't be certain" rather than "definitely AI".

You can easily build it with gpt3 api. And codex can help you if you don't know how to program :)

I also wanted to train my own GPT detection model but unfortunately there isn't any publicly available dataset about outputs.