A CompSci Student Built an App That Can Detect ChatGPT-Generated Text

[highplainsdem]

I missed this story - which IMO is very good news - when it broke the other day. Heard about it earlier this morning thanks to CBS Mornings' Talk Of The Table segment.

https://www.vice.com/amp/en/article/3admg8/a-compsci-student-built-an-app-that-can-detect-chatgpt-generated-text

Since the emergence of ChatGPT, a chatbot that can generate realistic text based on different prompts that a user enters, teachers and educators have been worried about the possibilities of cheating that machine learning helps enable. To combat AI plagiarism and keep the tool in check, Princeton computer science and journalism student Edward Tian spent his winter break building GPTZero, an app that attempts to detect whether or not a body of text was human-written or AI-written.

-snip-

Tian said that GPTZero was wrong less than 2 percent of the time when he tested the app on a dataset of BBC news articles, as well as machine-generated articles with the same prompt. Motherboard also tried the app using paragraphs from this article, which the app deemed were human-written, due to high “perplexity” levels.

OpenAI has acknowledged the fears around using ChatGPT to cheat. An OpenAI guest researcher Scott Aaronson said at a December lecture that the company was working on creating watermarks for the outputs so that people could see signs of a machine-generated text. This week, New York City’s education department has banned access to ChatGPT, out of concern for “safety and accuracy.”

Tian acknowledged that while there are benefits to using ChatGPT, there are a lot of aspects of human writing that machine learning cannot replicate. “Human writing can be so beautiful. There is beauty in the human prose that computers can never and should never co-opt,” he said.

Editing to link to another article about GPTZero, and a couple of threads I posted recently about ChatGPT.

The article is from Gizmodo - https://gizmodo.com/chatgpt-ai-essay-detector-college-princeton-edward-tian-1849946535 - and I particularly want to emphasize the final paragraph:

GPTZero is pretty cool—though it just goes to show that, in our dystopian present, not only are machines writing stuff for us, but they’re also telling us whether they wrote it or not. Will humans maintain any cognitive abilities in the future or are robots going to do literally all of our thinking for us? Things seem pretty grim for our future IQs.

OP that I posted a couple of weeks ago here in GD about AI being used - IMO misused - for writing fiction:

https://www.democraticunderground.com/100217495259

And another OP, cross-posted to Music Appreciation and the Lounge, about the effect of computerization including ChatGPT on music ( the video posted shows ChatGPT told to write song lyrics like Ed Sheeran's):

https://www.democraticunderground.com/103490480

https://www.democraticunderground.com/10181757981

See the replies in both. Especially the second link there, the thread in the Lounge, with one reply about a short film scripted and directed by ChatGPT.

[highplainsdem]

1. Kicking after adding to the OP

[Javaman]

2. Bots fighting bots. This the world of social media now.

I weep for our future

[Renew Deal]

5. It's more than just social media.

It's a lot of the computing world, especially in security.

[Javaman]

6. Oh I'm aware of that. Just commenting on the nature of our weird social fabric now.nt

[dalton99a]

3. Kick

[Renew Deal]

4. I have doubts about "watermarks"

Post the text in something like Microsoft's Notepad and the watermarks are either lost or become visible.

[ecstatic]

9. I'm thinking by watermarks they mean code words or something

Or maybe they'll inject some kind of syntax or punctuation pattern that is invisible to people but obvious to machines.

[Renew Deal]

10. That's my point

A simple text editor will not show hidden text and I believe it will strip fonts. The person could see the plain text.

[ecstatic]

11. To clarify, I mean a pattern hidden in plain sight.

It's physically there in the text but the pattern is too subtle or complex for non-machines to detect.

For example, a generated article that uses a word with an apostrophe on every 7th sentence in the 3rd position.

Or to detect computer science cheating, a generated computer/programming code that consistently adds in an unnecessary variable declaration or something like that. A slacker student might not notice, but a machine would.

[ecstatic]

7. Interesting, positive development.

I've tested the waters with openAI to generate articles / content for websites. It's not perfect and can sometimes be tedious, but it saves a lot of time. When I saw how good it was, the first thing I wondered about was the future of education. Would students ever have to write their own essays again?

I'm not surprised that a tool could be built to detect it because AI generated articles have a very cliche / superficial / generic/ formulaic feel. It's okay for random articles on the internet but not for everyday life. Some might object and ask why do students need to write essays anyway--in MY opinion, it's important for critical thinking and brain development.

[FakeNoose]

8. If teachers could have one-on-one facetime with their students

... I think that would reduce the chances of being fooled by students' using ChatGPT to cheat.

One of the aftereffects of the Covid pandemic has been that teachers are getting way less time to become familiar with their students. Many times the students are only faces on a computer screen, with text messages taking the place of real conversations. In a better, safer world where everyone is vaccinated and avoiding infections, I think teachers wouldn't need this kind of tool.

It makes me sad that it has come to this ...

[highplainsdem]

13. I'm sure that could help in some cases, but with introverted students in

particular, or kids who aren't introverted but like to clown around, it could be very hard to judge how well they might be able to write by face-to-face interactions.

[Hekate]

12. Kick and Rec

[andym]

14. Soon such detective apps will have a lot more trouble

Chatting programs will improve, injecting more complexity (and randomness) into their text. They will use the very same detective tools to drive improvement, the same way biological viruses evolve to defeat prior immune responses.