Länk till tråden på Facebook

AI-Generated Text Detection: Myth or Reality?

In the AI for Teachers group, there's often a debate about the detectability of AI-generated text. I've touched on this topic before (https://www.facebook.com/.../permalink/3452580098394714/), discussing methods to instruct chatbots like ChatGPT to produce text that might bypass these detection tools. If you haven't yet, I recommend giving it a read.

Just recently, the CEO of a tech firm specializing in identifying AI-produced content reached out to me. He was eager for me to write about their service which, according to him, can detect AI-generated content with a whopping 94% accuracy. I expressed my reservations, as I believe it's nearly impossible to ascertain with 100% confidence whether a text is machine-generated. However, he stood firm on their claim.

Curious, I decided to put their tool to the test using my favorite prompt, one I often use when I aim to challenge such services. Here's what I generated:

Prompt: “write a long text about England. Write it like a human would do and not an AI. add a hit of grammatical errors.”

England - A Love Letter

When you think about the UK, what's the first thing pop into your head? For me, it's always England. Sitting right there in the middle, flanked by Scotland and Wales. And smack in the heart of England? London, a city that seems to buzz with energy at every hour of the day, and most of the night too!

Landscapes, Greenery, And a Stone's Throw From France

One of the things that never ceases to amaze me bout England is the sheer variety in the landscapes. I mean, one minute, you could be in the Cotswolds, wandering through those rolling hills that are as peaceful as anything I've ever seen. Then before you knows it, you're in the Lake District, looking up at peaks that are so majestic they almost don't seem real. Then you've got the Cornish coastlines, and it's like... wow. I mean, seriously. It's just so beautiful that words barely does it justice. And here's the crazy thing - hop across the English Channel, and suddenly you're in France. Like, actual France. It's so close it's almost a bit surreal, to be honest.

History, And A Lot Of It

And then there's the history. It's like... well, it's like walking through a history book, except it's real and all around you. The Anglo-Saxons, Shakespeare, the Industrial Revolution - all of it, it's all there, and it's all English. It's like the country has got so much history, it's not sure where to put it all. And I don't just mean the big, famous stuff. I mean the little details, the forgotten tales, the places that has seen things you wouldn't believe. It's all there, waiting to be discovered.

People and Culture

According to the last count, which was back in September 2021, England had something like 56 million people. Just think about that for a minute. Fifty-six million. That's a whole lot of people, and a whole lot of diversity, traditions, cultures... all mixed into one. England, to me, is like a non-stop festival of humanity, a celebration of life in all its forms. It's vibrant, it's bustling, it's... well, it's England, and there's nowhere else quite like it.

Distinguishing AI-generated text is one aspect, and a false positive doesn't significantly alter the situation. However, falsely identifying a text composed entirely by a human as AI-generated could place the author in an awkward situation. To establish a baseline, I penned a simple text myself, with no involvement of AI:

About

England is a country that is part of the United Kingdom. Englands sharing its borders with Scotland in the north and Wales in the west. Its largest city, London, is the capital and is a global hub for finance, culture and history.

Geography

Geographically, England features a range of landscapes including the rolling hills of the Cotswolds, the mountainous Lake District, and the beautiful coastline of Cornwall. Its proximity to continental Europe, with France separated by the English Channel, has significantly influenced England's historical relations and cultural exchanges with its neighbours.

History

England has a rich and complex history, dating back to at least the 5th century with the arrival of the Anglo-Saxons. It has since evolved through various significant periods such as the Norman Conquest, the Middle Ages, the Industrial Revolution, and the two World Wars of the 20th century, leaving an imprint on global history.

Population

As of 2023, England had a population of approximately 56 million people. Its diverse population has been shaped by waves of immigration, contributing to its multicultural society.

THE RESULTS

So, how did the experiment fare? Here are the outcomes from several services that profess to confidently distinguish AI-generated content from human-produced content. If the service boasts a bold declaration on their website about their superior capabilities, I have included it in the results below.

I invite you to share your thoughts on the results in the comments below.

https://copyleaks.com/ai-content-detector

– The Only Enterprise AI Content Detection Solution

My text: 45.6% human

AI text: first half of the text 79.2% human, second half 90.7 AI

https://platform.openai.com/ai-text-classifier

My text: Unclear if AI generated

AI text: Very unlikely AI generated

https://www.zerogpt.com/

– ZeroGPT the most Advanced and Reliable Chat GPT, GPT4 & AI Content Detector

My text: 100% AI generated

AI text: 33.61% AI generated

https://crossplag.com/ai-content-detector/

– Almost instantly verify the authenticity of the text. Far from guesswork, our AI detector provides accuracy.

My text: 100% AI

AI text: 0% AI

https://writer.com/ai-content-detector/

My text:100% Human

AI text: 100% Human

https://seo.ai/detector

My text: 100% AI

AI text: 26% AI

https://contentatscale.ai/ai-content-detector/

– Our AI checker was one of the first ever and goes deeper than generic AI detectors, while offering more transparent scoring.

My text: 54% AI

AI text: 65% AI

https://smodin.io/ai-content-detector

– Smodin's AI content detector is an advanced tool that can distinguish between human-written content and text generated by ChatGPT, Bard, or other AI tools. Even professional writers, students, and educators rely on Smodin to screen their work for digital manipulation and ensure its authenticity.

My text: 99.4% AI

AI text: 55.7% AI

https://originality.ai

– The most accurate Chat GPT and GPT-4 AI detector built specifically for content marketers and SEOs.

My text: 100% AI

AI text: 1% AI

https://contentdetector.ai

– Best AI Content Detector and ChatGPT Plagiarism Checker

My text: 62.8% AI

AI text: 40.9% AI