Embedded Files
The Surveillance Squad
The Surveillance Squad
What do the detectors look for?
Linguistic Patterns: Repetition, word usage, absence of typos.
Predictability: AI text is generated based on probabilities, leading to structured or repetitive outputs.
Stylistic Traits: Consistent sentence lengths, fewer grammatical errors, lack of anecdotes.
Entropy: AI-generated text may be overly predictable or too random, depending on the model.
Semantic Coherence: Logical flow and depth of arguments.
Text Metrics: Sentence length, punctuation use, parts of speech.
Metadata: Timestamps, software used, and document history.
AI Detectors are NOT Perfect!
AI Detectors are NOT Perfect!
Is the Bible AI?
Is the Bible AI?
Is the U.S. Constitution AI?
Is the U.S. Constitution AI?
Who are you really catching?
Who are you really catching?
ESL learner, English is not spoken at home
Uses free AI models to translate assignment instructions
Writes the assignment himself, but uses free AI models to translate his work to English
Low socioeconomic background with limited access to technology
Low digital literacy
Writes the assignment by herself, but uses a free AI model to make her writing sound more "academic"
Native English speaker, wealthy household, parents work in tech
Has access to several advanced subscription AI models
Uses AI to write entire assignments
“Detector-proofs” her work by using adversarial techniques and multiple AI models
GenAI detection tools privilege students who are English first language, have access to paid large language models/applications, and are more digitally literate. -Leon Furze
https://leonfurze.com/2024/04/09/ai-detection-in-education-is-a-dead-end/ GenAI detection tools privilege students who are English first language, have access to paid large language models/applications, and are more digitally literate. -Leon Furze
AI Detectors are Biased Against Non-Native English Writers
AI Detectors are Biased Against Non-Native English Writers
TOEFL = Test of English as a Foreign Language
Dark bars: % of non-native English speaker essays misclassified as AI
Light bars: % of native English speakers essays misclassified as AI
Using AI to Outsmart AI-Detectors
Using AI to Outsmart AI-Detectors
It’s easy to circumvent AI-detection by using adversarial techniques.
It’s easy to circumvent AI-detection by using adversarial techniques.
Adversarial techniques are ways of prompting AI to write in a way that is not detectable to AI-detectors.
Instructions in the prompt to vary sentence structure
Instructions to incorporate deliberate errors and make the outputs more human-like
Ping-ponging from one model to another, laundering the outputs
Using more powerful models when the limitations of a specific tool are known
Creating system prompts designed to circumvent as many points of detection/prediction as possible
"AI paraphrasing tools, such as Quillbot AI, utilize artificial intelligence algorithms to rewrite and rephrase text, providing an alternative version while retaining the original meaning"
https://www.b12.io/resource-center/ai-how-to-guides/what-does-an-ai-paraphrasing-tool-do.html
Next: Set Expectations
Page updated
Google Sites
Report abuse