Real Or Fake Text
An online game called Real Or Fake Text* where participants are fed a 8 sentences one line at a time and asked to guess whether each line was written by a human or generated by a computer. While most will start out human-written, at some point the computer version will take over. When you guess "this sentence is computer generated" you are prompted to explain your reasoning.
*Real Or Fake Text was created as part of a research study:
Dugan, L., Ippolito, D., Kirubarajan, A., Shi, S., & Callison-Burch, C. (2022). Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2212.12672
The purpose
Test our assumptions about how accurately we can distinguish between human and computer generated text.
Reflect on what factors prompt us to identify text as computer generated.
Highlight that some computer generated text is more subtle and easy to miss.
What you need
Setting it up
Participants need the link to the Real Or Fake Text game and a personal device.
How long does it take?
We suggest 5 minutes individual game play and 3-5 minutes group discussion.
How it works
Participants individually open Real Or Fake Text game using a personal device.
They choose a category - Short Stories, Recipes, New York Times, Presidential Speeches or Random.
They are shown an initial human written sentence and a second sentence and asked to identify whether the second was written by a human or a computer.
If they choose "this sentence is computer generated" they are asked to suggest why (eg. grammar).
They are then told if they were correct or incorrect and will be given another sentence until they have either read all 8 or have correctly identified a computer generated sentence.
Suggested follow-up
After 5 minutes of playing the game bring everyone back for a Think-Pair-Share or group discussion.
Ask participants to reflect on what helped them to identify computer generated text and how this differs from identifying unethical Gen AI use among students.
Where it works well
Within small groups exploring the impact of Gen AI in Higher Education
As part of marking calibration sessions as a fun ice-breaker before a discussion on a shared approach suspected inappropriate use of Gen AI
What to watch out for
As a short game it does not fully represent the experience of faculty undertaking marking of assessments where the whole piece can be taken into consideration, not just a single line.
Authorship
This entry was written by Maria O'Hara.