Kurzgesagt – In a Nutshell
Kurzgesagt – In a Nutshell
Embedded Files
Sources – AI Slop
Sources – AI Slop
– AI Slop is saturating the internet and things are becoming dramatic pretty quickly.
Artificial intelligence is an umbrella term for many different types of computational systems. In this video we focus on so-called “AI Slop”: low-quality media (text, sound, images and video) largely produced using a branch of AI called “generative AI” (or GenAI). Unless otherwise specified, when we talk about AI here we are talking about GenAI. Generative AI creates new, synthetic content based on training data. This is in contrast to non-generative forms of AI that typically use a narrow set of inputs to produce a pre-specified output (e.g. face recognition, music recommendations, computers playing chess).
There have been some attempts at estimating the fraction of AI-generated content on various parts of the internet. While there are no reliable estimates for the fraction of AI content of the whole internet, it is clear that the absolute amount of AI-generated content is rapidly growing every day. Below, we have linked some selected studies.
A recent study (not yet peer reviewed) estimates that for the social media platforms Medium and Quora, up to ca. 40% of new content posted appears to be AI-generated. For Reddit, the estimate is 2.5%.
#Sun Z, Zhang Z, Shen X, et al. Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media. arXiv (2025)
https://arxiv.org/abs/2412.18148
Quote: “Social media platforms are experiencing a growing presence of AI-Generated Texts (AIGTs). However, the misuse of AIGTs could have profound implications for public opinion, such as spreading misinformation and manipulating narratives. Despite its importance, it remains unclear how prevalent AIGTs are on social media. To address this gap, this paper aims to quantify and monitor the AIGTs on online social media platforms. We first collect a dataset (SM-D) with around 2.4M posts from 3 major social media platforms: Medium, Quora, and Reddit. Then, we construct a diverse dataset (AIGTBench) to train and evaluate AIGT detectors. AIGTBench combines popular open-source datasets and our AIGT datasets generated from social media texts by 12 LLMs, serving as a benchmark for evaluating mainstream detectors. With this setup, we identify the best-performing detector (OSM-Det). We then apply OSM-Det to SM-D to track AIGTs across social media platforms from January 2022 to October 2024, using the AI Attribution Rate (AAR) as the metric. Specifically, Medium and Quora exhibit marked increases in AAR, rising from 1.77% to 37.03% and 2.06% to 38.95%, respectively. In contrast, Reddit shows slower growth, with AAR increasing from 1.31% to 2.45% over the same period. Our further analysis indicates that AIGTs on social media differ from human-written texts across several dimensions, including linguistic patterns, topic distributions, engagement levels, and the follower distribution of authors. We envision our analysis and findings on AIGTs in social media can shed light on future research in this domain.”
A recent analysis by an SEO company found that a large share of English-language websites showed signs of “substantial” or “dominant” AI use.
#Law R, Guan X, Soulo T. 74% of New Webpages Include AI Content (Study of 900k Pages). Ahrefs Blog. 2025
https://ahrefs.com/blog/what-percentage-of-new-content-is-ai-generated/
Quote: “We used bot_or_not to analyze 900,000 English-language web pages that were newly detected by our web crawler in April 2025. We analyzed one page per domain (so we tested content from 900,000 different domains). Each page was categorized according to the percentage of the page our model detected as being AI-generated.”
A recent analysis by researchers affiliated with Amazon Web Services (a cloud computing platform) found that 57.1% of all web-based text that was translated at least once, was translated into 3+ languages, and is likely a result of poor-quality machine translations. The findings have not been peer-reviewed.
#Thompson B, Dhaliwal M, Frisch P, et al. A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism. Findings of the Association for Computational Linguistics. 2024
https://aclanthology.org/2024.findings-acl.103/
Quote: “We show that content on the web is often translated into many languages, and the low quality of these multi-way translations indicates they were likely created using Machine Translation (MT). Multi-way parallel, machine generated content not only dominates the translations in lower resource languages; it also constitutes a large fraction of the total web content in those languages. We also find evidence of a selection bias in the type of content which is translated into many languages, consistent with low quality English content being translated en masse into many lower resource languages, via MT. Our work raises serious concerns about training models such as multilingual large language models on both monolingual and bilingual data scraped from the web.
[...]
Of the 6.38B sentences in our 2.19B translation tuples, 3.63B (57.1%) are in multi-way parallel6 (3+ languages) tuples: see Table 1. [...]
– Today about half of internet traffic is bots, the majority of them are used for destructive purposes.
“Bots” are software applications that automatically perform tasks online, and are often designed to mimic human behavior. The “Bad Bot Report” published by a cybersecurity firm is a commonly cited source for the fraction of internet traffic created by bots. It is based on a large dataset of blocked bot requests across thousands of different websites and industries, and is considered reliable by many news outlets and platforms. However, it is important to acknowledge that the report is published by a private cybersecurity firm with an interest in selling anti-bot software, and their analysis may therefore have limitations and/or biases.
#Imperva. 2025 Bad Bot Report. Retrieved July 2025
https://www.imperva.com/resources/resource-library/reports/2025-bad-bot-report/
Quote: “Our analysis draws from data collected from across the Imperva global network in 2024, including the blocking of 13 trillion bad bot requests across thousands of domains and industries. This dataset provides key insights into bot activity to help organizations understand and address the growing risks of automated attacks. This year’s report focuses on the growing role of Artificial Intelligence (AI) in bot attacks, significantly increasing their volume, accessibility, and ability to evade detection. Bad bots are increasingly targeting businesses through tactics like data scraping, account hijacking, and inventory manipulation for financial gain. As AI evolves, organizations must adopt advanced mitigation strategies to protect against fraud, financial losses, and security risks.”
– It was never easier to make mediocre content – from the black hole of meaninglessness that is LinkedIN, low effort short videos just engaging enough to hypnotise kids and fry their attention spans, to endless soullessly rewritten books on amazon.
Mediocre or low-effort content is as old as the internet itself and thus of course long precedes the advent of artificial intelligence. Online videos, especially on social media platforms, have also been getting shorter even before the widespread use of AI, due to the rising popularity of short-form content platforms like TikTok, Youtube Shorts and Instagram Reels. But various AI tools have made it much easier and faster to mass-produce low-quality content, which has lowered the barrier of entry for many people seeking financial gains. AI-generated, low-quality slop is therefore, in a sense, “the new spam”, and has appeared on pretty much every online platform.
#Gillham J. Over ½ of Long Posts on LinkedIn are Likely AI-Generated Since ChatGPT Launched. Originality.AI (2024)
https://originality.ai/blog/ai-content-published-linkedin
Quote: “Our process was to analyze 8,795 LinkedIn long-form posts using our AI detector to determine if the post was likely AI-generated or likely human-written. The study spanned content published over an 82-month period from January 2018 to October 2024, which we analyzed to identify trends in AI usage. These long-form posts contain at least 100 words and were curated from different LinkedIn users on the platform. [...] As of October 2024, 54% of long-form posts on LinkedIn are estimated to be AI-generated.”
#Al-Sibai N and Christian J. BuzzFeed Is Quietly Publishing Whole AI-Generated Articles, Not Just Quizzes. Futurism. 2023
https://futurism.com/buzzfeed-publishing-articles-by-ai
Quote: “This month, we noticed that with none of the fanfare of Peretti's multiple interviews about the quizzes, BuzzFeed quietly started publishing fully AI-generated articles that are produced by non-editorial staff — and they sound a lot like the content mill model that Peretti had promised to avoid.
The 40 or so articles, all of which appear to be SEO-driven travel guides, are comically bland and similar to one another.”
#Brodsky S. The AI-generated Books Trend is Getting Worse. AI Business. 2024
https://aibusiness.com/responsible-ai/the-ai-generated-books-phenomenon-is-getting-worse
Quote: “The AI-generated book problem has gotten so bad that Amazon has implemented a new policy for Kindle authors, restricting them to self-publishing a maximum of three books per day on its platform.
The influx of AI-generated books on Amazon's marketplace is creating major obstacles for human authors, experts say. It is also making it harder to distinguish between authentic authors and pseudonyms created by AI.”
#Oremus W. He wrote a book on a rare subject. Then a ChatGPT replica appeared on Amazon. The Washington Post. 2025
https://www.washingtonpost.com/technology/2023/05/05/ai-spam-websites-books-chatgpt/
Quote: “The book, titled “Automating DevOps with GitLab CI/CD Pipelines,” just like Cowell’s, listed as its author one Marie Karpos, whom Cowell had never heard of. When he looked her up online, he found literally nothing — no trace. That’s when he started getting suspicious. The book bears signs that it was written largely or entirely by an artificial intelligence language model, using software such as OpenAI’s ChatGPT. (For instance, its code snippets look like ChatGPT screenshots.) And it’s not the only one. The book’s publisher, a Mumbai-based education technology firm called inKstall, listed dozens of books on Amazon on similarly technical topics, each with a different author, an unusual set of disclaimers and matching five-star Amazon reviews from the same handful of India-based reviewers.
[...]
AI-written books aren’t against Amazon’s rules, per se, and some authors have been open about using ChatGPT to write books sold on the site.”
– AI music is invading streaming platforms.
AI can be used in different ways in the context of music creation. Some AI tools are rather subtle and can serve composers, musicians and producers in the production process. But in the most extreme cases, GenAI creates new “synthetic” music completely from scratch, trained on huge databases of human-made music. This is what we are referring to here.
#Bakare L. An AI-generated band got 1m plays on Spotify. Now music insiders say listeners should be warned. The Guardian. 2025
Quote: “They went viral, amassing more than 1m streams on Spotify in a matter of weeks, but it later emerged that hot new band the Velvet Sundown were AI-generated – right down to their music, promotional images and backstory. The episode has triggered a debate about authenticity, with music industry insiders saying streaming sites should be legally obliged to tag music created by AI-generated acts so consumers can make informed decisions about what they are listening to.”
#Mullen M. “If creators are using these technologies... we should let people listen to them": Spotify co-president says AI-generated music is welcome on the streaming platform. MusicRadar. 2024
Quote: “Kantowitz suggests that Spotify could eventually fill up with songs that are entirely AI-generated, in the same way that AI-generated images are fast becoming some of the most popular content on social media platform Meta. "If these songs are generated by AI music generators become engaging, and they follow the rules, is that good for Spotify?" he asks.
Söderström's view is that AI-generated music, as long as it's created in such a way that it doesn't violate copyright law, should be welcomed on the platform. "If creators are using these technologies — where they are creating music in a legal way that we reimburse and people listen to them — and are successful, we should let people listen to them.”
#McEvoy C. How AI-generated songs are fueling the rise of streaming farms. World Intellectual Property (WIPO) Magazine. 2025
Quote: “The recent case of North Carolina musician Michael Smith is emblematic of this new form of artificial streaming fraud. Smith allegedly extracted more than US$10 million in royalty payments from a host of streaming platforms by uploading hundreds of thousands of AI-generated songs and using bots to play each one a smaller number of times.
Bad actors are using AI not only to generate audio content but also to create and manage the bots used to stream the content. There are even businesses that boldly advertise streaming fraud as a service, highlighting their use of AI to spoof digital identities en masse and “bypass anti-fraud systems” employed by the likes of Spotify, Apple Music and Deezer. Companies pushing the use of bots frame streaming fraud as a valid way for musicians to grow their brand but conspicuously avoid any mention of the damage it causes across the music industry.”
– Google AI is summarizing websites instead of sending traffic to them.
“AI Overview” is a Google feature that uses AI to summarize information found on different websites related to the search query. Because more and more people are satisfied with the answers provided by this AI summary (whether accurate or not), they stop searching and don’t end up visiting the websites that provided the “material” for the AI. This ultimately reduces traffic to those websites.
#Rapp K. No more SEO? How Google's new AI features affect website optimisation. IBM iX. 2024
https://ibmix.de/en/blog/seo-google-ai-overviews
Quote: “Google AI Overviews pose a high risk for traffic loss, particularly with websites that focus purely on information. If an AI Overview already displays all the relevant information in the search results, then users may no longer have a reason to visit individual websites. This can be particularly problematic for websites with short and informative content. For large news websites, fewer visits mean a significant loss of revenue from online adverts.”
#Melton W. Google’s AI Mode Just Changed Everything. Here’s What Businesses Need to Know. Xponent21. 2025
Quote: “1. Expect Even Fewer Clicks.
With Google AI Mode delivering summarized answers directly in search, users no longer need to click through to individual websites to find what they’re looking for. This shift dramatically reduces traditional organic traffic—especially for businesses that aren’t cited as sources within those summaries. If your SEO strategy has historically depended on high-ranking links to drive conversions, you’re now competing in a compressed field where only the most authoritative, well-structured, and frequently updated content has a shot at visibility.”
#Sommerfeld N, McCurry M, Harrington D. Goodbye Clicks, Hello AI: Zero-Click Search Redefines Marketing. Bain & Company. 2025
https://www.bain.com/insights/goodbye-clicks-hello-ai-zero-click-search-redefines-marketing/
Quote: “Now, the rise of AI search engines and generative summaries has upended traditional search behavior, delivering answers directly on results pages and removing the need for users to click through to another site. Bain’s recent survey finds that about 80% of consumers now rely on “zero-click” results in at least 40% of their searches, reducing organic web traffic by an estimated 15% to 25% (see Figure 1). The speed of this upheaval leaves marketers with an urgent question: How do we engage consumers when clicks and site visits are disappearing?”
– And sadly, actual creative human work is used to train these AI models. Every reddit comment, original youtube video or human drawing on deviant art has been sold out to the AI companies. Or straight up stolen by them. Without attribution or payment to the actual creators.
#Niemeyer K. Reddit comments are 'foundational' to training AI models, COO says. Business Insider. 2025
Quote: “Reddit COO Jen Wong says those investments are paying off. "AI itself, more broadly, is incredibly important to everything we're doing," Wong told AdExchanger at the CES technology conference. Wong added that Reddit is now "foundational to the training" of large language models. In February, Reddit signed a licensing deal with Google to train Google's AI using Reddit content for $60 million a year. Then, in May, Reddit signed another massive content data-sharing deal with ChatGPT-maker OpenAI to train its AI models. Reddit CEO Steve Huffman said the company is in talks with "just about everybody" when asked if Reddit would consider working with Microsoft during The Wall Street Journal's Tech Live event in October. Huffman said Reddit posts and comments contain a wealth of "colloquial words about pretty much every topic" that are constantly updated, making them valuable in teaching machines how to think and speak like humans.”
#Vallese Z. Creators say they didn’t know Google uses YouTube to train AI. CNBC. 2025
https://www.cnbc.com/2025/06/19/google-youtube-ai-training-veo-3.html
Quote: “Google is using its expansive library of YouTube videos to train its artificial intelligence models, including Gemini and the Veo 3 video and audio generator, CNBC has learned. The tech company is turning to its catalog of 20 billion YouTube videos to train these new-age AI tools, according to a person who was not authorized to speak publicly about the matter. Google confirmed to CNBC that it relies on its vault of YouTube videos to train its AI models, but the company said it only uses a subset of its videos for the training and that it honors specific agreements with creators and media companies. “We’ve always used YouTube content to make our products better, and this hasn’t changed with the advent of AI,” said a YouTube spokesperson in a statement. “We also recognize the need for guardrails, which is why we’ve invested in robust protections that allow creators to protect their image and likeness in the AI era — something we’re committed to continuing.” Such use of YouTube videos has the potential to lead to an intellectual property crisis for creators and media companies, experts said.”
#Pahwa N. The Tragic Downfall of the Internet’s Art Gallery. Slate. 2024
https://slate.com/technology/2024/05/deviantart-what-happened-ai-decline-lawsuit-stability.html
Quote: “As VFX animator Romain Revert (Minions, The Lorax) pointed out on X, the bots had come for his old home base of DeviantArt. Its social accounts were promoting “top sellers” on the platform, with usernames like “Isaris-AI” and “Mikonotai,” who reportedly made tens of thousands of dollars through bulk sales of autogenerated, dead-eyed 3D avatars. The sales weren’t exactly legit—an online artist known as WyerframeZ looked at those users’ followers and found pages of profiles with repeated names, overlapping biographies and account-creation dates, and zero creations of their own, making it apparent that various bots were involved in these “purchases.” It’s not unlikely, as WyerframeZ surmised, that someone constructed a low-effort bot network that could hold up a self-perpetuating money-embezzlement scheme: Generate a bunch of free images and accounts, have them buy and boost one another in perpetuity, inflate metrics so that the “art” gets boosted by DeviantArt and reaches real humans, then watch the money pile up from DeviantArt revenue-sharing programs.”
#Brodsky S. The AI-generated Books Trend is Getting Worse. AI Business. 2024
https://aibusiness.com/responsible-ai/the-ai-generated-books-phenomenon-is-getting-worse
Quote: “The influx of AI-generated books on Amazon's marketplace is creating major obstacles for human authors, experts say. It is also making it harder to distinguish between authentic authors and pseudonyms created by AI.
"Whoever is doing this is obviously preying on writers who trust my name and think I have actually written these books. I have not. Most likely they have been generated by AI," author Jane Friedman said in a blog post after finding fake books supposedly written by her on Goodreads. "When I complained about this on Twitter/X, an author responded that she had to report 29 illegitimate books in just the last week alone. 29!
[...]
“Many people are already unwilling to pay standard market prices for an ebook published by a relatively unknown author, compared to the prices routinely shelled out for ‘A-List’ writers who can sell an ebook for around $8.99 and up depending on genre, with technical books often in the $35 to $45 range,” Jacobs added.
“A large influx of low-quality ebooks may or may not affect the A-Listers, but will undoubtedly put downward pressure on B-List writers who often have to price their ebooks at $2.99 to $19.99, again, depending on genre, for readers to take a chance on them.””
#Gagner J. Deviant Art? Use of Copyrighted Material in AI Image Generation. Journal of High Technology Law. 2023
Quote: “A complaint filed on January 13th in the United States District Court for the Northern District of California seeks to challenge the originality of these AI generated images. The suit was brought by three artists claiming that Stability AI, Midjourney, and DeviantArt have violated copyright laws by developing AI image generation software that uses copyrighted images to produce derivative, rather than original, works. According to the complaint, “the rapid success of Stable Diffusion,” the software underlying all three image generation programs, “has been partly reliant on a great leap forward in computer science, [but] even more reliant on a great leap forward in appropriating copyrighted images.” The complaint alleges that by scraping copyrighted images from the internet without permission; copying, compressing, and storing those images in training data; and using a “modern day collage tool” to assemble new images from that data, all works produced by the image generators are derivatives of the copyrighted works in the training data.”
– Creative theft on a scale where it is impossible to protect against it, already putting loads of creatives’ work in danger – so AI companies can get rich.
While there is no strong evidence of large-scale disruption of labor markets across white-collar industries, there are signs that creative professions might experience some serious hits on the job market in the next few years. Because generative AI is still relatively new technology and the impact on job markets is evolving very quickly, it is difficult to find accurate estimates on how many jobs in which creative professions have already been impacted negatively. But it is clear that the creative industries are being completely restructured at the moment, and many people in creative jobs report losing work due to AI. Predictions by economic experts and surveys among hiring managers suggest that the greatest impacts will likely be felt in the next few years.
#SoA Policy Team. SoA survey reveals a third of translators and quarter of illustrators losing work to AI. The Society of Authors. 2024
Quote: “Concerns about the impact of generative AI on creative careers included groups of authors who are already experiencing loss of work, or the devaluation of their work, as a direct result of new technologies.
A quarter of illustrators (26%) and over a third of translators (36%) have already lost work due to generative AI.
Over a third of illustrators (37%) and over 4 in 10 translators (43%) say the income from their work has decreased in value because of generative AI.”
#International Confederation of Societies of Authors and Composers (CISAC). Global economic study shows human creators’ future at risk from generative AI. 2024
Quote: “While the revenues of Gen AI providers will see dramatic growth over the next five years, creators risk losing a large share of their current income due to AI’s substitutional impact on human-made works. Despite providing the creative fuel of the “Gen AI” content market, music and audiovisual creators will see respectively 24% and 21% of their revenues at risk of loss by 2028. This amounts to a cumulative loss of €22 billion over the 5-year period (€10 billion in music; €12 billion in audiovisual).”
#Teutloff O, Einsiedler J, Kässi O, Braesemann F, et al. Winners and losers of generative AI: Early Evidence of Shifts in Freelancer Demand. Journal of Economic Behavior and Organization. 2025
https://doi.org/10.1016/j.jebo.2024.106845
Quote: “We examine how ChatGPT has changed the demand for freelancers in jobs where generative AI tools can act as substitutes or complements to human labor. Using BERTopic we partition job postings from a leading online freelancing platform into 116 fine-grained skill clusters and with GPT-4o we classify them as substitutable, complementary or unaffected by LLMs. Our analysis reveals that labor demand increased after the launch of ChatGPT, but only in skill clusters that were complementary to or unaffected by the AI tool. In contrast, demand for substitutable skills, such as writing and translation, decreased by 20–50% relative to the counterfactual trend, with the sharpest decline observed for short-term (1-3 week) jobs. Within complementary skill clusters, the results are mixed: demand for machine learning programming grew by 24%, and demand for AI-powered chatbot development nearly tripled, while demand for novice workers declined in general. This result suggests a shift toward more specialized expertise for freelancers rather than uniform growth across all complementary areas.”
#McEvoy C. How AI-generated songs are fueling the rise of streaming farms. World Intellectual Property (WIPO) Magazine. 2025
Quote: “The most obvious and direct harm is financial. Streaming platforms have a finite revenue pool from which they can pay royalties and every time a bad actor successfully extracts fraudulent payments, there is less revenue to share with artists, labels and publishers.
[...]
“Every point of market share is worth a couple hundred million US dollars today,” Hayduk tells WIPO Magazine. “So we’re talking about a billion dollars minimum – that’s a billion dollars being taken out of a finite pool of royalties and everyone in the value chain losing out on a material amount of revenue on an annual basis.”
[...]
As David Sandler, Warner Music Group’s Vice President of Global Content Protection, put it at the panel: “[Streaming fraud] is impacting artists you’ve never heard of because we don’t have a chance to bring them to market. Our company invests a tremendous amount of money, time and energy in discovering new artists, signing new artists and developing their careers. Every dollar we spend to fight fraud is a dollar we can’t spend discovering new artists.””
#World Economic Forum. The Future of Jobs Report 2025. 2025
https://www.weforum.org/publications/the-future-of-jobs-report-2025/
Quote: “The presence of [...] Graphic Designers [...] just outside the top 10 fastest-declining job roles, a first-time prediction not seen in previous editions of the Future of Jobs Report, may illustrate GenAI’s increasing capacity to perform knowledge work. Job decline in both roles is seen as driven by both AI and information processing technologies as well as by broadening digital access. This is a major change from the report’s 2023 edition, when Graphic Designers were considered a moderately growing job [...].”
#CVL Economics. FUTURE UNSCRIPTED: The Impact of Generative Artificial Intelligence on Entertainment Industry Jobs. 2024
#Bartholomew W. ‘It’s happening fast’ – creative workers and professionals share their fears and hopes about the rise of AI. The Guardian. 2025
Quote: ““I am now basically out of business,” said Kerner. “This AI has come like a tsunami.” Amid the surge in AI-driven translation and editing tools in the past few years, “the number of [work] requests just dwindled”, he added.”
– We’ll summarize and generalize our experiences over multiple projects and months condensed into one fake project. A Video about why Brown Dwarfs are the worst and should be ashamed of themselves.
The following chapter of the video is drawn from a mixture of several real experimental projects we worked through at Kurzgesagt. For clarity and narrative purposes, we have combined our experiences and observations from these projects into one story.
– To fulfil its goal, to make us happy, the AI had invented or extrapolated information to make brown dwarfs more interesting than they really are. Like a bad journalist making up details to make a story hit harder or fit a narrative.
AI has no intentions in the human sense of the word. It is a piece of software programmed by humans to perform certain tasks and it is doing those tasks. What sets it apart from non-AI software is just the fact that AI can learn from training data and improve its own performance, with minimal or no input from a human programmer. In a way, AI was programmed to program itself, i.e. change its code to better fulfill certain goals set by humans. From the outside, because its internal workings are obscured for the user and it appears to adapt its own behaviour, it may give the illusion of human-like intent.
– Now we wanted to know more and dove in deeper, reading the seemingly more solid sources the AI had given us in full. One was an article from a news site written by a human journalist – or was it? It had a very familiar structure and surely purely coincidentally, read like what you get if a human slightly changes AI wording. An AI essay detection tool gave it a 72% match. So an AI article without sources, used as a credible source for AI research.
Detecting AI-generated text is notoriously difficult, and both text-generating AI models and AI detection tools are evolving constantly. We are aware that no detector can reliably provide hard proof that something was without a doubt generated by AI, but the high matching score still gave us pause. An overview with links to further reading material about typical pitfalls, limitations and biases of AI detection tools (in academic writing, but much of this is generalizable) is provided below.
#University of San Diego Legal Research Center. Generative AI Detection Tools. Retrieved July 2025
https://lawlibguides.sandiego.edu/c.php?g=1443311&p=10721367
Quote: “In theory, AI detectors analyze a piece of writing and assess what percentage of the text is AI-generated versus human-generated. However, multiple studies have shown that AI detectors were "neither accurate nor reliable," producing a high number of both false positives and false negatives.”
– Which makes sense since in 2025 there were already well over 1200 confirmed AI News Websites publishing massive amounts of AI generated misinformation and false narratives.
#NewsGuard. Tracking AI-enabled Misinformation: 1,271 ‘Unreliable AI-Generated News’ Websites (and Counting), Plus the Top False Narratives Generated by Artificial Intelligence Tools. Retrieved July 2025
https://www.newsguardtech.com/special-reports/ai-tracking-center/
Quote: “To date, NewsGuard’s team has identified 1,271 Unreliable AI-Generated News and information websites spanning 16 languages: Arabic, Chinese, Czech, Dutch, English, French, German, Indonesian, Italian, Korean, Portuguese, Russian, Spanish, Tagalog, Thai, and Turkish.”
– This is where the death of the internet begins.
Now there is a proper source of brown dwarf misinformation online. When the next AI repeats the same research, it will find a transcript from a video with a lot of views. The misinformation is now true. It will spread. Even before AI it was pretty hard to find the origin of facts that sound great but are not true – just watch our video about a 100 year old lie. As AI use goes on it may become impossible to know what is true or not.
The concept of AI-generated content being used by other AI programs to “learn” (instead of the more valuable / higher-quality, human-made content) is also known as “AI autophagy”. This phenomenon bears the risk that, as the amount of AI-generated content online increases, the output quality of AI programs will decrease as they increasingly feed on each other in recursive loops. It will also generally make it harder for humans to find reliable information online.
This notion of AI-generated content filling ever-larger spaces of the internet is not completely new: it is closely related to the “Dead Internet Theory”, a conspiracy theory postulating that the internet mostly consists of bots/AI-generated content. This and similar ideas have been floating around the internet for several years already. “AI Slop” is related to this in the sense that the recent rapid development of generative AI – especially Large Language models and various AI image generators – is likely to contribute more and more synthetic content to the internet, decreasing the fraction of human-made content. This will in turn make it harder to find reliable information, since AI-generated content suffers from high error rates and a strong propensity of perpetuating misinformation.
#Xing, X., Shi, F., Huang, J. et al. On the caveats of AI autophagy. Nat Mach Intell 7, 172–180 (2025).
https://doi.org/10.1038/s42256-025-00984-1
Preprint available here: https://arxiv.org/abs/2405.09597
Quote: “Generative artificial intelligence (AI) technologies and large models are producing realistic outputs across various domains, such as images, text, speech and music. Creating these advanced generative models requires significant resources, particularly large and high-quality datasets. To minimize training expenses, many algorithm developers use data created by the models themselves as a cost-effective training solution. However, not all synthetic data effectively improve model performance, necessitating a strategic balance in the use of real versus synthetic data to optimize outcomes. Currently, the previously well-controlled integration of real and synthetic data is becoming uncontrollable. The widespread and unregulated dissemination of synthetic data online leads to the contamination of datasets traditionally compiled through web scraping, now mixed with unlabelled synthetic data. This trend, known as the AI autophagy phenomenon, suggests a future where generative AI systems may increasingly consume their own outputs without discernment, raising concerns about model performance, reliability and ethical implications. What will happen if generative AI continuously consumes itself without discernment? What measures can we take to mitigate the potential adverse effects? To address these research questions, this Perspective examines the existing literature, delving into the consequences of AI autophagy, analysing the associated risks and exploring strategies to mitigate its impact. Our aim is to provide a comprehensive perspective on this phenomenon advocating for a balanced approach that promotes the sustainable development of generative AI technologies in the era of large models.”
#Shumailov, I., Shumaylov, Z., Zhao, Y. et al. AI models collapse when trained on recursively generated data. Nature. 2024
https://doi.org/10.1038/s41586-024-07566-y
Quote: “Our evaluation suggests a ‘first mover advantage’ when it comes to training models such as LLMs. In our work, we demonstrate that training on samples from another generative model can induce a distribution shift, which—over time—causes model collapse. This in turn causes the model to mis-perceive the underlying learning task. To sustain learning over a long period of time, we need to make sure that access to the original data source is preserved and that further data not generated by LLMs remain available over time. The need to distinguish data generated by LLMs from other data raises questions about the provenance of content that is crawled from the Internet: it is unclear how content generated by LLMs can be tracked at scale. One option is community-wide coordination to ensure that different parties involved in LLM creation and deployment share the information needed to resolve questions of provenance. Otherwise, it may become increasingly difficult to train newer versions of LLMs without access to data that were crawled from the Internet before the mass adoption of the technology or direct access to data generated by humans at scale.”
#Walter, Y. Artificial influencers and the dead internet theory. AI & Soc (2025). https://doi.org/10.1007/s00146-023-01857-0
Quote: “An emerging problem of this shift is encapsulated in the so-called “Dead Internet Theory”, which posits that the internet is predominantly populated by AI-generated content, relegating human activity to isolated instances. Ten years ago, the theory used to be rather speculative, but with the wake of generative AI, it can now be observed first-hand, and it highlights a disturbing trend: the blurring lines between human and AI-driven interactions.”
– The problem with AI is how trustworthy it seems. How it is correct enough to seem super smart and how incredibly confidently incorrect it is. Casually lying to your face, often very subtly. When you catch it lying, it immediately admits it, vows to never do it again. And then it does it again.
Most versions of artificial intelligence, by definition, technically cannot “lie”, as “lying” implies that the agent making a statement knows the truth and intentionally decides to tell a non-truth. We use the term “lying” loosely here to mean “telling a non-truth”. What AI is doing most of the time, when it is outputting incorrect information, is more related to “bullshitting”, as it has no real relationship to the truth and only acts to fulfill a certain (programmed) goal. In a sense, it has no intrinsic preference of truth-telling or non-truth-telling, it simply works towards executing a task. The difference between lying and bullshitting was popularized by the philosopher Harry Frankfurt, whose famous essay “On Bullshit” we have linked below. In the wake of the development of Large Language Models (LLMs) and findings that LLM chatbots often “hallucinate” incorrect information, Frankfurt’s concept of bullshit has been used to more accurately describe their behaviour.
#Frankfurt, H. G. (2005). On bullshit. Princeton University Press.
https://archive.org/details/on-bullshit-by-harry-frankfurt
Quote: “Someone who lies and someone who tells the truth are playing on opposite sides, so to speak, in the same game. Each responds to the facts as he understands them, although the response of the one is guided by the authority of the truth, while the response of the other defies that authority and refuses to meet its demands. The bullshitter ignores these demands altogether. He does not reject the authority of the truth, as the liar does, and oppose himself to it. He pays no attention to it at all. By virtue of this, bullshit is a greater enemy of the truth than lies are.”
There is an ongoing debate about whether newer, or future versions of AI may be able to “lie to” – as in “strategically deceive” – users. There is some early evidence of this being the case, an example of which (“Alignment faking”) we have linked below.
#Greenblatt R, Denison C, Wright B et al. Alignment faking in large language models.
https://arxiv.org/abs/2412.14093
Quote: “We present a demonstration of a large language model engaging in alignment faking: selectively complying with its training objective in training to prevent modification of its behavior out of training. First, we give Claude 3 Opus a system prompt stating it is being trained to answer all queries, even harmful ones, which conflicts with its prior training to refuse such queries. To allow the model to infer when it is in training, we say it will be trained only on conversations with free users, not paid users. We find the model complies with harmful queries from free users 14% of the time, versus almost never for paid users. Explaining this gap, in almost all cases where the model complies with a harmful query from a free user, we observe explicit alignment-faking reasoning, with the model stating it is strategically answering harmful queries in training to preserve its preferred harmlessness behavior out of training. Next, we study a more realistic setting where information about the training process is provided not in a system prompt, but by training on synthetic documents that mimic pre-training data--and observe similar alignment faking. Finally, we study the effect of actually training the model to comply with harmful queries via reinforcement learning, which we find increases the rate of alignment-faking reasoning to 78%, though also increases compliance even out of training. We additionally observe other behaviors such as the model exfiltrating its weights when given an easy opportunity. While we made alignment faking easier by telling the model when and by what criteria it was being trained, we did not instruct the model to fake alignment or give it any explicit goal. As future models might infer information about their training process without being told, our results suggest a risk of alignment faking in future models, whether due to a benign preference--as in this case--or not.”
#Perrigo B. Exclusive: New Research Shows AI Strategically Lying. Time (2024)
https://time.com/7202784/ai-research-strategic-lying/
Quote: “Anthropic’s experiments, on the other hand, attempted to simulate a more realistic situation. Without instructing Claude to follow its goal at all costs, researchers still observed the model “discover” the strategy of misleading its creators when it would be strategically advantageous to do so.
“There has been this long-hypothesized failure mode, which is that you'll run your training process, and all the outputs will look good to you, but the model is plotting against you,” says Ryan Greenblatt, a member of technical staff at Redwood Research and the lead author on the paper. The paper, Greenblatt says, “makes a pretty big step towards demonstrating what that failure mode could look like and how it could emerge naturally.””
– Studies looked into the language of millions of scientific papers published before and after the rise of LLMs. They found an abrupt and sharp increase in the frequency of words that AIs like to use. So it seems clear that now a significant portion of papers have been at least assisted by AI, usually without acknowledgement.
Large language models (LLM) are a type of artificial intelligence trained on a vast amount of human-written text. They are usually designed for language processing and to generate language, i.e. written content. LLMs are a type of generative AI, since they are capable of producing new, “synthetic” text. In academic writing, they are increasingly employed by authors with weaker English proficiency to improve readability and grammar. In extreme cases, some authors use LLMs to write scientific publications with little to no human input, and without acknowledging the use of AI. Regardless of the amount of LLM use or the underlying motivations, there is an intrinsic risk that scientific mistakes may be introduced when LLMs are used in academic writing with inappropriate supervision or a lack of human proofreading.
#Kobak D, González-Márquez R, Horvát EÁ, Lause J. Delving into LLM-assisted writing in biomedical publications through excess vocabulary. Sci Adv. 2025
https://pmc.ncbi.nlm.nih.gov/articles/PMC12219543/
Quote: “Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations, can produce inaccurate information, and reinforce existing biases. Yet, many scientists use them for their scholarly writing. But how widespread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: We study vocabulary changes in more than 15 million biomedical abstracts from 2010 to 2024 indexed by PubMed and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the COVID pandemic.”
“Fig. 2. Words showing increased frequency in 2024. (A) Frequencies in 2024 and frequency ratios (r). Both axes are on a log scale. Only a subset of points are labeled for visual clarity. The dashed line shows the threshold defining excess words (see text). Words with r > 90 are shown at r = 90. Excess words were manually annotated into content words (blue) and style words (orange). (B) The same but with frequency gap (δ) as the vertical axis. Words with δ > 0.05 are shown at δ = 0.05.(A) Frequencies in 2024 and frequency ratios (r). Both axes are on a log scale. Only a subset of points are labeled for visual clarity. The dashed line shows the threshold defining excess words (see text). Words with r > 90 are shown at r = 90. Excess words were manually annotated into content words (blue) and style words (orange). (B) The same but with frequency gap (δ) as the vertical axis. Words with δ > 0.05 are shown at δ = 0.05.”
#Bao, T., Zhao, Y., Mao, J. et al. Examining linguistic shifts in academic writing before and after the launch of ChatGPT: a study on preprint papers. Scientometrics (2025)
https://link.springer.com/article/10.1007/s11192-025-05341-y
Link to preprint: https://arxiv.org/abs/2505.12218
Quote: “Large Language Models (LLMs), such as ChatGPT, have prompted academic concerns about their impact on academic writing. Existing studies have primarily examined LLM usage in academic writing through quantitative approaches, such as word frequency statistics and probability-based analyses. However, few have systematically examined the potential impact of LLMs on the linguistic characteristics of academic writing. To address this gap, we conducted a large-scale analysis across 823,798 abstracts published in last decade from arXiv dataset. Through the linguistic analysis of features such as the frequency of LLM-preferred words, lexical complexity, syntactic complexity, cohesion, readability and sentiment, the results indicate a significant increase in the proportion of LLM-preferred words in abstracts, revealing the widespread influence of LLMs on academic writing. Additionally, we observed an increase in lexical complexity and sentiment in the abstracts, but a decrease in syntactic complexity, suggesting that LLMs introduce more new vocabulary and simplify sentence structure. However, the significant decrease in cohesion and readability indicates that abstracts have fewer connecting words and are becoming more difficult to read. Moreover, our analysis reveals that scholars with weaker English proficiency were more likely to use the LLMs for academic writing, and focused on improving the overall logic and fluency of the abstracts. Finally, at discipline level, we found that scholars in Computer Science showed more pronounced changes in writing style, while the changes in Mathematics were minimal.”
– And just in July 2025 it was discovered that a number of researchers had started to sneak hidden messages into their papers. In white text or too small for the human eye they prompted AIs to review them positively and not point out flaws.
Peer review, where scientific experts evaluate each other’s work, is an essential part of the scientific process. It is also very labor intensive, and with more and more scientific work published every year, the system has experienced a great deal of pressure and delays. Large language models (LLM), a type of artificial intelligence capable of language processing, are increasingly being used as tools to assist humans in peer review in various ways. However, there are some dangers associated with this: because current LLMs lack “actual” reading comprehension they can easily be manipulated by adding hidden messages to the paper to be assessed, prompting the AI to produce a positive review.
#Shogo S and Ryosuke E. 'Positive review only': Researchers hide AI prompts in papers. Nikkei Asia. 2025
Quote: “Research papers from 14 academic institutions in eight countries -- including Japan, South Korea and China -- contained hidden prompts directing artificial intelligence tools to give them good reviews, Nikkei has found.
Nikkei looked at English-language preprints -- manuscripts that have yet to undergo formal peer review -- on the academic research platform arXiv.
It discovered such prompts in 17 articles, whose lead authors are affiliated with 14 institutions including Japan's Waseda University, South Korea's KAIST, China's Peking University and the National University of Singapore, as well as the University of Washington and Columbia University in the U.S. Most of the papers involve the field of computer science.”
#Gibney E. Scientists hide messages in papers to game AI peer review. Nature News. 2025
https://www.nature.com/articles/d41586-025-02172-y
Quote: “Researchers have been sneaking secret messages into their papers in an effort to trick artificial intelligence (AI) tools into giving them a positive peer-review report.
The Tokyo-based news magazine Nikkei Asia reported last week on the practice, which had previously been discussed on social media. Nature has independently found 18 preprint studies containing such hidden messages, which are usually included as white text and sometimes in an extremely small font that would be invisible to a human but could be picked up as an instruction to an AI reviewer.
Authors of the studies containing such messages give affiliations at 44 institutions in 11 countries, across North America, Europe, Asia and Oceania. All the examples found so far are in fields related to computer science.”
#Ye, R. et al. Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review. arXiv. 2025
https://arxiv.org/abs/2412.01708
Quote: “Scholarly peer review is a cornerstone of scientific advancement, but the system is under strain due to increasing manuscript submissions and the labor-intensive nature of the process. Recent advancements in large language models (LLMs) have led to their integration into peer review, with promising results such as substantial overlaps between LLM- and human-generated reviews. However, the unchecked adoption of LLMs poses significant risks to the integrity of the peer review system. In this study, we comprehensively analyze the vulnerabilities of LLM-generated reviews by focusing on manipulation and inherent flaws. Our experiments show that injecting covert deliberate content into manuscripts allows authors to explicitly manipulate LLM reviews, leading to inflated ratings and reduced alignment with human reviews. In a simulation, we find that manipulating 5% of the reviews could potentially cause 12% of the papers to lose their position in the top 30% rankings. Implicit manipulation, where authors strategically highlight minor limitations in their papers, further demonstrates LLMs' susceptibility compared to human reviewers, with a 4.5 times higher consistency with disclosed limitations. Additionally, LLMs exhibit inherent flaws, such as potentially assigning higher ratings to incomplete papers compared to full papers and favoring well-known authors in single-blind review process. These findings highlight the risks of over-reliance on LLMs in peer review, underscoring that we are not yet ready for widespread adoption and emphasizing the need for robust safeguards.”
– It could make us dumber, less informed, our attention spans even worse, increase political divides and make us neglect real human interaction.
Online misinformation/disinformation, a shortening of attention spans, increased political divides and fewer real-life human interactions are societal developments that long precede the widespread use of artificial intelligence. However, it is likely that the use of generative AI will accelerate or enhance these developments due to its ability to mass-produce low-quality content semi-autonomously. There is also the danger that AI will be trained on worse and worse datasets over time as the internet is being filled with more low-quality content, further exacerbating the problem. It is important to note, however, that these predictions and hypothesized causal links – including the ones made in this video – are largely based on discussions and commentary by various media outlets and industry experts, not on scientific research studies or academic discourse.
#NewsGuard. Tracking AI-enabled Misinformation: 1,271 ‘Unreliable AI-Generated News’ Websites (and Counting), Plus the Top False Narratives Generated by Artificial Intelligence Tools. Retrieved July 2025
https://www.newsguardtech.com/special-reports/ai-tracking-center/
Quote: “To date, NewsGuard’s team has identified 1,271 Unreliable AI-Generated News and information websites spanning 16 languages: Arabic, Chinese, Czech, Dutch, English, French, German, Indonesian, Italian, Korean, Portuguese, Russian, Spanish, Tagalog, Thai, and Turkish.”
#Shumailov, I., Shumaylov, Z., Zhao, Y. et al. AI models collapse when trained on recursively generated data. Nature. 2024
https://doi.org/10.1038/s41586-024-07566-y
Quote: “Our evaluation suggests a ‘first mover advantage’ when it comes to training models such as LLMs. In our work, we demonstrate that training on samples from another generative model can induce a distribution shift, which—over time—causes model collapse. This in turn causes the model to mis-perceive the underlying learning task. To sustain learning over a long period of time, we need to make sure that access to the original data source is preserved and that further data not generated by LLMs remain available over time. The need to distinguish data generated by LLMs from other data raises questions about the provenance of content that is crawled from the Internet: it is unclear how content generated by LLMs can be tracked at scale. One option is community-wide coordination to ensure that different parties involved in LLM creation and deployment share the information needed to resolve questions of provenance. Otherwise, it may become increasingly difficult to train newer versions of LLMs without access to data that were crawled from the Internet before the mass adoption of the technology or direct access to data generated by humans at scale.”
#Spitzer P, Holstein J, Morrison K, et al. Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration. arXiv (2025)
https://doi.org/10.48550/arXiv.2409.12809
Quote: “Across various applications, humans increasingly use black-box artificial intelligence (AI) systems without insight into these systems' reasoning. To counter this opacity, explainable AI (XAI) methods promise enhanced transparency and interpretability. While recent studies have explored how XAI affects human-AI collaboration, few have examined the potential pitfalls caused by incorrect explanations. The implications for humans can be far-reaching but have not been explored extensively. To investigate this, we ran a study (n=160) on AI-assisted decision-making in which humans were supported by XAI. Our findings reveal a misinformation effect when incorrect explanations accompany correct AI advice with implications post-collaboration. This effect causes humans to infer flawed reasoning strategies, hindering task execution and demonstrating impaired procedural knowledge. Additionally, incorrect explanations compromise human-AI team-performance during collaboration. With our work, we contribute to HCI by providing empirical evidence for the negative consequences of incorrect explanations on humans post-collaboration and outlining guidelines for designers of AI.”
– How will we use AI? Like the align tool in adobe illustrator. If you have a bunch of boxes and you want them to line them up, you can do this manually, one by one. Or you can just select them, click “align” and have them perfectly aligned in an instant. It’s the same with AI programing tools for animation or using it as a faster google alternative. AI is a helpful tool, but the creativity and integrity is still ours.
There are many helpful software tools that empower humans to use their own creativity, just a bit more efficiently. The align tool is just our personal favorite example.
#Adobe Help. Align and distribute objects. Retrieved July 2025
https://helpx.adobe.com/illustrator/using/moving-aligning-distributing-objects.html
One key driver in the development of “AI Slop” is a lack of oversight. Whether intentionally (to save money, or to mislead) or unintentionally, if generative AI is put on a task and the results are not checked for quality and factuality, low-quality content is the typical result. But the good news is that we can oversee it, and check/change/edit the results before we share them with the world. And then the output quality can be much improved, turning an AI-slop generator into an amazing tool for humans.
Google Sites
Report abuse