CS 218 – Final Project - Wikipedia Claim-Source Verification

Background

Wikipedia is one of the most widely used repositories of encyclopedic knowledge in the world. Articles are written and maintained by a global community of volunteer editors and rely on a core principle: verifiability. Claims made in Wikipedia articles must be supported by reliable sources through citations.

Because of this transparent citation structure, Wikipedia has become one of the most trusted public knowledge resources online. It also plays an influential role in the modern AI ecosystem: publicly available web content—including Wikipedia—is widely used as training data or reference material for large language models developed by organizations such as OpenAI, Google, and Meta.

Ensuring the accuracy of Wikipedia is therefore important not only for readers, but for the broader information ecosystem.

The Problem

Wikipedia articles contain millions of claims supported by citations.

The expectation is that the cited source actually supports the claim made in the article.

However, verifying this relationship is currently a manual process. Editors must locate the source, read it, and determine whether the claim is supported. This is often difficult because sources may be long documents such as books, research papers, or news articles.

Research on Wikipedia has shown that inaccuracies tend to arise when citations:

• do not support the claim at all (duh)

• misrepresent the source (get the claim totally wrong)

• only partially support the claim (more challenging)

• provide weak or indirect evidence (more challenging)

• AI slop. AI-generated text gets claims wrong, or fabricates them entirely.

Improving the efficiency of this verification process could significantly help Wikipedia editors maintain article quality. It is also an active area of investigation within the wikipedia community.  See: https://phabricator.wikimedia.org/T399642

A potentially rich source of problematic articles are those edited by students, such as those created through educational programs, since students are more likely to use AI tools to write articles. See: https://dashboard.wikiedu.org/campaigns/fall_2025/articles

Your Task

Your task is to explore how modern AI tools might help editors verify claims made in articles against their cited sources more efficiently, and to produce a working prototype that demonstrates progress, or some proof of concept toward solving the claim–source verification problem, or some aspect of it.

Project Goal

The goal is not to automatically edit Wikipedia (yet), but to assist human editors by identifying claims that may require closer inspection.  Your team should develop a prototype or tool that helps Wikipedia editors investigate whether article claims are supported by their cited sources.

The key objective is to make citation verification faster or easier for Wikipedia editors.

Business Value

For any organization that produces or relies on information, failures in verification lead to legal risk, reputational damage, and costly rework. A system that improves claim–source verification creates value by doing one or more of the following:

• Reducing risk (fewer false or unsupported claims)

• Improving efficiency (faster review of large documents)

• Increasing trust (higher confidence in published content)

• Enabling scale (verifying more content with the same resources)

Verifiable metrics can be applied to any of these areas that would help demonstrate the value.

Connections to other industries

While you are solving this problem for Wikipedia, this is not just a Wikipedia problem—it is emerging as a critical issue across industries using AI-generated content:

• Legal (Law Firms & Courts) -- AI-generated briefs have cited nonexistent cases or misrepresented rulings, creating serious professional and ethical risk. Verification tools could prevent these failures before submission.

• Journalism & Media -- Reporters increasingly use AI to draft content. Ensuring that sources actually support claims is essential to avoid misinformation and protect credibility.

• Healthcare & Medical Writing -- Incorrect or weakly supported claims in medical summaries can lead to patient harm and liability. Verification improves safety and compliance.

• Finance & Consulting -- Analysts produce reports backed by data and sources. Misrepresented evidence can lead to bad decisions and financial loss.

Deliverable

Each team should produce a working prototype that demonstrates progress, or some proof of concept toward solving the claim–source verification problem, or some aspect of it.  This might include: a software prototype, an AI agent pipeline, a reporting tool that evaluates article citations, etc.

Teams should demonstrate how their system works and explain how it could help Wikipedia editors investigate citation accuracy.  The team should also demonstrate how it improves efficiency.

Resources

• Q&A with Wikipedia about this project (contains links to suggested articles and other starting points)

• Class Data Collection Spreadsheet (crowd-sourced claim/source verification data)

Submission & Evaluation

Place the components shown below on your website. Each of these will be graded 0..3 as follows, for a total possible maximum of 12 points:
        0 points: Not done

        1 point:   Partially done, with significant parts incomplete

        2 points:  Almost everything is there
        3 points: Done completely, with all aspects addressed.  Includes appropriate visual elements.

Website Required Components:

1. Link or embedding of your group's planning document, indicating 

   • What exploratory (learning) you did or still need to do outside of class need to do

   • Timeline - Who/What/When – work-back from deadline, what needs to happen when, by whom

2. Link or embedding of your group's presentation slides.  

3. A stand-alone description of your solution.  This should have:

   • A one-sentence description of your solution

   • A statement (with metrics) about what/how you solution is an improvement (e.g. in terms of efficiency, or cost, time, accuracy, etc.)

   • A link to your solution, which could be a document, agent(s), or ML model.

4. Your personal contributions to the project as a STAR story (Situation, Task, Action, Result), the goal being that someday, when you are interviewing for a job, you could bring this up and use it to tell a compelling narrative about this project.

Getting started - things to think about. 

Even verifying a single claim against a single source is a challenging problem. Questions include:

• Can AI determine whether a cited source actually supports a claim?

• Which models perform best at this task?

• Can AI locate the passage in a source that provides evidence?

• Can AI summarize evidence for human review?

If this process works for individual claims, presumably you could scale it to entire articles or groups of articles.

Some ideas to get started:

• analyze individual claim–source pairs

• highlight supporting or contradictory passages in sources

• generate verification reports for an article

• compare the performance of different AI models

• propose a workflow that could accelerate citation verification

The exact approach is intentionally open-ended.

You may also want or need to find examples “in the wild” that represent various flavors of the claim-source verification problem