Embedded Files
Q&A with Wikipedia
Guidance for the Claim–Source Verification Project
Guidance for the Claim–Source Verification Project
The following responses come from conversations with Sage Ross, our collaborator at the Wikipedia foundation. You may have your own questions, but these are the ones we asked first.
The following responses come from conversations with Sage Ross, our collaborator at the Wikipedia foundation. You may have your own questions, but these are the ones we asked first.
1. Have we framed the problem and its value correctly?
1. Have we framed the problem and its value correctly?
Yes.
Yes.
One of the biggest issues with claim–source verification is that it is extremely manual, so editors simply cannot do enough of it.
One of the biggest issues with claim–source verification is that it is extremely manual, so editors simply cannot do enough of it.
Because verifying citations requires locating sources and checking them by hand, many mismatches between claims and sources go unnoticed. This includes cases where:
Because verifying citations requires locating sources and checking them by hand, many mismatches between claims and sources go unnoticed. This includes cases where:
citations do not actually support the claim
claims misrepresent the cited source
AI-generated edits introduce unsupported statements
Tools that can automatically identify clear cases of claim–source mismatch could help editors detect and fix systematic problems much more efficiently.
Tools that can automatically identify clear cases of claim–source mismatch could help editors detect and fix systematic problems much more efficiently.
2. Is claim–source verification a high-priority challenge for Wikipedia?
2. Is claim–source verification a high-priority challenge for Wikipedia?
Yes. This is an area of active interest within the Wikipedia community.
Yes. This is an area of active interest within the Wikipedia community.
There are ongoing discussions and exploratory efforts related to this challenge. One example is the technical discussion thread below:
There are ongoing discussions and exploratory efforts related to this challenge. One example is the technical discussion thread below:
Projects that experiment with methods for verifying claims against citations could contribute useful insights to these broader efforts.
Projects that experiment with methods for verifying claims against citations could contribute useful insights to these broader efforts.
3. Are there specific articles where citation problems are more common?
3. Are there specific articles where citation problems are more common?
A particularly interesting area to investigate is articles edited by students, such as those created through educational programs:
A particularly interesting area to investigate is articles edited by students, such as those created through educational programs:
Student editors are often:
Student editors are often:
new to Wikipedia editing
still learning how to properly use sources
more likely to rely on AI tools when writing content
Because of this, these articles may contain:
Because of this, these articles may contain:
sources that are accessible online
claims generated by AI that do not actually match the source material
This makes them a useful testing ground for claim–source verification tools.
This makes them a useful testing ground for claim–source verification tools.
4. What kinds of outputs would be most useful for editors?
4. What kinds of outputs would be most useful for editors?
A useful starting point would be to characterize the scope of the problem within an article.
A useful starting point would be to characterize the scope of the problem within an article.
For example, a tool might analyze an article and report:
For example, a tool might analyze an article and report:
how many citations the article contains
how many claim–source pairs can be extracted
how many cited sources are accessible for verification
among those, how many claims appear to be
verified by the source
contradicted by the source
unclear or unsupported
This type of analysis would help editors quickly understand how much verification work may be needed for a given article.
This type of analysis would help editors quickly understand how much verification work may be needed for a given article.
Another valuable direction is identifying patterns that make verification easier or more reliable.
Another valuable direction is identifying patterns that make verification easier or more reliable.
For example:
For example:
if a claim includes a specific date, the same date should appear in the source
if the source does not contain the date, that may signal the claim is unreliable
Signals like this could help prioritize which claims deserve closer inspection.
Signals like this could help prioritize which claims deserve closer inspection.
Ultimately, automated verification could act as a quality signal for articles, helping editors identify where deeper investigation is needed.
Ultimately, automated verification could act as a quality signal for articles, helping editors identify where deeper investigation is needed.
5. What data sources or tools should students use?
5. What data sources or tools should students use?
The Wikipedia API provides access to a wide variety of Wikipedia data.
The Wikipedia API provides access to a wide variety of Wikipedia data.
However, there are many different ways to retrieve Wikipedia-related data, and there is no single required approach.
However, there are many different ways to retrieve Wikipedia-related data, and there is no single required approach.
For this project, a useful starting point may be:
For this project, a useful starting point may be:
retrieving the HTML of Wikipedia articles, which contains structural information about claims, references, and citations.
If students encounter specific data needs during the project, additional guidance can help point them toward appropriate Wikipedia data resources.
If students encounter specific data needs during the project, additional guidance can help point them toward appropriate Wikipedia data resources.
Page updated
Report abuse