The mission of the lab is to foster the development of technology that would enable the automatic verification of claims. Automated systems for claim identification and verification can be very useful as supportive technology for investigative journalism, as they could provide help and guidance, thus saving time.
A typical fact-checking system includes the following steps.
This lab is the third edition of the CheckThat! lab. In 2020, the lab turns its attention to social media - in particular to Twitter - as information posted on that platform is not checked by an authoritative entity before publication and such information tends to disseminate very quickly.
Task 1 - Check-Worthiness on tweets: Predict which tweet from a stream of tweets on a topic should be prioritized for fact-checking.
Task 2 - Verified claim retrieval: Given a check-worthy tweet claim, and a set of previously-checked claims, determine whether the claim has been already fact-checked.
Task 3 - Evidence retrieval: Given a check-worthy claim on a specific topic and a set of text snippets extracted from potentially-relevant webpages, return a ranked list of evidence snippets for the claim.
Task 4 - Claim verification: Given a check-worthy claim in a tweet and a set of potentially-relevant Web pages, estimate the veracity of the claim.
Task 5 - Check-Worthiness on debates: Given a debate segmented into sentences, together with speaker information, prioritize sentences for fact-checking.
All Tasks will run in English. Additionally, Tasks 1, 3, and 4 will also run in Arabic and Spanish.
Information and datasets from past editions can be found here.