Tasks

Don't forget to register through CLEF2020 Lab Registration before 26 April 2020, using this link. Otherwise, your submission will NOT be considered!

Overview

The mission of the lab is to foster the development of technology that would enable the automatic verification of claims. Automated systems for claim identification and verification can be very useful as supportive technology for investigative journalism, as they could provide help and guidance, thus saving time.

A typical fact-checking system includes the following steps.

    1. Automatically identify check-worthy claims
    2. Make sure they have not been fact-checked already by a reputable fact-checking organization
    3. Identify documents that are potentially useful for humans to perform manual fact-checking of a claim.
    4. Estimate a claim veracity supported by evidence to increase the journalist's understanding and the trust in the system's decision

This lab is the third edition of the CheckThat! lab. In 2020, the lab turns its attention to social media - in particular to Twitter - as information posted on that platform is not checked by an authoritative entity before publication and such information tends to disseminate very quickly.

Tasks

Task 1 - Check-Worthiness on tweets: Predict which tweet from a stream of tweets on a topic should be prioritized for fact-checking.

Task 2 - Verified claim retrieval: Given a check-worthy tweet claim, and a set of previously-checked claims, determine whether the claim has been already fact-checked.

Task 3 - Evidence retrieval: Given a check-worthy claim on a specific topic and a set of text snippets extracted from potentially-relevant webpages, return a ranked list of evidence snippets for the claim.

Task 4 - Claim verification: Given a check-worthy claim in a tweet and a set of potentially-relevant Web pages, estimate the veracity of the claim.

Task 5 - Check-Worthiness on debates: Given a debate segmented into sentences, together with speaker information, prioritize sentences for fact-checking.

All Tasks will run in English. Additionally, Tasks 1, 3, and 4 will also run in Arabic and Spanish.

Past Editions

Information and datasets from past editions can be found here.