Task Definition: Given a piece of text, detect whether it is worth fact-checking.
Subtask 1A: Check-worthiness of tweets: Given a tweet, predict whether it is worth fact-checking. We will have not only check-worthiness labels, but also four auxiliary labels. i.e. used to facilitate annotation in 2020, which would enable multi-task learning. We will only evaluate with respect to check-worthiness. This is a classification task. This subtasks runs in 5 languages:
Arabic
Bulgarian
English
Spanish
Turkish
Subtask 1B: Check-Worthiness of Debates/Speeches: Given a political debate/speech, produce a ranked list of its sentences, ordered by their check-worthiness. This is a ranking task.
This task is evaluated as a ranking task. We will use Mean Average Precision (MAP) as the official evaluation measure and we will report reciprocal rank, and P@k for k ∈ {1,3,5,10,20,30} as well.
All scripts can be found on the main repo for the lab, CheckThat! Lab Task 1.
You can find the leaderboard and submission in this CodaLab Comptetion link.
Note: The leaderboard for the Spanish dataset can be found in a separate CodaLab Comptetion link.