Task Definition: Given a check-worthy claim in the form of a tweet, and a set of previously fact-checked claims, determine whether the claim has been previously fact-checked.
Subtask 2A: Detect Previously Fact-Checked Claims in Tweets: Given a tweet, detect whether the claim the tweet makes was previously fact-checked with respect to a collection of fact-checked claims. This is a ranking task, where the systems will be asked to produce a list of top-n candidates. This task is given in two languages:
Arabic
English
Subtask 2B: Detect Previously Fact-Checked Claims in Political Debates/Speeches: Given a claim in a political debate or a speech, detect whether the claim has been previously fact-checked with respect to a collection of previously fact-checked claims. This is a ranking task. This task is only given in English.
This task is evaluated as a ranking task. Ranked list per claim will be evaluated using ranking evaluation measures (MAP@k for k=1,3,5,10,20,all, MRR and Precision@k for k=1,3,5,10,20). Official measure is MAP@5.
You can find the Subtask-2b--English dataset and more details about it here.
All scripts can be found on the main repo for the lab, CheckThat! Lab Task 2.
You can find the leaderboard and submission in this CodaLab Comptetion link.