CLEF2021 - CheckThat! Lab

Invited talks

Lab Registration

To register in the CheckThat! lab, please visit: http://clef2021-labs-registration.dei.unipd.it/registrationForm.php

Tasks

Check the overview of the tasks here or in the following task specific task pages.

Task 1 - Check-Worthiness Estimation : Given a claim, detect whether it is worth fact-checking.

Datasets

• Subtask-1a--Arabic

• Subtask-1a--Turkish

• Subtask-1a--Spanish

• Subtask-1a--English

• Subtask-1a--Bulgarian

• Subtask-1b--English

Task 2 - Verified Claim Retrieval: Given a check-worthy claim, and a set of previously fact-checked claims, determine whether the claim has been previously fact-checked.

Datasets:

• Subtask-2a--English

• Subtask-1b--Arabic

Task 3 - Fake News Detection: Given the text of a news article, determine whether the claims made in the article are true, partially true, false or other (e.g., claims in dispute) and also detect the topical domain of the article.

Datasets:

• Subtask-3a--English

• Subtask-3b--English

You can also check out the main repo containing all the scripts and data, CheckThat! Lab-2021.

Discussion Group

Please join our discussion group clef-factcheck@googlegroups.com to receive announcements and participate in discussions.

Citation

You can find the overview papers on the CLEF2021-CheckThat! Lab as well as of the individual tasks below:

@InProceedings{​​​​​​​CheckThat:ECIR2021,

author = {​​​​​​​Preslav Nakov and

Da San Martino, Giovanni and

Tamer Elsayed and

Alberto Barr{​​​​​​​\'{​​​​​​​o}​​​​​​​}​​​​​​​n{​​​​​​​-}​​​​​​​Cede{​​​​​​​\~{​​​​​​​n}​​​​​​​}​​​​​​​o and

Rub\'{​​​​​​​e}​​​​​​​n M\'{​​​​​​​i}​​​​​​​guez and

Shaden Shaar and

Firoj Alam and

Fatima Haouari and

Maram Hasanain and

Nikolay Babulkov and

Alex Nikolov and

Shahi, Gautam Kishore and

Struß, Julia Maria and

Thomas Mandl}​​​​​​​,

title = {​​​​​​​The {​​​​​​​CLEF}​​​​​​​-2021 {​​​​​​​CheckThat}​​​​​​​! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News}​​​​​​​,

booktitle = {​​​​​​​Proceedings of the 43rd European Conference on Information Retrieval}​​​​​​​,

series = {​​​​​​​ECIR~'21}​​​​​​​,

pages = {​​​​​​​639--649}​​​​​​​,

address = {​​​​​​​Lucca, Italy}​​​​​​​,

month = {​​​​​​​March}​​​​​​​,

year = {​​​​​​​2021}​​​​​​​,

url = {​​​​​​​https://link.springer.com/chapter/10.1007/978-3-030-72240-1_75}​​​​​​​​,

}​​​​​​​

@InProceedings{clef-checkthat:2021:LNCS,

author = {Preslav Nakov and

Da San Martino, Giovanni and

Tamer Elsayed and

Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and

Rub\'{e}n M\'{i}guez and

Shaden Shaar and

Firoj Alam and

Fatima Haouari and

Maram Hasanain and

Watheq Mansour and

Bayan Hamdan and

Zien Sheikh Ali and

Nikolay Babulkov and

Alex Nikolov and

Shahi, Gautam Kishore and

Struß, Julia Maria and

Thomas Mandl and

Mucahid Kutlu and

Yavuz Selim Kartal},

title = "Overview of the {CLEF}-2021 {CheckThat}! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News",

year = {2021},

booktitle = "Proceedings of the 12th International Conference of the CLEF Association: Information Access Evaluation Meets Multiliguality, Multimodality, and Visualization",

series = {CLEF~'2021},

address = {Bucharest, Romania (online)},

url ={https://link.springer.com/chapter/10.1007/978-3-030-85251-1_19}

}

@InProceedings{clef-checkthat:2021:task1,

author = {Shaden Shaar and

Maram Hasanain and

Bayan Hamdan and

Zien Sheikh Ali and

Fatima Haouari and

Alex Nikolov,

Mucahid Kutlu and

Yavuz Selim Kartal,

Firoj Alam and

Da San Martino, Giovanni and

Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and

Rub\'{e}n M\'{i}guez and

Tamer Elsayed and

Preslav Nakov},

title = "Overview of the {CLEF}-2021 {CheckThat}! Lab Task 1 on Check-Worthiness Estimation in Tweets and Political Debates",

year = {2021},

booktitle = "Working Notes of CLEF 2021---Conference and Labs of the Evaluation Forum",

series = {CLEF~'2021},

address = {Bucharest, Romania (online)},

url={http://ceur-ws.org/Vol-2936/paper-28.pdf}

}

@InProceedings{clef-checkthat:2021:task2,

author = {Shaden Shaar and

Fatima Haouari and

Watheq Mansour and

Maram Hasanain and

Nikolay Babulkov and

Firoj Alam and

Da San Martino, Giovanni and

Tamer Elsayed and

Preslav Nakov},

title = "Overview of the {CLEF}-2021 {CheckThat}! Lab Task 2 on Detecting Previously Fact-Checked Claims in Tweets and Political Debates",

year = {2021},

booktitle = "Working Notes of CLEF 2021---Conference and Labs of the Evaluation Forum",

series = {CLEF~'2021},

address = {Bucharest, Romania (online)},

url={http://ceur-ws.org/Vol-2936/paper-29.pdf}

}

@InProceedings{clef-checkthat:2021:task3,

author = {Shahi, Gautam Kishore and

Struß, Julia Maria and

Thomas Mandl},

title = "Overview of the {CLEF}-2021 {CheckThat}! Lab Task 3 on Fake News Detection",

year = {2021},

booktitle = "Working Notes of CLEF 2021---Conference and Labs of the Evaluation Forum",

series = {CLEF~'2021},

address = {Bucharest, Romania (online)},

url={http://ceur-ws.org/Vol-2936/paper-30.pdf}

}