CLEF2022 - CheckThat! Lab
Fighting the COVID-19 Infodemic and Fake News Detection
Welcome to CheckThat! Lab at CLEF 2022, the 5th version of the lab!
This page has moved. Please visit our new website
Proceedings
Main Proceedings: Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum
Task 1 paper: http://ceur-ws.org/Vol-3180/paper-28.pdf
Task 2 paper: http://ceur-ws.org/Vol-3180/paper-29.pdf
Task 3 paper: http://ceur-ws.org/Vol-3180/paper-30.pdf
Lab Program
This year, lab activities will run on 6 and 7 September. All times follow CET timezone.
[6/9] CheckThat! Lab Overview: 10:50 - 12:30
[7/9] Session 1: 8:50 - 10:20 (Chair: Alberto Barron-Cedeno)
[7/9] Session 2: 13:30 - 15:00 (Chair: Giovanni Da San Martino)
[7/9] Session 3: 15:30 - 17:00 (Chair: Preslav Nakov)
[8/9] Closing: Andrea Galassi
Recent Updates
July 14, 2022: Program overview posted
May 16, 2022: Leaderboard for task 3 is made public.
May 9, 2022: Test set with gold label for task 1 and 2 released.
Submission deadline for Task-3 extended: May 9, 2022 (AoE)
May 3, 2022: Task-3 submission site released.
April 29, 2022: Task-3 test data released.
April 23, 2022: Task-2 submission site released.
April 22, 2022: Task-1 submission site released.
April 22, 2022: Test data released
March, 24, 2022: Baseline system for task 3 available: https://zenodo.org/record/6362498
February 24, 2022: Training data released
September 14, 2021: Website is up!
Lab Registration
CLOSED on April 22, 2022: To register in the CheckThat! lab, please visit: https://clef2022-labs-registration.dei.unipd.it/registrationForm.php
Leaderboards
Kindly check the following leaderboards:
Task 1: http://shorturl.at/nuCOS
Task 2: http://shorturl.at/dzX28
Task 3: http://shorturl.at/knBCG
Submission sites
To submit your results on the test data, kindly use the following leaderboard:
Task 1: https://codalab.lisn.upsaclay.fr/competitions/4230
Please check submission guidelines
Task 2: https://codalab.lisn.upsaclay.fr/competitions/4260
Please check submission guidelines
Task 3: https://codalab.lisn.upsaclay.fr/competitions/4633
Please check submission guidelines
Instructions to prepare your paper
The title of paper should be in the following format:
<Team Name> at CheckThat! 2022: <Name of the title>
For example:
LamdaX at CheckThat! 2022: Ensemble Models for Checkworthy Tweet Identification
Cite: Please check the citation section to cite the shared task papers.
Templates: The working notes are prepared using the CEUR-WS templates (both Word and LaTex) available at: https://drive.google.com/drive/folders/1atq0bC9hSDMicqm79Jeu6TWOR-q_BCRQ?usp=sharing
More information: https://clef2022.clef-initiative.eu/index.php?page=Pages/instructions_for_authors.html
Paper submission: https://www.easychair.org/my/conference?conf=clef2022
Tasks
Check the overview of the tasks here. Details of each task are given in the specific task pages.
Task 1: https://sites.google.com/view/clef2022-checkthat/tasks/task-1-identifying-relevant-claims-in-tweets
Task 3: https://sites.google.com/view/clef2022-checkthat/tasks/task-3-fake-news-detection
Datasets
You can check out the main repo containing all the scripts and data, CheckThat! Lab-2022. Liks to the Individual task dataset pages can be found below:
Task 1: https://gitlab.com/checkthat_lab/clef2022-checkthat-lab/clef2022-checkthat-lab/-/tree/main/task1
Task 2: https://gitlab.com/checkthat_lab/clef2022-checkthat-lab/clef2022-checkthat-lab/-/tree/main/task2
Important Dates
All times are Any where On Earth (AOE).
15 November 2021: Registration opens
22 April 2022: Registration closes and test set data released
6 May 2022: End of the evaluation cycle (test set submission closes)
9 May 2022: Results released
27 May 2022: Submission of participant papers [CEUR-WS]
13 June 2022: Deadline for reviews of participant papers
1 July 2022: Deadline for submission camera-ready version of the participant papers
18-22 July 2022: Preview of working notes by authors and lab organizers [CEUR-WS]
5-8 September 2022: CLEF 2022 Conference
Discussion Group
Please join our discussion group clef-factcheck@googlegroups.com to receive announcements and participate in discussions.
Organizers
Preslav Nakov, Qatar Computing Research Institute, HBKU
Alberto Barrón-Cedeño, Università di Bologna
Giovanni Da San Martino, University of Padova
Firoj Alam, Qatar Computing Research Institute, HBKU
Julia Maria Struß, Potsdam University of Applied Sciences
Thomas Mandl, University of Hildesheim
Rubén Míguez, Newtral Media Audiovisual, Spain
Tommaso Caselli, University of Groningen
Mucahid Kutlu, TOBB Economy and Technology University
Wajdi Zaghouani, HBKU
Chengkai Li, University of Texas at Arlington
Shaden Shaar, Cornell University
Gautam Kishore Shahi, University of Duisburg-Essen
Hamdy Mubarak, Qatar Computing Research Institute, HBKU
Alex Nikolov, Sofia University
Nikolay Babulkov, Sofia University
Yavuz Selim Kartal, TOBB Economy and Technology University
Javier Beltr ́an, Newtral Media Audiovisual, Spain
Juliane Köhler, Potsdam University of Applied Sciences
Michael Wiegand, University of Klagenfurt
Melanie Siegel, Darmstadt University of Applied Sciences
Citation
You can find the overview papers on the CLEF2021-CheckThat! Lab as well as of the individual tasks below:
@InProceedings{CheckThat:ECIR2022,
author="Nakov, Preslav and Barr{\'o}n-Cede{\~{n}}o, Alberto and Da San Martino, Giovanni and Alam, Firoj and Stru{\ss}, Julia Maria and Mandl, Thomas and M{\'i}guez, Rub{\'e}n and Caselli, Tommaso and Kutlu, Mucahid and Zaghouani, Wajdi and Li, Chengkai and Shaar, Shaden and Shahi, Gautam Kishore and Mubarak, Hamdy and Nikolov, Alex and Babulkov, Nikolay and Kartal, Yavuz Selim and Beltr{\'a}n, Javier",
editor="Hagen, Matthias and Verberne, Suzan and Macdonald, Craig and Seifert, Christin and Balog, Krisztian and N{\o}rv{\aa}g, Kjetil and Setty, Vinay",
title="The CLEF-2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection",
booktitle="Advances in Information Retrieval",
year="2022",
publisher="Springer International Publishing",
address="Cham",
pages="416--428",
isbn="978-3-030-99739-7"
}
@InProceedings{clef-checkthat:2022:LNCS,
author = {Nakov, Preslav and Barr\'{o}n-Cede\~{n}o, Alberto and Da San Martino, Giovanni and Alam, Firoj and Stru\ss{}, Julia Maria and Mandl, Thomas and M\'{\i}guez, Rub\'{e}n and Caselli, Tommaso and Kutlu, Mucahid and Zaghouani, Wajdi and Li, Chengkai and Shaar, Shaden and Shahi, Gautam Kishore and Mubarak, Hamdy and Nikolov, Alex and Babulkov, Nikolay and Kartal, Yavuz Selim and Beltr\'{a}n, Javier and Wiegand, Michael and Siegel, Melanie and K{\"o}hler, Juliane},
title = "Overview of the {CLEF}-2022 {CheckThat}! Lab on Fighting the {COVID-19} Infodemic and Fake News Detection",
year = {2022},
booktitle = "Proceedings of the 13th International Conference of the CLEF Association: Information Access Evaluation meets Multilinguality, Multimodality, and Visualization",
series = {CLEF~'2022},
address = {Bologna, Italy},
}
@InProceedings{clef-checkthat:2022:task1,
author = {Nakov, Preslav and Barr\'{o}n-Cede\~{n}o, Alberto and Da San Martino, Giovanni and Alam, Firoj and M\'{\i}guez, Rub\'{e}n and Caselli, Tommaso and Kutlu, Mucahid and Zaghouani, Wajdi and Li, Chengkai and Shaar, Shaden and Mubarak, Hamdy and Nikolov, Alex and Kartal, Yavuz Selim and Beltr\'{a}n, Javier},
title = "Overview of the {CLEF}-2022 {CheckThat}! Lab Task 1 on Identifying Relevant Claims in Tweets",
year = {2022},
booktitle = "Working Notes of CLEF 2022---Conference and Labs of the Evaluation Forum",
series = {CLEF~'2022},
address = {Bologna, Italy},
}
@InProceedings{clef-checkthat:2022:task2,
author = {Nakov, Preslav and Da San Martino, Giovanni and Alam, Firoj and Shaar, Shaden and Mubarak, Hamdy and Babulkov, Nikolay},
title = "Overview of the {CLEF}-2022 {CheckThat}! Lab Task 2 on Detecting Previously Fact-Checked Claims",
year = {2022},
booktitle = "Working Notes of CLEF 2022---Conference and Labs of the Evaluation Forum",
series = {CLEF~'2022},
address = {Bologna, Italy},
}
@InProceedings{clef-checkthat:2022:task3,
author = {K{\"o}hler, Juliane and Shahi, Gautam Kishore and Stru{\ss}, Julia Maria and Wiegand, Michael and Siegel, Melanie and Mandl, Thomas and Sch{\"u}tz, Mina},
title = "Overview of the {CLEF}-2022 {CheckThat}! Lab Task 3 on Fake News Detection ",
year = {2022},
booktitle = "Working Notes of CLEF 2022---Conference and Labs of the Evaluation Forum",
series = {CLEF~'2022},
address = {Bologna, Italy},
}