Welcome to Propaganda Detection in Arabic shared task at WANLP!
Propaganda is defined as an expression of opinion or action by individuals or groups deliberately designed to influence opinions or actions of other individuals or groups with reference to predetermined ends and this is achieved by means of well-defined rhetorical and psychological devices. Currently, propaganda (or persuasion) techniques have been commonly used on social media to manipulate or mislead social media users. More detail of the techniques can be found here: https://propaganda.qcri.org/annotations/definitions.html.
The goal of the shared task is to build models for identifying such techniques in the Arabic social media text (specifically Tweets).
Leaderboards
Please check the leaderboards here: https://docs.google.com/spreadsheets/d/1Iu544EkD-i2QRtsiNO-PcvH1voI0mh55MtMvgqeTOIM/edit?usp=sharing
Lab Registration
Please register here:https://shorturl.at/ftuz9
Important Dates
July 24, 2022: Release of training, dev and dev-test data, and evaluation scripts.September 10, 2022: Registration deadline.September 10, 2022: Release of test data (and final training and dev data).September 15, 2022: End of the evaluation cycle (test set submission closes)September 17, 2022: End of the evaluation cycle (test set submission closes)
September 20, 2022: Results releasedSeptember 23, 2022: Results released
October 5, 2022: System description paper submissions due.
October 15, October 17, 2022: Notification of acceptance.October 31, 2022: Camera-ready versions due.
December 7, 2022: WANLP 2022 workshop at EMNLP in Abu Dhabi
Submission
Submission link: https://codalab.lisn.upsaclay.fr/competitions/7274, please register to the codalab system to participate.
Submission instructions
Make sure that you create one account for each team, and submit it through one account only.
We will keep the leaderboard private till the end of the submission period, hence, results will not be available upon submission. All results will be available after the evaluation period.
You are allowed to submit max 200 submissions per day for each subtask.
The last file submitted to the leaderboard will be considered as the final submission.
Name of the output file should with json extension (e.g., team_name_run_no_subtask1.json); otherwise, you will get an error on the leaderboard.
You have to zip the json, `zip team_name_run_no_subtask1.zip team_name_run_no_subtask1.json` and submit it through the codalab submission page: https://codalab.lisn.upsaclay.fr/competitions/7274#participate
Instructions to prepare your paper
The title of paper should be in the following format:
<Team Name> at WANLP 2022 Shared Task: <Name of the title>
For example:
TeamX at WANLP 2022 Shared Task: Data Augmentation and Ensemble Models for Propaganda Detection in Arabic
More information about template and submission system: https://sites.google.com/view/propaganda-detection-in-arabic/paper-submission-guidelines
Tasks
We defined the following subtasks:
Subtask 1: Given the text of a tweet, identify the propaganda techniques used in it (multilabel classification problem).
Subtask 2: Given the text of a tweet, identify the propaganda techniques used in it together with the span(s) of text in which each propaganda technique appears. This is a sequence tagging task.
Evaluation Metrics
Subtask 1: The official evaluation metric for the task is micro-F1. However, the scorer also reports macro-F1.
Subtask 2: Task 2 is a multi-label sequence tagging task. We modify the standard micro-averaged F1 to account for partial matching between the spans.
Datasets
The Datasets are hosted on the Gitlab repository: https://gitlab.com/arabic-nlp/propaganda-detection/
Organizers
Firoj Alam, Qatar Computing Research Institute, HBKU
Hamdy Mubarak, Qatar Computing Research Institute, HBKU
Wajdi Zaghouani, HBKU
Preslav Nakov, Mohamed bin Zayed University of Artificial Intelligence
Giovanni Da San Martino, University of Padova
Citation
Please cite the following paper. In addition, in the following link (https://gitlab.com/arabic-nlp/propaganda-detection/) you can find a list of relevant work that might be useful to prepare your paper. Please see the section recommended reading.
We also shared bibliography files that might be useful: https://gitlab.com/arabic-nlp/propaganda-detection/-/tree/main/bibtex
@inproceedings{propaganda-detection:WANLP2022-overview,
title = "Overview of the {WANLP} 2022 Shared Task on Propaganda Detection in {A}rabic",
author = "Alam, Firoj and Mubarak, Hamdy and Zaghouani, Wajdi and Nakov, Preslav and Da San Martino, Giovanni",
booktitle = "Proceedings of the Seventh Arabic Natural Language Processing Workshop",
month = Dec,
year = "2022",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
}
CONTACT
For any questions related to this task, please post to this group: arabic-nlp-tasks@googlegroups.com
You are also encouraged to join the Google group https://groups.google.com/g/arabic-nlp-tasks