WANLP 2020: The Fifth Workshop for Arabic Natural Language Processing
Arabic has a widely varying collection of dialects. Many of these dialects remain under-studied due to rarity of resources. The goal of the shared task is to alleviate this bottleneck in the context of fine-grained Arabic dialect identification. Dialect identification is the task of automatically detecting the source variety of a given text or speech segment. Previous work on Arabic dialect identification has focused on coarse-grained regional varieties such as Gulf or Levantine (e.g., Zaidan and Callison-Burch, 2013; Elfardy and Diab, 2013; Elaraby and Abdul-Mageed, 2018) or country-level varieties (e.g., Bouamor et al., 2018; Zhang and Abdul-Mageed, 2019) such as the MADAR shared task in WANLP 2019 (Bouamor, Hassan, and Habash, 2019). The MADAR shared task also involved city-level classification on human translated data.
Shared Task:
The Nuanced Arabic Dialect Identification (NADI) shared task targets province-level dialects, and as such will be the first to focus on naturally-occurring fine-grained dialect at the sub-country level. The data covers a total of 100 provinces from all 21 Arab countries and come from the Twitter domain. Evaluation and task set up follow the MADAR 2019 shared task. The subtasks involved include:
Subtask 1: Country-level dialect identification: A total of 21,000 tweets, covering all 21 Arab countries. This is a new dataset created for this shared task.
Subtask 2: Province-level dialect identification. A total of 21,000 tweets, covering 100 provinces from all 21 Arab countries. This is the same dataset as in Subtask 1, but with province labels.
Unlabeled data:
Participants will also be provided with an additional 10M unlabeled tweets that can be used in developing their systems for either or both of the tasks.
Metrics: The evaluation metrics will include precision/recall/f-score/accuracy. Macro Averaged F-score will be the official metric.
Participants need to register below. Participating teams will be provided with a common training data set and a common development set. No external manually labelled data sets are allowed. A blind test data set will be used to evaluate the output of the participating teams. Each team is allowed a maximum of 3 submissions. All teams are required to report on the development and test sets (after results are announced) in their write-ups.
The shared task evaluation will be hosted through CODALAB.
CODALAB link for NADI Shared Task Subtask 1: https://competitions.codalab.org/competitions/24001?secret_key=66d8a9d9-3ac9-4ef2-bb74-6b8f3e287468
CODALAB link for NADI Shared Task Subtask 2: https://competitions.codalab.org/competitions/24002?secret_key=53f14ab8-7db2-4011-9418-c95ee3b7ea2c
Train, development, and test (unlabelled) dataset has already been released to registered participants via email. The evaluation stage is over but you can score your system on the Codalab by the post-evaluation phase.
By downloading the NADI Shared Task files from HERE you agree to the terms of the license.
We also organized the second NADI-shared task in WANLP 2021. Please find more information HERE.
December 1, 2019: First announcement of the shared task
January 30, 2020: Release of training data and scoring script
March 30, 2020 April 30, 2020: Registration deadline
March 30, 2020: Test set made available
April 15, 2020 June 5, 2020: Codalab system submission deadline
May 20, 2020 July 10, 2020: Shared task system paper submissions due.
Jun 24, 2020 July 30, 2020: Notification of acceptance.
July 11, 2020 September 7, 2020: Camera-ready version of shared task system papers due.
Sep 12, 2020 Dec 12, 2020: Workshop Dates
For any questions related to this task, please contact the organizers directly using the following email address: ubc.nadi2020@gmail.com or join google group: https://groups.google.com/d/forum/nadi_shared_task.
Muhammad Abdul-Mageed, Chiyu Zhang (The University of British Columbia, Canada), Nizar Habash (New York University Abu Dhabi) , and Houda Bouamor (Carnegie Mellon University, Qatar).