Qur'an QA 2022 Shared Task!
Answering Questions on the Holy Qur'an
@ OSACT 2022 Workshop, LREC 2022
Marseille, France, 20th June 2022
See you at Qur'an QA 2023 إن شاء الله!
Welcome to Qur'an QA 2022 Shared Task!
Qur'an QA 2022 Overview Paper & Talk
Our shared task concluded! The OSACT5 workshop has concluded on June 20, 2022 at Marseille, France!
Check out:
Recent Updates
20 June 2022: Qur'an QA 2022 concluded!
20 June 2022: Award winners are announced -- congratulations!
19 June 2022: Proceedings and published papers are available here.
10 June 2022: Program is available here.
31 May 2022: List of accepted papers is available here.
05 May 2022: Paper notifications have been sent out.
09 Apr 2022: Paper submission deadline has been extended to April 13 AOE.
06 Apr 2022: Code submission instructions are posted here.
02 Apr 2022: Results sent to individual teams.
01 Apr 2022: Instructions and guidelines for paper submission are available here.
29 Mar 2022: Submission of runs on test set closes.
09 Mar 2022: Submission on the dev set is open. Find the instructions here.
08 Mar 2022: Team registration is open. Find the instructions here.
01 Mar 2022: Awards are announced. Find details here.
26 Feb 2022: Submission checker and evaluation scripts are released.
08 Feb 2022: Training and dev sets are released.
07 Feb 2022: New pages for task details, dataset, submission, and FAQ are added to the website!
22 Jan 2022: Discussion group (https://groups.google.com/g/quran-qa) created!
15 Jan 2022: Website is up!
Information Hub!
All details about the task and how it is evaluated can be found on the Task Details page.
Registration is now open. Check the registration page.
Awards are announced.
Dataset format and download is now available on our Dataset page.
How to submit your results is described on our Run Submission page.
Instructions and guidelines for paper submission is available on the Paper Submission page.
You have any questions about the task? Check our FAQ page or post it on our Discussion Group.
Discussion Group
Please join our Google group at https://groups.google.com/g/quran-qa to receive announcements and participate in discussions.
Task Overview
Reading comprehension (RC) task, viewed as a type of question-answering (QA) tasks, is perceived as the ideal method to evaluate language understanding by computer systems. Given a passage of text, a machine reading comprehension system is required to answer a set of questions over the given passage. We propose a shared task of Arabic Reading Comprehension over the Holy Qur’an, aiming to trigger state-of-the-art question answering and reading comprehension research on a book that is sacredly held by more than 1.8 billion people across the world.
The Holy Qur’an is composed of 114 chapters (Suras) and 6,236 verses that comprise more than 80k words in Classical Arabic. The participating systems are expected to provide answers to questions (posed in Modern Standard Arabic (MSA)) on given passages (sets of consecutive verses) from the Holy Qur’an, where an answer is a span of text extracted from the given passage.
Our dataset for the shared task, called QRCD (Qur'anic Reading Comprehension Dataset), is composed of 1,093 tuples of question-passage pairs that are coupled with their extracted answers to constitute 1,337 question-passage-answer triplets. A question might have more than one answer in the passage; therefore, a typical RC system is expected to extract all of them and return a ranked list of answer spans. For this year, the task requires returning at least one correct answer, therefore, the evaluation will reward a system for returning any of the answers to the given question, even if the question actually has more than one answer in the accompanying passage. To evaluate the system performance, multiple evaluation measures, that adopt exact and partial matching of spans, will be used.
All details about the task can be found here.
Important Dates
All times are Anywhere On Earth (AOE).
06 February 2022: Train/dev set release
25 March 2022: Registration closes.
26-29 March 2022: Runs submission (Test set available)
31 March 2022: Announcing runs results
10 April 2022 13 April 2022: Shared-task paper submission deadline
1 May 2022: Notification of acceptance
25 27 May 2022: Camera-ready submission of manuscripts
20 June 2022: Workshop in Marseille, France.
Organizers
Tamer Elsayed, Qatar University
Rana Malhas, Qatar University
Watheq Mansour, Qatar University
Contact
For any questions or inquiries related to the shared task, please contact Dr. Tamer Elsayed (telsayed ~@~ qu DOT edu DOT qa), or (better) subscribe to https://groups.google.com/g/quran-qa and post your question there.