Qur'an QA 2022 Shared Task!
Answering Questions on the Holy Qur'an

Qur'an QA 2022 Overview Paper & Talk

Our shared task concluded! The OSACT5 workshop has concluded on June 20, 2022 at Marseille, France!

Check out:

• Qur'an QA 2022 overview talk

• Qur'an QA 2022 overview paper

• Qur'an QA 2022 overview slides

Information Hub!

• All details about the task and how it is evaluated can be found on the Task Details page.

• Registration is now open. Check the registration page.

• Awards are announced.

• Dataset format and download is now available on our Dataset page.

• How to submit your results is described on our Run Submission page.

• Instructions and guidelines for paper submission is available on the Paper Submission page.

• You have any questions about the task? Check our FAQ page or post it on our Discussion Group.

Task Overview

Reading comprehension (RC) task, viewed as a type of question-answering (QA) tasks, is perceived as the ideal method to evaluate language understanding by computer systems. Given a passage of text, a machine reading comprehension system is required to answer a set of questions over the given passage. We propose a shared task of Arabic Reading Comprehension over the Holy Qur’an, aiming to trigger state-of-the-art question answering and reading comprehension research on a book that is sacredly held by more than 1.8 billion people across the world.

The Holy Qur’an is composed of 114 chapters (Suras) and 6,236 verses that comprise more than 80k words in Classical Arabic. The participating systems are expected to provide answers to questions (posed in Modern Standard Arabic (MSA)) on given passages (sets of consecutive verses) from the Holy Qur’an, where an answer is a span of text extracted from the given passage. 

Our dataset for the shared task, called QRCD (Qur'anic Reading Comprehension Dataset), is composed of 1,093 tuples of question-passage pairs that are coupled with their extracted answers to constitute 1,337 question-passage-answer triplets. A question might have more than one answer in the passage; therefore, a typical RC system is expected to extract all of them and return a ranked list of answer spans. For this year, the task requires returning at least one correct answer, therefore, the evaluation will reward a system for returning any of the answers to the given question, even if the question actually has more than one answer in the accompanying passage. To evaluate the system performance, multiple evaluation measures, that adopt exact and partial matching of spans, will be used.

All details about the task can be found here.