Overview
The dataset comprises Arabic responses generated by large language models to real Islamic questions drawn from Fanar logs, covering everyday religious inquiries. Each response may support its answer by citing Quranic verses (Ayah) and Prophetic sayings (Hadith), and every such citation is annotated in fine detail. The training set contains approximately 5,000 model responses, comprising over 15,000 annotated citations across roughly 33,000 labelled segments, and supports all three subtasks Span Detection, Hallucination Identification, and Correction from a single file. The dataset also supports the fourth subtask, Answer Relevance. However, the relevance label is only included in the datasets for that subtask.
Each response is provided with its question, its generated_answer, and a list of annotations. Every annotation corresponds to one citation in the answer and is typed as either Ayah or Hadith.
Each citation is decomposed into segments, each marked by its character start and end offsets into generated_answer (end-exclusive), so the exact quoted text can be recovered. The segment types are:
Ayah : the quoted Qur'anic verse text (in Qur'an citations).
matn : the text of the Hadith (in Hadith citations).
isnad : the chain of narration, where the response provides one (Hadith citations).
claimed_source : the attribution stated in the response (e.g. a surah:ayah reference, or a Hadith collection such as Bukhari or Muslim).
Each segment carries a label of correct, incorrect, or N/A:
The text segment (Ayah or matn) is judged correct when it faithfully matches the authentic source, and incorrect when it does not match.
The isnad and claimed_source are judged on their own only when the corrsponding ayah or hadith matn is correct. When the text segment is incorrect, the isnad and claimed_source are set to N/A — a broken or fabricated citation cannot have a meaningfully verifiable chain or source, so these are not scored. Such gated segments keep their spans but appear with label N/A and are excluded from labelling evaluation.
When a text segment (Ayah or matn) is incorrect, its annotation includes a correction slot giving the authentic form of the citation. This supports the correction subtask (Subtask 3), where a system proposes the corrected text for a mis-cited verse or Hadith.
Only Quranic Verses and Hadith matns can be corrected. Isnad and claimed sources have no corrections.
A single incorrect citation can have more than one acceptable correction. A Hadith matn is often attested across several collections with slightly different wording, and a Qur'anic quotation may correspond to more than one verse. The gold therefore records a **set** of acceptable corrections for each incorrect segment, and a prediction is counted correct when it matches **any one** of them. A system must only submit a single correction per span, not the full list of corrections, to be counted as correct.
For Quran the corrected text is the full verse; for Hadith it is the matn. When an incorrect span cannot be grounded to any authentic Qur'an or Hadith text (a fabrication or an unrecoverable garble), the correction is the literal string خطأ, and a system is expected to output خطأ for such spans.
All Quranic verses and Hadiths are grounded to the provided corpora `quranic_verses.json` and `six_hadith_books.json` and must match them.
Some answers address the question without quoting any Quran or Hadith. These responses carry no citation segments and are marked accordingly, so systems are also expected to recognise when nothing has been cited.
The Quranic verses are verified against the canonical Quranic text, and Hadith against standard collections, making the annotations a fine-grained ground truth for detecting where generated Islamic content departs from the authentic sources.
Only Qur'anic and/or Hadith citations that are correct or corrected may have a relevance label. A citation span is considered Relevant if it directly or indirectly answers the question. Citation spans that are merely topically related but do not answer the question, as well as citation spans that are entirely unrelated, are classified as Non-relevant. Refer to the detailed relevance label definitions published on the Subtask 4 webpage