Task Details

 SubTask 1: Islamic Inheritance Reasoning

This task focuses on the domain of ʿlm al-mawārīth, the Islamic science of inheritance. The goal is to assess the ability of LLMs to accurately apply Islamic inheritance rules in realistic scenarios.

The questions for training and validation in this task were extracted from IslamWeb. These fatwas were transformed into multiple-choice questions (MCQs) using Gemini 2.5 and then carefully reviewed and validated by an expert in Islamic sciences to ensure accuracy and authenticity.

As part of the preprocessing phase, we rephrased ambiguous questions to ensure a single, unambiguous interpretation. We also revised the answer choices to remove semantic and numerical redundancies, such as equivalent options (e.g., 1/2 and 2/4). 

Each MCQ includes 6 answer options, with only one correct answer 

This sub-task requires LLMs to:

• Comprehend the inheritance scenario presented

• Identify eligible heirs based on their relationship to the deceased

• Apply the fixed-share rules (farāʾiḍ), including detailed calculations and priority logic
  
  

The task is organized into three levels of difficulty:

• Beginner: recognize the eligible heirs, their basic shares, and the non-eligible heirs (محجوب).                                            

• Intermediate: moderate complexity involving multiple heirs, residuary shares, and partial exclusions (الرد والعول).

• advanced: Complex inheritance scenarios with multiple deceased individuals, distribution of a defined amount of money, including nuanced cases and intricate fractional calculations.

Example of MCQ Level Beginner

 توفي عن أب، و2 أخ شقيق، و1 ابن أخ شقيق، و2 عم شقيق للأب، وأم، و2 بنت، و1 زوجة، ما هو نصيب الأم؟

☐    الثلث

☐    الربع

☒    السدس

☐    الثمن

☐   النصف

☐   لا شيء

Example of MCQ Level Intermediate

 توفي عن أخ من الأم، وبنت، وزوجة، وأختين من الأم: كم عدد أسهم البنت بعد الرد؟  

☐  سهم واحد

☐ سهمان

☐ ثلاثة أسهم

☐ أربعة أسهم

☒ سبعة أسهم

☐ ثمانية أسهم

Example of MCQ Level Advanced

 توفي عن زوجة وبنتين وأخ شقيق، والتركة 12000 درهم. ما هو النصيب النهائي لكل وارث من التركة؟
☒ الزوجة: 1500 درهم، البنتان: 8000 درهم، الأخ الشقيق: 2500 درهم

☐ الزوجة: 3000 درهم، البنتان: 8000 درهم، الأخ الشقيق: 1000 درهم

☐  الزوجة: 1500 درهم، البنتان: 6000 درهم، الأخ الشقيق: 4500 درهم

☐  الزوجة: 1500 درهم، البنتان: 8000 درهم، الأخ الشقيق: 3000 درهم

☐ الزوجة: 2000 درهم، البنتان: 7500 درهم، الأخ الشقيق: 2500 درهم

☐ الزوجة: 1000 درهم، البنتان: 8500 درهم، الأخ الشقيق: 2500 درهم

Participants will work with the following datasets:

• Multiple-Choice Questions (MCQs):

  • Training set: ≈  20.000 MCQs

  • Validation set: 1000 MCQs

  • Test set: the models will be evaluated at 1000 MCQs.

• Extra Data

Participants are also provided with the following additional resources, which may be used for fine-tuning purposes:

• A collection of 3165 fatwas from IslamWeb is available. These fatwas cover a broad spectrum of Islamic legal, ethical, and social issues and can serve as a valuable supplementary knowledge base.

Participants are also allowed to use any external data for training or fine-tuning their models, provided that it is publicly available and legally accessible.

SubTask 2: Islamic Assessment

The dataset consists of 1400 question-answer pairs, covering a wide range of topics within Islamic knowledge, including ʿulūm al-Qurʾān (Quranic studies), ʿulūm al-Ḥadīth (hadith criticism), fiqh (jurisprudence), ʾuṣūl al-fiqh (legal theory), sīrah (Prophetic Biography). It is organized into three progressively challenging difficulty levels: beginner, intermediate, and advanced.

The Sub-task aims to evaluate a model’s ability to produce accurate answers to questions in the field of classical Islamic scholarship, covering seven specialized disciplines. This is achieved through a rigorous, domain-specific question-answering benchmark. The evaluation dataset consists of multiple-choice questions (MCQs) organized into three progressively challenging difficulty levels: beginner, intermediate, and advanced.

We split the data into 700 MCQs for validation and 700 MCQs for final test.  Each question is accompanied by four options, with only one correct choice. All questions and answers in our dataset are extracted from 25 traditional Islamic reference works and have been rigorously reviewed and validated by five experts specializing in Islamic studies. Additionally, each question is carefully crafted to elicit a single, unambiguous correct answer, ensuring clarity and consistency in the evaluation process.

We provide participants with a large collection of relevant classical Islamic texts (unsupervised data), which serve as a foundational resource. The answers to the multiple-choice questions in the validation and test sets are derived from these books. As such, this corpus can be leveraged either for fine-tuning language models on Islamic studies or as part of a Retrieval-Augmented Generation (RAG) system to enhance the model’s ability to generate accurate and contextually grounded responses.

Example of MCQ Level Beginner

 ما مدة المسح على الخفين للمقيم؟

☒ يوم وليلة

☐ ثلاثة أيام بلياليهن

☐ يومان وليلتان

☐ أسبوع كامل

Example of MCQ Level Intermediate

 من شروط الأصل في القياس؟

☐ أن يكون الأصل فرعا لأصل آخر

☐ أن لا يكون الحكم ثابتا في الأصل بطريق سمعي شرعي

☒ ألا يكون الأصل فرعا لأصل آخر

☐ ألا تعرف طريقة الاستنباط

Example of MCQ Level Advance

 ما هو طريق الحكماء لإثبات وجود الواجب؟

☐ عن طريق اعتبار العالم قديمًا.

☐ عن طريق إثبات أن العالم واجب لذاته.

☒ عن طريق امتناع التسلسل والدور.

☐ عن طريق إثبات حدوث العالم.