📢  27 July 2025  Results announced

📅  26 July 2025  —  End of the test phase 

📅  21 July 2025  —  Beginning of the test phase

📅  1 June 2025  —  Dataset released and available for download

       📂  To access the dataset, register here. 

       ⬇️  Download data 

📅 22 May 2025  — The official QIAS 2025 website is now live! 

📅 8 May 2025   — The QIAS 2025 proposal has been accepted! 

QIAS 2025 invites participants to test and advance the capabilities of Large Language Models (LLMs) in understanding and reasoning within Islamic knowledge. 

This shared task evaluates how well LLMs can comprehend Islamic content, solve complex problems, and accurately answer questions across diverse areas of Islamic scholarship.

The challenge is structured into two focused subtasks, each consisting exclusively of multiple-choice questions (MCQs). These MCQs are organized into three levels of difficulty—beginner, intermediate, and advanced—allowing for a progressive assessment, from foundational knowledge to deep, complex reasoning. 

SubTask 1: Islamic Inheritance Reasoning  

Focuses on inheritance-related issues (ʿilm al-mawārīth), requiring models to apply precise reasoning and rule-based calculations grounded in Islamic jurisprudence. 

SubTask2: Islamic Assessment 

Evaluates general Islamic knowledge across a wide range of topics and disciplines. The increasing difficulty levels reflect the progressive depth and complexity of both legal and theological aspects in Islamic tradition.

• Participants can choose to take part in one or both subtasks.

• Participants are free to use any approach, such as prompting, fine-tuning, Retrieval-Augmented Generation (RAG), or a combination of methods. This flexibility encourages experimentation and helps evaluate how different methods perform on Islamic Question Answering.

We provide a complete baseline implementation. This includes code to perform predictions using Fanar API and ALLAM (Arabic LLM open source model), using prompting techniques only (no fine-tuning). 

📂 The baseline code is available here: GitLab

Participants are free to build upon this baseline and explore a variety of approaches, including:

• 🧠 Prompting strategies (zero-shot, few-shot, etc.)

• 🔧 Fine-tuning with the provided training data

• 📚 Retrieval-Augmented Generation (RAG)  with the provided training data 

• June 1, 2025: Release of training, validation, and evaluation scripts

• July 20, 2025: Registration deadline and release of test data

• July 25, 2025: Test Set published

• July 30, 2025: Final results released

• August 15, 2025: System description paper submissions due

• August 25, 2025: Notification of acceptance

• September 5, 2025: Camera-ready versions due

• November 5-9, 2025: ArabicNLP Conference

• Abdessalam BOUCHEKIF (Hamad Bin Khalifa University, Qatar)

• Samer RASHWANI (Hamad Bin Khalifa University, Qatar)

• Mohammed GHALY (Hamad Bin Khalifa University, Qatar)

• Mutaz A. AL-KHATIB (Hamad Bin Khalifa University, Qatar)

• Emad Mohamed  (Nazarbayev University, Kazakhstan )

• Wajdi ZAGHOUANI (Northwestern University, Qatar)

• Aiman M ERBAD (Qatar University, Qatar)