We encourage participants to focus on building and refining their systems during the solution submission phase and the integration during the system integration phase.

Solution Submission Phase

Based on our experience from previous editions of the challenge, we observed that leaderboard-driven platforms can sometimes encourage participants to focus primarily on incremental score improvements rather than on the broader scientific and methodological goals of the challenge. To foster deeper analysis, reproducibility, and methodological innovation, we have decided to adopt a simplified submission process this year.

Therefore, the challenge will not use CodaBench or a public leaderboard. Instead, participants will submit their predictions through a dedicated Google Form.

Test Sets:

The test sets for tasks are as follows.  

• Task A:  The Flagship Task test case is available at https://github.com/sciknoworg/LLMs4OL-Challenge/tree/main/2026/TaskA-Flagship (2018 test case)

• Task B: The Reuse Task test case is available at: https://github.com/sciknoworg/LLMs4OL-Challenge/tree/main/2026/TaskB-Reuse  (1799 test case)

• Task C: The Taxonomy Learning consists of multiple test cases, and the test cases are available at https://github.com/sciknoworg/LLMs4OL-Challenge/tree/main/2026/TaskC-Taxonomy. We have 4 test cases named as Phyra, Archeon, Biora, and Hylex. The prediction should be submitted for each test case separately.

Expected Output Formats:

For tasks, we expect strict formatting to ease the in-house evaluation and avoid any errors, so please take them seriously into consideration:

• Task A: For this task, we expect the following JSON format - a list of your system predictions, where each test case should be formatted as follows:

[

{

"id": "..." ,                                                        # ID of the test case - the same as the input sample id

"primitive-ontology-triples": [[subject, predicate, object], ...]    # List of triplets

   },

...

]

• Task B: Similar to Task A, we expect the following JSON format, but with the following key and values.

[

{

"id": "..." ,                                                                 # ID of the test case - the same as the input sample id

"extended-primitive-ontology-triples": [[subject, predicate, object], ...]    # List of new triplets 

   },

...

]

• Task C:   For this task, we expect the following JSON format - a list of your system predictions, which consist of parent and child pairs and should be formatted as follows:

[

{

"parent": "..." , 

"child": "..."    

   },

...

]

• Task D [Optional Ontologizer Task]: For submissions to this task, please add your final merge request URL to the OntoLearner in the submission form only.

Submission Policy

• Participants may submit up to five (5) runs per task until 17 June as an intermediate evaluation phase.

• A single intermediate evaluation will be conducted, and the results will be communicated privately to each participating team. This feedback is intended to help teams assess their approach and make improvements before the final submission.

• After 17 June, each team will be allowed to submit two (2) final runs for the official evaluation.

• No public rankings or leaderboards will be displayed during the challenge evaluation phase.

• The test set will remain unchanged throughout the challenge period. The intermediate evaluation and the final evaluation will be performed on the same test cases.

• Teams joining after 17 June will not be eligible for the intermediate evaluation feedback.

We believe this format will encourage participants to focus on developing robust and generalizable solutions while still providing an opportunity to receive feedback and refine their systems before the final evaluation.

Reporting Potential Issues with Test Cases

If you identify any potential issues with the test cases, please feel free to report them by providing the task name and the corresponding test case ID. We will carefully review each report and verify whether the test case contains an error or inconsistency.

While we have made every effort to ensure the quality and correctness of all test cases, minor issues may still remain. We greatly appreciate your feedback and collaboration in helping us maintain the highest possible quality of the evaluation data.

System Integration Phase 

Please note that participation in Phase 2 is mandatory for teams participating in Phase 1.

In this phase, the focus shifts toward system integration and finalization. The process will be kept simple to reduce overhead and support easy submission of working systems. Participants will be able to submit their system either as a single notebook or script, or via an OntoLearner issue, depending on what is most convenient. The goal of this phase is to ensure smooth integration of the developed approaches rather than introducing additional complexity. So, there will be more updates on this in a short time. 

Submission Form

• The submission form can be accessed via: https://forms.gle/cgLWLhj6zzZTTayj6 

⚠️ Please note that, due to unforeseen technical issues, the release of the evaluation scripts may be delayed by a couple of days. We appreciate your understanding and patience. In the meantime, you may refer to the task pages and OntoLearner for the evaluation metrics.