LLMs4OL 2026: Large Language Models for Ontology Learning

The 3rd LLMs4OL Challenge @ ISWC 2026

‌ISWC 2026, Bari, Italy | 25-29 October

LLMs4OL Paradigm

OL tasks revolve around ontology primitives, where:

1. Lexical entries L

2. Conceptual types T

3. A hierarchical taxonomy H_T

4. Non-taxonomic relations R within a heterarchy H_R

5. Axioms A for constraints and rules.

Key OL activities include corpus preparation, terminology extraction, term typing, taxonomy construction, relationship extraction, and axiom discovery. Together, these six tasks constitute the LLMs4OL task framework (see the following figure), aligning with the previously outlined LLMs4OL conceptual model.

Assuming the corpus preparation step is done by reusing ontologies publicly released in the community, the LLMs4OL paradigm is structured around three primary tasks essential for developing a primitive ontology:

Term Typing: Discover the generalized type for a lexical term

The process of assigning a generalized type to each lexical term involves mapping lexical items to their most appropriate semantic categories or ontological classes. For example, in the biomedical domain, the term aspirin should be classified under Pharmaceutical Drug. This task is crucial for organizing extracted terms into structured ontologies and improving knowledge reuse.

Example: Assign the type "disease" to the term "myocardial infarction".

Type Taxonomy Discovery: Discover the taxonomic hierarchy between type pairs.

Taxonomy discovery focuses on identifying hierarchical relationships between types, enabling the construction of taxonomic structures (i.e., is-a relationships). Given a pair of terms or types, the task determines whether one is a subclass of the other. For example, discovering that Sedan is a subclass of Car contributes to structuring domain knowledge in a way that supports reasoning and inferencing in ontology-driven applications.

Example: Recognize that "lung cancer" is a subclass of "cancer", which is a subclass of "disease".

Non-Taxonomic Relation Extraction: Identify non-taxonomic, semantic relations between types.

This task aims to extract non-hierarchical (non-taxonomic) semantic relations between concepts in an ontology. Unlike taxonomy discovery, which deals with is-a relationships, this task focuses on other meaningful associations such as part-whole (part-of), causal (causes), functional (used-for), and associative (related-to) relationships. For example, in a medical ontology, discovering that Aspirin treats Headache adds valuable relational knowledge that enhances the utility of an ontology.

Example: Identify that “virus” causes “infection” or “aspirin” treats “headache”.

More to this, in the 2nd iteration of the LLMs4OL challenge, we took a deliberate step back—refocusing attention also on a foundational phase of the OL pipeline, while maintaining its core paradigm. We introduced the Text2Onto task, which serves as an important building block in the OL process, bridging between unstructured text and the LLMs4OL paradigm. This task is defined as follows:

Text2Onto: Extract ontological terminologies and types from a raw text.

This task focuses on extracting ontological types and terms from unstructured text. Given an unstructured text corpus/documents, the goal is to identify foundational elements for ontology construction by recognizing domain-relevant vocabulary and categorizing it appropriately. We aim to extract:

Terms (or Entities): These are specific terms that form the basis of an ontology. They populate the ontology by instantiating the defined classes.

Example: For instance, COVID-19 is a term of the type Disease, and Paris is a term of the type City.

Types (or Classes): These are abstract categories or groupings that represent general concepts within a domain. They form the backbone of an ontology's structure.

Example: Types include Disease, Vehicle, or City.

By identifying and extracting these elements, the task helps bridge the gap between unstructured natural language and structured ontological knowledge. This process is critical for building knowledge representations that support reasoning, semantic integration, and advanced information retrieval.

OntoLearner Statement

Turn your submission system into lasting, reusable scientific tools!

Imagine this: the solution you build during this competition doesn’t end when the deadline passes. Instead, it becomes something any researcher, student, or engineer in the world can run, reuse, and build upon — next week, next year, and years from now. This is our mission!

OntoLearner is not just a toolkit to help you solve a task. It is a shared, living infrastructure for ontology learning where your method, once integrated, becomes part of a public, reusable ecosystem.

The OntoLearner Guide page is available to help participants advance the OL within this challenge. The library already includes integrated implementations of LLM-based, retrieval-based, and RAG-based workflows, along with approaches contributed by four previous challenge participant teams. These serve as strong starting points for you to:

Build your method on top of existing pipelines
Improve and extend prior approaches
Explore ideas and directions worth investigating

Rather than starting from scratch, you can leverage these foundations to move faster, experiment smarter, and focus on innovation.

3rd Edition of LLMs4OL Tasks

Primary tasks of the 3rd edition of LLMs4OL challenge include:

A. Flagship Task

Given raw text, construct a primitive ontology including terms, types, taxonomy, and relations.

B. Reuse Task

Given a partial ontology, add new concepts and relations extracted from text.

C. Taxonomy Task

Given ontologies from source domains, induce a hierarchical taxonomy for an unseen domain.

D. Ontologizer Task

The Ontologizer task is optional and not part of the evaluation phase. The aim here is to extend the list of ontologies within OntoLearner.

How to do this? Please refer to the OntoLearner documentation page at https://ontolearner.readthedocs.io/ontologizer/new_ontologies.html and proceed with the mentioned steps.
- Note: only a single PR is anticipated to be submitted for the collection.
A minimum of 10 ontologies is expected. We encourage consideration of a domain ontology or a topic-based ontology modularization.
- Note-1: if an ontology already exists in OntoLearner, it will not be considered.
- Note-2: duplicate submissions are not avoidable, as one team might work on ontology Y while others are working on it as well. We will consider both as correct submissions. Unless two teams' ontologies are correlated with more than 3 ontologies. (If a team adds more than 10 ontologies, this can be avoided).
- Note-3: Before the second phase ends, all the ontologies should be sent to the OntoLearner via PR. Look at the documentation guideline on how to add a new Ontologizer.
Participants should submit a paper addressing the analysis and benchmarking effort -- discuss the ontologies that are included, metrics, their nature, if you experimented with any modeling, point out the results, etc.
More info on the Ontologizer module of OntoLearner is available at: https://ontolearner.readthedocs.io/ontologizer/ontology_modularization.html
You can also learn more about ontological metrics at: https://ontolearner.readthedocs.io/ontologizer/metrics.html#complexity-score
We don't expect participants of this task to be available for presentation during the challenge session, but we will require participants to submit a short video presentation of 5 minutes on the analysis. This video will also be monitored on the OntoLearner documentation website. More information on this will be shared with participants of the task via email.

Challenge Phases

1. Solution Submission Phase: Participants should develop and submit system outputs addressing one or more of the challenge tasks. Submissions are evaluated using standardized metrics.

2. System Integration Phase: Participants integrate their systems with the OntoLearner platform to enable reproducible benchmarking. This phase is open to both new participants and previous years’ participants who wish to contribute their system implementations to the platform. However, this is mandatory for participants of phase 1.

On the OntoLearner Guide page, you can read more on how to integrate your system.
Ontologizer Task -- This is an optional task related to the extension of ontologies within the OntoLearner.

Challenge Policy

The ontologies provided for each task are publicly available sources, and participants are strictly required not to recreate new datasets from these ontologies unless the whole ontology is in the training set. The challenge organizers reserve a portion of each ontology for testing, and training on re-created datasets would lead to unfair evaluations, as it would no longer constitute a valid OL problem.
Participants may introduce additional contextual information from the World Wide Web, but this must be explicitly documented in the system submission, either in the accompanying publication or the README file, as it will be considered part of the methodology.
Final papers should be treated as research papers rather than reports.
Participant should they chose write a paper for their own work and present during the challenge workshop at ISWC.
By submitting results to this challenge, you consent to the public release of your scores at the challenge website and in the associated proceedings.
The task organizers are under no obligation to release scores, and scores may be withheld if it is the task organizers' judgment that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission, system, or science.
This challenge will have two phases. You can choose to participate in at least one task from phase 1, but participating in phase 2 is mandatory, except for the Ontologizer task.
Each task participant will be assigned at least one other team's system description paper for review. The papers will thus be peer-reviewed.
Participants are expected to adhere to the ISWC Code of Conduct, which governs professional and respectful behavior during the challenge.
Participants are required to share their code with the research community via GitHub under an appropriate license.

References

Babaei Giglou, H., D'Souza, J., Mihindukulasooriya, N., & Auer, S. (2025). LLMs4OL 2025 Overview: The 2nd Large Language Models for Ontology Learning Challenge. Open Conference Proceedings, 6. https://doi.org/10.52825/ocp.v6i.2913
Babaei Giglou, H., D’Souza, J., & Auer, S. (2024). LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge. Open Conference Proceedings, 4, 3–16. https://doi.org/10.52825/ocp.v4i.2473
Babaei Giglou, H., D’Souza, J., Auer, S. (2023). LLMs4OL: Large Language Models for Ontology Learning. In: Payne, T.R., et al. The Semantic Web – ISWC 2023. ISWC 2023. Lecture Notes in Computer Science, vol 14265. Springer, Cham. https://doi.org/10.1007/978-3-031-47240-4_22.

The 1st LLMs4OL proceedings: https://www.tib-op.org/ojs/index.php/ocp/issue/view/169

The 2nd LLMs4OL proceedings: https://www.tib-op.org/ojs/index.php/ocp/issue/view/185

Page updated

Google Sites

Report abuse