Novel evaluation approaches for LMs in generative KE tasks include (but are not limited to) ontology description and summarization generation.
Novel evaluation approaches for LMs in completion tasks in KE include (but are not limited to) subsumption inference.
Challenges and limitations of existing evaluation methods for LMs in KE through review, revisit, and comparison of different methods
Examination of human factors in the evaluation of LMs in KE tasks, novel human-centered evaluation principles, strategies, paradigms, and interfaces
Exploration of hybrid evaluation methods that consider both quantitative and qualitative results
Datasets and benchmarks for LMs evaluation in KE tasks include novel methods for dataset and benchmark creation.
Methods and metrics for evaluating the trustworthiness, interpretability, and explainability of LM-generated results
Methods for detecting and mitigating bias
Techniques for detecting and evaluating hallucinations and inconsistencies in the output of language models, with a focus on enhancing experiment replicability
Examination of data leakage and its impact on the evaluation of language models
Criteria for identifying effective benchmarks, evaluations, and metrics
Tasks and scenarios within knowledge engineering where LMs can potentially contribute, along with the associated evaluation methods
Accepted papers will be published at CEUR-WS.org proceedings!
All submissions must be formatted using the template for submissions to the CEUR Workshop Proceedings (single column format). An Overleaf template for LaTeX users is available at here.
All papers have to be submitted electronically via EasyChair.
We encourage submissions in two formats including:
Full papers: 10 - 15 pages, including references
Short papers: 5 - 9 pages, including references
Submissions to workshops must be original. Papers that have been previously published or are under review for another journal, conference or workshop must not be considered for publication.
Further, at least one author of each accepted workshop paper has to register for the conference. Please note that workshop attendance is only granted to registered conference participants.