Our Mission

We believe libraries are at the heart of our society and education, therefore we believe in impacting technological innovation in these age-old institutions as well as modern digital libraries when we can! We believe in transforming workflows in modern digital library systems... We believe AI can make things faster, we also believe AI can be built to be reliable! Watch our short promotional video below to learn more about our mission and task.

... The way we propose to transform workflows in modern digital libraries is to start with one of the core tasks in libraries, i.e. subject indexing. Subject specialists at libraries currently undergo a tedious manual process of memorizing large subject taxonomies and then figuring out a way to ensure that each record submitted to the library has a good coverage of its representative subject tags. This takes a lot of time and effort. Instead, bringing in AI, especially modern technologies like LLMs, to the table can make the process smart, simple, and user-friendly. This goes a long way in making a mark of technological innovation in society. 

This is why we created the LLMs4Subjects shared task to provide the opportunity in the community to come up with many innovative solutions around using AI i.e. LLMs for subject indexing. We give you a large-scale annotated dataset, compiled over many years, to build your systems. What we need are creative ideas to this problem. Would you build a RAG approach, or finetune a model, or maybe innovate around few-shot or chain-of-thought prompting? 

Explore this website to find detailed information on the task description, dataset, evaluation criteria, key dates, how to participate, and the link to the official LLMs4Subjects Google Group for questions and announcements. 

Organizers

Jennifer D'Souza, Sameer Sadruddin, Holger Israel, Mathias Begoin, and Diana Slawig.

All organizers are affiliated with the  TIB Leibniz Information Centre for Science and Technology in Hannover, Germany.

Contact

llms4subjects [at] gmail.com

Funding Statement

The LLMs4Subjects shared task organized as SemEval 2025 Task 5 is jointly supported by the SCINEXT project (BMBF, German Federal Ministry of Education and Research, Grant ID: 01lS22070) and the NFDI4DataScience initiative (DFG, German Research Foundation, Grant ID: 460234259).