23 November 2023

Hybrid Workshop 

by SAIL 

on Fundamental Limits of 

Large Language Models

at Universität Bielefeld

From popular media to scientific discourse, examples of large language models (short: LLMs; like ChatGPT, Bard, etc.) exhibiting both impressive capabilities and stunning failure cases are abound. These inconsistent results beg the question: What are the fundamental limits of LLMs? In this workshop, we try to spotlight theoretical work from machine learning, linguistics, and cognitive science and thus wish to provide insight into the underlying principles behind the capabilities and limits of LLMs. The program will contain a series of invited talks by esteemed researchers around the world, featuring each topic. 

INVITED SPEAKERS

Leonie Weissweiler,

LMU Munich

Leonie Weissweiler is a fourth-year PhD student at the Center for Information and Language Processing at LMU Munich, supervised by Prof. Dr. Hinrich Schütze. She is interested in leveraging methods from Natural Language Processing to contribute to the empirical study of the emergent structure of Language, its evolution and processing in the brain. Her current research focuses on multilingual unsupervised Morphosyntax and probing language models for Construction Grammar.

William Merrill,

New York University

William Merill is a Ph.D. student at the CDS at NYU, where he is advised by Tal Linzen and supported by an NSF graduate research fellowship and by AI2. His research leverages linguistic and computational theory to better understand deep learning. He has worked on characterizing the computational power of transformers for representing linguistic structure and solving reasoning problems. He has also analyzed the types of semantics that can be learned distributionally, i.e., the language decipherment problem faced by today's large language models.

Iris van Rooij 

(Donders Institute / Radboud University Nijmegen, Netherlands) 

Iris van Rooij is a Professor of Computational Cognitive Science at Radboud University, the Netherlands and Guest Professor at Aarhus University, Denmark. Her research interests lie at the interface of psychology, philosophy and theoretical computer science, with a focus on the theoretical foundations of computational explanations of cognition.

The work she will be presenting at the workshop is joint work with Olivia Guest, Federico Adolfi, Ronald de Haan, Antonina Kolokolova, and Patricia Rich.

Nouha Dziri 

Allen Institute for AI, USA)

Nouha Dziri is a research scientist at AI2 working with Yejin Choi and the Mosaic team. She earned her PhD from the University of Alberta in 2022. Her work revolves around three main axes: 1) Understanding the limits of Transformers and their inner workings. 2) Building smaller LMs that can learn more efficiently. And 3) Better aligning LMs with human values and ethical principles. In the past, she worked at Google Research, Microsoft Research, and Mila. Her work has been published in top-tier venues including Neurips, ICML, TACL, ACL, NAACL and EMNLP. She actively serves as a reviewer and area chair for NLP conferences, journals, and workshops and was recognized among the best reviewers at ACL 2021.

Venue 

CITEC, Room 1.204, University of Bielefeld

https://maps.app.goo.gl/TLSZxtMSE69DQrmG6

Gefördert durch Ministerium für Kultur und Wissenschaft des Landes Nordrhein-Westfalen.