Bio: Mor Geva is an Assistant Professor (Senior Lecturer) at the School of Computer Science at Tel Aviv University and a Visiting Researcher at Google Research. Her research focuses on understanding the inner workings of large language models, to increase their transparency and efficiency, control their operation, and improve their reasoning abilities. Mor completed a Ph.D. in Computer Science and a B.Sc. in Bioinformatics at Tel Aviv University, and was a postdoctoral researcher at Google DeepMind and the Allen Institute for AI. She was nominated as one of the MIT Rising Stars in EECS and is a laureate of the Séphora Berrebi Fellowship in Computer Science. She was awarded the Dan David Prize for graduate students in the field of AI and received an Outstanding Paper Award at EACL 2023.

Bio: Dieuwke Hupkes is a research scientist at Meta. Among other things, she works on better understanding how (large) language models generalise, what they (don't) understand and what that even means, and more generally on how they can reasonably be evaluated. She is excited about the new opportunities such models bring us and the new scientific challenges that go hand in hand with that.

Latent Space Exploration for Safe and Trustworthy AI

Abstract: The recent performance advancement in deep neural network models has substantially increased their dissemination to vast application areas. Given this widespread adoption, ensuring the safety and trustworthiness of AI models is more critical than ever before. A standard way for assessing AI model performance involves extrinsic evaluation across a set of downstream tasks. While effective for advancing state-of-the-art, these evaluations offer limited insights into how models learn and solve a task. In this talk, I advocate for a deeper exploration of model internals, particularly their latent space, to fully test their capabilities, to build better models and to increase trust in them. I will present a few use cases to support this stance. For instance, the intrinsic dimensionality trend of models explains the robustness-generalization tradeoff during adversarial training, informing the design of robust and scalable adversarial methods without compromising generalization. Moreover, the study of the structure and representation of knowledge within latent space is effective in evaluating the language comprehension capabilities of models and enables interpretation of their predictions.

Bio: Hassan Sajjad is an Associate Professor in the Faculty of Computer Science at Dalhousie University, Canada, and the director of the HyperMatrix lab. His research focuses on natural language processing (NLP) and safe and trustworthy AI, particularly text generation, robustness, generalization, alignment, interpretation, and explainability of NLP models. His research work is recognized at several prestigious venues such as NeurIPS, ICLR, and ACL and is featured in prominent tech blogs including MIT News. 

Efficiency as an Inductive Bias: Towards Tokenizer-free and Dynamically Sparse Language Models

Abstract: Efficiency in language models is often hailed as a solution to democratise access to AI technology and to make it more environmentally sustainable. In this talk, I emphasise an additional and sometimes neglected advantage of efficiency: namely, providing an inductive bias for language use and acquisition. Firstly, I will explore how dynamically compressing token representations and/or the key-value cache in Transformer LLMs boosts memory and time efficiency. In addition, this process also discovers abstractions from raw data and results in tokenizer-free models. Secondly, I will demonstrate how fine-tuning subnetworks in LLMs allows for adapting them with limited memory and parameter budgets. In addition, learning how to route information through such neural pathways also leads to better generalisation in new tasks.

Bio: Edoardo M. Ponti is an assistant professor in natural language processing at the University of Edinburgh and a visiting professor at NVIDIA. Previously, he was a visiting postdoctoral scholar at Stanford University and a postdoctoral fellow at Mila and McGill University in Montreal. In 2021, he obtained a PhD in computational linguistics from the University of Cambridge, St John’s College. His main research foci are modular deep learning, efficient neural architectures, and computational typology. His research earned him a Google Research Faculty Award and 2 Best Paper Awards at EMNLP 2021 and RepL4NLP 2019. He is a (terrible) violinist, football and tennis player, and an aspiring practitioner of heroic viticulture.