Schedule

Abstract

tbd

Abstract

Large language models can generate scientific text, solve complex problems, and engage in coherent dialogue. Despite these capabilities, their internal mechanisms remain difficult to understand. This is not only a technical limitation but a fundamental scientific challenge, as it restricts our ability to predict model behavior, detect errors, and justify their use in critical settings.

In this talk, we take a critical perspective on how current methods attempt to explain these models. We examine what common approaches such as feature attribution and attention analysis actually reveal, and where their interpretations become unreliable. We then focus on probing methods, questioning what it really means for a model to represent a concept, and why apparent signals in the model do not necessarily reflect how it makes decisions.

We also discuss emerging directions that move beyond passive interpretation. Concept-based methods aim to connect internal representations to human-understandable concepts, while newer approaches such as representation engineering and activation steering allow controlled modification of model behavior. These methods shift the focus from observing models to actively testing and influencing them.

Finally, we reflect on recent developments in mechanistic interpretability, which attempt to uncover how models compute by identifying internal structures and causal pathways. While promising, these approaches still face challenges in scaling and validation.

The goal of this talk is to provide a clear and grounded understanding of the field, separating well-supported insights from open questions, and highlighting directions that are likely to shape future research.