Embedded Files
Theory of Explainable Machine Learning Workshop
Auditorium 15
ELLIS UnConference, Bella Center, Copenhagen, Denmark
co-located with EurIPS
Explainable AI (XAI) is now deployed across a wide range of settings, including high-stakes domains in which misleading explanations can cause real harm. For example, explanations are required by law to safeguard the fundamental rights of individuals subject to algorithmic decision-making, and they inform decisions in sensitive areas such as medicine. It is therefore essential that XAI methods reliably fulfill their intended purpose rather than offering persuasive but incorrect explanations. Achieving this reliability demands a theory of explainable AI: a framework that clarifies which methods answer which questions, and under what assumptions they deliver valid, provably correct answers. Equally important is a systematic account of common failure modes—well-constructed counterexamples that reveal pitfalls and help practitioners avoid them in practice.
With this ELLIS UnConference workshop, we aim to strengthen and connect the community working on the theory of XAI, advancing foundations, curating instructive counterexamples, and translating theoretical insights into practical guidance for responsible deployment.
Invited Speakers
Invited Speakers
Shahaf
Bassan
Shahaf
Bassan
Hebrew University of Jerusalem
Jessica Hullman
Jessica Hullman
Northwestern University
Mateja
Jamnik
Mateja
Jamnik
University of Cambridge
Dolores Romero Morales
Dolores Romero Morales
Copenhagen Business School
Tiago
Pimentel
Tiago
Pimentel
ETH Zurich
Bernt
Schiele
Bernt
Schiele
MPI for Informatics Saarland University
Schedule (tentative)
Schedule (tentative)
The ELLIS UnConference workshop is co-located with EurIPS Copenhagen and will take place on December 2, 2025.
When you click on the title of a talk the abstract expands.
8:00 - 9:00
Registration
9:00 - 9:30
Jessica Hullman
Explanations are a means to an end
Modern methods for explainable machine learning are often based on developers' intuitions about what makes an explanation valuable to a user. We show how taking a statistical decision theoretic perspective provides a more rigorous basis for characterizing the value of explanation methods. In this view, the value of an explanation technique is the expected improvement in performance of a decision-maker who gains access to those explanations. We show how this functionally-grounded approach can be applied across diverse use cases, such as clinical decision support, providing recourse, or debugging. We demonstrate its use to characterize the maximum "boost" in performance on a particular task that an explanation could provide an idealized decision-maker. We argue that evaluation should meld theoretical and empirical perspectives on the value of explanation, and present a workflow that combines theoretic, human complementary, and behavioral definitions of the value of explanation.
9:30 - 10:00
Shahaf Bassan
Explanation, Guaranteed! Provable Certificates for Machine Learning Explanations
Despite significant advances in explainability methods in recent years, many widely used techniques still rely on heuristic principles and lack rigorous mathematical guarantees. In safety-critical domains, where incorrect explanations can undermine trust or lead to harmful decisions, this poses a fundamental challenge. In this talk, I will present emerging approaches for generating explanations with formal, provable guarantees and certified reliability. I will discuss the algorithmic and scalability barriers that arise in this setting, highlight recent techniques that address these challenges, and explore related theoretical questions, including the computational complexity of producing sound and trustworthy explanations.
10:00 - 10:30
Kiet Vo
Explanation Design in Strategic Learning: Sufficient Explanations that Induce Non-harmful Responses
We study the design of explanations in algorithmic decision-making with strategic agents---individuals who may modify their inputs in response to explanations of a decision maker's (DM's) predictive model. While the demand for algorithmic transparency has led much prior work to assume full model disclosure, in practice DMs typically provide only partial information via explanations, which can cause agents to misinterpret the model and take actions that unintentionally reduce their own utility. A central open question is therefore how DMs should communicate explanations that avoid harming strategic agents while still supporting their own goals, e.g., minimising predictive error. In this work, we analyse widely used explanation methods and establish a necessary condition to prevent explanations from inducing self-harming responses. Furthermore, we show that action recommendation-based explanations (ARexes), which encompass counterfactual explanations, are sufficient to induce all non-harmful responses. Under a conditional homogeneity assumption, this sufficiency extends to ARex-generating methods, echoing the revelation principle in information design. To demonstrate their practical utility, we introduce a simple learning procedure that jointly optimises the predictive model and the explanation-generating policy. Experiments on both synthetic and real-world tasks show that ARexes enable DMs to achieve high predictive performance while preserving agents' utility, offering a principled strategy for safe and effective partial model disclosure.
Magamed Taimeskhanov
Feature Attribution from First Principles
Feature attribution methods are a popular approach to explain the behavior of machine learning models. They assign importance scores to each input feature, quantifying their influence on the model’s prediction. However, evaluating these methods empirically remains a significant challenge. To bypass this shortcoming, several prior works have proposed axiomatic frameworks that any feature attribution method should satisfy. In this work, we argue that such axioms are often too restrictive, and propose in response a new feature attribution framework, built from the ground up. Rather than imposing axioms, we start by defining attributions for the simplest possible models, i.e., indicator functions, and use these as building blocks for more complex models. We then show that one recovers several existing attribution methods, depending on the choice of atomic attribution. Subsequently, we derive closed-form expressions for attribution of deep ReLU networks, and take a step toward the optimization of evaluation metrics with respect to feature attribution.
10:30 - 11:00
Coffee break
11:00 - 11:30
Bernt Schiele
Inherent Interpretability for Deep Learning in Computer Vision
Computer Vision has been revolutionized by Machine Learning and in particular Deep Learning. End-to-end trainable models allow to achieve top performance across a wide range of computer vision tasks and settings. While recent progress is remarkable, current deep learning models are hard to interpret. In this talk discuss a new class of neural networks with a high degree of inherent interpretability. In particular, these novel networks leverage a series of input-dependent linear transformations, that outperform existing attribution methods both quantitatively as well as qualitatively.
11:30 - 12:00
Tiago Pimentel
Leveraging Causality for Better Machine Learning Interpretability
Causality has the potential to offer a principled framework for understanding why machine learning models behave as they do, enabling explanations that go beyond surface correlations to uncover the underlying mechanisms driving model predictions. In this talk, I will present recent work demonstrating how causal inference can illuminate core aspects of language model behaviour. I will first discuss how econometric tools—e.g., difference-in-differences and regression discontinuity design—allow us to estimate causal effects that are otherwise inaccessible, revealing how training dynamics and tokeniser design choices systematically influence model outputs. I will then turn to mechanistic interpretability, focusing on causal abstraction as a formal framework for aligning neural networks’ computations with higher-level algorithms that they may implement. Building on recent theoretical and empirical results, I will present the non-linear representation dilemma, which shows that without constraints on how information is encoded, causal abstraction becomes vacuous: arbitrarily powerful alignment maps can make any model appear to implement any algorithm. Together, these perspectives illustrate both the promise and the limits of causality as a tool for ML interpretability, pointing toward a path that integrates causality to develop more faithful explanations of model behaviour.
12:00 - 12:30
Amir-Hossein Karimi
Explainable AI is Causality in Disguise
Explainable AI (XAI) has intrigued researchers since the earliest days of artificial intelligence. However, with the surge in AI-based applications—especially deep neural network models—the complexity and opacity of AI models have intensified, renewing the call for explainability. As a result, an overwhelming number of methods have been introduced, reaching a point where surveys now summarize other surveys on XAI. Yet, significant challenges persist, including unresolved debates on accuracy-explainability tradeoffs, conflicting evaluation metrics, and repeated failures in sanity checks. Further complications arise from fairness violations, robustness issues, privacy concerns, and susceptibility to manipulation. While there’s broad agreement on the importance of XAI, expert panels and major conferences continue to reveal that the only consensus on how to achieve it is a lack of one. This has led some to question whether the discord stems from a fundamental absence of ground truth for defining “the” correct explanation. We argue that explainable AI is, in fact, a supervised problem—albeit with a target rooted in a profound, often elusive, understanding of reality. In this sense, XAI is causality in disguise. By reframing XAI queries as causal inquiries—whether about data, models, or decisions—we prove the necessity and sufficiency of causal models for XAI, encouraging community convergence around advanced methods for concept and causal discovery, potentially through interactive, approximate causal inference.
Amin Parchami
Faithful Concept Traces for Explaining Neural Network Decisions
Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge. Many post-hoc concept-based approaches have been introduced to understand their workings, yet they are not always faithful to the model. Further, they make restrictive assumptions on the concepts a model learns, such as class-specificity, small spatial extent, or alignment to human expectations. In this work, we put emphasis on the faithfulness of such concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations. Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced. We also leverage foundation models to propose a new concept-consistency metric, C2-Score, that can be used to evaluate concept-based methods. Compared to prior work, we show that our concepts are quantitatively more consistent and that users find them to be more interpretable, while retaining competitive ImageNet performance.
12:30 - 13:30
Lunch
13:30 - 14:00
Dolores Romero Morales
Making LIME Explanations Collective with Mathematical Optimization
Local Interpretable Model-Agnostic Explanations (LIME) is a popular tool in Explainable Artificial Intelligence used to shed light on black-box machine learning models for tabular data. When LIME is applied to a group of instances, independent linear models are obtained, often overlooking global properties, such as smoothness, cost-sensitive feature selection or fairness. In this talk, we propose a novel framework, called Collective LIME (CLIME), where the surrogate models built for the different instances are linked, being smooth with respect to the coordinates of the instances. The surrogate models built are Generalized Linear Models (GLMs), enabling us to address with the very same methodology different prediction tasks: classification (with, e.g., logistic regression surrogate models), regression (with, e.g., linear regression surrogate models) and regression of counting data (with, e.g., Poisson regression surrogate models). With this, our approach enables one to control global sparsity, i.e., which features are used ever, even if sparse models are built for each instance. Since the combinatorial optimization problem controlling the features that can be used by CLIME is typically intractable, we instead consider a heuristic approach that repeatedly linearizes the global error around the current feature set and solves a knapsack subproblem to choose the next feature set. We illustrate CLIME on a collection of benchmark datasets.
14:00 - 14:30
Mateja Jamnik
Explainable AI: From Black Boxes to Concept-Based Understanding
Current AI systems achieve high predictive accuracy, yet their decision processes remain opaque, difficult to interrogate, and impossible to correct. This limits their deployment in safety-critical applications and undermines human trust. In this talk I describe concept-based models that build human-aligned concepts into the decision process. This makes explanations part of the model itself, enables targeted interventions when concepts go wrong, and maintains strong predictive accuracy. I will show how these models support “what-if” queries, cope with incomplete concept annotations, and generalise across realistic application conditions. I conclude by arguing that such approaches offer a practical route to AI systems that can be deployed responsibly and work with human experts as genuine partners.
14:30 - 15:00
Abhijeet Mulgund
Theoretical Aspects of Deep-Learned Error-Correcting Codes
Deep learning has been applied recently to obtain error-correcting codes, a crucial component of communication systems. Deep-learned error-correcting codes (DL-ECC) are competitive with classical codes and sometimes outperform them. However, there are significant challenges for the approach to become a broadly applicable methodology, and the lack of interpretability is a major issue. In our experiments we interpreted previously published DL-ECCs which neuralize Turbo codes and Reed-Muller codes, and deep-learned feedback codes. Observations include the presence of dominant Fourier coefficients for Turbo codes and the possibility of simplifying the neuralized Reed-Muller encoder to get an interpretable code.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
Amir Mehrpanah
Spectral Analysis as a Basis for a Theory of XAI
Establishing trust in AI systems requires interpretable explanations of their decisions. Yet, effective models are often highly nonlinear, yielding complex explanations that conflict with humans’ preference for simplicity. This tension defines a central dilemma in Explainable AI (XAI): should simplicity be enforced ante-hoc (by designing inherently interpretable models) or post-hoc (by explaining complex ones)? Both strategies rely on heuristics and lack a rigorous foundation. We propose spectral analysis as a principled framework for studying XAI, enabling a formal examination of explanation and model complexity. Using image data as a case study, we analyze the implicit assumptions underlying ante- and post-hoc approaches and identify key research directions. Spectral methods, we argue, offer a unifying theoretical basis for formal XAI across modalities.
15:00 - 20:00
ELLIS Unconference Program
Organizers
Organizers
Nicolò
Cesa-Bianchi
Nicolò
Cesa-Bianchi
University of Milan
Tim
van Erven
Tim
van Erven
University of Amsterdam
Ulrike
von Luxburg
Ulrike
von Luxburg
University of Tübingen
Robi Bhattacharjee
Robi Bhattacharjee
University of Tübingen
Emmanuel Esposito
Emmanuel Esposito
University of Milan
Gunnar
König
Gunnar
König
University of Tübingen
Call for Contributions (closed)
Call for Contributions (closed)
We invite submissions to the workshop; Accepted submissions will be selected for presentation as a talk or a poster. Contributions are submitted as extended abstracts (up to 2 pages) and can contain already published as well as new unpublished work.
We welcome contributions that address the theoretical underpinnings of explainability methods. Submissions may, for example, consist of provable guarantees for explanation methods, identified limitations and impossibility results, illuminating examples and counterexamples, or formal conjectures and work in progress. We are agnostic to the explanation paradigm (feature importance, concept-based, causal, mechanistic interpretability) and modality (tabular data, images, text, and beyond).
Submission Guidelines
Format: NeurIPS style (use the linked official NeurIPS LaTeX template)
Length: Up to 2 pages (excluding references, which may extend beyond the limit). Since the entire submission is an abstract, there is no need to use the abstract environment.
Review: Submissions will be lightly reviewed for relevance and quality. Accepted abstracts will be selected for presentation as posters or short talks. You can specify your preferred presentation format along with the submission and we will try to accommodate your preference.
Archival Policy: The workshop is non-archival. Authors are encouraged to submit work that is preliminary, in progress, or recently published elsewhere.
Important Dates
Submission Deadline: October 15, 2025, AoE
Accept/Reject Notifications: October 31, 2025, AoE
Workshop: December 2, 2025
You can submit your extended abstract via the following form: Submission Form.
The submission form is now closed.
Location
Location
Center Blvd. 5, 2300 København S
Center Blvd. 5, 2300 København S
Our workshop is located in Auditorium 15
Our workshop is located in Auditorium 15
Sponsors
Sponsors
Supported by the Tübingen AI Center
and the Tübingen Research Cluster
"Machine Learning: New Perspectives for Science"
Supported by the Tübingen AI Center
and the Tübingen Research Cluster
"Machine Learning: New Perspectives for Science"
Page updated
Google Sites
Report abuse