Embedded Files
Theory of Explainable Machine Learning Workshop
ELLIS UnConference, Bella Center, Copenhagen, Denmark
co-located with EurIPS
Explainable AI (XAI) is now deployed across a wide range of settings, including high-stakes domains in which misleading explanations can cause real harm. For example, explanations are required by law to safeguard the fundamental rights of individuals subject to algorithmic decision-making, and they inform decisions in sensitive areas such as medicine. It is therefore essential that XAI methods reliably fulfill their intended purpose rather than offering persuasive but incorrect explanations. Achieving this reliability demands a theory of explainable AI: a framework that clarifies which methods answer which questions, and under what assumptions they deliver valid, provably correct answers. Equally important is a systematic account of common failure modes—well-constructed counterexamples that reveal pitfalls and help practitioners avoid them in practice.
With this ELLIS UnConference workshop, we aim to strengthen and connect the community working on the theory of XAI, advancing foundations, curating instructive counterexamples, and translating theoretical insights into practical guidance for responsible deployment.
Invited Speakers
Invited Speakers
Shahaf
Bassan
Shahaf
Bassan
Hebrew University of Jerusalem
Jessica Hullman
Jessica Hullman
Northwestern University
Mateja
Jamnik
Mateja
Jamnik
University of Cambridge
Dolores Romero Morales
Dolores Romero Morales
Copenhagen Business School
Tiago
Pimentel
Tiago
Pimentel
ETH Zurich
Bernt
Schiele
Bernt
Schiele
MPI for Informatics Saarland University
Workshop Agenda
Workshop Agenda
The ELLIS UnConference workshop is co-located with EurIPS Copenhagen and will take place on December 2, 2025.
Time Session
8:00 - 9:00 Registration
9:00 - 10:30 Talks
10:30 - 11:00 Coffee break
11:00 - 12:30 Talks
12:30 - 13:30 Lunch
13:30 - 15:00 Talks
15:00 - 20:00 ELLIS Unconference Program
Detailed Schedule (tentative)
Detailed Schedule (tentative)
Click the title to expand the abstract.
9:00 - 9:30
Jessica Hullman
Explanations are a means to an end
Modern methods for explainable machine learning are often based on developers' intuitions about what makes an explanation valuable to a user. We show how taking a statistical decision theoretic perspective provides a more rigorous basis for characterizing the value of explanation methods. In this view, the value of an explanation technique is the expected improvement in performance of a decision-maker who gains access to those explanations. We show how this functionally-grounded approach can be applied across diverse use cases, such as clinical decision support, providing recourse, or debugging. We demonstrate its use to characterize the maximum "boost" in performance on a particular task that an explanation could provide an idealized decision-maker. We argue that evaluation should meld theoretical and empirical perspectives on the value of explanation, and present a workflow that combines theoretic, human complementary, and behavioral definitions of the value of explanation.
9:30 - 10:00
Shahaf Bassan
Explanation, Guaranteed! Provable Certificates for Machine Learning Explanations
Despite significant advances in explainability methods in recent years, many widely used techniques still rely on heuristic principles and lack rigorous mathematical guarantees. In safety-critical domains, where incorrect explanations can undermine trust or lead to harmful decisions, this poses a fundamental challenge. In this talk, I will present emerging approaches for generating explanations with formal, provable guarantees and certified reliability. I will discuss the algorithmic and scalability barriers that arise in this setting, highlight recent techniques that address these challenges, and explore related theoretical questions, including the computational complexity of producing sound and trustworthy explanations.
10:00 - 10:30
Kiet Vo
Explanation Design in Strategic Learning: Sufficient Explanations that Induce Non-harmful Responses
We study the design of explanations in algorithmic decision-making with strategic agents---individuals who may modify their inputs in response to explanations of a decision maker's (DM's) predictive model. While the demand for algorithmic transparency has led much prior work to assume full model disclosure, in practice DMs typically provide only partial information via explanations, which can cause agents to misinterpret the model and take actions that unintentionally reduce their own utility. A central open question is therefore how DMs should communicate explanations that avoid harming strategic agents while still supporting their own goals, e.g., minimising predictive error. In this work, we analyse widely used explanation methods and establish a necessary condition to prevent explanations from inducing self-harming responses. Furthermore, we show that action recommendation-based explanations (ARexes), which encompass counterfactual explanations, are sufficient to induce all non-harmful responses. Under a conditional homogeneity assumption, this sufficiency extends to ARex-generating methods, echoing the revelation principle in information design. To demonstrate their practical utility, we introduce a simple learning procedure that jointly optimises the predictive model and the explanation-generating policy. Experiments on both synthetic and real-world tasks show that ARexes enable DMs to achieve high predictive performance while preserving agents' utility, offering a principled strategy for safe and effective partial model disclosure.
Taimeskhanov Magamed
Feature Attribution from First Principles
Feature attribution methods are a popular approach to explain the behavior of machine learning models. They assign importance scores to each input feature, quantifying their influence on the model’s prediction. However, evaluating these methods empirically remains a significant challenge. To bypass this shortcoming, several prior works have proposed axiomatic frameworks that any feature attribution method should satisfy. In this work, we argue that such axioms are often too restrictive, and propose in response a new feature attribution framework, built from the ground up. Rather than imposing axioms, we start by defining attributions for the simplest possible models, i.e., indicator functions, and use these as building blocks for more complex models. We then show that one recovers several existing attribution methods, depending on the choice of atomic attribution. Subsequently, we derive closed-form expressions for attribution of deep ReLU networks, and take a step toward the optimization of evaluation metrics with respect to feature attribution.
11:00 - 11:30
Tiago Pimentel
TBD
TBD
11:30 - 12:00
Dolores Romero Morales
TBD
TBD
12:00 - 12:30
Amir-Hossein Karimi
Explainable AI is Causality in Disguise
Explainable AI (XAI) has intrigued researchers since the earliest days of artificial intelligence. However, with the surge in AI-based applications—especially deep neural network models—the complexity and opacity of AI models have intensified, renewing the call for explainability. As a result, an overwhelming number of methods have been introduced, reaching a point where surveys now summarize other surveys on XAI. Yet, significant challenges persist, including unresolved debates on accuracy-explainability tradeoffs, conflicting evaluation metrics, and repeated failures in sanity checks. Further complications arise from fairness violations, robustness issues, privacy concerns, and susceptibility to manipulation. While there’s broad agreement on the importance of XAI, expert panels and major conferences continue to reveal that the only consensus on how to achieve it is a lack of one. This has led some to question whether the discord stems from a fundamental absence of ground truth for defining “the” correct explanation. We argue that explainable AI is, in fact, a supervised problem—albeit with a target rooted in a profound, often elusive, understanding of reality. In this sense, XAI is causality in disguise. By reframing XAI queries as causal inquiries—whether about data, models, or decisions—we prove the necessity and sufficiency of causal models for XAI, encouraging community convergence around advanced methods for concept and causal discovery, potentially through interactive, approximate causal inference.
Amin Parchami
FaCT: Faithful Concept Traces for Explaining Neural Network Decisions
Deep learning has been applied recently to obtain error-correcting codes, a crucial component of communication systems. Deep-learned error-correcting codes (DL-ECC) are competitive with classical codes and sometimes outperform them. However, there are significant challenges for the approach to become a broadly applicable methodology, and the lack of interpretability is a major issue. In our experiments we interpreted previously published DL-ECCs which neuralize Turbo codes and Reed-Muller codes, and deep-learned feedback codes. Observations include the presence of dominant Fourier coefficients for Turbo codes and the possibility of simplifying the neuralized Reed-Muller encoder to get an interpretable code.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
13:30 - 14:00
Bernt Schiele
TBD
TBD
14:00 - 14:30
Mateja Jamnik
Tbd
TBD
14:30 - 15:00
Abhijeet Mulgund
Theoretical Aspects of Deep-Learned Error-Correcting Codes
Deep learning has been applied recently to obtain error-correcting codes, a crucial component of communication systems. Deep-learned error-correcting codes (DL-ECC) are competitive with classical codes and sometimes outperform them. However, there are significant challenges for the approach to become a broadly applicable methodology, and the lack of interpretability is a major issue. In our experiments we interpreted previously published DL-ECCs which neuralize Turbo codes and Reed-Muller codes, and deep-learned feedback codes. Observations include the presence of dominant Fourier coefficients for Turbo codes and the possibility of simplifying the neuralized Reed-Muller encoder to get an interpretable code.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
We present two results which illustrate theoretical research directions raised by the experiments. Tolerant property testing is a relevant model for post hoc interpretation of DL-ECCs, and there is an efficient testing algorithm for a variant of convolutional codes. In addition,deep learning error-correcting codes can be viewed as an optimization problem over spaces of codes. In order to interpret the learned codes and extract interpretable representations it is useful to understand the loss landscape and the learning process. We show that under any parameterization of the space of codes, linear codes induce critical points of the loss landscape.
Amir Mehrpanah
Spectral Analysis as a Basis for a Theory of XAI
Establishing trust in AI systems requires interpretable explanations of their decisions. Yet, effective models are often highly nonlinear, yielding complex explanations that conflict with humans’ preference for simplicity. This tension defines a central dilemma in Explainable AI (XAI): should simplicity be enforced ante-hoc (by designing inherently interpretable models) or post-hoc (by explaining complex ones)? Both strategies rely on heuristics and lack a rigorous foundation. We propose spectral analysis as a principled framework for studying XAI, enabling a formal examination of explanation and model complexity. Using image data as a case study, we analyze the implicit assumptions underlying ante- and post-hoc approaches and identify key research directions. Spectral methods, we argue, offer a unifying theoretical basis for formal XAI across modalities.
Organizers
Organizers
Nicolò
Cesa-Bianchi
Nicolò
Cesa-Bianchi
University of Milan
Tim
van Erven
Tim
van Erven
University of Amsterdam
Ulrike
von Luxburg
Ulrike
von Luxburg
University of Tübingen
Robi Bhattacharjee
Robi Bhattacharjee
University of Tübingen
Emmanuel Esposito
Emmanuel Esposito
University of Milan
Gunnar
König
Gunnar
König
University of Tübingen
Call for Contributions (closed)
Call for Contributions (closed)
We invite submissions to the workshop; Accepted submissions will be selected for presentation as a talk or a poster. Contributions are submitted as extended abstracts (up to 2 pages) and can contain already published as well as new unpublished work.
We welcome contributions that address the theoretical underpinnings of explainability methods. Submissions may, for example, consist of provable guarantees for explanation methods, identified limitations and impossibility results, illuminating examples and counterexamples, or formal conjectures and work in progress. We are agnostic to the explanation paradigm (feature importance, concept-based, causal, mechanistic interpretability) and modality (tabular data, images, text, and beyond).
Submission Guidelines
Format: NeurIPS style (use the linked official NeurIPS LaTeX template)
Length: Up to 2 pages (excluding references, which may extend beyond the limit). Since the entire submission is an abstract, there is no need to use the abstract environment.
Review: Submissions will be lightly reviewed for relevance and quality. Accepted abstracts will be selected for presentation as posters or short talks. You can specify your preferred presentation format along with the submission and we will try to accommodate your preference.
Archival Policy: The workshop is non-archival. Authors are encouraged to submit work that is preliminary, in progress, or recently published elsewhere.
Important Dates
Submission Deadline: October 15, 2025, AoE
Accept/Reject Notifications: October 31, 2025, AoE
Workshop: December 2, 2025
You can submit your extended abstract via the following form: Submission Form.
The submission form is now closed.
Location
Location
Center Blvd. 5, 2300 København S
Center Blvd. 5, 2300 København S
Detailed information about the location of the workshop will be provided as soon as possible
Detailed information about the location of the workshop will be provided as soon as possible
Sponsors
Sponsors
Supported by the Tübingen AI Center
and the Tübingen Research Cluster
"Machine Learning: New Perspectives for Science"
Supported by the Tübingen AI Center
and the Tübingen Research Cluster
"Machine Learning: New Perspectives for Science"
Page updated
Google Sites
Report abuse