Keynote Speakers

Evaluating Reliability in Concept-Based XAI: Challenges and Prospects for Verifying Deep Neural Networks

Abstract:

As a subfield of explainable artificial intelligence (XAI), concept-based explanations (C-XAI) aim to associate human-interpretable concepts with representations in a deep neural network's (DNN) latent space. This promises to bridge the gap between human-defined symbolic knowledge and the powerful yet opaque representations learned by DNNs. However, for the use in verification and correction of DNN knowledge in critical applications like medical engineering, the obtained associations must be of high quality. This talk argues that more measures than just performance against a ground truth are necessary to ensure reliability of the obtained explanations. To illustrate this, we walk through recent work on evaluating C-XAI method and prediction quality. Looking outward, I will place these efforts in the broad landscape of XAI evaluation metrics—a cornerstone of future high-quality and human-aligned AI.

Bio:

Since September 2024, Gesina Schwalbe has led the junior research group Correctable Hybrid Artificial Intelligence (chAI) at the University of Lübeck, where she joined as a postdoctoral researcher in 2024. Her group investigates how techniques from explainable artificial intelligence can support the integration, verification, and correction of symbolic knowledge in deep neural networks.
This work builds on her previous research on safety assurance of deep neural networks for automated driving vision, conducted during her time as a researcher (2021–2023) and doctoral student (2018–2022) in the automotive industry. She received her doctorate from the University of Bamberg in 2022 and holds a Master’s (2018) and Bachelor’s (2015) degree in Mathematics from the University of Regensburg. 

Explaining Is Not Enough: How to Evaluate the Explainers 

Abstract:

Explainable Artificial Intelligence (XAI) has become a cornerstone of responsible AI, yet explaining is not enough, we must also learn how to evaluate the explainers. This talk revisits the foundations of XAI taxonomy and explores the spectrum of evaluation measures for local and counterfactual explanations, from functionally grounded metrics to human-centered assessments. Through examples spanning model explanation, medical diagnosis, and interpretable clustering, it highlights the limitations of current proxy-based evaluations and the urgent need for systematic, objective, and application-grounded frameworks.

Bio:

Riccardo Guidotti is an Associate Professor at University of Pisa. In 2013 and 2010 he graduated cum laude in Computer Science (MS and BS) at University of Pisa. He received the PhD in Computer Science with a thesis on Personal Data Analytics in the same institution. He is currently anAssociate Professor at the Department of Computer Science University of Pisa, Italy, and a member of the Knowledge Discovery and Data Mining Laboratory (KDDLab), a joint research group with the Information Science and Technology Institute of the National Research Council in Pisa. He won the IBM fellowship program and has been an intern in IBM Research Dublin, Ireland in 2015. He also won the DSAA New Generation Data Scientist Award 2018, and the Marco Somalvico Award 2021. His research interests are in explainable artificial intelligence, interpretable machine learning, quantum computing, fairness, and bias detection, time series analysis, data generation, personal data mining, clustering, and analysis of transactional data. 

Beyond the Black Box: XAI Evaluation Metrics, Frameworks, and Upcoming Trends

Abstract:

In the rapidly evolving landscape of artificial intelligence and machine learning, the need for transparency and interpretability is critical as AI systems increasingly influence decision-making across various sectors, making it essential to understand how these black-box models arrive at their conclusions to foster trust and accountability through explanations. Extracting explanations may not always be sufficient; these must be meaningful, exhibiting essential properties such as correctness, especially when feedback and actionable insights from explanations are sought in real-world scenarios. This keynote talk will explore methodologies for evaluating and analyzing explanations generated by AI/ML models, focusing on metrics and properties for evaluation, including an overview of quantitative metrics, existing frameworks for measuring interpretability, and techniques for analyzing the effectiveness and clarity of different explanation methods. It will also delve into emerging trends in AI explainability, particularly in the telecom domain, discussing industry standards and case studies that demonstrate successful applications in real-world scenarios, illustrating challenges and need for quantifiable analysis of explainable AI with Quality of Service (QoS) considerations related to Service-Level Agreement (SLA) violations. Furthermore, the talk will present the evolution of explainability evaluation methods and frameworks and their implications for AI/ML research and practice.