In high-stakes domains like finance and healthcare, we often deploy a single machine learning model and trust it as the “best” possible solution. This tutorial challenges that assumption by exploring the Rashomon Effect -- the idea that many different models can achieve equally high performance on the same dataset. Far from being a mere theoretical curiosity, the Rashomon Effect represents a fundamental, yet often overlooked, paradigm shift in how we should build, evaluate, and trust AI systems. By revealing model multiplicity, the Rashomon Effect enables alignment: the ability to select among high-performing models those that best match user or societal preferences.

This quarter-day tutorial will provide a comprehensive introduction to the Rashomon Effect in machine learning, moving from its foundations to practical implications. 

The tutorial is organized into four parts: 

Part 1: Introduction to the Rashomon Effect and its impact on research and real-world decision-making

Part 2: Recent algorithms for constructing Rashomon sets

Part 3: How the Rashomon Effect enables alignment

Part 4: Open problems and future directions

By merging theory, algorithms, real-world examples, and interactive exploration, this tutorial equips participants with new insights and tools to recognize, analyze, and harness the Rashomon Effect, ultimately enabling the development of more robust, interpretable, and trustworthy machine learning systems.

Part 1: Introduction to the Rashomon Effect and Its Properties (30 mins)

• What the Rashomon Effect is and how it changes data science pipelines and approaches to machine learning

• Examples of its consequences in machine learning research and real-world applications

• Causes of the Rashomon Effect and why it is inevitable

Part 2: Constructing and Exploring the Rashomon Set (30 mins)

• Algorithms for constructing Rashomon sets across model classes, such as decision trees, generalized additive models, and neural networks

• Hands-on activity using TreeFARMS and TimberTrek to obtain interpretable and accurate trees

Break (5 mins)

Part 3: From Model Multiplicity to Alignment (30 mins)

• How the Rashomon Effect leads to prediction multiplicity and its impact on applications

• Trustworthy AI: interpretability, fairness, and robustness

• Implications for regulation and policy

Part 4: Future of the Field and Open Problems (10 mins)

• The Rashomon Effect in complex model classes, including large language models