Overview

This is a tutorial designed to equip PhD students and early-career researchers with the tools and best practices needed to critically assess and validate AI models for real-world applications. As Deep Learning systems increasingly move from research labs to real deployment, robust evaluation and uncertainty quantification become essential. This tutorial addresses the most common mistakes in model validation and guides participants through key questions to ask for realistic and reliable implementation. By the end of the tutorial, attendees will walk away with a practical and theoretical foundation for performing robust, real-world-ready Deep Learning validation.

Best practices you will learn from this tutorial:

Through a combination of lectures and interactive hands-on sessions, participants will learn how to:

Ensure data integrity through comprehensive preprocessing and validation.
Choose appropriate evaluation metrics and validation strategies to avoid misleading conclusions, especially when multiple criteria are involved, in order to gain a comprehensive understanding of model performance.
Assess model feature selection and interpretability to enhance trust and accountability.
Leverage external benchmarks and standardized datasets to improve comparability, generalizability, and reproducibility.
Evaluate potential biases and address ethical considerations in model assessment.
Quantify uncertainty arising from noisy or missing data, or from incomplete dataset coverage, by incorporating Bayesian uncertainty estimation techniques into AI models.
Control uncertainty due to incomplete data coverage using conformal prediction methods rooted in game theory (e-values).
Apply techniques to correctly size the model and understand when and how to introduce bias through regularization.
Guide models to learn more effectively by embedding prior knowledge in the form of inductive bias.

Page updated

Google Sites

Report abuse