There is growing interest in making deep neural networks more reliable. Challenges arise when models receive samples drawn from outside the training distribution. For example, a neural network tasked with classifying handwritten digits may assign high confidence predictions to cat images. Anomalies are frequently encountered when deploying ML models in the real world. Well-calibrated predictive uncertainty estimates are indispensable for many machine learning applications, such as self-driving vehicles and medical diagnosis systems. Generalization to unforeseen and worst-case inputs is also essential for robustness to distributional shift. In order to have ML models reliably predict in open environments, we must deepen technical understanding in the emerging areas of:
- Learning algorithms that can detect changes in data distribution (e.g., out-of-distribution examples).
- Mechanisms to estimate and calibrate confidence produced by neural networks in typical and unforeseen scenarios.
- Methods to improve out-of-distribution generalization, including generalization to temporal, geographical, hardware, adversarial, stylization, and image-quality changes.
- Benchmark datasets and protocols for evaluating model performance under distribution shift.
- Key applications of robust and uncertainty-aware deep learning (e.g., computer vision, robotics, self-driving vehicles, medical imaging) as well as broader machine learning tasks.
This workshop will bring together researchers and practitioners from the machine learning communities, and highlight recent work that contributes to addressing these challenges. Our agenda will feature contributed papers with invited speakers. Through the workshop we hope to help identify fundamentally important directions on robust and reliable deep learning, and foster future collaborations.
Topics of interest include but not limited to:
• Model uncertainty estimation and calibration
• Probabilistic (Bayesian and non-Bayesian) neural networks
• Robustness to distribution shift and out-of-distribution generalization
• Anomaly detection and out-of-distribution detection
• Model misspecification
• Quantifying different types of uncertainty (known unknowns, unknown unknowns, contextual anomalies, ambiguities)
• Open world recognition and open set learning
• Connections between out-of-distribution generalization and adversarial robustness
• New datasets and protocols for evaluating uncertainty and robustness
Please see the call for papers for formatting instructions and deadlines.
- Eric Nalisnick (University of Cambridge)
- Jiefeng Chen (University of Wisconsin-Madison)
- Kimin Lee (UC Berkeley)
- Matthias Hein (University of Tübingen)
- Maksym Andriushchenko (EPFL)
- Sina Mohseni (Texas A&M University)
- Alicia Y. Tsai (UC Berkeley)
- Jeremiah Liu (Google)
- Polina Kirichenko (NYU)
- Ananya Kumar (Stanford)
- Jie Ren (Google Brain)
- Kibok Lee (University of Michigan)
- Dong Yin (DeepMind)
- Wendy Shang (University of Amsterdam)
- Rodolphe Jenatton (Google Brain)
- Zack Nado (Google Brain)
- Robert Geirhos (University of Tübingen)
- Tim Pearce (University of Cambridge)
- Andrey Malinin (University of Cambridge)
- Sunil Thulasidasan (Los Alamos National Laboratory)
- Bhavya kailkhura (Lawrence Livermore National Laboratory)
- Sevvandi Kandanaarachchi (RMIT University)
- Priyanga Dilini Talagala (Monash University)
- Alexander Alemi (Google)
- Karan Goel (Stanford University)
- Disi Ji (UC Irvine)
- Florian Wenzel (Google Brain Berlin)
- Andreas Kirsch (University of Oxford)
- Pavel Izmailov (NYU)
- Joost van Amersfoort (University of Oxford)
- Lukas Ruff (TU Berlin)
- Marius Kloft (TU Kaiserslautern)
- Mike Dusenberry (Google)
- Raphael Lopes (Google)
- Siddharth Swaroop (University of Cambridge)
- Aditya Grover (Stanford University)
- Shiori Sagawa (Stanford University)
- Andrew Foong (University of Cambridge)