As we increasingly rely on ML models to contend with unstructured and unpredictable environments and tasks, it is paramount that we also acknowledge the shortcomings of our models: In practice, robots often fail to meet our expectations when we deploy them in the real world, where distributions subtly shift from training data, and where we will always continue discovering rare corner cases and failure scenarios not represented at train/design time. While reliability concerns in the face of distributional shifts are well-known, a comprehensive roadmap to address these issues at all levels of a learned autonomy stack is absent. How do we unblock ourselves, and build reliable systems for the real world?
This workshop aims to bring together a diverse group of researchers and industry practitioners to chart a roadmap for 1) addressing the disruptive impact of distributional shifts and out-of-distribution (OOD) observations on the performance of robots and 2) examining opportunities to enable generalization to unseen domains. Therefore, this workshop broadly aims to address gaps between academia and practice by igniting discussions on research challenges and their synergies at all timescales crucial to improving reliability and deploying robust systems:
Identifying the right, i.e., task-relevant, definition or viewpoint: There are many ways to define what makes data out-of-distribution. Specific choices are often left implicit and vary between problem formulations and application contexts. So, what are the most task-relevant definitions for common robotics problems? How can we quantify our definitions, and how will they influence our approach?
Real-time decision-making: How can we leverage full-stack sensory information to anticipate model errors and take actions to mitigate the consequences?
Episodic interaction with an environment: Can we develop methods that mitigate or account for shifted conditions that consistently degrade a learning-enabled robot's performance?
A continual data lifecycle as we deploy, evaluate, and retrain learning-enabled robots: How can we efficiently collect data during deployment and develop learning algorithms that maximally improve system robustness and performance?
We are taking questions for the panel discussion here!