former Safe Artificial Intelligence for Automated Driving, in conjunction with CVPR 2025
Date: June 12th
Room: 104 D
After the success of ML and AI-based approaches in outperforming traditional vision algorithms, recently a lot of research effort is dedicated to understanding of the limitations and the general behavior of AI methods in a broad range of computer vision applications. Specifically for a successful introduction of ML and AI in a wider range of products, safety is often a top priority. Being able to ensure safety of ML based computer vision is key to unlock its potential in a broad range of safety related applications and future products. In domains like automotive, aviation and the medical domain, it paves the way towards systems with a greater degree of autonomy and assistance for humans.
The workshop focuses on bringing together researchers, engineers, and practitioners from academia, industry, and government to exchange ideas, share their latest research, and discuss the latest trends and challenges in this field. The workshop also aims to foster collaboration between different stakeholders, including computer vision researchers, machine learning experts, robotics engineers and safety experts, to create a comprehensive framework for developing safe AI systems for all domains.
Overall, SAIAD aims to advance the state-of-the-art in safe AI, address the most pressing challenges, and provide a platform for networking and knowledge sharing among the experts in this field.
This workshop will focus on safe artificial intelligence in a wide range of application domains of Computer Vision and Pattern Recognition. Compared to previous iterations of this workshop (2019-2022, except 2023 and 2024) that were exclusively focused on the automotive domain, the organizers have decided to expand the thematic field to include (non-exhaustive listing):
Automated Driving
Applied Robotics
Aerospace
Rail
Medical Applications
Ethics
Irrespective of the application domain, safety mechanisms for AI are implemented throughout the entire development pipeline of safety-related AI-based systems. This includes stages such as specification, data and model selection, training, evaluation/testing, monitoring, and assurance argumentation. Each stage presents opportunities to ensure that AI systems remain trustworthy, robust, and aligned with safety requirements.
Model behavior (e.g., CNN, Transformers, World Models): How can the behavior of complex models, such as world models (e.g., PRISM-1, GAIA-1, LINGO-1), be described in a way that ensures safety and aligns with the system's operational goals?
Key Performance Indicators (KPIs): How can KPIs be derived from or aligned with safety requirements?
Dataset specification: How should training and test datasets be specified to ensure full coverage of the input space?
Generative AI models for data generation: What is the role of advanced generative models in generating synthetic data, simulating complex environments, and creating realistic edge cases for testing safety-critical systems?
Synthetic data and augmentation: How can synthetic data generation and augmentation methods contribute to the robustness of models, particularly in rare or extreme situations, such as adversarial attacks or unforeseen operational conditions?
Special model design: What architectural techniques (e.g., attention mechanisms, multi-modal fusion in transformers) can be used to enhance the reliability and trustworthiness of AI models?trustworthiness of model output?
Model redundancy strategies: How can redundancy, such as ensemble methods, sensor fusion, or the use of multiple architectures, be applied to increase resilience to failure or adversarial inputs?
Transparent model training: How can the training process be made more transparent to ensure that models are learning the right features? How can prior domain knowledge or safety-critical information be incorporated into the training process?
New loss functions for safety: What new or adapted loss functions can be introduced to focus on safety-related objectives, such as robustness to noise, interpretability, or fairness?
Meta-classification techniques: How effective are meta-classifiers that rely on uncertainty modeling (e.g., confidence intervals) or attention-based heatmaps in predicting safety issues during deployment?
Robustness to anomalies: How can training be optimized to increase robustness to input anomalies, sensor failures, and adversarial attacks?
Domain robustness: What methods can be used to increase the robustness of AI systems across different domains or datasets, ensuring generalizability while preserving safety?
Novel evaluation schemes: What innovative testing and evaluation strategies can be applied to assess AI models' performance under safety-critical conditions, particularly in simulated environments that mimic real-world complexities (e.g., through world models)?
Interpretable and explainable AI: What new diagnostic tools can provide insight into how models function, including intermediate layers and feature maps, and how do these tools improve interpretability for stakeholders?
Evaluation of diagnostic techniques: How can we rigorously evaluate and compare various techniques for interpreting and explaining models? What metrics should be used to determine the most reliable method?
Test methods and coverage criteria: What testing methods (e.g., search-based testing, simulation-based testing) are effective in uncovering a model's weaknesses, and what criteria should be used to define test completion?
Uncertainty modeling: How can uncertainties during inference be modeled and quantified (e.g., using Monte Carlo dropout or Bayesian techniques) to ensure safe decision-making during deployment?
Detection of anomalies: How can we detect anomalies or outliers in real-time, including adversarial attacks, sensor errors, and out-of-distribution inputs, to maintain system reliability?
Plausibility checks of model outputs: How can we implement automated plausibility checks (e.g., for identifying implausible object sizes or positions in vision tasks) to detect when a model may be generating unsafe outputs?
Field monitoring: How can we effectively monitor AI systems in the field, especially in decentralized and dynamic environments like fleets of autonomous vehicles or sensor networks, to ensure that performance remains within safety parameters?
Risk reduction and traceability: How can we trace safety mechanisms back to their contributions to overall system risk reduction? How do these mechanisms align with safety requirements across the system’s lifecycle?
Combination of evidence: How can we combine qualitative and quantitative evidence (e.g., from formal verification, statistical testing, and field monitoring) to build a compelling assurance argument for the safety of AI systems?
