A Framework for Generating Dangerous Scenes for Testing Robustness

Shengjie Xu, Lan Mi, and Leilani H. Gilpin

{sxu88, lmi6, lgilpin}@ucsc.edu

University of California, Santa Cruz

Paper | Code | Data

36th Conference on Neural Information Processing Systems Workshop Progress and Challenges in Building Trustworthy Embodied AI

(NeurIPS 2022 Workshop TEA)


Benchmark datasets for autonomous driving, such as KITTI, nuScenes, Argoverse, or Waymo are realistic but designed to be faultless. These datasets do not contain errors, difficult driving maneuvers, or other corner cases. We propose a framework for perturbing autonomous vehicle datasets, the DANGER framework, which generates edge-case images on top of current autonomous driving datasets. The input to DANGER are photorealistic datasets from real driving scenarios. We present the DANGER algorithm for vehicle position manipulation and the interface towards the renderer module, and present five scenario-level dangerous primitives generation applied to the virtual KITTI and virtual KITTI 2 datasets. Our experiments prove that DANGER can be used as a framework for expanding the current datasets to cover generative while realistic and anomalous corner cases.

Scene 0001

Exit parking lot

Scene 0002

Cut-in opposite

Scene 0006

Lane Change

Scene 0018


Scene 0020





title={A Framework for Generating Dangerous Scenes for Testing Robustness},

author={Shengjie Xu and Lan Mi and Leilani H. Gilpin},

booktitle={Progress and Challenges in Building Trustworthy Embodied AI},