Unexpected Events: Testing Current Capability of Autonomous Vehicles

Logan Sampath

Mentor: Dr. Carolina Pacheco Oñate

Mathematical Institute for Data Science, Johns Hopkins University

Self-driving vehicles use sensors and onboard computers to automate transportation. They have the capability to drastically reduce road accidents while also increasing vehicle efficiency. Most vehicle crashes are caused by human error, and vehicles driven autonomously remove human error and can be shared more easily. However, autonomous vehicles cannot be allowed freely on the roads because they are not sufficiently accurate. The main challenge for self-driving cars is to adapt to novel situations. Many situations can be captured by training data, but at some point the vehicle must adapt to a new, rare situation. A “heavy tail distribution” can represent this issue, where there is a long list of possible events, each with a low probability, creating a long “tail” in a probability distribution graph. To quantify the extent of this issue, we will characterize the capability of current systems to generalize to novel situations. To test the ability of state-of-the-art detection systems for vehicles, we will use a Python-based, pretrained Mask R-CNN and the NuImages dataset. The NuImages dataset has detailed annotations of specific objects, important for testing unexpected events. We will compare the annotated and predicted bounding boxes and object categories. When testing the Mask R-CNN on the NuImages dataset, we can study the accuracy of the system and single out certain errors for further study. By observing the situations where mistakes are made, we can gain a greater understanding of what specifically needs to be improved in the field of autonomous vehicles.