SDAS 2023

CVPR 2023 Workshop

Synthetic Data for Autonomous Systems (SDAS)

Omar Maher, Alex Zook, Rares Ambrus, Dengxin Dai

Sunday June 18th, 2023

Room: West 302-305

Live Streaming: CVPR registrants can access live streaming of the sessions through this link.

Overview

This workshop considers challenges and opportunities at the frontier of using synthetic data to bootstrap and improve the performance of autonomous systems (inclusive of robots and autonomous vehicles) in varied environments and across tasks. For autonomous systems to be able to operate intelligently in the real world, they need to be able to safely perform a wide range of tasks under varied perceptual inputs. Deep learning based methods have greatly surpassed traditional bespoke algorithms for tasks such as object detection, instance, and semantic segmentation, tracking, etc. However, to achieve robust performance, which is critical for safe autonomous operation, learning-based methods require vast amounts of data for training. Moreover, a significant portion of the data collected needs to be manually labeled to provide the training signal required to optimize the neural networks for a particular task.

The workshop’s main theme will be to identify, characterize, and investigate how to qualitatively go beyond the current limitations of machine learning and computer vision methods through the use of synthetic data for autonomous systems. This is a half-day hybrid workshop, where all talks will be in person and streamed online, with the ability to receive and answer questions from online attendees. We plan to have several keynotes by leading figures in academia and industry focusing on the uses and applications of synthetic data in autonomous driving and robotics. Our goal is to use this workshop to disseminate state-of-the-art technical knowledge related to the use and usefulness of simulation for autonomous systems and to provide a venue for meaningful and vigorous debate on current challenges and ways to overcome them. Below are examples of the topics the speakers will be covering:

Synthetic data for embodied foundations: practices and success stories for generating and using synthetic data to train and test autonomous robots and vehicles.
Synthesizing Humans for Outdoor Environments: large-scale data synthesis with parameterized human 3D model.
The impact of dataset design on model performance: lessons learned and best practices for generating data that moves the needle on various ML tasks.
Synthetic Data Generation Pipelines: best practices and learned lessons for autonomous driving and robotics.
Unsupervised Domain Adaptation: critical factors impacting the success of unsupervised domain adaptation such as the network architecture, general training strategies, image resolution, and crop size.
Sim2Real Gap: the impact of different factors driving domain gaps between real and synthetic data.
Generative AI for Synthetic Data: how NeRFs and Diffusion Models could impact Synthetic Data.
Democratization of Synthetic Data, learning-based synthetic data generation, the importance of realism vs. domain randomization, closed vs. open worlds.