ROAD++: The Second Workshop & Challenge on Event Detection for Situation Awareness in Autonomous Driving

Co-hosted by ICCV 2023

October 2 , Hybrid

Workshop Agenda

Overview

The accurate detection and anticipation of actions performed by multiple road agents (pedestrians, vehicles, cyclists and so on) is a crucial problem to solve for enabling autonomous vehicles with the capability to support reliable and safe autonomous decision making. While the task of teaching an autonomous vehicle how to drive can be tackled in a brute-force, direct reinforcement learning approach, a sensible and attractive alternative is to first provide the vehicle with situation awareness capabilities, to then feed the resulting intermediate representations of road scenarios (in terms of agents, events and scene configuration) to a suitable decision-making strategy. This, in particular, has the advantage to allow the modelling of the reasoning process of road agents in a theory-of-mind approach, inspired by the behavior of the human mind in similar contexts. We propose a change of paradigm towards action detection in that the focus in not on the objects/actors themserlves and their appearance, but on what they do and the meaning of their behaviour. Accordingly to our previous successful experience in hosting ROAD workshop in ICCV2021 (link), the goal of ROAD++ workshop is to put to the forefront of the research in autonomous driving the topic of situation awareness, intended as the ability to create semantically useful representations of dynamic road scenes in terms of the notion of ‘road event’.
This includes the following tasks:
• Detecting and modelling ‘atomic’ events, intended as simple actions performed by a single agent. • Detecting and modelling complex activities, contributed to by several agents over an extended period of time. • Predicting agent intentions. • Predicting the trajectory of pedestrians, vehicles and other road users. • Forecasting future road events (both atomic and complex). • Based on all these elements, deciding what action the autonomous vehicle should perform next. • Modelling the reasoning processes of road agents in terms of goals or mental states. • Realistic simulation to generate training data for road event action detection. • Realistic simulation to generate training data for road event action detection.

We invite both paper contributions on these topics, as well as submissions of entries to a challenge specifically designed to test situation awareness capabilities in autonomous vehicles.

Paper submission

Submitted papers will follow the standard ICCV 2023 template. Authors are welcome to submit a supplementary material document with details on their implementation; however, reviewers are not required to consult this additional material when assessing the submission. The Workshop will allow for the submission of papers concurrently submitted elsewhere, with the aim of aggregating all relevant efforts in this area.

Accepted papers will get a poster at the main conference, and the very best ones also an oral.

Challenge

The accurate detection and anticipation of actions performed by multiple road agents (pedestrians, vehicles, cyclists and so on) is a crucial task to address for enabling autonomous vehicles to make autonomous decisions in a safe, reliable way. While the task of teaching an autonomous vehicle how to drive can be tackled in a brute-force fashion through direct reinforcement learning, a sensible and attractive alternative is to first provide the vehicle with situation awareness capabilities, to then feed the resulting semantically meaningful representations of road scenarios (in terms of agents, events and scene configuration) to a suitable decision-making strategy. In perspective, this has also the advantage of allowing the modelling of the reasoning process of road agents in a theory-of-mind approach, inspired by the behaviour of the human mind in similar contexts.

Accordingly, the goal of this Challenge is to put to the forefront of the research in autonomous driving the topic of situation awareness, intended as the ability to create semantically useful representations of dynamic road scenes, in terms of the notion of a road event.

Dataset

To allow the research community to thoroughly investigate situation awareness for autonomous driving, this workshop introduces ROAD++, an extension of the first ROad event Awareness in Autonomous Driving Dataset, built upon (a fraction of) the Waymo Dataset (https://waymo.com/open/).

ROAD++ is the result of annotating ~55k carefully selected, relatively short-duration (20 second each) videos from the Waymo dataset in terms of what we call road events (REs), as seen from the point of view of the autonomous vehicle capturing the video. REs are defined as triplets E = (Ag;Ac; Loc) composed by a moving agent Ag, the action Ac it performs, and the location Loc in which this takes place. Agent, action and location are all classes in a finite list compiled by surveying the content of the 22 videos.

More information about the dataset can be found the the dataset tab or directly from here (TBA)