ROAD-R 2023: the Road Event Detection with Requirements Challenge

Overview

Given our successful experience on hosting a workshop and a competition on ROAD dataset at ICCV2021, and our ongoing qualified workshop and competition on ROAD++ at ICCV2023, we are proud to propose the ROAD-R challenge for Neurips2023.
In the ROAD-R challenge, we provide the participants with a dataset comprising 22 (~8-minutes long) videos annotated with road events together with a set of 243 requirements expressing hard facts about the world that are commonly known by humans (e.g., “a traffic light cannot be red and green at the same time”). We present them with two challenging tasks:

In the first task of our challenge, we ask the participants to develop the best-performing model with only a subset of the annotated data, thus encouraging them to exploit the requirements to facilitate training on the unlabelled portion of the dataset.
In the second task of our challenge, we ask the participants to create systems whose predictions are compliant with the requirements.

We invite researchers, students, practioners (and literally anybody interested in machine learning) to participate to our challenge!

Important Dates

Challenges open for registration: June 13, 2023
Training and validation data release: June 13, 2023
Test data release: August 15, 2023
Deadline for results submission: September 30, 2023 October 25, 2023
Announcement of results: October 7, 2023 October 30, 2023

Dataset Overview

This challenge is built around ROAD-R, an extension of the ROad event Awareness in Autonomous Driving (ROAD) Dataset which is based on (a fraction of) the Oxford RobotCar Dataset. ROAD-R is the first real-world dataset equipped with requirements written in propositional logic that capture background knowledge on autonomous driving.

ROAD-R contains 22 carefully selected, relatively long-duration (~8 minutes each) videos from the RobotCar dataset annotated with road events, as seen from the point of view of the autonomous vehicle capturing the video. Road events are defined as a series of bounding boxes linked in time annotated with:

the label associated to the agent (e.g., “Pedestrian”),
the action(s) the agent is doing (e.g., “Pushing Object”, “Moving Away”), and
the location(s) where the agent is placed (e.g., “Right Pavement”, “Bus Stop”).

More information about the dataset can be found in the Dataset tab or directly from here.

Furthermore, ROAD-R comes with 243 logical requirements asserting which sets of labels can be associated to the bounding boxes. The requirement "not MoveAway or not MoveForward", for example, expresses the fact that an agent cannot move away and towards the vehicle at the same time.