Precognition: Seeing through the Future

in conjunction with

The 34th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)

Virtual conference, June 19th - 25th, 2021

Topics of the Workshop

Vision-based detection and recognition studies have been recently achieving highly accurate performance and were able to bridge the gap between research and real-world applications. Beyond these well-explored detection and recognition capabilities of modern algorithms, vision-based forecasting will likely be one of the next big research topics in the field of computer vision. Vision-based prediction is one of the critical capabilities of humans, and potential success of automatic vision-based forecasting will empower and unlock human-like capabilities in machines and robots.

One important application is in autonomous driving technologies, where vision-based understanding of a traffic scene and prediction of movement of traffic actors is a critical piece of the autonomous puzzle. Various sensors such as camera and lidar are used as "eyes" of a vehicle, and advanced vision-based algorithms are required to allow safe and effective driving. Another area where vision-based prediction is used is medical domain, allowing deep understanding and prediction of future medical conditions of patients. However, despite its potential and relevance for real-world applications, visual forecasting or precognition has not been in the focus of new theoretical studies and practical applications as much as detection and recognition problems. 

Through organization of this workshop we aim to facilitate further discussion and interest within the research community regarding this nascent topic. This workshop will discuss recent approaches and research trends not only in anticipating human behavior from videos but also precognition in multiple other visual applications, such as: medical imaging, health-care, human face aging prediction, early even prediction, autonomous driving forecasting, etc.

In this workshop, the topics of interest include, but are not limited to:

• Early event prediction

• Activity and trajectory forecasting

• Multi-agent forecasting

• Human behavior and pose prediction

• Human face aging prediction

• Predicting frames and features in videos and other sensors in autonomous driving

• Traffic congestion anomaly prediction

• Automated Covid-19 prediction in medical imaging

• Visual DeepFake prediction

• Short- and long-term prediction and diagnoses in medical imaging

• Prediction of agricultural parameters from satellite imagery

• Databases, evaluation and benchmarking in precognition

This is the third Precognition workshop organized at CVPR. It follows a very successful workshops organized in 2019 and 2020, which featured talks from researchers across a number of industries, insightful presentations, and large attendance. For full programs, slides, posters, and other resources, please visit 2019 and 2020 workshop websites.

Important Dates

Paper submission deadline: March 28th, 2021 (anywhere on Earth)

Notification to authors: April 12th, 2021

Camera-ready deadline: April 17th, 2021

Video presentation submission: June 1st, 2021

Workshop: June 19th, 2021

Invited Speakers

Program (times are in PT timezone)

• 12:00PM - Workshop kick-off

• 12:05PM - Invited talk: Nassir Navab, "Machine Learning, Robotic Imaging, and Augmented Reality for Computer-Assisted Interventions"

• 12:40PM - Lightning talks (full papers)

  • "End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving", Hengli Wang (Hong Kong University of Science and Technology); Peide Cai (Hong Kong University of Science and Technology), Rui R. Fan (UC San Diego), Yuxiang Sun (The Hong Kong Polytechnic University), Ming Liu (HKUST) [open access]

  • "Long-term Head Pose Forecasting Conditioned on the Gaze-guiding Prior", Shentong Mo (Carnegie Mellon University); Miao Xin (Institute of Automation, Chinese Academy of Sciences) [Best Paper Award] [open access]

• 12:55PM - Invited talk: Richard Baraniuk, "Spline Insights into Deep Networks"

• 1:30PM - Extended abstract session

  • "Panoptic Forecasting Based on Features-to-Motion-and-Features Models", Josip Šarić (UniZg-FER), Sacha Vrazic (Rimac Automobili d.o.o.), Sinisa Segvic (UniZg-FER)

  • "End-to-End Object Detection and Trajectory Prediction using Spatio-Temporal Fusion of RADAR with LiDAR and HDMap", Ankit Laddha (Uber ATG), Carlos Vallespi-Gonzalez (Uber ATG), Meet Shah (Uber ATG)

  • "Motion Prediction Metrics for Self-Driving Safety and Comfort", Skanda Shridhar (Aurora), Yuhang Ma (Aurora), Tara Stentz (Aurora), Zhengdi Shen (Aurora), Neil Traft (Aurora), Galen Clark Haynes (Aurora)

• 1:50PM - Invited talk: Anelia Angelova, "Learning to see beyond" [video]

• 2:25PM - Lightning talks (full papers)

  • "Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videos", Olga Zatsarynna (University of Bonn), Yazan Abu Farha (University of Bonn), Jurgen Gall (University of Bonn) [open access]

  • "Glaucoma Precognition Based on Confocal Scanning Laser Ophthalmoscopy Images of the Optic Disc Using Convolutional Neural Network", Krati Gupta (UTHSC), Michael Goldbaum (UCSD), Siamak Yousefi (University of Tennessee Health Science Center) [open access]

• 2:40PM - AlODrive trajectory forecasting challenge (Xinshuo Weng and Jiachen Li)

• 3:05PM - Invited talk: Drago Anguelov, "Long-term prediction in complex interactive environments"

• 3:40PM - Lightning talks (full papers)

  • "Spatio-temporal Predictive Network For Videos With Physical Properties", Yuka Aoyagi (Waseda University), Noboru Murata (Waseda University), Hidetomo Sakaino (Weathernews Inc.) [open access]

  • "Panoptic Segmentation Forecasting", Colin Graber (UIUC), Grace Tsai (Niantic), Michael Firman (Niantic), Gabriel Brostow (University College London), Alexander Schwing (UIUC) [open access]

• 3:55PM - Workshop wrap-up

• 4:00PM - End of workshop

Submission Instructions

All submitted work will be assessed based on their novelty, technical quality, potential impact, insightfulness, depth, clarity, and reproducibility. For each accepted submission, at least one author must attend the workshop and present the paper. There are two ways to contribute submissions to the workshop:

• Extended abstracts submissions are single-blind peer-reviewed, and author names and affiliations should be listed. Extended abstract submissions are limited to a total of four pages (including references). Extended abstracts of already published works can also be submitted. Accepted abstracts will be presented at the poster session, and will not be included in the printed proceedings of the workshop.

• Full paper submissions are double-blind peer-reviewed. The submissions are limited to eight pages, including figures and tables, in the CVPR style. Additional pages containing only cited references are allowed (additional information about formatting and style files is available here). Accepted papers will be presented at the poster session, with selected papers also being presented in an oral session. All accepted papers will be published by the CVPR in the workshop proceedings.

Submission website: https://cmt3.research.microsoft.com/Precognition2021

Organizers

• Khoa Luu (University of Arkansas)

• Nemanja Djuric (Aurora Innovation)

• Kris Kitani (Carnegie Mellon University)

• Minh Hoai Nguyen (Stony Brook University)

• Hien Van Nguyen (University of Houston)

• Utsav Prabhu (Google Research)

For questions please contact the organizers at precognition.organizers@gmail.com.

Workshop Sponsors

Program Committee members

• Radu Timofte, ETH Zurich

• Arun Ross, MSU

• Marios Savvides, CMU

• Ankit Laddha, Aurora Innovation

• Carlos Vallespi-Gonzalez, Aurora Innovation

• Chen Sun, Google

• Chi Nhan Duong, PDActive, Inc.

• David Ross, Google

• De-An Huang, Stanford

• Fang-Chieh Chou, Uber ATG

• Minh Triet Tran, University of Science, HCM

• Henggang Cui, Motional

• Thi Hoang Ngan Le, UA

• Kaushik Roy, NCAT

• Kha Gia Quach, PDActive, Inc.

• Meet Shah, Aurora Innovation

• Namhoon Lee, Oxford

• Nick Rhinehart, UC Berkeley

• Rowan McAllister, UC Berkeley

• Slobodan Vucetic, Temple University

• Vladan Radosavljevic, Spotify

• Wei-Chiu Ma, MIT

The Second Precognition CVPR Workshop - 2020

It was a virtual workshop for the paper presentations, the posters and the talks. Uber ATG generously sponsored to reward the authors of the best paper and the best student paper.

The First Precognition CVPR Workshop - 2019

There were about 300 attendants for the paper presentations, the posters and the talks. Uber ATG generously sponsored to reward the authors of the best paper and the best student paper.