Dr. Holger Caesar, Assistant Professor

Intelligent Vehicles Lab, TU Delft

Autonomous vehicles (AVs) have come a long way in the past two decades. Yet they are nowhere close to safely carrying our children to an arbitrary location. Until AVs become a net provider of safety, comfort and reduced carbon emissions, a lot of work remains to be done. The real world is full of challenging corner-cases and unexpected behavior. My research focuses on finding more scalable approaches to address these corner cases by collecting, mining, labeling, training from and evaluating sensor data more cleverly.

Latest news

  • 10.2022: Yancong joins the group as postdoc

  • 07.2022: One PhD position available

  • 06.2022: Two keynotes at the 33rd IEEE Intelligent Vehicles (IV) Symposium at workshops on unsupervised learning and map-less driving. See you there!

  • 05.2022: One paper published at the 2022 International Conference on Robotics and Automation (ICRA)

  • 05.2022: One PhD and one PostDoc position available

  • 04.2022: Started as Assistant Professor at TU Delft




Master students:

Education & Vacancies


  • Only students from TU Delft are accepted for Master theses and internships

  • I do not supervise Bachelor theses

  • Please send me an email with your CV and any other relevant information if you are interested

  • Collaborations with industry partners are welcome

    • See the list of collaboration partners below

    • The primary supervisor needs to be in the company


  • Outperforming the state-of-the-art in

    • 3d detection/tracking

    • Prediction/planning

    • Segmentation/mapping

  • Use of prior knowledge to solve these tasks

  • Lesser forms of supervision (self, weakly or semi supervised)

  • Active learning and partial labeling techniques to select which data to annotate

  • Domain adaptation and transfer learning

  • Sensor fusion

  • Collaborative perception

  • Semantic mapping

Internal Master Thesis Positions

Collaboration with Andras Palffy

"Pose estimation and reidentification with 3+1D radar"

"Creation of a 3d library of object models for autonomous driving"

External Master Thesis Positions

Delft Haptics Lab, TU Delft

Collaboration with Assistant Professor Arkady Zgonnikov

"Automated discovery of scenario-specific prediction metrics"

Computer Graphics and Visualization lab, TU Delft

Collaboration with Assistant Professor Michael Weinmann

"Pointcloud compression for autonomous driving data"


Dr. Holger Caesar is an Assistant Professor at the Intelligent Vehicles group of TU Delft in the Netherlands. The Intelligent Vehicles group is led by Prof. Dr. Dariu Gavrila and part of the Cognitive Robotics department. Holger's research interests are in the area of Autonomous Vehicle perception and prediction, with a particular focus on scalability of learning and annotation approaches. Previously Holger was a Principal Research Scientist at an autonomous vehicle company called Motional (formerly nuTonomy). There he started 3 teams with 20+ members that focused on Data Annotation, Autolabeling and Data Mining. Holger also developed the influential autonomous driving datasets nuScenes and nuPlan and contributed to the commonly used PointPillars baseline for 3d object detection from lidar data.

Holger received a PhD in Computer Vision from the University of Edinburgh in Scotland under Prof. Dr. Vittorio Ferrari and studied in Germany and Switzerland (KIT Karlsruhe, EPF Lausanne, ETH Zurich). In his spare time he likes to hike with his small family, as well as sing, run or cross the Alps by mountainbike.

Workshops & Events

[06/2022] 3rd workshop on Unsupervised Learning for Automated Driving

Keynote speaker "A farewell to human labeled datasets"

IV 2022

[06/2022] Workshop on Bridging the gap between map-based and map-less driving

Keynote speaker "Semantic mapping towards Level 5 autonomy"

IV 2022

[12/2021] Podcast interview

Interview about nuPlan


[11/2021] Tech.AD USA

Highlighted speaker

Tech.AD 2021

[12/2020] Autonomy Talks

Invited speaker

ETH Zurich

[06/2021] AI Driving Olympics (AIDO) 6 Workshop

Workshop co-organizer, presented results of four nuScenes challenges

ICRA 2021

[12/2020] AI Driving Olympics (AIDO) 5 Workshop

Workshop co-organizer, presented nuScenes detection & lidarseg challenge results

NeurIPS 2020

[11/2020] Advanced Analytics, Beautifully Engineered in Singapore

Invited speaker

QuantumBlack, McKinsey

[10/2020] Workshop on Benchmarking Progress in Autonomous Driving

Workshop co-organizer, submitted position paper

IROS 2020

[12/2019] AI Driving Olympics (AIDO) 3 Workshop

Workshop co-organizer

NeurIPS 2019

[12/2019] Workshop on Autonomous Driving (WAD)

Challenge chair, presented nuScenes detection challenge results

CVPR 2019

[10/2019] COCO + Mapillary Joint Recognition Challenge Workshop

Workshop co-organizer, supported panoptic challenge

ICCV 2019

[05/2019] Advanced Robotics Centre

Invited seminar speaker

National University of Singapore (NUS), Singapore

[03/2019] Mobility Futures Seminar Series

Invited speaker at SMART Mobility Program

Nanyang Technological University (NTU), Singapore

[07/2018] Featured in CVPR News

Interview about my COCO-Stuff work

CVPR 2018

[10/2017] COCO + Places Joint Workshop

Workshop co-organizer, presented stuff challenge results

ICCV 2017


Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking

Whye-Kit Fong, (...), Holger Caesar, (...) Abhinav Valada

ICRA 2022

nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles

Holger Caesar, Juraj Kabzan, Kok Seang Tan et al.

CVPR workshop 2021

The Efficacy of Neural Planning Metrics: A Meta-Analysis of PKL on nuScenes

Yiluan Guo, Holger Caesar, Oscar Beijbom et al.

IROS workshop 2021

nuScenes: A Multimodal Dataset for Autonomous Driving

Holger Caesar, Varun Bankiti, Alex H. Lang et al.

CVPR 2020

PointPillars: Fast Encoders for Object Detection from Point Clouds

Alex H Lang, Sourabh Vora, Holger Caesar et al.

CVPR 2019

COCO-Stuff: Thing and Stuff Classes in Context

Holger Caesar, Jasper Uijlings, Vittorio Ferrari

CVPR 2018

Weakly Supervised Object Localization Using Things and Stuff Transfer

Miaojing Shi, Holger Caesar, Vittorio Ferrari

ICCV 2017

Region-Based Semantic Segmentation with End-to-End Training

Holger Caesar, Jasper Uijlings, Vittorio Ferrari

ECCV 2016

Joint Calibration for Semantic Segmentation

Holger Caesar, Jasper Uijlings, Vittorio Ferrari

BMVC 2015

MediaParl: Bilingual Mixed Language Accented Speech Database

David Imseng; Hervé Bourlard; Holger Caesar et al.

SLT workshop 2012