Wolfram Burgard is VP for Automated Driving Technology at the Toyota Research Institute. He is on leave from his professorship at the University of Freiburg where he heads the research group for Autonomous Intelligent Systems. Wolfram Burgard is known for his contributions to mobile robot navigation, localization and SLAM (simultaneous localization and mapping). He has published more than 350 papers in the overlapping area of robotics and artificial intelligence.

At the Toyota Research Institute we are following the one-system-two-modes approach to building truly automated cars. More precisely, we simultaneously aim for the L4/L5 chauffeur application and the the guardian system, which can be considered as a highly advanced driver assistance system of the future that prevents the driver from making any mistakes. TRI aims to equip more and more consumer vehicles with guardian technology and in this way to turn the entire Toyota fleet into a giant data collection system. To leverage the resulting data advantage, TRI performs substantial research in machine learning and, in addition to supervised methods, particularly focuses on unsupervised and self-supervised approaches. In this presentation, I will present three recent results regarding self-supervised methods for perception problems in the context of automated driving. I will present novel approaches to inferring depth from monocular images and a new approach to panoptic segmentation.

This talk presents recent work at TU Delft on 3D semantic scene analysis in urban traffic using video and/or LiDAR. First, I discuss fast and compact stereo image segmentation using Instance Stixels [1]. These augment single-frame stixels with instance information, which can be extracted by a CNN from the RGB image input. As a result, the novel Instance Stixels method efficiently computes stixels that account for boundaries of individual objects, and represents instances as grouped stixels that express connectivity. Second, I discuss the outcome of an experimental study on video- and LiDAR-based 3D person detection (i.e. pedestrians and cyclists) [2]. I report how the detection performance depends on distance, number of LiDAR points, amount of occlusion, and the optional use of LiDAR intensity cues. I include results on the new EuroCity Persons 2.5D (ECP2.5D) dataset, which is about one order of magnitude larger than KITTI regarding persons. Finally, I cover domain transfer experiments between the KITTI and ECP2.5D datasets, and discuss future challenges.

[1] T. Hehn, J.F.P. Kooij and D.M. Gavrila. “Fast and Compact Image Segmentation using Instance Stixels”. Under review at IEEE Trans. on Intelligent Vehicles, 2020.

[2] J. van der Sluis, E.A.I. Pool and D.M. Gavrila. “An Experimental Study on 3D Person Localization in Traffic Scenes”. Under review at IEEE Trans. on Intelligent Vehicles, 2020.

Zhijian Liu is pursuing his Ph.D. degree at the MIT EECS Department, under the supervision of Prof. Song Han. Zhijian received his B.Eng. degree in computer science from Shanghai Jiao Tong University. His research mainly focuses on efficient and hardware-friendly machine learning and its applications in vision and language.

3D neural networks are widely used in real-world applications (e.g., AR/VR headsets, self-driving cars). They are required to be fast and accurate; however, limited hardware resources on edge devices make these requirements rather challenging. Previous work processes 3D data using either voxel-based or point-based neural networks; while both types of 3D models are not hardware-efficient because of the large memory footprint and random memory access. In this thesis, we study 3D deep learning from the efficiency perspective. We first systematically analyze the bottlenecks of previous 3D methods. We then combine the best from point-based and voxel-based models together and propose a novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We evaluate our proposed method on various tasks including 3D part segmentation (for objects), 3D semantic segmentation (for indoor and outdoor scenes), and 3D object detection (for outdoor scenes). Across all four benchmarks, our proposed method achieves state-of-the-art performance with a 2.8x measured speedup on average. Furthermore, our model has been deployed to the autonomous racing vehicle of MIT Driverless, achieving a larger detection range, higher accuracy, and lower latency for efficient LiDAR perception.

Felix Heide has co-authored over 50 publications and filed 6 patents. He received his Ph.D. from the University of British Columbia under the advisement of Professor Wolfgang Heidrich. He obtained his MSc from the University of Siegen, and was a postdoc at Stanford University. His doctoral dissertation won the Alain Fournier Ph.D. Dissertation Award and the SIGGRAPH outstanding doctoral dissertation award.

Imaging has become an essential part of how we communicate with each other, how autonomous agents sense the world and act independently, and how we research chemical reactions and biological processes. Today's imaging and computer vision systems, however, often fail for the ``edge cases'', for example in low light, fog, snow, or highly dynamic scenes. These edge cases are a result of ambiguity present in the scene or signal itself, and ambiguity introduced by imperfect capture systems. In this talk, I will present several examples of computational imaging methods that resolve this ambiguity by jointly designing sensing and computation for domain-specific applications. Instead of relying on intermediate image representations, which are often optimized for human viewing, these cameras are designed end-to-end for a domain-specific task. In particular, I will show how to co-design optics, sensors and ISP for automotive HDR ISPs, detection and tracking (beating Tesla's latest OTA Model S Autopilot), how to optimize thin freeform lenses for wide field of view applications, and how to extract accurate dense depth from three gated images (beating scanning lidar, such as Velodyne's HDL64). Finally, I will present computational imaging systems that extract domain-specific information from faint measurement noise using domain-specific priors, allowing us to use conventional intensity cameras or conventional Doppler radar to image ``hidden'' objects outside the direct line of sight at ranges of more than 20m.

PhD student at Key Lab of Machine Perception (MOE), Peking University

Qingyong Hu is a second-year D.Phil student (Oct 2018 - ) in the Department of Computer Science at the University of Oxford, supervised by Niki Trigoni and Andrew Markham. His research goal is to build intelligent systems that are able to achieve an effective and efficient perception and understanding of 3D scenes. In particular, his research focuses on large-scale point cloud segmentation, dynamic point cloud processing, and point cloud tracking.

We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Extensive experiments show that our RandLA-Net can process 1 million points in a single pass with up to 200$\times$ faster than existing approaches. Moreover, our RandLA-Net clearly surpasses state-of-the-art approaches for semantic segmentation on two large-scale benchmarks Semantic3D and SemanticKITTI.

Thomas Chaton holds a double master degree from Telecom Paristech in Security / Data Science, with a specialization in Machine Learning.After graduation, he worked 2 years at Fujitsu AI Labs in London on ML CV applied to anomaly detection for manufacturing processes and 1 year at HELIX RE working on state-of-the-art AI point cloud models for segmentation at scale, which became the starting point for TorchPoints3d framework.He is currently working as a Senior Research Engineer at Sky, London.

Research in Deep Learning for point cloud data has boomed in the past few years resulting in rapid improvements of state of art (SOTA) results on common benchmarks. Yet, developing new architecture and evaluating the real impact of new contributions remains a challenge. We propose an open source framework that enables easy reproduction of the existing SOTA models and exploration of new models.

In this talk, I talk about how we can use 3D data to self-supervise existing problems, and how in particular panoramic images such as from the Matterport3D dataset can fuel many computer vision tasks. I will further talk about how to leverage these ideas in the context of 3D shape reconstruction / completion, getting high-quality 3D models even when no ground truth data is available in real-world scan.

Dr. Jens Behley is post-doctoral researcher at the Photogrammetry and Robotics Lab since 2015. He obtained his Ph.D. in computer science from University of Bonn in 2014. The Ph.D. thesis addresses LiDAR-based perception in urban environments and was supervised by Prof. Dr. Armin B. Cremers. His research interest includes semantic scene understanding in urban environments and semantic interpretation in the agricultural domain with a strong focus on Deep Learning.

Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly.

In this talk, we present our extension of SemanticKITTI, which is a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark, for training and evaluation of laser-based panoptic segmentation. We present our approach to get temporally-consistent instance annotations and our strong two-stage baselines combining state-of-the-art LiDAR-based semantic segmentation approaches with a state-of-the-art detector enriching the segmentation with instance information.

Furthermore, we present our novel, single-stage, and real-time capable panoptic segmentation approach using a shared encoder with a semantic and instance decoder. We leverage the geometric information of the LiDAR scan to perform a novel, distance-aware tri-linear upsampling, which allows our approach to use larger output strides than using transpose convolutions leading to substantial savings in computation time.