Abstract: To get clear street-view and photo-realistic simulation in autonomous driving, we present an automatic video inpainting algorithm that can remove traffic agents from videos and synthesize missing regions with the guidance of depth/point cloud. By building a dense 3D map from stitched point clouds, frames within a video are geometrically correlated via this common 3D map. In order to fill a target inpainting area in a frame, it is straightforward to transform pixels from other frames into the current one with correct occlusion. Furthermore, we are able to fuse multiple videos through 3D point cloud registration, making it possible to inpaint a target video with multiple source videos.

Dataset

The large-scale dataset consists of synchronized Labeled image and LiDAR scanned point clouds. It captured by HESAI Pandora All-in-One Sensing Kit. It can be used for inpainting and other tasks. [Download]

Publication

DVI: Depth Guided Video Inpainting for Autonomous Driving.  

Miao Liao, Feixiang Lu, Dingfu Zhou, Sibo Zhang, Wei Li, Ruigang Yang. 

European Conference on Computer Vision (ECCV 2020).

[PDF] [Result Video] [10 min Presentation] [Inpainting Dataset] [Github]

Media:

实现最强自动驾驶街景仿真，百度ECCV 2020视频修复论文解读 

ECCV2020论文收录揭晓，百度AI入选10篇论文，涵盖众多研究领域