Sensor fusion for non-rigid 3D SLAM from MAV

Project supported by Kakenhi (young scientist)

Depth Image fusion with Variational Message Passing

Video

Real-time 3D mapping of large indoor scenes

Summary of the project:

The frame-to-global-model approach is widely used for accurate 3D modeling from sequences of RGB-D images. Because still no perfect camera tracking system exists, the accumulation of small errors generated when registering and integrating successive RGB-D images causes deformations of the 3D model being built up. In particular, the deformations become significant when the scale of the scene to model is large. To tackle this problem, we propose a two-stage strategy to build in details a large-scale 3D model with minimal deformations where the first stage creates accurate small-scale 3D scenes in real-time from short subsequences of RGB-D images while the second stage re-organises all the results from the first stage in a geometrically consistent manner to reduce deformations as much as possible. By employing planar patches as the 3D scene representation, our proposed method runs in real-time to build accurate 3D models with minimal deformations even for large-scale scenes.

Publications:

D. Thomas and A. Sugimoto, Modeling large-scale indoor scenes with rigid fragments using RGB-D cameras. Computer Vision and Image Understanding (CVIU 2016).

D. Thomas and A. Sugimoto, Parametric Surface Representation with Bump Image for Dense 3D Modeling Using an RBG-D Camera. International Journal of Computer Vision (IJCV 2016).

Tutorial at 3DV 2014: Dense 3D reconstruction using an RGB-D camera.

D. Thomas and A. Sugimoto. A two-stage strategy for real-time dense 3D reconstruction of large-scale scenes. In Proc. of CDC4CV’14, International workshop of ECCV'14 on Consumer Depth Cameras for Computer Vision, 2014.

D. Thomas and A. Sugimoto. A Flexible Scene Representation Using a RGB-D Camera. In Proc. of the IEEE International Conference on Computer Vision (ICCV’13), 2013.

Demo source code (C++)