RGB-Depth Camera Network

From object tracking to 3D reconstruction, RGB-Depth (RGB-D) camera networks play an increasingly important role in many vision and graphics applications. Practical applications often use sparsely-placed cameras to maximize visibility, while using as few cameras as possible to minimize cost. In general, it is challenging to calibrate sparse camera networks due to the lack of shared scene features across different camera views. In this paper, we propose a novel algorithm that can accurately and rapidly calibrate the geometric relationships across an arbitrary number of RGB-D cameras on a network. Our work has a number of novel features. First, to cope with the wide separation between different cameras, we establish view correspondences by using a spherical calibration object. We show that this approach outperforms other techniques based on planar calibration objects. Second, instead of modeling camera extrinsic calibration using rigid transformation, which is optimal only for pinhole cameras, we systematically test different view transformation functions including rigid transformation, polynomial transformation and manifold regression to determine the most robust mapping that generalizes well to unseen data. Third, we reformulate the celebrated bundle adjustment procedure to minimize the global 3D reprojection error so as to fine-tune the initial estimates. Finally, our scalable client-server architecture is computationally efficient: the calibration of a five-camera system, including data capture, can be done in minutes using only commodity PCs. Our proposed framework is compared with other state-of-the-arts systems using both quantitative measurements and visual alignment results of the merged point clouds.

Selected Publications:

• Su, P.-C., J. Shen, W. Xu, and S.-C. Cheung. 2018. A Fast and Robust Extrinsic Calibration for RGB-D Camera Networks. Sensors: special issue on “Imaging Depth Sensors—Sensors, Algorithms and Applications", 18(1), 235; doi:10.3390/s18010235.
• Shen, J., W. Xu, Y. Luo, P.-C. Su, and S.-C. Cheung. 2014. Extrinsic Calibration for Wide-baseline RGB-D Camera Networks. In IEEE International Workshop on Multimedia Signal Processing (MMSP 2014), Sept. 22-24, 2014, DOI: 10.1109/MMSP.2014.6958798.
• Su, P.-C., J. Shen, and S.-C. Cheung. 2013. A Robust RGB-D SLAM System for 3D Environment with Planar Surfaces. In IEEE International Conference on Image Processing (ICIP 2013), Sept. 15-18, 2013, pp. 275-176.