Self-Supervised Camera
Self-Calibration from Video
Jiading Fang Igor Vasiljevic Vitor Guizilini Rares Ambrus
Greg Shakhnarovich Adrien Gaidon Matthew Walter

Abstract. Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams. In practice, calibration is a laborious procedure requiring specialized data collection and careful tuning. This process must be repeated whenever the parameters of the camera change, which can be a frequent occurrence for mobile robots and autonomous vehicles. In contrast, self-supervised depth and ego-motion estimation approaches can bypass explicit calibration by inferring per-frame projection models that optimize a view-synthesis objective. In this paper, we extend this approach to explicitly calibrate a wide range of cameras from raw videos in the wild. We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models. Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods. We validate our approach on a wide variety of camera geometries, including perspective, fisheye, and catadioptric. Finally, we show that our approach leads to improvements in the downstream task of depth estimation, achieving state-of-the-art results on the EuRoC dataset with greater computational efficiency than contemporary methods.

Contributions:

• We propose to self-calibrate a variety of generic camera models from raw video using self-supervised depth and pose learning as a proxy objective, providing for the first time a calibration evaluation of camera model parameters learned purely from self-supervision.

• We demonstrate the utility of our framework on challenging and radically different datasets, learning depth and pose on perspective, fisheye, and catadioptric images without architectural changes.

• We achieve state-of-the-art depth evaluation results on the challenging EuRoC MAV dataset by a large margin, using our proposed self-calibration framework.

Citation

@inproceedings{tri_self-calib_icra20,

author = {Jiading Fang and Igor Vasiljevic and Vitor Guizilini and Rares Ambrus and Greg Shakhnarovich and Adrien Gaidon and Matthew Walter},

title = {Self-Supervised Camera Self-Calibration from Video},

booktitle = {International Conference on Robotics and Automation (ICRA)},

year = {2022},

}