Abstract

Recently, self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. However, the inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. Therefore, most research relies on additional self-supervision sources such as well-lit RGB images, generative models, and Lidar information. In this paper, we conduct an in-depth analysis of thermal image characteristics that degenerates self-supervision from thermal images. Based on the analysis, we propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency. The proposed method shows outperformed depth and pose results than previous state-of-the-art networks without leveraging additional RGB guidance.

Contribution

• We provide an in-depth analysis of raw thermal image properties from a self-supervised learning perspective.

  • Raw thermal image is a suitable choice to preserve temporal consistency.

  • Sparsely distributed thermal radiation values dominate image reconstruction loss.

  • High- and low- temperature objects degenerate self-supervision from a thermal image.

• Based on the in-depth analysis, we propose a temporal consistent image mapping method that maximizes self-supervision from thermal images by rearranging thermal radiation values and boosting image information while preserving temporal consistency..

• We demonstrate that the proposed method shows outperformed results than previous state-of-the-art networks without leveraging additional guidance.

Publication

"Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion" [PDF]

Ukcheol Shin, Kyunghyun Lee, Byeong-Uk Lee, and In So Kweon

Robotics and Automation Letters 2022 and IROS 2022