Hybrid Thermal-EO Camera

Panoramic Dual-spectrum (Thermal-RGB) Stereo Vision 

In this work, a novel paradigm is proposed to design a hybrid RGB/thermal camera, whose thermal and RGB frames are pixel-wisely aligned and temporally synchronized. Compared with the existing schemes, five innovations are introduced to make it more compact in dimension, and thus more practical and extendable for real-world applications. The first is a redesign of the structure layout of the hybrid camera. The second is a removable adjustment system to detach the hybrid camera. The third is on obtaining a pixel-wise spatial registration of the thermal and RGB frames by a two-step alignment. The forth is on extending one single hybrid camera to a hybrid camera array, through which we can obtain wide-view spatially aligned thermal, RGB and disparity images simultaneously. The fifth is to weaken the ghost effect from the rolling-shutter-based thermal camera. The experimental results demonstrate the accuracy of the hybrid camera and the effectiveness of our designed stereo wide-view hybrid camera system.

Reference paper:

• Yigong Zhang, Yicheng Gao, Shuo Gu, Yubin Guo, Minghao Liu, Zezhou Sun, Zhixing Hou, Hang Yang, Ying Wang, Jian Yang, Jean Ponce, and Hui Kong, DIY your own hybrid thermal/EO camera for autonomous vehicle, IEEE International Conference on Robotics and Automation (ICRA) 2019 (spotlight presentation)

De-Jittering Stitched Hybrid Videos