Thermal-Infrared Remote-Target Detection System for Maritime Rescue Using 3-D Game-Based Data Augmentation With GAN

Abstract

This paper proposes a deep learning-based thermal-infrared (TIR) remote target detection system for maritime rescue with self-collected real TIR dataset and corresponding data augmentation method based on GAN. We have collected and established a real field TIR dataset consisting of multiple scenes imitating actual human rescue scenarios using a TIR camera (FLIR M364C). Additionally, synthetic TIR data from a game (ARMA3) to augment the real TIR data is further collected to address dataset scarcity and improve model performance. However, a significant domain gap exists between the real and synthetic TIR datasets. Hence, a proper domain adaptation algorithm is essential to overcome the gap. We suggest a target-background separation scheme during the domain adaptation to mitigate this gap while preserving the shapes and locations of the small-size targets even after the domain transfer. Furthermore, a fixed-pattern kernel module inserted at the network front is proposed to improve signal-to-noise ratio (SNR) as TIR remote targets inherently suffer from unclear boundaries and heavy clutters. Experimental results reveal that the segmentation network trained on both real and domain-translated synthetic TIR data shows improved performance compared to that trained on only real TIR data. Moreover, the segmentation network with the fixed-weight kernel module shows better performance than state-of-the-art methods in terms of every evaluation metric.