Publication‎ > ‎

Infrared action dataset

Chenqiang Gao, Yinhe Du, Jiang Liu, Jing Lv, Luyu Yang, Deyu Meng, Alexander G. Hauptmann


Action recognition (AR) is one of the most important tasks in video analysis and computer vision. Recently a large number of related methods have been proposed. While most of these methods are investigated on AR datasets collected from the visible spectrum, the AR problem under infrared scenarios still has not attracted much attention. There is even few public infrared datasets available for supporting the fundamental evaluation requirements of this research. To this issue, this work aims to emphasize the importance of the infrared AR problem in applications and arouse researchers' attention on this task. Specifically, we construct a new Infrared Ation Recognition (InfAR) dataset captured at different times, including in summer and winter, and explore how discriminable actions in our InfAR dataset are with the state-of-the-art pipelines based on low-level features and deep convolutional neural network (CNN), respectively. Our results reveal: (1) In all, dense trajectory feature can achieve the best performance while the appearance features, e.g., HOG, have relatively poorer performance; (2) The encoding method of vector of locally aggregated descriptors is evidently better than that of the widely-used Fisher Vector; (3) The late fusion facilitates a better performance than early fusion; (4) Action videos captured in winter is more discriminable than in summer; (5) Compared to appearance information, the motion information is more essential for infrared action recognition and utilizing this information through deep CNN can improve greatly the performance. The best performance achieved on our dataset is 76.66% (Average Precision), leaving a reasonable space for further exploring the insights underlying such type of infrared AR problem and accordingly designing proper techniques to further promote the performance on this specifically constructed InfAR dataset.

InfAR dataset

Evaluation Pipeline
Our evaluation framework for low-level features
                            Our evaluation framework for low-level features

                                    Our two-stream CNN framework


Infrared_dataset_1.0 (717M)


  title={InfAR dataset: Infrared action recognition at different times},
  author={Gao, Chenqiang and Du, Yinhe and Liu, Jiang and Lv, Jing and Yang, Luyu and Meng, Deyu and Hauptmann, Alexander G},

title={A New Dataset and Evaluation for Infrared Action Recognition},
author={Gao, Chenqiang and Du, Yinhe and Liu, Jiang and Yang, Luyu and Meng, Deyu},
booktitle={Computer Vision},