Weighted Low-Rank Decomposition for Robust Grayscale-Thermal Foreground Detection

Chenglong Li* , Xiao Wang* , Lei Zhang, Jin Tang, Hejun Wu, and Liang Lin ( * denotes equal contribution )

1. Abstract

This paper investigates how to fuse grayscale and thermal video data for detecting foreground objects in challenging scenarios. To this end, we propose an intuitive yet effective method called weighted low-rank decomposition (WELD), which adaptively pursues the cross-modality low-rank representation. Specifically, we form two data matrices by accumulating sequential frames from the grayscale and the thermal videos, respectively. Within these two observing matrices, WELD detects moving foreground pixels as sparse outliers against the low-rank structure background and incorporates the weight variables to make the models of two modalities complementary to each other. The smoothness constraints of object motion are also introduced in WELD to further improve the robustness to noises. For optimization, we propose an iterative algorithm to efficiently solve the low-rank models with three subproblems. Moreover, we utilize an edge-preserving filtering-based method to substantially speed up WELD while preserving its accuracy. To provide a comprehensive evaluation benchmark of grayscale-thermal foreground detection, we create a new data set including 25 aligned grayscale-thermal video pairs with high diversity. Our extensive experiments on both the newly created data set and the public data set OSU3 suggest that WELD achieves superior performance and comparable efficiency against other state-of-the-art approaches.

2. Model and Optimization

3. Benchmark for Multi-modal Moving Object Detection

4. Dataset, Baseline Methods and Evaluation Code

【Note】Please feel free to contact me, if you find the code can not run correctly. Email: wangxiaocvpr@foxmail.com

If you use this dataset, please cite this paper:

  title={Weighted Low-Rank Decomposition for Robust Grayscale-Thermal Foreground Detection},
  author={Li, Chenglong and Wang, Xiao and Zhang, Lei and Tang, Jin and Wu, Hejun and Lin, Liang},
  journal={IEEE Transactions on Circuits Systems for Video Technology},