Role and Importance of Features for Background Modeling and Foreground Detection
The purpose of this study is to initiate a rigorous and comprehensive survey of features used within background modeling and foreground detection. Further, this paper presents a systematic experimental and statistical analysis of techniques that provide valuable insight on the trends in background modeling and use it to draw meaningful recommendations for practitioners. In this paper, a preliminary review of the key characteristics of features based on the types and sizes is provided in addition to investigating their intrinsic spectral, spatial and temporal properties. Furthermore, improvements using statistical and fuzzy tools are examined and techniques based on multiple features are benchmarked against reliability and selection criterion. Finally, a description of the different resources available such as datasets and codes is provided.
T. Bouwmans, C. Silva, C. Marghes, M. Zitouni, H. Bhaskar, C. Frelicot, “On the Role and the Importance of Features for Background Modeling and Foreground Detection”, Computer Science Review, Volume 28, pages 26-91, May 2018.
Robust Texture Features
We propose an eXtended Center-Symmetric Local Binary Pattern (XCS-LBP) descriptor for background modeling and subtraction in videos. By combining the strengths of the original LBP and the similar CS ones, it appears to be robust to illumination changes and noise, and produces short histograms, too. The experiments conducted on both synthetic and real videos (from the Background Models Challenge) of outdoor urban scenes under various conditions show that the proposed XCS-LBP outperforms its direct competitors for the background subtraction task. (More information)
C. Silva, T. Bouwmans, C. Frelicot, 'An eXtended Center-Symmetric Local Binary Pattern for Background Modeling and Subtraction in Videos", VISAPP 2015, Berlin, Germany, March 2015.
Feature Selection
This works present an Online Weighted Ensemble of One-Class SVMs (Support Vector Machines) able to select suitable features for each pixel to distinguish the foreground objects from the background. In addition, our proposal uses a mechanism to update the relative importance of each feature over time. Moreover, a heuristic approach is used to reduce the complexity of the background model maintenance while maintaining the robustness of the background model. Results on two datasets show the pertinence of the approach.
C. Silva, T. Bouwmans, C. Frelicot, "Online Weighted One-Class Ensemble for Feature Selection in Background/Foreground Separation", International Conference on Pattern Recognition, ICPR 2016, December 2016.
C. Pacheco, T. Bouwmans, C. Frelicot,"Superpixel-based online wagging one-class ensemble for feature selection in foreground/background separation", Pattern Recognition Letters, pages 144-151, December 2017.
Multi-Feature Fusion
This works propose a multi-feature fusion scheme to background subtraction for video sequences with strong background changes. We reconstruct the whole videos frame by frame by fusing several video features. In this fusing step, we design an energy function based on enforcing every features with an equal weight. By comparing reconstruction videos with the original videos, pixels with small differences are classified as background pixels. Thus, we can identify background areas in advance and then we construct a contour-based mask combining mechanism.
Z. Huang, R. Hu, T. Bouwmans, S. Chen, "Multi-Feature Fusion based Background Subtraction for Video Sequences with Strong Background Changes", IEEE International Conference on Image Processing, ICIP 2017, Beijing, China, September 2017