1) Survey on sub-categories
Mixture of Gaussians is a widely used approach for background modeling to detect moving objects from static cameras. Numerous improvements of the original method developed by Stauffer and Grimson have been proposed over the recent years and the purpose of this paper is to provide a survey and an original classification of these improvements. We also discuss relevant issues to reduce the computation time. Firstly, the original MOG are reminded and discussed following the challenges met in video sequences. Then, we categorize the different improvements found in the literature. We have classified them in term of strategies used to improve the original MOG and we have discussed them in term of the critical situations they claim to handle. After analyzing the strategies and identifying their limitations, we conclude with several promising directions for future research. (more information)
T. Bouwmans, F. El Baf, B. Vachon, “Background Modeling using Mixture of Gaussians for Foreground Detection: A Survey”, Recent Patents on Computer Science, Volume 1, No 3, pages 219-237, November 2008.
Background modeling is often used to detect moving object in video acquired by a fixed camera. Recently, subspace learning methods have been used to model the background in the idea to represent online data content while reducing dimension significantly. The first method using Principal Component Analysis (PCA) was proposed by Oliver et al. [1] and a representative patent using PCA concerns the detection of cars and persons in video surveillance [2]. Numerous improvements and variants were developed over the recent years. The purpose of this paper is to provide a survey and an original classification of these improvements. Firstly, we classify the improvements of the PCA in term of strategies and the variants in term of the used subspace learning algorithms. Then, we present a comparative evaluation of the variants and evaluate them with the state-of-art algorithms (SG, MOG, and KDE) by using the Wallflower dataset. (more information)
T. Bouwmans, “Subspace Learning for Background Modeling: A Survey”, Recent Patents on Computer Science, Volume 2, No 3, pages 223-234, November 2009.
Foreground detection is the first step in video surveillance system to detect moving objects. Recent research on subspace estimation by sparse representation and rank minimization represents a nice framework to separate moving objects from the background. Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit decomposes a data matrix A in two components such that A= L+S, where L is a low-rank matrix and S is a sparse noise matrix. The background sequence is then modeled by a low-rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. To date, many efforts have been made to develop Principal Component Pursuit (PCP) methods with reduced computational cost that perform visually well in foreground detection. However, no current algorithm seems to emerge and to be able to simultaneously address all the key challenges that accompany real-world videos. This is due, in part, to the absence of a rigorous quantitative evaluation with synthetic and realistic large-scale dataset with accurate ground truth providing a balanced coverage of the range of challenges present in the real world. In this context, this work aims to initiate a rigorous and comprehensive review of RPCA-PCP based methods for testing and ranking existing algorithms for foreground detection. For this, we first review the recent developments in the field of RPCA solved via Principal Component Pursuit. Furthermore, we investigate how these methods are solved and if incremental algorithms and real-time implementations can be achieved for foreground detection. Finally, experimental results on the Background Models Challenge (BMC) dataset which contains different synthetic and real datasets show the comparative performance of these recent method. (more information)
T. Bouwmans, E. Zahzah, “Robust PCA via Principal Component Pursuit: A Review for a Comparative Evaluation in Video Surveillance”, Special Isssue on Background Models Challenge, Computer Vision and Image Understanding, CVIU 2014, Volume 122, pages 22–34, May 2014.
T. Bouwmans, E. Zahzah,"Robust Principal Component Analysis via Decomposition into Low-rank and Sparse Matrices: An overview", Handbook on "Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing", CRC Press, Taylor and Francis Group, May 2016.
C. Guyon, T. Bouwmans, E. Zahzah, “Robust Principal Component Analysis for Background Subtraction: Systematic Evaluation and Comparative Analysis”, INTECH, Principal Component Analysis, Book 1, Chapter 12, page 223-238, March 2012.
2) Survey on categories
Developing a background subtraction method, researchers must design each step and choose the features in relation to the critical situations they want to handle. All these critical situations generates imprecision and uncertainties in all the process of background subtraction. Therefore, some authors have introduced fuzzy concepts in the different steps of background subtraction.
T. Bouwmans, “Background Subtraction For Visual Surveillance: A Fuzzy Approach”, Handbook on Soft Computing for Video Surveillance, Taylor and Francis Group, Chapter 5, March 2012.
Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate
moving objects from the background. This work aims to initiate a rigorous and comprehensive review of the similar problem formulations in
robust subspace learning and tracking based on decomposition into low-rank plus additive matrices for testing and ranking existing algorithms for background/foreground separation. For this, we first provide a preliminary review of the recent developments in the different problem formulations which allows us to define a unified view that we called Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine carefully each method in each robust subspace learning/tracking frameworks with their decomposition, their loss functions, their optimization problem and their solvers. Furthermore, we investigate if incremental algorithms and real-time implementations can be achieved for background/foreground separation. Finally, experimental results on a large-scale dataset called Background Models Challenge (BMC 2012) show the comparative performance of 32 different robust subspace learning/tracking methods. (more information)
T. Bouwmans, A. Sobral, S. Javed, S. Jung, E. Zahzah, "Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset", Computer Science Review, Volume 23, pages 1-71, February 2017.
Background modeling is currently used to detect moving objects in video acquired from static cameras. Numerous statistical methods have been developed over the recent years. The aim of this paper is firstly to provide an extended and updated survey of the recent researches and patents which concern statistical background modeling and secondly to achieve a comparative evaluation. For this, we firstly classified the statistical methods in term of category. Then, the original methods are reminded and discussed following the challenges met in video sequences. We classified their respective improvements in term of strategies used. Furthermore, we discussed them in term of the critical situations they claim to handle. Finally, we conclude with several promising directions for future research. The survey also discussed relevant patents. (more information)
T. Bouwmans, "Recent Advanced Statistical Background Modeling for Foreground Detection: A Systematic Survey", Recent Patents on Computer Science, Volume 4, No. 3, pages147-176, September 2011.
Background modeling is often used in the context of moving objects detection from static cameras. Numerous methods have been developed over the recent years and the most used are the statistical ones. The purpose of this chapter is to provide a recent survey of these different statistical methods. For this, we have classified them in term of generation following the years of publication and the statistical tools used. We then focus on the first generation methods: Single Gaussian, Mixture of Gaussians, Kernel Density Estimation and Subspace Learning using PCA. These original methods are reminded and then we have classified their different improvements in term of strategies. After analyzing the strategies and identifying their limitations, we conclude with several promising directions for future research. (more information)
T. Bouwmans, F. El Baf, B. Vachon, “Statistical Background Modeling for Foreground Detection: A Survey”, Handbook of Pattern Recognition and Computer Vision, World Scientific Publishing, Volume 4, Part 2, Chapter 3, pages 181-199, January 2010.
3) Survey on all the categories
Background modeling for foreground detection is often used in different applications to model the background and then detect the moving objects in the scene like in video surveillance. The last decade witnessed very significant publications in this field. Furthermore, several surveys can be found in literature but none of them addresses an overall review in this field. So, the purpose of this paper is to provide a complete survey of the traditional and recent approaches. First, we categorize the different approaches found in literature. We have classified them in terms of the mathematical models used and we have discussed them in terms of the critical situations that they claim to handle. Furthermore, we present the available resources, datasets and libraries. Then, we conclude with several promising directions for future research.
T. Bouwmans, “Traditional and Recent Approaches in Background Modeling for Foreground Detection: An Overview”, Computer Science Review, 2014.
T. Bouwmans, “Traditional Approaches in Background Modeling for Static Cameras”, Chapter 1, Handbook on "Background Modeling and Foreground Detection for Video Surveillance: Traditional and Recent Approaches, Implementations, Benchmarking and Evaluation", CRC Press, Taylor and Francis Group, July 2014.
T. Bouwmans,“Recent Approaches in Background Modeling for Static Cameras”, Chapter 2, Handbook on "Background Modeling and Foreground Detection for Video Surveillance: Traditional and Recent Approaches, Implementations, Benchmarking and Evaluation", CRC Press, Taylor and Francis Group, July 2014.