Low-rank and block-sparse matrix decomposition
Background subtraction is the first step in video surveillance system to detect moving objects. Recent research on reconstructive subspace learning by Robust Principal Component Analysis (RPCA) shows a nice framework to separate moving objects from background. RPCA decomposes a data matrix A in two components such that A=L+S, where L is a low-rank matrix and S is a noise matrix. The background sequence is then modeled by a low-rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. In this work, we investigate how these methods can be robustly applied to detect moving objects. (more information)
C. Guyon, T. Bouwmans, E. Zahzah, "Foreground detection based on low-rank and block-sparse matrix decomposition", IEEE International Conference on Image Processing, ICIP 2012, Orlando, Florida, September 2012.
Low-rank and sparse matrix decomposition via IRLS
This works proposes to use a low-rank matrix factorization with IRLS scheme (Iteratively Reweighted Least Squares), and to address in the minimization
process the spatial connexity and the temporal sparseness of moving objects. (more information)
C. Guyon, T. Bouwmans, E. Zahzah, “Foreground Detection via Robust Low Rank Matrix Decomposition including Spatio-Temporal Constraint”, International Workshop on Background Model Challenges, ACCV 2012, pages 315-320, Daejeon, Korea, November 2012.
C. Guyon, T. Bouwmans, E. Zahzah, “Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted Regression”, International Conference on Pattern Recognition, ICPR 2012, Tsukuba, Japan, November 2012.
C. Guyon, T. Bouwmans, E. Zahzah, “Moving Object Detection via Robust Low Rank Matrix Decomposition with IRLS scheme”, International Symposium on Visual Computing, ISVC 2012,pages 665–674, Rethymnon, Crete, Greece, July 2012.
Low-rank and sparse matrix decomposition via ADM
This works proposes to use a low-rank matrix factorization with ADM scheme (Alternating Direction Method) for moving objects detection. (more information)
C. Guyon, T. Bouwmans, E. Zahzah, “Moving Object Detection by Robust PCA solved via a Linearized Symmetric Alternating Direction Method”, International Symposium on Visual Computing, ISVC 2012, pages 427-436, Rethymnon, Crete, Greece, July 2012.
C. Guyon, T. Bouwmans, E. Zahzah, "Foreground Detection by Robust PCA solved via a Linearized Alternating Direction Method", International Conference on Image Analysis and Recognition, ICIAR 2012, pages 115-122, Aveiro, Portugal, June 2012.
Stochastic RPCA
This paper presents a robust foreground detection algorithm via Online Robust PCA (OR-PCA) using image decomposition along with continuous constraint such as Markov Random Field (MRF). OR-PCA with good initialization scheme using image decomposition approach improves the accuracy of foreground detection and the computation time as well. Moreover, solving MRF with graph-cuts exploits structural information using spatial neighborhood system and similarities to further improve the foreground segmentation in highly dynamic backgrounds. Experimental results on challenging datasets such as Wallflower, I2R, BMC 2012 and Change Detection 2014 dataset demonstrate that our proposed scheme signi.cantly outperforms the state of the art approaches and works e.ectively on a wide range of complex background scenes. (more information)
S. Javed, S. Oh, T. Bouwmans, S. Jung, "Stochastic RPCA for Background/Foreground Separation", Handbook on "Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing", CRC Press, Taylor and Francis Group, May 2016.
S. Javed, S. Oh, T. Bouwmans, S. Jung, "Robust Background Subtraction to Global Illumination Changes via Multiple Features based OR-PCA with MRF", Journal of Electronic Imaging, 2015.
S. Javed, T. Bouwmans, S. Jung, “Improving OR-PCA via Smoothed Spatially-Consistent Low-rank Modeling for Background Subtraction”, ACM Symposium on Applied Computing, SAC 2017, 2017.
S. Javed, T. Bouwmans, S. Jung, “Combining ARF and OR-PCA Background Subtraction of Noisy Videos”, International Conference in Image Analysis and Applications, ICIAP 2015, Genova, Italy, September 2015.
S. Javed, T. Bouwmans, S. Jung, "Depth Extended Online RPCA with Spatiotemporal Constraints for Robust Background Subtraction", Korea-Japan Workshop on Frontiers of Computer Vision, FCV 2015, Mokpo, South Korea, January 2015.
S. Javed, A. Sobral, T. Bouwmans, S. Jung, "OR-PCA with Dynamic Feature Selection for Robust Background Subtraction", ACM Symposium On Applied Computing, SAC 2015, Salamanca, Spain, April 2015.
S. Javed, A. Sobral, S. Oh, T. Bouwmans, S. Jung, “OR-PCA with MRF for Robust Foreground Detection in Highly Dynamic Backgrounds”, Asian Conference on Computer Vision, ACCV 2014, Singapore, November 2014.
Double-constrained RPCA
This paper, a double-constrained Robust Principal Component Analysis (RPCA), named SCM-RPCA (Shape and Confidence Mapbased RPCA), is proposed to improve the object foreground detection in maritime scenes. The sparse component is constrained by shape and confidence maps both extracted from spatial saliency maps. The experimental results in the UCSD and MarDT data sets indicate a better enhancement of the object foreground mask when compared with some related RPCA methods. (more information)
A. Sobral, T. Bouwmans, E. Zahzah, “Double-constrained RPCA based on Saliency Maps for Foreground Detection in Automated Maritime Surveillance”, ISBC 2015 Workshop conjunction with AVSS 2015, Karlsruhe, Germany, 2015.
Structured RPCA
When the size of the input data grows and due to the lack of sparsity constraints, RPCA methods cannot cope with the real-time challenges and always show a weak performance due to the erroneous foreground regions. In order to address the above mentioned issues, this paper presents a superpixel based matrix decomposition method together with maximum norm (max-norm) regularizations and structured sparsity constraints. The low-rank component estimated from each homogeneous region is more perfect, reliable, and efficient, since each superpixel provides different characteristics with a reduced value of rank. Online max-norm based matrix decomposition is employed on each segmented superpixel to separate the low rank and initial outliers support. And then, the structured sparsity constraints such as the generalized fussed lasso (GFL) are adopted for exploiting structural information continuously as the foreground pixels are both spatially connected and sparse. We propose an online single unified optimization framework for detecting foreground and learning the background model simultaneously. (more information)
S. Javed, S. Oh, A. Sobral, T. Bouwmans , S. Jung, "Background Subtraction via Superpixel-based Online Matrix Decomposition with Structured Foreground Constraints", Workshop on Robust Subspace Learning and Computer Vision, ICCV 2015, Santiago, Chile, December 2015.
Structured Sparse RPCA
This works proposes a spatiotemporal structured sparse RPCA algorithm for moving objects detection, where we impose spatial and temporal regularization on the sparse component in the form of graph Laplacians. Each Laplacian corresponds to a multi-feature graph constructed over superpixels in the input matrix. We enforce the sparse component to act as eigenvectors of the spatial and temporal graph Laplacians while minimizing the RPCA objective function. These constraints incorporate a spatiotemporal subspace structure within the sparse component. Thus, we obtain a novel objective function for separating moving objects in the presence of complex backgrounds. The proposed objective function is solved using a linearized alternating direction method of multipliers based batch optimization. Moreover, we also propose an online optimization algorithm for real-time applications.
S. Javed, A. Mahmood, T. Bouwmans, S. Jung, "Superpixels based Manifold Structured Sparse RPCA for Moving Object Detection", International Workshop on Activity Monitoring by Multiple Distributed Sensing, BMVC 2017, London, UK, September 2017.
S. Javed, A. Mahmood, S. Al-Maadeed, N. Rajpoot, T. Bouwmans, S. Jung,"Moving Object Detection in Complex Scene using Spatio-temporal Structured-Sparse RPCA", IEEE Transactions on Image Processing, 2018.
Graph Regularized Spatiotemporal RPCA
This works investigates the performance of online Spatiotemporal RPCA (SRPCA) algorithm for moving object detection using RGB-D videos. SRPCA is a graph regularized algorithm which preserves the low-rank spatiotemporal information in the form of dual spectral graphs. This graph regularized information is then encoded into the objective function which is solved using online optimization.
S. Javed, T. Bouwmans, M. Sultana, S. Jung, "Moving Object Detection on RGB-D Videos using Graph Regularized Spatiotemporal RPCA", International Workshop on Background learning for detection and tracking from RGBD videos, ICIAP 2017, Catani, Italy, September 2017.
Spatio-temporal Sparse Subspace Clustering
This works proposes to incorporate the spatial and temporal sparse subspace clustering into the RPCA framework. To that end, we compute a spatial and temporal graph for a given sequence using motion-aware correlation coefficient. The information captured by both graphs is utilized by estimating the proximity matrices using both the normalized Euclidean and geodesic distances. The low-rank component must be able to efficiently partition the spatiotemporal graphs using these Laplacian matrices. Embedded with the RPCA objective function, these Laplacian matrices constrain the background model to be spatially and temporally consistent,both on linear and nonlinear manifolds. The solution of the proposed objective function is computed by using the LADMAP optimization scheme. (more information)
S. Javed, A. Mahmood, T. Bouwmans, S. Jung, "Background-Foreground Modeling Based on Spatio-temporal Sparse Subspace Clustering", IEEE Transactions on Image Processing, Volume 26, No. 12, pages 5840-5854, December 2017