Online Robust PCA for Background/Foreground Separation

Introduction

Robust Principal Components Analysis (RPCA) shows a very nice framework for moving object detection. The background sequence is modeled by a low-dimensional subspace called low-rank matrix and sparse error con-stitutes the foreground objects. But RPCA presents the limitations of computational complexity and memory storage due to batch optimization methods, as a result it is di.cult to apply for real-time system. To handle these challenges, this work presents a robust foreground detection algorithm via Online Robust PCA (OR-PCA). OR-PCA with good initialization scheme using image decomposition approach improves the accuracy of foreground detection and the computation time as well. Moreover, solving MRF with graph-cuts exploits structural information using spatial neighborhood system and similarities to further improve the foreground segmentation in highly dynamic backgrounds.

Principle

Our methodology consists of four main stages: decomposition, background modeling, integration and continuous MRF. Initially, the input video frames are decomposed into Gaussian and Laplacian images using a set of two Gaussian kernels. Then, OR-PCA is applied to each of Gaussian and Laplacian images with di.erent parameters to model the background, separately. In the background modeling stage, we have proposed an alternative initialization scheme to speed up the stochastic optimization process. Finally, the integration stage, which combines low-rank and sparse components obtained via OR-PCA to recover the background model and foreground image, is performed. The reconstructed sparse matrix is then thresholded to get the binary foreground mask. In order to improve the foreground segmentation, a MRF is applied which exploits structural information and similarities continuously.

Figure 1 : Overview of the proposed framework

Publication

Chapter

S. Javed, S. Oh, T. Bouwmans, S. Jung, "Stochastic RPCA for Background/Foreground Separation", Handbook on "Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing", CRC Press, Taylor and Francis Group, May 2016. [pdf]

Journal

S. Javed, S. Oh, T. Bouwmans, S. Jung, "Robust Background Subtraction to Global Illumination Changes via Multiple Features based OR-PCA with MRF", Journal of Electronic Imaging, 2015.[Cited by]

Conference

S. Javed, S. Oh, A. Sobral, T. Bouwmans , S. Jung, "Background Subtraction via Superpixel-based Online Matrix Decomposition with Structured Foreground Constraints", Workshop on Robust Subspace Learning and Computer Vision, ICCV 2015, Santiago, Chile, December 2015. [pdf]

S. Javed, T. Bouwmans, S. Jung, “Combining ARF and OR-PCA Background Subtraction of Noisy Videos”, International Conference in Image Analysis and Applications, ICIAP 2015, Genova, Italy, September 2015. [pdf]

S. Javed, T. Bouwmans, S. Jung, “Stochastic Decomposition into Low Rank and Sparse Tensor for Robust Background Subtraction”, ICDP 2015, July 2015. [pdf]

S. Javed, T. Bouwmans, S. Jung, "Depth Extended Online RPCA with Spatiotemporal Constraints for Robust Background Subtraction", Korea-Japan Workshop on Frontiers of Computer Vision, FCV 2015, Mokpo, South Korea, January 2015. [pdf]

S. Javed, A. Sobral, T. Bouwmans, S. Jung, "OR-PCA with Dynamic Feature Selection for Robust Background Subtraction", ACM Symposium On Applied Computing, SAC 2015, Salamanca, Spain, April 2015. [pdf]

S. Javed, A. Sobral, S. Oh, T. Bouwmans, S. Jung, “OR-PCA with MRF for Robust Foreground Detection in Highly Dynamic Backgrounds”, Asian Conference on Computer Vision, ACCV 2014, Singapore, November 2014. [pdf]