Alasdair Newson, Mariano Tepper, Guillermo Sapiro
Abstract
Robust Principal Component Analysis (RPCA) has generated a great amount of interest for background/foreground estimation in videos. The central hypothesis in this setting is that a video’s background can be well-represented by a low-rank model. However, in the presence of complex lighting conditions this model is only accurate in localised spatio-temporal regions. Following this observation, we propose to model the
background with a piecewise low-rank approximation. To achieve this, we introduce the piecewise low-rank segmentation problem. Starting from a carefully designed cost function which assesses the low-rank coherence of two video regions, the segmentation is obtained with an efficient graph-clustering algorithm. We show that this segmentation, when used to establish a local RPCA per segment, leads to improved quantitative and qualitative results for background/foreground estimation in challenging videos.
Download the one-page abstract
Download the supplementary material
Visual results (segmentation and foreground estimation)
Example 1
Original video
Automatic low-rank segmentation
Foreground detection comparison
Example 2
Automatic low-rank segmentation
Original video
Foreground detection comparison
Segmentation of a timelapse video
Original video
Video segmentation
Segmentation of a scene transition
Scene transition
Automatic segmentation