Research

Background estimation is a fundamental task of computer vision. Indeed, in problems such as tracking or object recognition in videos, a good background estimation is usually a prerequisite. This is because these subsequent tasks are much easier if we know where to look for interesting objects, in particular if we know where the foreground is (when we track objects, we are generally interested in foreground objects).

Publications in this research area

Low-Rank Spatio-Temporal Video Segmentation

A. Newson, M. Tepper, G. Sapiro

BMVC, 2015, Project webpage

Multi-temporal Foreground Detection in Videos

M. Tepper, A. Newson, P. Sprechmann, G. Sapiro

ICIP, 2015, Project webpage

Here is an example of background/foreground estimation (thanks to Jose Lezama for appearing in this video) :

Robust Principle Component Analysis (RPCA)

Candès et al. introduced an approach which is widely known as Robust Principal Component Analysis [1] (even if other works had previously proposed robustified PCAs [2]), which could be used for tasks such as background estimation. The central idea of this work is to robustify traditional Principal Component Analysis, a data analysis method which is very popular, but can break down in the presence of outliers. This is done by decomposing an input matrix (the video, in our case) into the sum of a low-rank and a sparse component. More precisely, we solve the following convex optimisation problem :

where X is the input matrix (our video), L is the low-rank background component and S is the sparse foreground component. The norms and correspond to the Frobenius matrix norm, the nuclear norm and the norm, respectively. The two parameters and are tuning parameters.