Abstract

We address an essential problem in computer vision, that of unsupervised foreground object segmentation in video, where a main object of interest in a video sequence should be automatically separated from its background. An efficient solution to this task would enable large-scale video interpretation at a high semantic level in the absence of the costly manual labeling. We propose an efficient unsupervised method for generating foreground object soft masks based on automatic selection and learning from highly probable positive features. We show that such features can be selected efficiently by taking into consideration the spatio-temporal appearance and motion consistency of the object in the video sequence. We also emphasize the role of the contrasting properties between the foreground object and its background. Our model is created over several stages: we start from pixel level analysis  and move to descriptors that consider information over groups of pixels combined with efficient motion analysis. We also prove theoretical properties of our unsupervised learning method, which under some mild constraints is guaranteed to learn the correct classifier even in the unsupervised case. We achieve competitive and even state of the art results on the challenging YouTube-Objects and SegTrack datasets, while being at least one order of magnitude faster than the competition. We believe that the strong performance of our method, along with its theoretical properties, constitute a solid step towards solving unsupervised discovery in video. 

ICCV 2017: E. Haller, M. Leordeanu, "Unsupervised object segmentation in video by efficient selection of highly probable positive features", PDF

Qualitative results

Below you can find qualitative results of our method, on YouTube-Objects [1] and SegTrack [2] datasets.

Demo videos of our method are available here.

Code

Matlab code is available here.

Python repo https://github.com/emanuelahaller/HPP 

Team

References

[1] A. Prest, C. Leistner, J. Civera, C. Schmid, and V. Ferrari. Learning object class detectors from weakly annotated video. CVPR 2012

[2] D. Tsai, M. Flagg, A.Nakazawa, and J.M. Rehg. Motion coherent tracking using multi-label mrf optimization. ICCV 2012

Acknowledgements

This work was supported by UEFISCDI, under project PN-III-P4-ID-ERC-2016-0007.