VideoMatch: Matching based Video Object Segmentation

Yuan-Ting Hu*, Jia-Bin Huang**, Alex Schwing*

UIUC*, Virginia Tech**


Video object segmentation is challenging yet important in a wide variety of applications for video analysis. Recent works formulate video object segmentation as a prediction task using deep nets to achieve appealing state-of-the-art performance. Due to the formulation as a prediction task, most of these methods require fine-tuning during test time, such that the deep nets memorize the appearance of the objects of interest in the given video. However, fine-tuning is time-consuming and computationally expensive, hence the algorithms are far from real time. To address this issue, we develop a novel matching based algorithm for video object segmentation. In contrast to memorization based classification techniques, the proposed approach learns to match extracted features to a provided template without memorizing the appearance of the objects. We validate the effectiveness and the robustness of the proposed method on the challenging DAVIS-2016, DAVIS-2017, Youtube-Objects and JumpCut datasets. Extensive results show that our method achieves comparable performance without fine-tuning and is much more favorable in terms of computational time.


Performance vs Speed


Paper [Link]

Supplementary [Download]

Poster [Download]

Precomputed masks: [DAVIS16-val] [DAVIS017-val]


  author = {Hu, Yuan-Ting and Huang, Jia-Bin and Schwing, Alexander G.},
  title = {{VideoMatch: Matching based Video Object Segmentation}},
  booktitle = {European Conference on Compute Vision},
  year = {2018}