Embedded Files
Abstract
Abstract
This paper studies the problem of Video Object Segmentation which aims at segmenting objects of interest throughout entire videos, when provided with initial ground truth annotation. Although, variety of works in this field have been done utilizing Convolutional Neural Networks (CNNs), adversarial training techniques have not been used in spite of their effectiveness as a holistic approach. Our proposed architecture consists of a Generative Adversarial framework for the purpose of foreground object segmentation in videos coupled with Intersection-over-union and temporal information based loss functions for training the network. The main contribution of the paper lies in formulation of the two novel loss functions: (i) Inter-frame Temporal Symmetric Difference Loss (ITSDL) and (ii) Intra-frame Temporal Loss (IFTL), which not only enhance the segmentation quality of the predicted mask but also maintain the temporal consistency between the subsequent generated frames. Our end-to-end trainable network exhibits impressive performance gain compared to the state-of-the-art model when evaluated on three popular real-world Video Object Segmentation datasets viz. DAVIS 2016, SegTrack-v2 and YouTube-Objects dataset.
This paper studies the problem of Video Object Segmentation which aims at segmenting objects of interest throughout entire videos, when provided with initial ground truth annotation. Although, variety of works in this field have been done utilizing Convolutional Neural Networks (CNNs), adversarial training techniques have not been used in spite of their effectiveness as a holistic approach. Our proposed architecture consists of a Generative Adversarial framework for the purpose of foreground object segmentation in videos coupled with Intersection-over-union and temporal information based loss functions for training the network. The main contribution of the paper lies in formulation of the two novel loss functions: (i) Inter-frame Temporal Symmetric Difference Loss (ITSDL) and (ii) Intra-frame Temporal Loss (IFTL), which not only enhance the segmentation quality of the predicted mask but also maintain the temporal consistency between the subsequent generated frames. Our end-to-end trainable network exhibits impressive performance gain compared to the state-of-the-art model when evaluated on three popular real-world Video Object Segmentation datasets viz. DAVIS 2016, SegTrack-v2 and YouTube-Objects dataset.
Paper
Paper
TempSeg-GAN: Segmenting Objects in Videos Adversarially using Temporal Information
TempSeg-GAN: Segmenting Objects in Videos Adversarially using Temporal Information
In 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP part of VISIGRAPP), Prague, Czech Republic, 2019.
In 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP part of VISIGRAPP), Prague, Czech Republic, 2019.
Architecture
Architecture
Fig.1: Proposed TempSeg-GAN architecture. GT denotes the ground-truth and <- - -> refers to the Inter-frame Temporal Symmetric Difference Loss estimation using GT mask at time t, predicted mask at time (t − 1) and optical flow vectors between RGB input images at time (t −1) and t.
Quantitative Results
Quantitative Results
Qualitative Results
Qualitative Results
Video result
Video result
RGB
OnAVOS (BMVC, 2017)
TempSeg-GAN (Ours)
(VISAPP, 2019)
Ground Truth
Page updated
Google Sites
Report abuse