Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations

Andrii Zadaianchuk, Matthäus Kleindessner, Yi Zhu, Francesco Locatello, Thomas Brox

ICLR 2023 (Oral, notable 25%)

Abstract

In this paper, we show that recent advances in self-supervised representation learning enable unsupervised object discovery and semantic segmentation with a performance that matches the state of the field on supervised semantic segmentation 10 years ago. We propose a methodology based on unsupervised saliency masks and self-supervised feature clustering to kickstart object discovery followed by training a semantic segmentation network on pseudo-labels to bootstrap the system on images with multiple objects. We show that while being conceptually simple, our proposed baseline is surprisingly strong. We present results on PASCAL VOC that go far beyond the current state of the art (50.0 mIoU), and we report for the first time results on MS COCO for the whole set of 81 classes: our method discovers 34 categories with more than 20% IoU, while obtaining an average IoU of 19.6 for all 81 categories.

Overview of COMUS

Object Proposals Clustering

COMUS Performance on PASCAL

Citation

Please use the following bibtex entry to cite us:

@inproceedings{

zadaianchuk2023unsupervised,

title={Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations},

author={Andrii Zadaianchuk and Matthaeus Kleindessner and Yi Zhu and Francesco Locatello and Thomas Brox},

booktitle={International Conference on Learning Representations},

year={2023},

url={https://openreview.net/forum?id=1_jFneF07YC}
}

Page updated

Google Sites

Report abuse