Abstract

Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results. Prediction fusion is therefore important to achieve accurate panoptic segmentation. In this paper, we present REFINE, pREdiction FusIon NEtwork for panoptic segmentation, to achieve high-quality panoptic segmentation by improving cross-task prediction fusion, and within-task prediction fusion. Our single-model ResNeXt-101 with DCN achieves PQ=51.5 on the COCO dataset, surpassing state-of-the-art performance by a convincing margin and is comparable with ensembled models. Our smaller model with a ResNet-50 backbone achieves PQ=44.9, which is comparable with state-of-the-art methods with larger backbones.

Paper

Jiawei Ren*, Cunjun Yu*, Zhongang Cai*, Mingyuan Zhang, Chongsong Chen, Haiyu Zhao, Shuai Yi, Hongsheng Li

REFINE: Prediction Fusion Network for Panoptic Segmentation

AAAI 2021, [PDF], [Supp], [Code]

BibTex

@inproceedings{

Ren2021REFINE,

title={REFINE: Prediction Fusion Network for Panoptic Segmentation},

author={Jiawei Ren and Cunjun Yu and Zhongang Cai and Mingyuan Zhang and Chongsong Chen and Haiyu Zhao and Shuai Yi and Hongsheng Li},

booktitle={Thirty-Fifth AAAI Conference on Artificial Intelligence},

month = {February},

year={2021}

}