Contour-Aware Equipotential Learning for Semantic Segmentation

Xu Yin; Dongbo Min; Yuchi Huo and Sung-Eui Yoon

Korea Advanced Institute of Science and Technology (KAIST)

Abstract

With increasing demands for high-quality semantic segmentation in the industry, hard-distinguishing semantic boundaries have posed a significant threat to existing solutions. Inspired by real-life experience, i.e., combining varied observations contributes to higher visual recognition confidence, we present the equipotential learning (EPL) method. This novel module transfers the predicted/ground-truth semantic labels to a self-defined potential domain to learn and infer decision boundaries along customized directions. The conversion to the potential domain is implemented via a lightweight differentiable anisotropic convolution that neither incurs the parameter overhead of the neural networks nor changes to the architectures. Besides, the designed two loss functions, the point loss and the equipotential line loss implement anisotropic field regression and category-level contour learning, respectively, enhancing prediction consistencies in the inter/intra-class boundary areas. More importantly, EPL is agnostic to network architectures, and thus it can be plugged into most existing segmentation models. This paper is the first attempt to address the boundary segmentation problem with field regression and contour learning. Meaningful performance improvements on Pascal Voc 2012 and Cityscapes demonstrate that the proposed EPL module can benefit the off-the-shelf fully convolutional network models when recognizing semantic boundary areas. Besides, intensive comparisons and analysis show the favorable merits of EPL for distinguishing semantically-similar and irregular-shaped categories.

Motivations

(a) This work takes inspiration from the daily visual observation, i.e., changing the relative distance D and the perspective S benefits better the object-contour recognition. (b) The category-level contour information is an important cue for image classification. For example, people would categorize tigers and leopards into cat species by external contours, even though they have different textures.

Architecture Overview

Assemble FCN model with EPL for K-class semantic segmentation. After proceeding with the anisotropic convolution, along any direction in S, the point loss and the equipotential line loss respectively enable the anisotropic field regression and the category-level contour learning.