Generalized Boundary Detection

Overview

Boundary detection is a fundamental computer vision problem that is essential for a variety of tasks, such as contour and region segmentation, symmetry detection and object recognition and categorization. We propose a generalized formulation for boundary detection, with closed-form solution, applicable to the localization of different types of boundaries, such as object edges in natural images and occlusion boundaries from video. Our generalized boundary detection method (Gb) simultaneously combines low-level and mid-level image representations in a single eigenvalue problem and solves for the optimal continuous boundary orientation and strength. The closed-form solution to boundary detection enables our algorithm to achieve state of the art results at a significantly lower computational cost than current methods. We also propose two complementary novel components that can seamlessly be combined with Gb: first, we introduce a soft-segmentation procedure that provides region input layers to our boundary detection algorithm for a significant improvement in accuracy, at negligible computational cost; second, we present an efficient method for contour grouping and reasoning, which when applied as a final post-processing stage, further increases the boundary detection performance.

Our proposed step/ramp boundary model can be seen in different layers of real-world images. Left: A step is often visible in the low-level color channels. Middle: In some cases, no step is visible in the color channels yet the edge is clearly present in the output of a soft segmentation method.  Right: In video, moving boundaries are often seen in the optical flow layer. More generally, a strong perceptual boundary at a given location may be visible in several layers, with consistent orientation across layers. Our multi-layer ramp model covers all these cases.

Overview of the contour reasoning framework: First, an edge map is obtained from the input image using a Gb model with color and soft segmentation layers. Then edges are linked to form contours. Next, using two logistic classifiers we label the edge pixels based on different  properties of their contours: appearance (boundary strength) or geometry (length and smoothness). The outputs of these two classifiers are then combined to obtain the final boundary map.

Soft-segmentation results from our method: The first 3 dimensions of the soft-segmentations are shown on the RGB channels. Computation time for soft-segmentation is ≈2.5 seconds per 0.15 MP image in MATLAB.

Boundaries in images: output of a Gb model using color and soft segmentation layers, without contours (second column) and with contours (third column)after thresholding at the optimal F-measure. The use of global contour reasoning produces a cleaner output.

Boundaries in Video: Gb results on the CMU Motion Dataset.

Papers

[1] Marius Leordeanu, Rahul Sukthankar and Cristian Sminchisescu. "Efficient closed-form solution to generalized boundary detection." European Conference on Computer Vision (ECCV), Florence, Italy, 2012.  PDF 

[2] Marius Leordeanu, Rahul Sukthankar and Cristian Sminchisescu. "Generalized Boundaries from Multiple Image Interpretations", IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014. Version January 9th, 2014. PDF 

[3] Marius Leordeanu, Rahul Sukthankar and Cristian Sminchisescu, "Generalized Boundaries from Multiple Image Interpretations", arXiv:1202.3684 (2012). PDF

Code

MATLAB code (Version: 20 August 2014) for Gb Boundary Detector, Soft-Segmentation and Contour Reasoning are available here.

 for questions or bugs you could email Marius Leordeanu at leordeanu at gmail.

Related Work using Gb

[1] E. Trulls, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer, "Dense Segmentation-aware Descriptors", Computer Vision and Pattern Recognition (CVPR), 2013.  Access the project website here.

[2] F. Li, T. Kim, A. Humayun, D. Tsai and J.M. Rehg, "Video Segmentation by Tracking Many Figure-Ground Segments", International Conference on Computer Vision (ICCV), 2013.

[3] Shugao Ma, Jianming Zhang, Nazli Ikizler-Cinbis and Stan Sclaroff, "Action Recognition and Localization by Hierarchical Space-Time Segments", International Conference on Computer Vision (ICCV), 2013.

[4] A. Humayun, F. Li and J.M. Rehg, "RIGOR: Reusing Inference in Graph Cuts for generating Object Regions", Computer Vision and Pattern Recognition (CVPR), 2014.

[5] Trulls, E., Tsogkas, S., Kokkinos, I., Sanfeliu, A., & Moreno-Noguer, F. "Segmentation-aware Deformable Part Models", Computer Vision and Pattern Recognition (CVPR), 2014.