Deep Image Matting

Ning Xu, Brian Price, Scott Cohen, Thomas Huang

Beckman Institute for Advanced Science and Technology

University of Illinois at Urbana-Champaign

Adobe Research

Abstract

Image matting is a fundamental computer vision problem and has many applications. Previous algorithms have poor performance when an image has similar foreground and background colors or complicated textures. The main reasons are prior methods 1) only use low-level features and 2) lack high-level context. In this paper, we propose a novel deep learning based algorithm that can tackle both these problems. Our deep model has two parts. The first part is a deep convolutional encoder-decoder network that takes an image and the corresponding trimap as inputs and predict the alpha matte of the image. The second part is a small convolutional network that refines the alpha matte predictions of the first network to have more accurate alpha values and sharper edges. In addition, we also create a large-scale image matting dataset including 49300 training images and 1000 testing images. We evaluate our algorithm on the image matting benchmark, our testing set, and a wide variety of real images. Experimental results clearly demonstrate the superiority of our algorithm over previous methods.

New Matting Dataset

We create a large-scale matting dataset using composition. Images with objects on simple backgrounds were carefully extracted and were composited onto new background images to create a dataset with ~~49300~~ (45500) training images and 1000 test images.

Method

We address the image matting problem using deep learning. We are the first to demonstrate the ability to learn an alpha matte end-to-end given an image and trimap. Given our new dataset, we train a neural network to fully utilize the data. The network consists of two stages. The first stage is a deep convolutional encoder-decoder network which takes an image patch and a trimap as input and is penalized by the alpha prediction loss and a novel compositional loss. The second stage is a small fully convolutional network which refines the alpha prediction from the first network with more accurate alpha values and sharper edges.