Interactive Visual Hull Refinement for Specular and

Transparent Object Surface Reconstruction

International Conference on Computer Vision (ICCV) 2015

Figure 1: Original image(left); Reconstruction with traditional visual hull(middle left); Reconstruction with our method(middle right); Rendered model with our method(right).

Abstract:

In this paper we present a method of using standard multi-view images for 3D surface reconstruction of non-Lambertian objects. We extend the original visual hull concept to incorporate 3D cues presented by internal occluding contours, i.e., occluding contours that are inside the object’s silhouettes. We discovered that these internal contours, which are results of convex parts on an object’s surface, can lead to a tighter fit than the original visual hull. We formulated a new visual hull refinement scheme – Locally Convex Carving that can completely reconstruct concavity caused by two or more intersecting convex surfaces.

In addition we develop a novel approach for contour tracking given labeled contours in sparse key frames. It is designed specifically for highly specular or transparent objects, for which assumptions made in traditional contour detection/tracking methods, such as highest gradient and stationary texture edges, are no longer valid. It is formulated as an energy minimization function where several novel terms are developed to increase robustness.

Based on the two core algorithms, we have developed an interactive system for 3D modeling. We have validated our system, both quantitatively and qualitatively, with four datasets of different object materials. Results show that we are able to generate visually pleasing models for very challenging cases.

Citation:

References:

[1] Y. Furukawa and J. Ponce. Carved visual hulls for image based modeling. In ECCV, 2006.

[2] E. Boyer and M. O. Berger. 3d surface reconstruction using occluding contours. In IJCV, 1997.

[3] Q. Shan, B. Curless, Y. Furukawa, C. Hernandez, and S. M.Seitz. Occluding contours for multi-view stereo. In CVPR, 2014.