Visual Hull in MVS

under construction!

Visual Hull (Shape From Silhouette)-based Scene Reconstruction

Multiple View Reconstruction

•3-D volumetric approach;

•Voxel coloring;

•Surface evolution;

•Space carving;

•Feature extraction and expansion;

•Patch-based MVS;

•Depth map-based fusion.

•KinectFusion is fusion of a monocular depth image sequence.

Visual hull: shape from silhouette

•Visual hull is the maximal shape consistent with an object’s silhouettes as seen from multiple viewpoints;

•VS is constructed by intersecting the cones generated by back projecting the object silhouettes;

•Camera calibration;

•Methods:

•Voxels-based;

•Polyhedral methods;

•Marching cubes-based methods;

•Image-based methods.

Testing data of multiple view reconstruction

•UIUC Jean Ponce group’s data:

•http://www-cvr.ai.uiuc.edu/ponce_grp/data/visual_hull/index.html;

•http://www-cvr.ai.uiuc.edu/ponce_grp/data/mview/;

•Middlebury’s data:

•http://vision.middlebury.edu/mview/data/.

Voxel-based visual hull method's pros and cons

•Background subtraction is critical for silhouette extraction;

•Object volume’s bounding box is also a factor to decide reconstruction resolution;

•Voxel number can be reduced by the efficient volume structure: Octree;

•option: voxel hashing;

•Surface rendering by marching cubes;

•option: Poisson surface reconstruction;

•Texture mapping: vertex coloring with the closest view's pixel searched by vertex normal;

•photo consistency: multiple reference views with fusion weights calculation;

•Pros: handle much better texture-less regions which are difficult for other MVS methods;

•Cons: sensitive to calibration errors, require enough number of views, no 3-d points reconstructed, hard to cope with concavity in object surface, may cause “phantom” volume artifacts;

•Trend: fusion of visual hull with stereo matching or depth map.

************************ DEMO 1 ************************

Input images and silhouettes: 48 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping

************************ DEMO 2 ************************

Input images and silhouettes: 24 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping

************************ DEMO 3 ************************

Input images and silhouettes: 24 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping

************************ DEMO 4 ************************

Input images and silhouettes: 24 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping

************************ DEMO 5 ************************

Input images: 48 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping

************************ DEMO 6 ************************

Input images: 47 views with camera calibration parameters

3-D Reconstructed Model

3-D Model with Color/Texture Mapping