Members:

Mauricio Hess-Flores, Daniel Knoblauch, Mark A. Duchaineau, Kenneth I. Joy, Falko Kuester

Abstract

An algorithm that shows how ray divergence in multi-view stereo scene reconstruction can be used towards improving bundle adjustment weighting and conditioning is presented. Starting with a set of feature tracks, ray divergence when attempting to compute scene structure for each track is first obtained. Assuming accurate feature matching, ray divergence reveals mainly camera parameter estimation inaccuracies. Due to its smooth variation across neighboring feature tracks, from its histogram a set of weights can be computed that can be used in bundle adjustment to improve its convergence properties. It is proven that this novel weighting scheme results in lower reprojection errors and faster processing times than others such as image feature covariances, making it very suitable in general for applications involving multi-view pose and structure estimation.

Overview

Multi-view scene reconstruction is currently a prominent research area, with many applications. Despite accurate feature matching, reconstruction accuracy relies on accurate camera calibration. In the absence of ground-truth, algorithms usually resort to bundle adjustment to reduce reprojection error, which is very expensive and needs a good starting point. Our main contribution is to show how simple reconstruction 'ray divergence' is an inexpensive yet powerful tool that can aid in bundle adjustment convergence.

The algorithm can be summarized as follows. For a set of sparse feature matches between two images, ray divergence d when attempting to compute scene structure is first obtained, as shown in Figure 1 for one feature match seen between two cameras C1 and C2.

It had been shown in our earlier ISVC 2009 publication that such ray divergence can be decomposed into a smooth surface corresponding to camera parameter inaccuracies and a high-frequency component due to inaccurate feature matches. Examples of such smooth surfaces obtained for both sparse and dense feature matches are shown in Figure 2. Assuming highly-accurate feature matching, the entire ray divergence error corresponds to any inaccuracies related to the camera intrinsics, extrinsincs or radial distortion.

Figure 2: Sparse (left) and dense (right) ray divergence surfaces.

Given the smooth variation in ray divergence, its histogram, where an example is shown in Figure 3, can be modeled as a Gaussian probability density function. Based on this histogram, a set of weights can be derived for weighted bundle adjustment, such that lower weights correspond to higher ray divergences.

Results

Results show that our proposed method, PPBA, provides the best combination of processing time, final reprojection error and computational complexity in computing weights, with respect to common methods in the literature such as image feature covariances (CBA), reconstructed point confidence ellipsoid roundness with (UWBA) and without including image feature covariances (UIBA), and Gaussian-pdf with ray divergences (RDBA).

The resulting high-quality sparse reconstructions allow for other algorithms to be applied, such as dense reconstructions with the PMVS algorithm [Furukawa07] as shown in the following images.

Figure 5: Obtained sparse reconstructions (left) and after applying PMVS (right), for the 'Medusa' (top) and 'Stockton' (bottom) datasets.

Related Publications

• Daniel Knoblauch, Mauricio Hess-Flores, Mark A. Duchaineau, Falko Kuester: Factorization of Correspondence and Camera Error for Unconstrained Dense Correspondence Applications. In: ISVC(1)2009: 720-729.

• Mauricio Hess-Flores, Daniel Knoblauch, Mark A. Duchaineau, Kenneth I. Joy, Falko Kuester. Ray Divergence-Based Bundle Adjustment Conditioning for Multi-View Stereo. In: PSIVT 1, Vol. 7087 Springer (2011), p. 153-164.

• [Furukawa07] Furukawa, Y., Ponce, J.: Accurate, Dense, and Robust Multi-View Stereopsis. In: IEEE Conference on Computer Vision and Pattern Recognition. (2007) 1-8.

Return Home