Embedded Files
Members:
Mauricio Hess-Flores, Mark A. Duchaineau, Kenneth I. Joy
Overview
We present a novel method for multi-view sequential scene reconstruction scenarios, such as in aerial surveillance video, that exploits the constraints imposed by the path of a moving camera to detect and correct inaccuracies in the feature tracking and structure computation processes. The main contribution is to show that for short, planar segments of a continuous and known camera trajectory, parallax movement corresponding to a viewed scene point should ideally form a scaled and translated version of this trajectory when projected onto a parallel plane, as shown in Figures 1-3, with cameras rendered as blue dots and their plane projections in green.
Figure 1: Parallax paths concept.
Figure 2: Traced parallax paths.
Figure 3: Top view of parallax paths.
When such paths are translated to a position-invariant reference (Figure 4), this creates an intra-camera best-fit consensus path constraint (Figure 5) as well as an inter-camera constraint such that path positions viewed by the same camera (red triangle in Figure 2) should lie along the same line (Figure 6), and such that the position-invariant paths only vary in scale. These constraints differ from those of standard factorization [TomasiKanade92]. The intersection of the consensus path at different scales with these lines results in a direct prediction of where each parallax path position should lie given the constraints.
Figure 4: Position invariance.
Figure 5: Consensus path.
Figure 6: Line constraint.
Any inaccurate feature tracks will result in position-invariant parallax paths which do not coincide with these predictions (Figure 7) but can then be corrected to fit the model, resulting in an improved structure (Figure 9) as opposed to the original (Figure 8). Results have been demonstrated on real and synthetic aerial video and turntable sequences, where the proposed method was shown to detect and correct outlier tracks and tracking drift, improve scene structure and also improve convergence for bundle adjustment optimization.
Figure 7: Path differences from predictions.
Figure 8: Initial reconstruction.
Figure 9: Updated reconstruction.
For further details, the AIPR 2012 poster corresponding to this work can be found here.
Related Publications
Mauricio Hess-Flores, Mark A. Duchaineau, Kenneth I. Joy, "Sequential Reconstruction Segment-Wise Feature Track and Structure Updating Based on Parallax Paths", in "to be published in: K.M. Lee et al. (Eds.): ACCV 2012, Part III, LNCS 7726", Volume 7726, pp 636--649, 2012.
Mauricio Hess-Flores, Mark A. Duchaineau, Kenneth I. Joy, "Path-Based Constraints for Accurate Scene Reconstruction from Aerial Video", in "Applied Imagery Pattern Recognition (AIPR) Workshop", 2012.
[TomasiKanade92] Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision 9 (1992) 137-154.
Page updated
Google Sites
Report abuse