DAISY Filter Flow: A Generalized Discrete Approach to Dense Correspondences

CVPR 2014

Hongsheng Yang^{1, 2} Wen-Yan Lin¹ Jiangbo Lu¹

¹Advanced Digital Sciences Center, Singapore ²University of North Carolina at Chapel Hill, USA

Abstract

Establishing dense correspondences reliably between a pair of images is an important vision task with many applications. Though significant advance has been made towards estimating dense stereo and optical flow fields for two images adjacent in viewpoint or in time, building reliable dense correspondence fields for two general images still remains largely unsolved. For instance, two given images sharing some content exhibit dramatic photometric and geometric variations, or they depict different 3D scenes of similar scene characteristics. Fundamental challenges to such an image or scene alignment task are often multifold, which render many existing techniques fall short of producing dense correspondences robustly and efficiently. This paper presents a novel approach called DAISY filter flow (DFF) to address this challenging task. Inspired by the recent PatchMatch Filter technique, we leverage and extend a few established methods: 1) DAISY descriptors, 2) filter-based efficient flow inference, and 3) the Patch-Match fast search. Coupling and optimizing these modules seamlessly with image segments as the bridge, the proposed DFF approach enables efficiently performing dense descriptor-based correspondence field estimation in a generalized high-dimensional label space, which is augmented by scales and rotations. Experiments on a variety of challenging scenes show that our DFF approach estimates spatially coherent yet discontinuity-preserving image alignment results both robustly and efficiently.

Paper: [PDF]

Notice: Code is available now on the github repository. The code is for research and education purpose only.

The authors would plan to release the clean code during summer time. If you are interested in our work, please check it later after August 2014; also feel free to send email to the authors or post issue in the github repository. We appreciate your feedback, and thank you for interested in our research!

References

[1] PMF - PatchMatch Filter: Efficient Edge-Aware Filtering Meets Randomized Search for Fast Correspondence Field Estimation, J. Lu et al., CVPR 2013.

[2] GPM - The Generalized PatchMatch Correspondence Algorithm, C. Barnes et al., ECCV 2010.

[3] SIFT Flow - SIFT Flow: Dense Correspondence across Scenes and its Applications, C. Liu et al., TPAMI 2011.

[4] NRDC - NRDC: Non-Rigid Dense Correspondence with Applications for Image Enhancement, Y. HaCohen et al., SIGGRAGH 2011.

[5] SLS - On SIFTs and their Scales, T. Hassner et al., CVPR 2012.

[6] SID - Scale Invariance without Scale Selection, I. Kokkinos et al., CVPR 2008.

[7] SSID - Dense Segmentation-Aware Descriptors , E. Trulls et al., CVPR 2013.

[8] DSP - Deformable Spatial Pyramid Matching for Fast Dense Correspondences, J. Kim et al., CVPR 2013.