Tutorial

About

Local image descriptors play a key role in relevant computer vision applications dealing with image matching, such as 3D reconstruction, image stitching, and visual localization and tracking. 3D reconstruction systems based on SfM pipelines and visual autonomous navigation systems based on SLAM are maybe the most noticeable examples of this kind of applications.

Although the steps involving in the process are almost standardized (patch extraction, local descriptor computation and matches assignment) and several off-the-shelf libraries exist, it is not easy for non-expert developers to combine the various blocks, due to non-homogeneous code interfaces, or to find the best setup for their particular task. Under these circumstances it can be difficult to take advantage of the best pieces of code available on the web and experimenting with them, with the result that people usually tend to fall-back to old well-established methods (such as standard SIFT).

The proposed tutorial is intended to cover some practical aspects dealing with the various steps of the image matching pipeline, in order to allow a better understating of the different issues one have to deal with in real applications, including possible enhancements and limitations in terms of computational efficiency, matching accuracy and robustness. In the meantime, some practical guidelines and examples about how to effectively interface non-homogeneous code and how to critically interpret state-of-the-art results and benchmarks will be presented too.

Course outline

• keypoint detection: differences among detectors, input image preprocessing, output keypoint filtering, extraction scale;

• patch extraction: scaled-only vs affine patches, extended support region length, orientation estimation and when it is not needed;

• local image descriptor: a taxonomy of descriptors and their properties, how to interface non-compatible code between patches and descriptors;

• matching: how to match (NN, NNR, 1-to-1, 1-to-many, mutual and symmetric matches), distance metrics (L1, L2, Hamming), improving matches (cascade filtering, quantization, context exploitation);

• benchmarks: results, protocol bias, available benchmarks.

Downloads

• PDF slides

• Video presentation - part 1/3 (introduction to the topic; patch normalization: orientation estimation, affine normalization, support region selection)

• Video presentation - part 2/3 (SIFT quantization; descriptor distance computation and optimization; NN-based matching strategies and ranking)

• Video presentation - part 3/3 (planar scene matching hints; benchmark analysis and evaluation)

Brief tutorial, for intermediate audience

Organizer

Fabio Bellavia