Image-based Relocalization in SLAM

•Key Frame and its Camera Pose Stored;

•Frame id, image feature and camera pose.

•Option: Extract Local Features;

•Feature original or coded.

•Establish 2D-2D or 3D-to-2D Matches;

•Single camera: 3d-to-2d;

•Stereo or depth camera: 2d-to-2d;

•Camera Pose Estimation.

•Refine by 3-d point estimation;

•2-d features along with their depths.

•Direct image matching:

•Gaussian blurred and downsampled templates for key frame matching;

•Feature matching-based:

•Corner (Harris, FAST, Good) + Descriptor (SIFT, SURF, BRIEF, FREAK, ORB, Daisy, …);

•BovW-based image matching:

•Build codebook or vocabulary, then map features to BovW;

•Learning the mapping from the features or BoVWs to camera pose:

•Camera pose estimation is formulated as a regression problem;

•Feature learning for keyframe matching:

•Hierarchical feature representation by deep learning;

•Metric learning for nonlinear distance optimization;

•Manifold learning for optimized dimension reduction of feature space.

•Descriptor Extraction;

•Dimension reduction: PCA;

•Nearest Neighbor Search:

•Brute force: K-d tree or radius search;

•ANN;

•Outlier Removal:

•Ratio test;

•RANSAC for geometric consistency check

•Homography;

•Fundamental matrix;

•Feature detection and descriptor for key frames or new frames which require relocalization:

•Harris corner;

•SURF (64-d vector);

•Feature matching for pose recovery:

•Brute force search;

•Pose difference is small enough.

•Key frame selection:

•Pose change enough big;

•Codebook building from training data:

•Feature extraction: FAST corner detection + SURF feature descriptor;

•Feature clustering for visual vocabulary: incremental K-means;

•Construct co-occurrence statistics of visual words by Chow Liu tree.

•A naïve Bayes approximation by a maximum weight spanning tree.

•Visual pattern from key frames or new frames for relocalization:

•Feature extraction: same as above;

•Mapping descriptors to bags of words by ANN-based (kd-tree) methods;

•Pattern matching for pose recovery: descriptor mapped to codebook;

•A sampling method in training data to find the best match.