Aerial image geolocalization from recognition and matching of roads and intersections

Team members

Dragos Costea

Supervisors

Prof. dr. Marius Leordeanu

Idea

Use the unique road pattern surrounding intersections for geolocalization

Implementation

Detect roads, intersections>>make a descriptor for the road map around the intersections>>compare descriptors>>refine results by geometric alignment >>geolocalization

Details

We propose a complete pipeline for aerial image geolocalization based on roads and intersections (Fig.1). The main steps are road detection, followed by intersection detection and intersection matching with publicly available road vectors from the OpenStreetMap project (OSM). An initial localization is proposed, further improved by geometric alignment of roads.

Fig 1. Framework overview

The main goal of this project is to provide drones a reliable localization system in the areas we expect most autonomous deployments will be made - that is, urban and suburban areas.

A key insight of our approach is the observation that intersections tend to have a unique road pattern surrounding them and thus can play a key role in localization, by reducing this difficult task to a sparse feature matching problem followed by a local refined road map alignment.

For pixel-wise road detection we used a dual-stream local-global CNN model proposed here. Alexnet was used for intersection detection based on the detected road pattern.

For intersection matching, we introduce a novel dataset consisting of images centered on intersections from two cities (one for training and the other one for testing), totalling 7204 600x600px images. A 4096-element descriptor is generated for each intersection using the surrounding detected roads and a neural network trained for intersection detection, in a way that is similar to the method described here. We further fine-tune the network in a Siamese-like fashion in order to improve matching performance.

After an initial set of corresponding intersections is returned, we pick the best one by geometrical alignment(GA) of road maps for intersection candidates using shape context and RANSAC. Further road enhancements for OSM are possible once the location has been determined.

The detection radius translates in the number of intersections seen. A larger radius means more intersections available for comparison. Fig 2. shows the performance boost of geometrical alignment and overall localization errors for different radii.

Results

Fig2. Performance and errors

We notice that most errors (around or above 90% of them) are below 2.5 meters, that is below 3 pixels for the image resolution available in our experiments. We believe that our results demonstrate

high level of localization accuracy for our system, which could be very effective in most cases when the GPS signal is lost, for both day and nighttime.

Conclusions

We have presented a complete system for geo-localization from aerial images in the absence of GPS information. Our proposed pipeline includes many contributions with efficient methods for road and intersection detection, intersection recognition with geometric alignment for accurate localization, followed by road detection enhancement.

There are many potential applications for our approach in areas such as urban planning, tracking structural changes, updating of existing maps and environmental monitoring.

Our system could also be used in the context of unmanned aerial vehicles, in order to correct their GPS localization or to make their flight possible even when GPS signal is lost.

For nighttime use for example, the roads are generally 'extracted' by means of street lightning, which could make the problem of road and intersection detection easier - thus even more accessible for on-board processing.

Future work

Since the main goal is to have an embedded localization system, speed is very important. The work published at BMVC shows that such system should work well, but it is currently not tailored for an embedded platform. Although obtaining and comparing intersection descriptors is fast, road and intersection detection takes a lot of valuable time.

We aim to reduce the computational time by using custom neural network architectures for road and intersection detection. The performance tradeoff will be explored.

Furthermore, localization can be helped by some other image cues, such as buildings or trees. We aim to generalize our approach to fit a wide spectrum of applications.

Paper

paper

Accepted at British Machine Vision Conference, 2016

Dataset

here

Cite

Dragos Costea and Marius Leordeanu. Aerial image geolocalization from recognition and matching of roads and intersections. In Richard C. Wilson, Edwin R. Hancock and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 118.1-118.12. BMVA Press, September 2016.

@inproceedings{BMVC2016_118,

title={Aerial image geolocalization from recognition and matching of roads and intersections},

author={Dragos Costea and Marius Leordeanu},

year={2016},

month={September},

pages={118.1-118.12},

articleno={118},

numpages={12},

booktitle={Proceedings of the British Machine Vision Conference (BMVC)},

publisher={BMVA Press},

editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},

doi={10.5244/C.30.118},

isbn={1-901725-59-6},

url={https://dx.doi.org/10.5244/C.30.118}

}