Semantic Navigation for Robot-Human Teams
Most of the work done in localization, mapping, and navigation for both ground and aerial vehicles has been done by means of point landmarks or occupancy grids, using vision or lasers range finders. However, to make these robots one day able to cooperate with humans in complex scenarios, we need to build semantic maps of the environment.
My recent work (submitted to ICRA'12) considers the problem of map-based robot localization using "soft" object detection. Soft object detection differs from "hard" object detection in that we do not extract an "affirmative/negative" response (i.e., "hard") about the presence of the object but rather we compute, for each pixel in the current frame, the probability that the object under consideration is there. This gives raise to many false positive (see the multiple peaks in the object "heat-map") that are disambiguated during motion by the particle filter.
In the following video, the left panes shows: (top) the original panoramic image (captured with a Ladybug camera) of the environment, (second) the "heat-map" corresponding to the clock object class, (third)
the "heat-map" corresponding to the trashcan object class, (fourth) the "heat-map" corresponding to the ticket-machine object class, (fifth) the "heat-map" corresponding to the ATM-machine object class.
I was invited to wrote a two-part tutorial on visual odometry for IEEE RAM:
Feature Tracking on Omnidirectional Images for Robot Navigation
Here a novel method for robustly tracking vertical features taken by omnidirectional images is developed. Matching robustness is achieved by means of a feature descriptor which is invariant to rotation and slightly changes of illumination.
Omnidirectional Camera Modelling and Calibration
A unified model for central omnidirectional cameras (both with mirror or fisheye) and a novel calibration method which uses checkerboard patterns.
Omnidirectional Camera Calibration Toolbox for MATLAB: Download the OCamCalib Toolbox
This toolbox is the only one which has an automatic checkerboard extraction.
Works in Windows and Linux.