Research
My research field is field robotics with particular focus on perception and computer vision. These are the current problems I am focusing on:
Vision based Navigation for Micro Aerial Vehicles (MAVs)
My main research goal is to develop a teams of MAVs which can fly autonomously in city-like environments and which can be used to assist humans in tasks like rescue and monitoring. Since 2009, I am the coordinator of the EU project sFly. Below, you can see the relevant videos of my research that I realized together with my three PhD students at ETH Zurich. My main objectives focus on enabling autonomous navigation using vision and IMU as sole sensor modalities (i.e., no GPS, no laser). In 2009, I led a group of seven students with which we ranked second at the European MAV competition (EMAV) in September 2009 with our team called EMAVERICK. Our vehicle was the sole navigating using vision only. We performed 10 meters and entered a small apartment through it s window using only natural features extracted from the environment. See video below.
Relevant publications:
ICRA'10-a
Links:
Semantic Navigation for Robot-Human Teams
Most of the work done in localization, mapping, and navigation for both ground and aerial vehicles has been done by means of point landmarks or occupancy grids, using vision or lasers range finders. However, to make these robots one day able to cooperate with humans in complex scenarios, we need to build semantic maps of the environment.
My recent work (submitted to ICRA'12) considers the problem of map-based robot localization using "soft" object detection. Soft object detection differs from "hard" object detection in that we do not extract an "affirmative/negative" response (i.e., "hard") about the presence of the object but rather we compute, for each pixel in the current frame, the probability that the object under consideration is there. This gives raise to many false positive (see the multiple peaks in the object "heat-map") that are disambiguated during motion by the particle filter.
In the following video, the left panes shows: (top) the original panoramic image (captured with a Ladybug camera) of the environment, (second) the "heat-map" corresponding to the clock object class, (third)
the "heat-map" corresponding to the trashcan object class, (fourth) the "heat-map" corresponding to the ticket-machine object class, (fifth) the "heat-map" corresponding to the ATM-machine object class.
Visual Odometry and Mapping in Urban Environments
The car is equipped with an omnidirectional camera and the motion of the vehicle is purely recovered from salient features tracked over time. My algorithm can runs at 800 Hz on a normal laptop. To my knowledge, this is the most efficient visual odometry algorithm. This is possible thanks to a novel method for removing the outliers of the feature matching process. While standard algorithms are based on the 5-point RANSAC, our algorithm uses 1-point RANSAC (actually histogram voting) by exploiting the non-holonomic constraints of the vehicle.
I strongly believe that this algorithm will be integrated in the next generation cars
Relevant publications:
I was invited to wrote a two-part tutorial on visual odometry for IEEE RAM:
Self Calibration between a Camera and a 3D Scanner from Natural Scenes
This problem deals with the extrinsic calibration between an omnidirectional camera and 3D laser range finder. The aim is to get a precise mapping of the color information given by the camera onto the 3D points.
Relevant publications:
Videos:
[Laser & camera calibration demo]
Feature Tracking on Omnidirectional Images for Robot Navigation
Here a novel method for robustly tracking vertical features taken by omnidirectional images is developed. Matching robustness is achieved by means of a feature descriptor which is invariant to rotation and slightly changes of illumination.
Relevant publications:
[Scaramuzza, Int. Journal Robotics Research 09]
Omnidirectional Camera Modelling and Calibration
A unified model for central omnidirectional cameras (both with mirror or fisheye) and a novel calibration method which uses checkerboard patterns.
Omnidirectional Camera Calibration Toolbox for MATLAB: Download the OCamCalib Toolbox
This toolbox is the only one which has an automatic checkerboard extraction.
Works in Windows and Linux.
Relevant publications: