Abstract. Event-based cameras are bio-inspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging  scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (3D reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semi-dense 3D map of the scene by fusing depth estimates from multiple local  viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source license to foster research in the emerging topic of event-based SLAM.

• 2024/10/15 ESVO2 has been released!!! You may find it at https://github.com/NAIL-HNU/ESVO2.

• 2021/2/23 ESVO is integrated into the modular iniVation DV software platform. Download here: https://lnkd.in/deuRKSK 

• 2021/2/16 ESVO accepted by IEEE Transactions on Robotics (T-RO).

The original rpg (University of Zurich) and upenn (University of Pennsylvania) datasets can be download from

• rpg: http://rpg.ifi.uzh.ch/ECCV18_stereo_davis.html

• upenn: https://daniilidis-group.github.io/mvsec/

For convenience, we provide the edited rosbag files used in the paper. The proposed edition is as described in the "readme" file under /events_repacking_helper.

The software is released under a GPLv3 license. For commercial use, please contact the authors.

List of Event-based Vision Resources

Event-based Vision Survey paper

If you have any questions about this project, please contact

Yi Zhou (eeyzhou@hnu.edu.cn)

Guillermo Gallego

Shaojie Shen