EMSGC: Event-based Motion Segmentation with Spatio-Temporal Graph Cuts

Abstract. Identifying independently moving objects is an essential task for dynamic scene understanding. However, traditional cameras used in dynamic scenes may suffer from motion blur or exposure artifacts due to their sampling principle. By contrast, event-based cameras are novel bio-inspired sensors that offer advantages to overcome such limitations. They report pixelwise intensity changes asynchronously, which enables them to acquire visual information at exactly the same rate as the scene dynamics.
We develop a method to identify independently moving objects acquired with an event-based camera, i.e., to solve the event-based motion segmentation problem. We cast the problem as an energy minimization one involving the fitting of multiple motion models. We jointly solve two subproblems, namely eventcluster assignment (labeling) and motion model fitting, in an iterative manner by exploiting the structure of the input event data in the form of a spatio-temporal graph. Experiments on available datasets demonstrate the versatility of the method in scenes with different motion patterns and number of moving objects. The evaluation shows state-of-the-art results without having to predetermine the number of expected moving objects. We release the software and dataset under an open source licence to foster research in the emerging topic of event-based motion segmentation.



Yi Zhou, Guillermo Gallego, Xiuyuan Lu, Siqi Liu, Shaojie Shen, "Event-based Motion Segmentation with Spatio-Temporal Graph Cuts", IEEE Transactions on Neural Network and Learning Systems, 2021.

@article{Zhou21tnnls, title={Event-based Motion Segmentation with Spatio-Temporal Graph Cuts}, author={Zhou, Yi and Gallego, Guillermo and Lu, Xiuyuan and Liu, Siqi and Shen, Shaojie}, journal={IEEE Transactions on Neural Network and Learning Systems}, year={2021}}

X. Lu, Y. Zhou, S. Shen. Event-based motion segmentation by cascaded two-level multi-model fitting. In Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, Sept. 2021.

Bib not available yet.

Source Code

Dataset Download


The software is released under a GPLv3 license. For commercial use, please contact the authors.


If you have any questions about this project, please contact

Yi Zhou (eeyzhou@hnu.edu.cn or cavatinajoey@gmail.com)