Event Sensors

Principle

• Event cameras also called retinoptic cameras or dynamic vision sensors (DVS) capture “events”, which are triggered when the cumulative brightness change of a pixel reaches a certain threshold. An event has three elements: timestamp, pixel coordinate, and polarity. 

• An increase or decrease in brightness occurs when an event is detected or not.

• Event camera imaging principle guarantees that as long as the brightness change exceeds the threshold value, there will be an output, and it requires small bandwidth. If there are objects moving very fast in the  camera’s field of view, it will generate multiple events per second. If there is no object motion or brightness change, there are no events generated. At the same time, since the event camera is better at capturing the brightness change, it performs equally in dark and intense light scenes. 

• Event cameras offer the advantages of low latency, high dynamic range (140dB vs. 60dB), low power consumption, and are not affected by motion blur compared to regular frame-based.

A. Survey (1)

G. Gallego, T Delbrüc, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. Davidson, J. Conradt, K. Daniilidis, D. Scaramuzza,  "Event-Based Vision: A Survey", IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 44, No. 1, pages 154-180, Januray 2022 .

B. Image Reconstruction (7)

1. Asynchronous Event Processing 

C. Brandli, L. Muller, T. Delbruck, ”Real-time, high speed video decompression using a frame-and event-based DAVIS sensor",  IEEE International Symposium on Circuits and Systems, ISCAS 2014, 2014.

G. Munda, C. Reinbacher,  T. Pock, ”Real-time intensity-image reconstruction for event cameras using manifold regularisation", International Journal of Computer Vision, Volume 126, No. 12, pages 1381-1393, 2018.

C. Scheerlinck, N. Barnes, R. Mahony, ”Continuous- time intensity estimation using event cameras”, Asian Conference on Computer Vision, ACCV 2018, 2018.

2. Synchronous Batch Processing 

L. Wang, L. Ho, K. Yoon, ”Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks", Conference on Computer Vision and Pattern Recognition, CVPR 2019, 2019. 

E2VID

H. Rebecq, R. Ranftl, V. Koltun, D. Scaramuzza, ”High speed and high dynamic range video with an event camera", IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 43, No. 6, pages 1964-1980, 2019.

H. Rebecq, R. Ranftl, V. Koltun, D. Scaramuzza, ”Events-to-video: Bringing modern computer vision to event cameras", Computer Vision and Pattern Recognition, CVPR 2019, pages 3857-3866, 2019.

SPADE-E2VID

P. Cadena, Y. Qian, C. Wang, M. Yang, "SPADE-E2VID: Spatially-Adaptive Denormalization for Event-Based Video Reconstruction", IEEE Transactions on Image Processing, Volume 30, pages 2488-2500, 2021.

C. Moving Objects Detection (5)

S. Schaefer, D. Gehrig, D. Scaramuzza, "AEGNN: Asynchronous Event-based Graph Neural Networks ", IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2022,New Orleans, USA, 2022.

A. Mondal, R. Shashant, J. Giraldo, T. Bouwmans, A. Chowdhury, "Moving Object Detection for Event-based Vision using Graph Spectral Clustering", Workshop on "When Graph Signal Processing meets Computer Vision", ICCV 2021, Montréal, Canada, October 2021.

S. Zhang, W. Wang, H. Li, S. Zhang, "EventMD: High-Speed Moving Object Detection based on Event-based Video Frames", Pattern Recognition Letters, 2022.

J. Zhao , S. Ji, Z. Cai , Y. Zeng, Y. Wang,"Moving Object Detection and Tracking by Event Frame from Neuromorphic Vision Sensors", MDPI Biomemetics, 2022.

A. Mitrokhin, C. Fermüller, C. Parameshwara, Y. Aloimonos, "Event-Based Moving Object Detection and Tracking," 2018 IEEE International Conference on Intelligent Robots and Systems, IROS 2018, pages 1-9, 2018.

D. Tracking (1)

R. Wang, L. Wang, Y. He, L. Li, D. Lin, "Asynchronous event stream space point object motion trajectory detection", ELSEVIER Signal Processing, 2022.

E. 3D Reconstruction (1)

Y. Zhou, G. Gallego, H. Rebecq, L. Kneip,H. Li, D. Scaramuzza, "Semi-dense 3D Reconstruction with a Stereo Event Camera, European Conference on Computer Vision, ECCV 2018 pages 242-258, Munich, Germany, September 2018.