DeNoising Dynamic vision sensors

Low Cost and Latency Event Camera
Background Activity Denoising

DND21 is the dataset and methods for dynamic vision sensor event camera background activity denoising. DND21 targets comparative benchmarking for improved denoising algorithms.

See the top menu for Denoising examples, Datasets and Benchmarking

DND21 was developed by the Sensors Group of the Inst. of Neuroinformatics, Univ. of Zurich and ETH Zurich.

Information about other datasets and code are on the Sensors Group webpage.

News

Please see our companion paper Rios-Navarro, A., S. Guo, G. Abarajithan, K. Vijayakumar, A. Linares-Barranco, T. Aarrestad, R. Kastner, and T. Delbruck. “Within-Camera Multilayer Perceptron DVS Denoising.” In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 3933–42. IEEE, 2023. https://doi.org/10.1109/CVPRW59228.2023.00409 . Open access link.

This paper shows how to implement the Multilayer Perceptron Noise Filter (MLPF) within event camera logic circuits. This super quick tiny MLP neural network (40ns inference time, 4nJ/event) can accurately discriminate between signal and noise events from DVS event cameras as they are produced by the camera and can be economically integrated either to the sensor ASIC or to the external FPGA used for most event cameras. By using this denoiser, the system-level power consumption can be dramatically reduced (from 1k to 10 interrupts/sec) because the processor of events is much less busy processing noninformative noise events. The paper and hardware implementation is openly available on https://tub-rip.github.io/eventvision2023/ as accepted poster/code #24: Within-Camera Multilayer Perceptron DVS Denoising, Suppl mat, Code, Poster

Paper citation

S. Guo and T. Delbruck, “Low Cost and Latency Event Camera Background Activity Denoising,”, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2022.

Submitted Feb 2021; revised Aug 2021, accepted 10 Feb. 2022, early access posted 23 Feb. 2022, See preprint.

Available https://ieeexplore.ieee.org/document/9720086

Don't forget to check the long IEEE Supplementary Material extra media. Or else see preprint version TPAMI-2021-02-0169.R1 of accepted paper that includes links between main and supplementary material.

We recommend downloading and viewing with Acrobat or Sumatra PDF v3.0 to properly display the acronym tool tips.

Readers can check the complete thread of reviews and responses which further discuss some aspects of the work in detail.

Creators and contact

Shasha Guo <guoshasha13@nudt.edu.cn> (NUDT, Changsha, China),

Tobi Delbruck <tobi@ini.uzh.ch> (Sensors Group, INI, UZH-ETH Zurich)

Bibtex

@ARTICLE{Guo2022-vh,

title = "Low Cost and Latency Event Camera Background Activity Denoising",

author = "Guo, Shasha and Delbruck, Tobi",

journal = "IEEE Trans. Pattern Anal. Mach. Intell.",

volume = "PP",

month = feb,

year = 2022,

url = "http://dx.doi.org/10.1109/TPAMI.2022.3152999",

language = "en",

issn = "0162-8828",

pmid = "35196224",

doi = "10.1109/TPAMI.2022.3152999"

}

Video overview

DND21 overview -rev1_2_2.mp4

Acknowledgements

We gratefully acknowledge numerous comments on draft version of our paper from SC Liu and G Gallego, and further comments from C. Li, E. Culurciello, T. Aarrestad, and R Berner. S. Guo's visit to Zurich was supported by the Chinese Scholarship Council and her PhD advisors Weixia Xu and Lei Wang. We also acknowledge the contributions of M Milde, D Karpul, C Bartolozzi, E Chicca, and G Orchard in discussions leading to STCF, and L Wang for discussions leading to DWF. We thank A Khodamoradi and R Kastner for sharing source code and data for ONF, and technical support from Y Hu.

We thank the Swiss National Science Foundation for funding via the projects VIPS (40B2-0_181010), SciDVS (185069) and the visiting scientist travel grant for Prof. R. Kastner (210473).

Page updated

Google Sites

Report abuse