Video 1. What's in the box: Dynamics videos reconstructed from neuronal spikes. Here the neuronal signal is a population of spikes reordered form the ganglion cells of the Salamander retina (see Ref 2 for experimental data descriptions). As a comparison, videos reconstructed from the neuronal signal of fMRI is listed (see Ref 3 for fMRI reconstruction).
Video 2. What's in the box: Arbitrary images and videos obtained with simulated spikes. Here neural spikes are generated by an encoder, and then another decoder is to reconstruct visual scenes with these spikes. The encoder and decoder are based on a model of the retinal ganglion cells .
Video 3. What's in the box: Real-time encoding and decoding of videos. Similar to Video 2, here a few stacked encoders and decoders are used to obtain high-resolution videos.
References:
Ref 1: Zheng Y, Jia S, Yu Z, Huang T, Liu J.K., Tian Y. Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation. Neural Networks. 126:42-51 (2020) [PDF] DOI: 10.1016/j.neunet.2020.03.003
Ref 2: Onken A., Liu J. K., Karunasekara C. R., Delis I., Gollisch T. and Panzeri S. Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains. PLoS Comput Biol 12(11): e1005189 (2016) [PDF]. Data Availability: http://dx.doi.org/10.5061/dryad.4ch10
Ref 3: Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., Gallant, J.C. Reconstructing visual experiences from brain activity evoked by natural movies Curr Biol. 21(19): 1641–1646 (2011). Demo: http://gallantlab.org/index.php/publications/nishimoto-et-al-2011/