Research

Natural image processing in the visual cortex

Funding: NIH EY030578 

Signals from the natural environment are processed by neuronal populations in the cortex. Understanding the relationship between those signals and cortical activity is central to understanding normal cortical function and how it is impaired in psychiatric and neurodevelopmental disorders. Substantial progress has been made in elucidating cortical processing of simple, parametric stimuli, and computational technology is improving descriptions of neural responses to naturalistic stimuli. However, how cortical populations encode the complex, natural inputs received during every day perceptual experience is largely unknown. This project aims to elucidate how natural visual inputs are represented by neuronal populations in primary visual cortex (V1). Progress to date has been limited primarily by two factors. First, during natural vision, the inputs to V1 neurons are always embedded in a spatial and temporal context, but how V1 integrates this contextual information in natural visual inputs is poorly understood. Second, prior work focused almost exclusively on single-neuron firing rate, but to understand cortical representations one must consider the structure of population activity— the substantial trial-to-trial variability that is shared among neurons and evolves dynamically—as this structure influences population information and perception. The central hypothesis of this project is that cortical response structure is modulated by visual context to approximate an optimal representation of natural visual inputs. To test the hypothesis, this project combines machine learning to quantify the statistical properties of natural visual inputs, with a theory of how cortical populations should encode those images to achieve an optimal representation, to arrive at concrete, falsifiable predictions for V1 response structure. We then test the predictions with measurements of population activity in V1 of awake monkeys viewing natural images and movies. This project will provide the first test of a unified functional theory of contextual modulation in V1 encoding of natural visual inputs, and shed light on key aspects of natural vision that have been neglected to date.

(2023) X. Pan, R. Coen-Cagli, O. Schwartz, Probing the Structure and Functional Properties of the Dropout-induced Correlated Variability in Convolutional Neural Networks. Neural Computation (accepted; bioRxiv)

(2022) J. Botsen, R. Coen-Cagli, A. Franklin, S. Solomon, M. Webster. Calibrating the visual system. Vision Research 201:108-132

(2021) D. Festa, A. Aschner, A. Davila, A. Kohn, R. Coen-Cagli. Neuronal variability reflects probabilistic inference tuned to natural image statistics. Nature Communications 12:3635 (preprint bioRxiv) 

(2016) M. Snow, R. Coen-Cagli, O. Schwartz, Specificity and timescales of cortical adaptation as inferences about natural movie statistics. Journal of Vision 16(13):1.

(2015) R. Coen-Cagli, A. Kohn*, O. Schwartz*, Flexible Gating of Contextual Modulation During Natural Vision. Nature Neuroscience, 18: 1648–1655

(2013) R. Coen-Cagli, O. Schwartz, The Impact on Mid-Level Vision of Statistically Optimal Divisive Normalization in V1. Journal of Vision, 13(8):13

(2013) O. Schwartz, R. Coen-Cagli, Visual Attention and Flexible Normalization Pools. Journal of Vision, 13(1):25

(2012) R. Coen-Cagli, P. Dayan, O. Schwartz, Cortical Surround Interactions and Perceptual Salience Via Natural Scene Statistics. PLoS Computational Biology, 8(3): e1002405.

Probabilistic models of perceptual grouping and segmentation in natural vision 

Funding: NIH EY031166 (CRCNS)

To understand and navigate the environment, sensory systems must solve simultaneously two competing and challenging tasks: the segmentation of a sensory scene into individual objects and the grouping of elementary sensory features to build these objects. Understanding perceptual grouping and segmentation is therefore a major goal of sensory neuroscience, and it is central to advancing artificial perceptual systems that can help restore impaired vision. To make progress in understanding image segmentation and improving algorithms, this project combines two key components. First, a new experimental paradigm that allows for well-controlled measurements of perceptual segmentation of natural images. This addresses a major limitation of existing data that are either restricted to artificial stimuli, or, for natural images, rely on manual labeling and conflate perceptual, motor, and cognitive factors. Second, this project involves developing and testing a computational framework that accommodates bottom-up information about image statistics and top-down information about objects and behavioral goals. This is in contrast with the paradigmatic view of visual processing as a feedforward cascade of feature detectors, that has long dominated computer vision algorithms and our understanding of visual processing. The proposed approach builds instead on the influential theory that perception requires probabilistic inference to extract meaning from ambiguous sensory inputs. Segmentation is a prime example of inference on ambiguous inputs: the pixels of an image often cannot be labeled with certainty as grouped or segmented. This project will test the hypothesis that human visual segmentation is a process of hierarchical probabilistic inference, and will offer a unified framework to understand the integration of bottom-up and top-down influences in human segmentation of natural inputs. 

(2023) J. Vacher, C. Launay, P. Mamassian**, R. Coen-Cagli**. Measuring uncertainty in human visual segmentation. PLoS Computational Biology 19(9): e1011483  

(2022) C. Launay, J. Vacher, R. Coen-Cagli. Unsupervised video segmentation algorithms based on flexibly regularized mixture models. IEEE ICIP.

(2022) J. Vacher, C. Launay, R. Coen-Cagli. Flexibly regularized mixture models and application to image segmentation. Neural Networks 149:107-123 (preprint on arxiv:1905.10629)

(2021) D. Herrera, R. Coen-Cagli**, L. Gomez-Sena**. Flexible contextual modulation of naturalistic texture perception in peripheral vision. Journal of Vision 21(1):1

(2021) D. Herrera, L. Gomez-Sena, R. Coen-Cagli. Redundancy between spectral and higher-order statistics for natural image segmentation. Vision Research 187:55-65 (preprint bioRxiv)  

(2020) J. Vacher, A. Davila, A. Kohn, R. Coen-Cagli. Texture interpolation for probing visual perception. NeurIPS 33.

(2019) J. Vacher, P. Mamassian, R. Coen-Cagli. Probabilistic model of visual segmentation. arxiv:1806.00111.

Computational Tools for assessing mechanisms and functional relevance of divisive normalization

Funding: NIH DA056400

Divisive normalization is a long-standing theory, supported by empirical observations across species and brain areas, of how neurons in a circuit modulate each other’s activity. This project aims to develop statistical and computational tools to precisely monitor and perturb normalization, from single neurons to large neural populations. The models and software toolbox developed in this project will thus enable new quantitative and causal studies of the mechanisms of a widespread phenomenon—normalization—and their impact on behavior.

(2023) O. Weiss, H.A. Bounds, H. Adesnik, R. Coen-Cagli, Modeling the diverse effects of divisive normalization on noise correlations. PLoS Computational Biology 19(11):e1011667

(2019) R. Coen-Cagli, S. S. Solomon. Relating divisive normalization to neuronal response variability. Journal of Neuroscience 39(37):7344 (preprint on biorxiv).

Probabilistic population codes, Fisher information 

(2022) M.A. Frechou, S.S. Martin, C.D McDermott, S. Gokhan, W. Tome, R. Coen-Cagli, T.J. Goncalves, Adult Neurogenesis improves spatial information encoding in the mouse hippocampus. bioRxiv

(2021) S. Sokoloski, A. Aschner, R. Coen-Cagli. Modeling the neural code in large populations of correlated neurons. eLife (preprint on bioRxiv)

(2021) G. Dehaene, R. Coen-Cagli, A. Pouget. Investigating the representation of uncertainty in neuronal circuits. PLoS Computational Biology 17(1):e1008138.

(2016) A. Kohn, R. Coen-Cagli, I. Kanitscheider, A. Pouget, Correlations and neuronal population information. Annual Reviews of Neuroscience. 39:237-256.

(2015) I. Kanitscheider*, R. Coen-Cagli*, A. Pouget, The origin of information-limiting noise correlations. PNAS, 112(50): E6973-E6982

(2015) I. Kanitscheider*, R. Coen-Cagli*, A. Kohn, A. Pouget, Measuring Fisher Information Accurately in Correlated Neural Populations. PLoS Computational Biology, 11(6): e1004218