Undergraduate Thesis

Feature Integration in Visual Object Representations at the Perceptual and Neural Levels

Undergraduate thesis done with Prof. SP Arun, Centre for Neuroscience, Indian Institute of Science.

The relatively simple task of visual object perception, one we perform at each awake moment without much effort, is a non-trivial feat achieved by the brain. How the brain encodes various visual features that lead to object perception is a big open question, and attempts have been made at answering many small parts of this question. From a reductionist perspective, feature separability in the visual representation of objects is an essential property the visual system should display since it would allow the system to encode multiple objects by just encoding for the parts, or features, that comprise them. But for successful representations of objects as the combination of its features, while maintaining the separability of the parts, an efficient feature integration mechanism is also required. In this thesis, we investigated a few aspects of feature separability and integration mechanism, in the scale of single neurons, neural populations, and human perception, and tried to link the properties observed at these three scales. The specific questions answered in different chapters are mentioned below.

Does Surface Invariance in Single Neurons Manifest in Perception?

In this study we tried to understand the separability and integration of surface shapes and that of the shapes of patterns on it, in human perception. We investigated whether a certain set of surface-pattern combinations, termed congruent, are perceived to be more similar to each other in perception, compared to the corresponding set of incongruent combinations. We then compared the perceptual results with the previously reported results from our lab observed in single neural representation (Ratan Murty and Arun, 2017). We found that in a simple visual search paradigm, surfaces and patterns on them combine linearly, and the congruent set is perceived no different from the incongruent set. But in a different experimental setting, the complex visual search, where subtle differences are more detectable, we observed that congruent stimuli are more similar to each other than incongruent ones. We went on to show that the perception of congruence depend on local surface and pattern shapes, and breaks down if the local pattern shape does not match the local surface shape.

Does Feature Separability in Single Neurons Manifest at a Population Level?

In this study, we investigated how various features combine in their neural representation, and whether feature integration properties observed at the scale of single neurons hold at a population level. A previous study from our lab showed that in single neurons in IT, objects and its attributes (like size, shape, view angle, etc) combine multiplicatively, while physical parts of objects combine additively. We showed here that representation (and separability) of objects in neural population can predict whether the underlying features get combined additively or multiplicatively at the level of single neurons. We also found through simulations that a very small number of neurons are required to create a system that displays the representation and feature integration properties experimentally observed in neural populations and single neurons.

Does Feature Separability in Single Neurons Manifest in Perception?

In this study, we investigate whether the multiplicative feature integration for objects and attributes, and additive feature integration for object parts can be predicted by measuring perceptual distances between objects. Since perceptual distances in humans are known to be correlated with neural population distances between them in IT (Kriegeskorte et al., 2008; Sripati and Olson, 2009, 2010; Zhivago and Arun, 2014), we tested whether the perceptual distances themselves are sufficient in predicting the underlying mode of feature combination in single neurons. Though we were able to predict multiplicative encoding for the combination of object and attributes, we could not recover additive encoding for the combination of physical object parts.

Does 3D Integration Differ in Possible and Impossible Objects?

In this study, we inspected how 2-dimensional visual cues cause 3-dimensional perception by contrasting cases where 2D cues integrate to create meaningful 3D objects, with cases where the cues form a meaningless 3D object (termed Impossible Objects; example: Penrose Triangles). We predicted that objects are perceived to be 'impossible' if the local 2D cues for 3D shapes are connected together in a manner which does not make sense globally i.e. there is conflict in the integration of local perceived 3D shape to form a meaningful 3D shape. We tried to inspect whether these impossible objects are systematically different from the corresponding possible objects, in its neural and perceptual representation. We also investigated whether by removing the conflicting links between the local cues that are predicted to cause the perception of impossibility, this difference between the representation of possible and impossible objects vanish.