My Vision Projects

Human Vision

An Intuitive Model of Perceptual Grouping for HCI Design

Ruth Rosenholtz, Nathaniel Twarog, Nadja Schinkel-Bielefeld, Martin Wattenberg

Understanding and exploiting the abilities of the human visual system is an important part of the design of usable user interfaces and information visualizations. Good design enables quick, easy and veridical perception of key components of that design. An important facet of human vision is its ability to seemingly effortlessly perform “perceptual organization”; it transforms individual feature estimates into perception of coherent regions, structures, and objects. We perceive regions grouped by proximity and feature similarity, grouping of curves by good continuation, and grouping of regions of coherent texture. In this paper, we discuss a simple model for a broad range of perceptual grouping phenomena. It takes as input an arbitrary image, and returns a structure describing the predicted visual organization of the image. We demonstrate that this model can capture aspects of traditional design rules, and predicts visual percepts in classic perceptual grouping displays.

A: Original text and different hierarchical groupings predicted by our perceptual organization model. Different groups are indicated by different colors. B: Variations on a graph and the grouping found by the model. As Tufte (1983) points out, it is much easier to see the peaks in the upper plot than in the lower one where the dashed lines have been removed. This is reflected in the model predictions.

Contour Integration Models Predicting Human Behavior

in conjunction with Udo Ernst, Simon Neitzel, Sunita Mandon, Andreas Kreiter and Klaus Pawelzik

Contour integration is believed to be a fundamental process in object recognition and image segmentation. However, its neuronal mechanisms are still not well understood. Psychophysical experiments showed that humans are remarkably efficient in integrating contours even if these are jittered or partially occluded. Therefore the brain requires a reliable algorithm for extracting contours from stimuli. Several recent publications demonstrated that the brain often uses optimal strategies to integrate sensory information. Here I want to tackle the question which contour integration model describes human contour integration best.

Mathematically, contour ensembles can be characterized by a conditional link probability density between oriented edge elements, termed an association field. This association field can be used to generate contours or vice versa to extract a contour from a stimulus. While in most neuronal network models all inputs to a neuron are summed up, in such a probabilistically motivated neural network for contour integration the afferent input due to the visual stimuli and the lateral input from horizontal network interactions are multiplied.

Long-range horizontal interactions in primary visual cortex link orientation columns with similar preferred orientations and are often assumed to be the neuronal substrate for the association field. Experimental findings in monkeys suggest isotropic long-range horizontal connections, spreading symmetrically into all directions from an orientation column. In contrast, probabilistic models require unidirectional lateral interactions, linking orientation columns in only one direction, in order to get optimal contour detection performance.

Using stimuli generated from given association fields, our numerical simulations show that contour detection performance for both,probabilistic-multiplicative as well as additive models reaches human performance. Hence detection performance alone is insufficient to rule out either model class. However, psychophysical experiments with humans reveal that contour detection errors are not made randomly, but are highly correlated among different subjects. Thus a model describing contour integration in the brain should not only explain human contour detection performance, but should also reproduce these systematic errors made by humans. Comparison between misdetections of humans and mispredictions of the models on a trial-by-trial basis was used to evaluate different model dynamics and association fields. This suggests that unidirectional multiplicatively coupled horizontal interactions are required in order to explain human behavior. Furthermore, cortical magnification factors have to be taken into account and a fixed association field geometry for all stimuli is preferable instead of using for each contour the association field employed for the generation of this contour.

Optimal Contour Integration: When Additive Algorithms Fail

Nadja Schinkel-Bielefeld, Klaus R. Pawelzik, Udo A. Ernst

Contour integration is a fundamental computation during image segmentation. Psychophysical evidence shows that contour integration is performed with high precision in widely differing situations. Therefore, the brain requires a reliable algorithm for extracting contours from stimuli. While according to statistics, contour integration is optimal when using a multiplicative algorithm, realistic neural networks employ additive operations. Here we discuss potential drawbacks of additive models. In particular, additive models require a subtle balance of lateral and afferent input for reliable contour detection. Furthermore, they erroneously detect an element belonging to several jittered contours instead of a perfectly aligned and thus more salient contour.