We investigate the mechanisms of the human visual system in our lab. You may wonder why we are investigating it because we belong to Department of Information and Communications Engineering. However, our brains and neural systems are exceptionally efficient information processors. Even though our brain's computational resources cannot compete with that of computers, human eyes play a vital role in assessing product quality, material impressions, and defects. How do our eyes and brains manage to perform so well? On the other hand, the visual information presented to us must match our perceptual abilities. What types of visual stimuli, for example, do humans find comfortable?

To address these questions, Nagai Laboratory aims to elucidate the mechanisms of "image information processing" in the visual system. In particular, we are interested in mechanisms underlying material and color perception, which are directly related to object recognition and affections driven by such perception.

* Our primary experimental method is the sophisticated psychophysical experiment technique, which explores the relationship between controlled physical stimuli (images) and the observer's visual perception to experimentally deduce the underlying information processing in the brain and neural system. We also utilize various human measurement techniques, including electroencephalography and eye movement tracking. Conducting these experiments demands skills in statistical analysis, programming, image processing, and logical thinking, all of which can be adequately learned after joining our lab.

1. Surface quality, material, and affections

We human beings can easily perceive the material features of objects and feel various affections on them. Recently, the brain mechanisms underlying the material perception and affections from visual information have been thought to be useful for perceptive image processing.

Process for material perception and affections in the visual system

Various material impressions and affections arise from visual information at a glance. How does our visual system capture these material impressions and affections?

We examine 1. image features contributing to material impressions and affections and 2. the process of such image features in the brain by precisely measuring the dynamic properties of such impressions and their relations to image features.

What image features are used to distinguish various material classes?　 (Nagai et al., 2015, Kiyokawa et al., 2018)

Mechanisms of glossiness and translucency perception

Retinal images are created by complex combinations of light fields, object shapes, and their reflection properties related to perceptual gloss and translucency. However, human beings can effortlessly perceive their material features.

Therefore, we investigate mechanisms to perceive the glossiness and translucency with psychophysics, imaging techniques, and statistical analysis. Image features contributing to glossiness and transparency, if they are identified, should help us develop imaging techniques to manipulate perceptual material impressions freely.

How does the brain identify specular highlights to perceive glossiness?（Nohira et al., 2023）Are there any visual cues for glossiness other than specular highlights? (Kiyokawa et al., 2021)

2. Color perception

Color is a sensation that reflects information about light wavelengths entering the eye. Based on color information, we can easily infer what material objects are made of. The mechanism for color perception (i.e., color vision) is a crucial basis of current color engineering, and understanding color vision is a fundamental issue for various visual media.

Color representations in the brain

Color representation in the lower-level visual system (e.g., cones and ganglion cells) has been clarified. However, there are a lot of color perception phenomena that cannot be explained by these low-level color representations alone.

We aim to elucidate such unknown color representations in the brain color based on psychophysical experiments, EEG measurements, and the influence of experience from multiple perspectives and to revise the principles of color engineering. From these results, we'd like to contribute to various color-related problems, such as color barrier-free and quantification of color unevenness.

The way you perceive and name colors can significantly change after you repeatedly discriminate specific colors, like cherry farmers (Horiuchi et al., 2024).

Interaction of color and material perception

In conventional color vision research, experiments using uniformly colored stimuli have been used typically. However, in many cases, the knowledge from such conventional color vision research cannot explain color perception on object images with a glossy appearance.

We aim to elucidate the interaction between colors and surface qualities in perception by strictly controlling the color information of images generated by computer graphics.

Specular highlights modulate the color appearance of objects (Nagai et al., 2017). Conversely, adding color can increase perceived glossiness (Koizumi et al., 2023).

3. Development of methods for affection measurement 

Measuring human perception and affection is often very time-consuming. For instance, repeating mouse-clicking dozens of times regarding the observer's perception is required when measuring the properties of visual perception in visual psychophysics. If such properties could be measured more quickly with higher accuracy, such measurements should effectively contribute to understanding the process of perception and affection.

Development of highly efficient/accurate psychophysical methods

In our laboratory, we measure human perception in psychophysical experiments. However, such experiments often require a significant investment of time.

Thus, we develop highly efficient/accurate psychophysical methods based on statistical techniques such as the reverse correlation method, maximum likelihood scaling, and adaptive staircase procedure. Also, we apply such newly developed approaches to elucidate the mechanisms of our visual system.

(Left) Identify image regions related to translucency perception from psychophysical experiments (Nagai et al., 2014). (Right) Contrast sensitivity properties can be measured quickly and accurately (Hayasaka et al., 2022).

Uncovering unconscious perceptual strategies

For instance, we unconsciously and effortlessly read emotions and health states from other people's faces. However, do you know what facial image features you rely on for such recognition? We develop psychophysical experimental methods to uncover such unconscious visual strategies.

From psychophysical experiments on noisy face images like the left image, facial components related to facial impression can be visualized as in the right. This figure shows an example of the result of facial skin lightness perception (Nagai et al., 2022).