CAREER: Computational Model of Perceived Color and Appearance in Augmented Reality

Abstract

This project aims to build a robust computational model of visual appearance in AR systems that takes into account visual adaptation, cognitive interpretations, and the optical interaction between virtually displayed content and real objects and illumination in the environment. Visual experiments will employ psychophysical scaling, color matching via adjustment, and constant stimuli tasks to understand the influence of the luminance, color, contrast, complexity, and depth of both the AR virtual foreground and the real-world background. A model of color and material appearance in transparent AR environments will build on the working hypothesis that the perceived color is a non-physical addition of foreground and background whose weightings depend on cognitive discounting of the layers. Additional experiments will measure visual luminance and chromatic adaptation in temporally changing AR viewing environments, for instance, asking observers to adjust displayed AR and real-world stimuli to be achromatic in different lighting situations and at different adaptation times. The physical lighting environment, including the location, intensity, and color of light sources and bright objects, will be sensed with cameras and color sensors as input to responsive display algorithms utilizing the developed model of visual adaptation and appearance in AR. The responsive algorithms will ensure robust, predictable color appearance and display. During the project, several AR education applications incorporating the research results will be developed for validation and testing. Classroom assessments by students and faculty will evaluate these applications for validating the research results. AR will be adopted in the research group?s graduate color science courses as a tool for demonstrating adaptation and surround effects and as an environment for practicing psychophysical methods with experiments related to the ongoing research. The researchers will share their computational model and experimental findings via publication and professional organizations, with the goal of meaningful implementation both in AR system design and AR applications. Further, the impact of AR and the fascinating topic of color science will be shared with the community through the University?s open houses and the recruiting of students and faculty.

Journal & Conference Publications

M. J. Murdoch, “Keynote: Transparency and Scission in Augmented Reality,” in Human Vision and Electronic Imaging, Proc. IS&T Electronic Imaging, San Francisco: IS&T: 2025. https://doi.org/10.2352/EI.2025.37.11.HVEI-193 

M. J. Murdoch, “From Polaroid to Augmented Reality: The Enduring Advantages of White Borders,” in Human Vision and Electronic Imaging, Proc. IS&T Electronic Imaging, San Francisco: IS&T: 2025. https://doi.org/10.2352/EI.2025.37.11.HVEI-195 

S. R. Herbeck, M. J. Murdoch, and C. A. Thorstenson, "Adjusting Transparency Toward Optimizing Face Appearance in Optical See-Through Augmented Reality," Journal of Perceptual Imaging,  pp 1 – 12,  2024. – Best Student Paper Award, 32nd Color and Imaging Conference: https://doi.org/10.2352/J.Percept.Imaging.2024.7.000404 

Z. Li & M. J. Murdoch. “Improving Naturalness in Transparent Augmented Reality with Image Gamma and Black Level” in 30th Color & Imaging Conference, Scottsdale, AZ: IS&T, 2022. https://doi.org/10.2352/CIC.2022.30.1.27 

M. J. Murdoch. "Brightness matching in optical see-through augmented reality." Journal of the Optical Society of America A 37. 12 (2020): 1927-1936. https://doi.org/10.1364/JOSAA.398931 

T. Downs & M. J. Murdoch. “Color Layer Scissioning in See-Through Augmented Reality” in 29th Color & Imaging Conference (online): IS&T, 2021. https://doi.org/10.2352/issn.2169-2629.2021.29.60 

L. Zhang & M. J. Murdoch. "Perceived Transparency in Optical See-Through Augmented Reality." in 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct): IEEE, 2021. https://doi.ieeecomputersociety.org/10.1109/ISMAR-Adjunct54149.2021.00033 

L. Zhang, M. J. Murdoch, " How Bright Should It Be: Diffuse White in Optical See-Through Augmented Reality," 6th Frameless XR Symposium, Rochester, NY: Frameless Labs, 2021.

M. J. Murdoch, “Invited Paper: Color and Brightness in Optical See-Through Augmented Reality Display Systems,” in International Display Week (IDW 2020). Japan (online): SID, 2020.

Related Presentations

M. J. Murdoch,  “Invited: Color and Transparency in XR," in 2024 AR/VR Symposium, Rochester, NY, USA: University of Rochester Studio X, 2024.

E. Pei, S. Farnand, & M. J. Murdoch, “Simultaneous Lightness Contrast Effect in Augmented Reality,” in 2024 AR/VR Symposium, Rochester, NY, USA: University of Rochester Studio X, 2024 – 3rd Place Best Poster Award

M. J. Murdoch, “Keynote: Mastering Light: Reproduction, Reality, and Augmentation,” in 31st Color and Imaging Conference, Paris, France: IS&T, 2023.

S. Herbeck, M. J. Murdoch, C. Thorstenson "Demo: Faces in Spaces: Transparency Perception in AR," in 8th Annual Frameless XR Symposium, Rochester. November 17, 2023.

M. J. Murdoch,  "Color Science and Augmented Reality," Voices of XR Speaker Series, Studio X, University of Rochester (Online). March 1, 2023. Link to Video Recording

M. J. Murdoch,  “Color Appearance in Optical See-Through Augmented Reality," Color Impact 2023: Color and Human Experience, Rochester, NY, USA: ISCC, 2023.

M. J. Murdoch, “Keynote: Color in Layers: From Pepper’s Ghost to Augmented Reality,” in AIC 2022 Sensing Colour. Toronto (online): AIC, June 15, 2022.  Link to Video Recording 

M. J. Murdoch,  “Keynote: Color from Real Reality to Extended Reality," 3rd International Symposium for Color Science and Art, Tokyo Polytechnic University (Online). Mar 12, 2022.

M. J. Murdoch,  “Invited: Color Appearance in Augmented Reality Imaging Systems," Society for Imaging Science and Technology (IS&T) Imaging for XR Workshop and Panel Discussion (Online). March 4, 2022.