I am a faculty at the Department of Computer Science at American University in DC and the head of the Computational Material Perception Laboratory My research focuses on human and computer vision and how to apply principles of human cognition to improve robustness in artificial intelligence and whether and how machine algorithms can inform and assist humans. Specifically, I study perception and reasoning of material properties of objects in complex and dynamic scenes. I use a combination of human psychophysics, crowd-sourcing, image synthesis and analysis, machine learning, computer vision and VR techniques. Read more of on my bio and project page.
06/07/2021 I am one of CVPR2021 outstanding reviewers. Nice to be recognized.
05/22/2021 Xiao lab is attending V-VSS. Check out our poster on individual difference in material perception.
05/12/2021 Xiao lab welcomes CS major, Jesse Schwarz, joining the lab. Jesse will work on visual perception in VR.
05/2/2021 Xiao lab welcomes Shengkai Liao, Master student in CS, joining the lab. Shengkai will do his capstone project on machine learning to predict visual preferences.
02/22/2021. Our abstract " Individual Differences in Classification of Translucent Materials Using Photos of Real-world Objects has been accepted as a Poster presentation at the 2021 V-VSS Annual Meeting. Congratulations to first author Chenxi Liao.
09/16/2020. Our paper "Towards More Accurate Uncertainty Estimation In Text Classification" is accepted by EMNLP 2020! Congratulations to first author Jianfeng He. The paper will be presented at EMNLP 2020 conference.
08/31/2020 Our paper "Exploring the determinants of color perception using #thedress and its variants: the role of spatio-chromatic context, chromatic illumination, and material-light interaction" is accepted to publish in journal Perception.
7/26/2020 A new review chapter on Color Constancy written by Bei is published in Springer's Encyclopedia on Color Science and Technology.
7/15/2020 Our paper "Effect of Geometric Sharpness on Translucent Material Perception" is published in Journal of Vision. Thanks for all the co-authors.
06/23/2020 Xiao lab welcomes Chenxi Liao joining the lab. Chenxi will start his PhD studies in Brain, Cognition, and Neuroscience program at AU this Fall. Before joining Xiao lab, Chenxi was a master student in Industrial Engineering at Columbia University.
05/11/2020 Former lab member Gaurav Kumar has accepted an offer from UCSD for graduate study in CS. Congratulations Gaurav!
1/01/2020 Wenyan Bi accepted a postdoc position at Department of Psychology at Yale University. Congratulations!
12/20/2019 CVSS 2019 Lecture Videos online.
The lab currently has an opening for a PhD fellowship (Application deadline Dec 1, 2021). The main topic of this PhD studentship is to understand multi-sensory material perception from images and videos using large-scale human psychophysics, deep learning, and VR/AR methods. The ideal candidate should have a strong technical background and have experience in at least one or two of the following methods: machine learning, applied math or statistics , computational modeling, image processing, psychophysics, computer graphics. Solid programming skills with Python and MATLAB is plus . Prospective graduate student should contact me directly and are required to apply to the graduate program Behavior, Cognition, & Neuroscience Graduate Program at AU. International students must pass the TOFEL exam.
The lab also has openings for master and undergraduate independent studies/capstone projects. The projects can be one of the following: 1) material editing with computer grpahics, 2) large-scale online behavioral experiments data collection and analysis 3) develop VR/AR game and experiments in Unity 3D, tactile device, and other physics engine 4) Computer vision algorithms of material perception using deep learning. 5) Unsupervised learning with small image dataset. Interested students please send a CV and transcript and schedule a meeting with me.
CSC 476 Introduction to Computer Vision, Spring, 2021
CSC 435 Web Programming, Spring, 2021.
CSC148 Introduction to Programming in Python, Fall, 2020
CSC 469/696 Deep learning in computer vision, Fall, 2019
I am interested in human perception, VR, multi-sensory perception, computer vision, and graphics. Specifically, I am interested in various aspects of perception of object material properties (especially complex materials such as cloth, skin, liquid, wax and stone): how humans perceive materials with multiple senses, how machine algorithm estimate material properties, and how to simulate realistic material appearance using graphics.
Current main projects in the lab include:
Automatic image and video generation and editing from GANs with scene graphs, and texts, and multimedia inputs.
Interaction and integration between tactile and visual perception of object properties in VR.
Machine inference of scene, object, and material properties of objects from images and videos.
Perception and rendering of translucent appearance.
Fields: Multi-sensory perception, computer inference of material properties and dynamic scenes, perception-driven computer graphics, computer vision.
Techniques: Human psychophysics, computer vision, computer graphics (3D modeling, rendering, animations), machine learning, haptics, VR.
Read more on project page.
He,. J. Zhang, X., Shuo L. Wang, S, Huang, Q., Lu, C-T, Xiao, B. (2020) Semantic Editing On Segmentation Map Via Multi-Expansion Loss. arxiv preprint. 2010.08128. PDF
We automatically edit segmentation label map conditioned on semantic inputs by proposing MExGAN, which uses a novel Multi-Expansion (MEx) loss implemented by adversarial losses on a series of expanded areas to improve alignment in the boundary areas between manipulated object and the surrounding scene. This method can also be applied in natural image inpainting.
Aston S., Denisowa, K., Hurrlbert, A., Olkkonen,M., Pearce, B., Rudd, M, Werner, A., and Xiao, B. (2020). Exploring the Determinants of Color Perception Using #Thedress and Its Variants: The Role of Spatio-Chromatic Context, Chromatic Illumination, and Material– Light Interaction. Perception. PDF.
We show how spatial and temporal manipulations of light spectra affect people’s perceptions of material colors and illustrated the variability in individual color perception.
He, J., Zhang, X, Lei, S. Chen, Z, Chen, F, Alhamadani, A. Xiao. B and Lu, CT. (2020) Towards More Accurate Uncertainty Estimation In Text Classification. EMNLP 2020. PDF. Code.
We propose a model, MSD (Mix-up self-ensembling, and distinctiveness score), to improve accuracy of estimating uncertainty of text classification of DNN by reducing the effect of overconfidence of winning score and considering the impact of different categories of uncertainty simultaneously. Our methods can be flexibly applied to several DNNs.
Using simulated relief objects of translucent materials with varying shapes and optical properties under different illuminations, we find that the level of geometric sharpness significantly affects perceived translucency by the observers.
We isolated the dynamic deformation using dot stimuli and found directly manipulating the pattern of dynamic deformation using the method we proposed can alter the perceived stiffness.
Wijntes M., Xiao, B. and Volcic, R. (2019). Visual communication of how fabrics feel. Journal of Vision, Feb, 2019. PDF.
We study which visual media (images versus videos) better convey haptic properties of fabrics and explore what psychophysical task is appropriate to address this issue.
Bi, WY., Newport, J. Xiao, B. (2018). Interaction between static visual cues and force-feedback on the perception of mass of virtual objects. ACM Symposium of Applied Perception (SAP'18). PDF. Project Page.
We use force-feedback device and a game engine to measure the effects of material appearance on the perception of mass of virtual objects. We find static visual appearance influence perceived mass and the effect is opposite from the classical "material weight illusion".
Bi, WY, Jin,P. Nienborg, H and Xiao, B. (2018). Estimating mechanical properties of cloth from videos using dense motion trajectories: human psychophysics and machine learning. Journal of Vision, 18(5), 12-12. PDF. Project Page.
We discover that long-range spatiotemporal information across video frames plays an important role on how humans estimate bending stiffness of cloth from animations. A model based on the features of dense motion trajectories can predict human perceptual scale of bending stiffness of cloth.
We study how humans achieve perceptual constancy when estimating mechanical properties of cloth varying under external forces. We discuss our results in the context of optical flow statistics.
Xiao, B., Bi, W.Y., Jia, X-D, Wei, HH, and Adelson, E. (2016). Can you see what you feel? Color and folding properties affects visual-tactile material discrimination of fabrics. Journal of Vision. PDF. Project Page.
We use tactile perception as ground truth to measure visual material perception. Using fabrics as our stimuli, we measure how observers match what they see (photographs of fabric samples) with what they feel (physical fabric samples).
Heasly, B.S., Cottaris, N.P., Lichtman, D.P., Xiao, B., Brainard, D.H. (2014). RenderToolbox3:MATLAB tools that facilitate physically-based stimulus rendering for vision research. Journal of Vision, Vol. 14, 2. PDF. GitHub.
We describe and release RenderToolbox3, a set of MATLAB utilities, and prescribes a workflow that should be useful to researchers who want to employ computer graphics in the study of perception.
Xiao, B., Walter, B.W., Gkioulekas, I., Zickler, T., Adelson, E, and Bala, K. (2014). Looking against the light: how perception of translucency depends on lighting direction. Journal of Vision. 14(3): 17. PDF
We study the effects of lighting direction on perception of translucency. In particular, we explore the interaction of shape, illumination, and degree of translucency constancy in variation of lighting direction by including in our analysis the variations in translucent appearance that are induced by the shape of the scattering phase function.
Akkayanak, D. Treibitz, T., Xiao,B.,Gurkan, U.A., Allen, J.J.,Demirci, U., and Hanlon, R. (2014). Use of commercial off the shelf (COTS) digital cameras for scientific data acquisition and scene-specific color calibration. Journal of Optical Society of America A (JOSA A), Vol. 31, Issue 2, pp. 312-321. PDF, Online, MATLAB code.
We develop a new computer vision algorithm based on motion statistics extracted from videos that can accurately estimate mechanical properties of real cloth. We find the model prediction is highly correlated with human judgements.
Gkioulekas, I., Xiao,B., Zhao, S., Adelson, E.H., Zickler, T., and Bala, K.(2013). Understanding the Role of Phase Function in Translucent Appearance. ACM Transactions on graphics (TOG). Volume 32, Issue 5. PDF, Supplemental Materials, Media coverage. This work was presented at SIGGRAPH 2013.
We generalize scattering phase function models, demonstrate an expanded translucent appearance space, and discover perceptually-meaningful translucency controls by analyzing thousands of images with computation and psychophysics.
We measure human color constancy of 3D objects in computer rendered complex 3D scenes. More specifically, we find there is an interaction between the test object's three-dimensional shape and spectral changes in the contextual scene.
Xiao, B. and Wade, A.R. (2010). Measurements of Long-range suppression in human opponent S-cone and achromatic luminance channels. Journal of Vision 10(13):10. PDF
We use a combination of neuroimaging data from source-imaged EEG and two different psychophysical measures of surround suppression to study contrast normalization in stimuli containing achromatic luminance and S-cone-isolating contrast.
Xiao, B. and Brainard, D.H. (2006). Color Perception of 3D objects: constancy with respect to variation of surface gloss. Proceedings of ACM Symposium on Applied Perception in Graphics and Visualization (APGV06), 63-68. PDF
Brainard, D.H., Longere, P., Delahunt, P.B., Freeman, W.T., Kraft, J.M., and Xiao, B. (2006). Bayesian model of human color constancy. Journal of Vision, 6, 1277-1281. PDF
We develop a model of human color constancy which includes an explicit link between psychophysical data and illuminant estimates obtained via a Bayesian algorithm.
Xiao, B. (2015). The Science Behind #the Dress. American University News Column. Full Article.
Peer-reviewed Conference Presentations
Bi, W., Kumar, G., Nienborg, H., and Xiao, B. (2019). Understanding Information Processing Mechanisms for Estimating Material Properties of Cloth in Deep Neural Networks. VSS 2019, St. Pete Beach, FL.
Xiao, B. Shuang, Z., Gkioulekas, I, Bi, WY, and Bala, K. (2018). Does geometric sharpness affect perception of translucent material perception? VSS 2018. St. Pete's Beach. Poster.
Xiao, B. (2017). Seeing materials from movements: motion cues in perception of cloth in dynamic scenes. Symposium on "Beyond translation: Image deformation and dynamics in material and shape perception”. ECVP 2017. Berlin. Talk.
Wijntjes, M, and Xiao, B (2016). Visual communications of haptic material properties. VSS 2016. St. Pete's Beach, Poster.
Bermudez, L. and Xiao, B. (2016). Estimating material properties of cloth from dynamic silhouettes. VSS 2016, St.Pete's Beach, Florida. Poster.
Xiao, B., and Kistler, W. (2015). Perceptual dimensions of material properties of fabrics in dynamic scenes. VSS 2015, St. Pete's Beach, Florida. Talk.
Xiao, B., and Kistler, W. (2014). Perceptual dimensions of material properties of moving fabrics. European Conference of Visual Perception (ECVP), Belgrade, Serbia. Poster.
Xiao, B., Walter, B., Gkioulekas, I., Adelson,E., Zickler, T. and Bala, K. (2014). Looking against the light: how perception of translucency depends on lighting direction and phase function. Vision Science Society Annual Meeting, St.Pete's beach, FL. Abstract, Talk Slides.
Xiao, B., Adelson, E. (2013). Multi-sensory understanding of material properties. Prism2, The Science of Light and Shade. Bordeaux, France.
Xiao, B., Jia, X.D., and Adelson, E. (2013). Can you see what you feel? Visual and Tactile perception of fabrics. Vision Science Annual Meeting, Naples, FL. Poster.
Xiao, B., Gkioulekas, I., Dunn, A, Zhao, S. Adelson,E., Zickler, T. and Bala, K. (2012). Effects of shape and color on the perception of translucency. Vision Science Society Annual Meeting, Naples, FL. Talk Slides.
Xiao, B., Wade, A.R. (2010). Interactions between S-cone and luminance signals in surround suppression. Vision Science Society Annual Meeting, Naples, FL. Abstract.
Xiao, B., Wade, A.R. (2009). Surround suppression between S-cone and luminance channels measured with psychophysics and source-localized EEG. OSA Fall Vision Meeting, Seattle, WA. Abstract.
Xiao, B., Brainard, D.H. (2009). Surface material properties and color constancy of 3D objects. Vision Science Society Annual Meeting, Naples, FL. Abstract.
I had great fun playing a piano piece as part of the sound-track of my colleague Krzysztof Pietroszek's melancholic short movie "Vera" and its volumetric version. Here is a complete recording of Tchaikovsky's The seasons - June (barcarolle).
Trained mostly in classical music, I play the piano and the harpsichord. I am always looking for opportunities to play chamber music with other musicians, especially other pianists (four hands), vocalists and string musicians.
I listen to wide varieties of music but I am extremely interested in new music composed by living composers, baroque, Jazz and world music.
DC has vibrant concert series. I especially like chamber music in small venues. My favorite contemporary classical/world music venues in town are:
Lisner auditorium in GWU ( world music, contemporary ensembles).
Phillips Collection ( high quality chamber music, mostly classical, not free)
Washington Performing Art Society (Most classical music, not free)
Freer/Sackler Gallery Concerts (world music, free)
Library of Congress concerts (high quality old and contemporary music, free)
930 Club (Popular music, Contemporary music)
Six and I concert series (american contemporary music and jazz, not free)
I am always up for a board game break in the lab and at home. Some favorites: