Director, Computational Material Perception Laboratory
Contact: bxiao at american dot edu
I am a faculty at the Department of Computer Science at American University and the director of Computational Material Perception Laboratory. My research focuses on human and machine perception, especially perception of material properties of objects in complex and dynamic scenes.
Before joining AU, I was a postdoctoral fellow in the perceptual science group under the supervision of Professor Ted Adelson in the Brain and Cognitive Sciences Department at MIT. Between 2009 and 2011, I worked with Professor Alex Wade at the Smith-Kettlewell Eye Research Institute in San Francisco, where I investigated early visual processing of color and luminance signals using frequency-tagging and source-imaged EEG methods. I did my graduate work in human color vision with Professor David Brainard at the University of Pennsylvania. I received my undergraduate degree in physical chemistry from Tsinghua University in Beijing, China. I conducted my bachelor thesis research on thermodynamics of lipid bilayers with Professor Zhiwu Yu.
- 06/18 Our paper "Interaction between static visual cues and force feedback on perception of mass of visual objects" is conditionally accepted by ACM SAP 2018. Congratulations on the first author Wenyan Bi! Preprint is now published on Biorxiv.
- 05/18 Wenyan Bi received a competitive Center of Behavioral Neuroscience Summer Research Award. This award will support her research on haptic and visual perception of material properties this summer. Congratulations, Wenyan!
- 04/18 Bei Xiao received American University Mellon Fellowship for projects in haptics and visual perception in VR.
- 03/18 Wenyan's new paper," Estimating mechanical properties of cloth from videos using dense motion trajectories: human psychophysics and machine learning" is accepted for publication in Journal of Vision. Congratulations, Wenyan!
The lab currently has an opening for a PhD fellowship (Application deadline Dec 1, 2018). The main topic of this PhD studentship is to understand material perception from images and videos using human psychophysics, VR and computer vision tools. The ideal candidate should have a strong technical background and have experience in at least one of the following methods: computational modeling, psychophysics, computer graphics, visual psychophysics, and applied math. Prospective graduate student should contact me directly and are required to apply to the graduate program Behavior, Cognition, & Neuroscience Graduate Program at AU.
In all inquiries, please send me your detailed CV, a brief description of research interests, your GPA, and specify your programming and analytical skills.
I am interested in human perception, multi-sensory perception, computer vision, and computer graphics. I am particularly interested in various aspects of perception of object material properties: how humans perceive materials with multiple senses, how machine algorithm estimate material properties, and how to simulate realistic material appearance using graphics. Current main projects in the lab include:
- Interaction and integration between tactile and visual perception of object properties in real and virtual environments.
- Perceptual inference of material properties of deformable objects in dynamic scenes.
- Machine inference of material properties of objects from images and videos.
- Perception and rendering of translucent appearance.
Fields: Multi-sensory perception, computer inference of material properties and dynamic scenes, perception-driven computer graphics, computer vision.
Techniques: Human psychophysics, computer vision, computer graphics (3D modeling, rendering, animations), machine learning, VR.
Wijntes M., Xiao, B. and Volcic, R. (In Revision). Visual communication of how fabrics feel. PDF.
We study which visual media (images versus videos) better convey haptic properties of fabrics and explore what psychophysical task is appropriate to address this issue.
Bi, WY., Newport, J. Xiao, B. (In Press). Interaction between static visual cues and force-feedback on the perception of mass of virtual objects. ACM Symposium of Applied Perception (SAP'18). PDF.
We use force-feedback device and a game engine to measure the effects of material appearance on the perception of mass of virtual objects. We find static visual appearance influence perceived mass and the effect is opposite from the classical "material weight illusion".
Bi, WY, Jin,P. Nienborg, H and Xiao, B. (2018). Estimating mechanical properties of cloth from videos using dense motion trajectories: human psychophysics and machine learning. Journal of Vision, 18(5), 12-12. PDF, Project Github.
We discover that long-range spatiotemporal information across video frames plays an important role on how humans estimate bending stiffness of cloth from animations. A model based on the features of dense motion trajectories can predict human perceptual scale of bending stiffness of cloth.
We study how humans achieve perceptual constancy when estimating mechanical properties of cloth varying under external forces. We discuss our results in the context of optical flow statistics.
Xiao, B., Bi, W.Y., Jia, X-D, Wei, HH, and Adelson, E. (2016). Can you see what you feel? Color and folding properties affects visual-tactile material discrimination of fabrics. Journal of Vision. PDF. Project Page.
We use tactile perception as ground truth to measure visual material perception. Using fabrics as our stimuli, we measure how observers match what they see (photographs of fabric samples) with what they feel (physical fabric samples).
Heasly, B.S., Cottaris, N.P., Lichtman, D.P., Xiao, B., Brainard, D.H. (2014). RenderToolbox3:MATLAB tools that facilitate physically-based stimulus rendering for vision research. Journal of Vision, Vol. 14, 2. PDF. GitHub.
We describe and release RenderToolbox3, a set of MATLAB utilities, and prescribes a workflow that should be useful to researchers who want to employ computer graphics in the study of perception.
Xiao, B., Walter, B.W., Gkioulekas, I., Zickler, T., Adelson, E, and Bala, K. (2014). Looking against the light: how perception of translucency depends on lighting direction. Journal of Vision. 14(3): 17. PDF
We study the effects of lighting direction on perception of translucency. In particular, we explore the interaction of shape, illumination, and degree of translucency constancy in variation of lighting direction by including in our analysis the variations in translucent appearance that are induced by the shape of the scattering phase function.
Akkayanak, D. Treibitz, T., Xiao,B.,Gurkan, U.A., Allen, J.J.,Demirci, U., and Hanlon, R. (2014). Use of commercial off the shelf (COTS) digital cameras for scientific data acquisition and scene-specific color calibration. Journal of Optical Society of America A (JOSA A), Vol. 31, Issue 2, pp. 312-321. PDF, Online, MATLAB code.
We develop a new computer vision algorithm based on motion statistics extracted from videos that can accurately estimate mechanical properties of real cloth. We find the model prediction is highly correlated with human judgements.
Gkioulekas, I., Xiao,B., Zhao, S., Adelson, E.H., Zickler, T., and Bala, K.(2013). Understanding the Role of Phase Function in Translucent Appearance. ACM Transactions on graphics (TOG). Volume 32, Issue 5. PDF, Supplemental Materials, Media coverage. This work was presented at SIGGRAPH 2013.
We generalize scattering phase function models, demonstrate an expanded translucent appearance space, and discover perceptually-meaningful translucency controls by analyzing thousands of images with computation and psychophysics.
We measure human color constancy of 3D objects in computer rendered complex 3D scenes. More specifically, we find there is an interaction between the test object's three-dimensional shape and spectral changes in the contextual scene.
Xiao, B. and Wade, A.R. (2010). Measurements of Long-range suppression in human opponent S-cone and achromatic luminance channels. Journal of Vision 10(13):10. PDF
We use a combination of neuroimaging data from source-imaged EEG and two different psychophysical measures of surround suppression to study contrast normalization in stimuli containing achromatic luminance and S-cone-isolating contrast.
Xiao, B. and Brainard, D.H. (2006). Color Perception of 3D objects: constancy with respect to variation of surface gloss. Proceedings of ACM Symposium on Applied Perception in Graphics and Visualization (APGV06), 63-68. PDF
Brainard, D.H., Longere, P., Delahunt, P.B., Freeman, W.T., Kraft, J.M., and Xiao, B. (2006). Bayesian model of human color constancy. Journal of Vision, 6, 1277-1281. PDF
We develop a model of human color constancy which includes an explicit link between psychophysical data and illuminant estimates obtained via a Bayesian algorithm.
Selected Conference Presentations
- Xiao, B. Shuang, Z., Gkioulekas, I, Bi, WY, and Bala, K. (2018). Does geometric sharpness affect perception of translucent material perception? VSS 2018. St. Pete's Beach. Poster.
- Xiao, B. (2017). Seeing materials from movements: motion cues in perception of cloth in dynamic scenes. Symposium on "Beyond translation: Image deformation and dynamics in material and shape perception”. ECVP 2017. Berlin. Talk.
- Wijntjes, M, and Xiao, B (2016). Visual communications of haptic material properties. VSS 2016. St. Pete's Beach, Poster.
- Bermudez, L. and Xiao, B. (2016). Estimating material properties of cloth from dynamic silhouettes. VSS 2016, St.Pete's Beach, Florida. Poster.
- Xiao, B., and Kistler, W. (2015). Perceptual dimensions of material properties of fabrics in dynamic scenes. VSS 2015, St. Pete's Beach, Florida. Talk.
- Xiao, B., and Kistler, W. (2014). Perceptual dimensions of material properties of moving fabrics. European Conference of Visual Perception (ECVP), Belgrade, Serbia. Poster.
- Xiao, B., Walter, B., Gkioulekas, I., Adelson,E., Zickler, T. and Bala, K. (2014). Looking against the light: how perception of translucency depends on lighting direction and phase function. Vision Science Society Annual Meeting, St.Pete's beach, FL. Abstract, Talk Slides.
- Xiao, B., Adelson, E. (2013). Multi-sensory understanding of material properties. Prism2, The Science of Light and Shade. Bordeaux, France.
- Xiao, B., Jia, X.D., and Adelson, E. (2013). Can you see what you feel? Visual and Tactile perception of fabrics. Vision Science Annual Meeting, Naples, FL. Poster.
- Xiao, B., Gkioulekas, I., Dunn, A, Zhao, S. Adelson,E., Zickler, T. and Bala, K. (2012). Effects of shape and color on the perception of translucency. Vision Science Society Annual Meeting, Naples, FL. Talk Slides.
- Xiao, B., Sharan. R, Rosenholtz. R. and Adelson,E. (2011). Speed of material vs. object recognition depends upon viewing conditions. OSA Fall Vision Meeting, Seattle, WA. Abstract, Slides.
- Xiao, B., Wade, A.R. (2010). Interactions between S-cone and luminance signals in surround suppression. Vision Science Society Annual Meeting, Naples, FL. Abstract.
- Xiao, B., Wade, A.R. (2009). Surround suppression between S-cone and luminance channels measured with psychophysics and source-localized EEG. OSA Fall Vision Meeting, Seattle, WA. Abstract.
- Xiao, B., Brainard, D.H. (2009). Surface material properties and color constancy of 3D objects. Vision Science Society Annual Meeting, Naples, FL. Abstract.
- Gaurav Kumar, BS, Computer Science, Indian Institute of Technology at Kanpur. Summer research student. Deep learning on cloth perception.
- Wenyan, Bi. BS, Psychology, Beijing Normal University. Ph.D student. Cloth Simulation, Perception and Machine Inference.
Former Group Members
- William Kistler, MA, American University. Research Assistant & Scientific Programmer. Now at NIH.
- Simin Li. High School Student Researcher, Montgomery Blair High School. Machine learning. Now undergraduate student at University of Maryland.
- Alex Perepechko, Senior, CS & Audio Technology major, American University. Now at Amazon.
- Laura Uribe, Senior, CS major, American University. Now at Adobe.
- Yuuki Kpa, Sophomore, CS & Neuroscience double major, American University.
- Luis Bermudez, BS, Carnegie Mellon University. Research Assistant & Scientific Programmer. Now graduate student at Clemson University.
- Abhinav Swaminathan, Freshman, CS major, American University. Computer graphics.
- Eric Whipp, Senior, Computer Science. Haptic device and VR.
- George Gu, Senior, Graphic Design. VR and Perception.
Main current and past collaborators:
- Maarten Wijntjes, TU Delft
- Ted Adelson, MIT
- Ruth Rosentholtz, MIT
- Kavita Bala, Cornell
- Todd Zickler, Harvard
- Bruce Walter, Cornell
- Ioannis Gkioulekas, CMU
- Katie Bouman, MIT
- Lavanya Sharan, Netflix
- Derya Akkayanak, Princeton
- Shuang Zhao, UC Irvine
- Xiaodan Jia, Apple
- Alex Wade, University of York, UK
- David Brainard, University of Pennsylvania
CSC 589, Introduction to Computer Vision. Introduction course in computer vision. The course will survey both low-level image processing methods such as filters, edge detection and color imaging, and also mid-to high-level tasks such as segmentation, clustering, and objects and scene understanding. Course Github Page.
CSC 280, Introduction to computer science. This courses focused on introduction to programing and problem solving using Python.
Music plays an important role in my life. Trained mostly in classical music, I play the piano and the harpsichord. I am always interested in playing chamber music with other musicians, especially vocalists and string musicians.
DC has vibrant concert series. I especially like chamber music in small venues. My favorite contemporary classical/world music series in town are:
- Phillips Collection ( high quality chamber music, mostly classical, not free)
- National Gallery of Art West Wing (free)
- Washington Performing Art Society (Most classical music, not free)
- Freer/Sackler Gallery Concerts (world music, free)
- Smithsonian chamber music society
- Library of Congress concerts (high quality old and contemporary music, free)
- 930 Club .(Popular music, Contemporary music)
- Six and I concert series (american contemporary music and jazz, not free)