Research‎ > ‎

Eye Contact

Abstract:
We propose a system for detecting bids for eye contact directed from a child to an adult who is wearing a point-of-view camera. The camera captures an egocentric view of the child-adult interaction from the adult's perspective. We detect and analyze the child's face in the egocentric video in order to automatically identify moments in which the child is trying to make eye contact with the adult. We present a learning-based method that couples a pose-dependent appearance model with a temporal Conditional Random Field (CRF). We present encouraging findings from an experimental evaluation using a newly collected dataset of 12 children. Our method outperforms state-of-the-art approaches and enables measuring gaze behavior in naturalistic social interactions.

Demo:


Media:
http://www.wsbtv.com/news/news/local/tech-uses-high-tech-glasses-autism-research/nSdWB/ (WSBTV)

Dataset:
coming soon... [more info]

Related Publication(s):
  1. Zhefan Ye, Yin Li, Yun Liu, Chanel Bridges, Agata Rozga, James M. Rehg. Detecting Bids for Eye Contact Using a Wearable Camera, IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2015. (Best Student Paper Award) [ paper ] [ project page ] [ bibtex ]
  2. James M. Rehg et al. Decoding Children's Social Behavior, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013 [ paper ] [ dataset ]
  3. Zhefan Ye, Yin Li, Alireza Fathi, Yi Han, Agata Rozga, Gregory D. Abowd, James M. Rehg. Detecting Eye Contact using Wearable Eye-Tracking Glasses2nd International Workshop on Pervasive Eye Tracking and Mobile Eye-Based Interaction (PETMEI) in conjunction with (UbiComp) 2012, 699-704 [ paper ] [ poster ] [ award ]