EXTRA EYE
Egocentric and eXocenTRic views for An object-level human bEhavior analYsis and undErstanding through tracking in complex space
Project Goal
The EXTRA EYE project aims to decode human interactions in diverse environments. We plan to combine first-person and third-person views to provide a holistic understanding of human's activities and behavior. EXTRA EYE will rely on three modules: a First-Person View (FPV) module which aims to understand take/release user-object interactions from egocentric vision, a Third-Person View (TPV) module which processes exocentric signals captured from fixed cameras, accompanied with an aggregation unit that also integrates information from the other components to perform context-specific behavioral analysis, and an Object Tracking (OT) module which supports the former two components by tracking objects of interest from both the third- and first-person points of view. The system is will be useful in many scenarios in which human behavior is largely determined by the way in which people interact with the objects available in the scene. As a clear application context for the development and test of EXTRA EYE, we will consider retail stores as goal-driven, interaction-rich scenarios. The technologies emerging from this project aim to offer a new perspective on human behavior understanding from Computer Vision, informing the construction of systems able to user experiences in a variety of environments.
Technologies
First Person Vision
The first-person vision module will be able to understand take/release human-object interactions from visual signals collected by the unique perspective of the user.
Third Person Vision
The Third-Person Vision Module provides a broader context by integrating data from fixed cameras. It reveals spatial relationships and interactions across the entire environment.
Object Tracking
The Object Tracking Module supports the system by tracking objects of interest from both viewpoints, ensuring comprehensive and precise behavioral analysis.
Behavioral Analysis
By modeling human-object interactions from first- and third-person vision, EXTRA EYE will provide a holistic understanding of human behavior.
Research Units
University of Macerata
University of Catania
University of Udine
Project Grant
The project is funded by the Italian Ministry of University and Research (MUR) through the PRIN 2022 measure.