Embedded Files
Subtopics: Probabilistic Activity Recognition Social Behavior Analysis Child-Robot Interaction
Probabilistic Human Daily Activity Recognition towards Robot-assisted Living
Objectives:
- Develop a probabilistic framework for Human-Robot Interaction useful for robot-assisted living;
- Develop an autonomous system with machine learning techniques in order to exploit different sensory information;
- Extract human actions through body motion patterns;
- Decision making to offer proactive initiative to prompt and support a human in an indoor environment.
Achievements: A novel framework for human Activity Recognition (AR) is addressed, where a probabilistic ensemble of classifiers called Dynamic Mixture Model (DBMM) was proposed. The DBMM relies on the computed confidence belief from multiple base classifiers, combining these likelihoods into a single form by assigning weights from an uncertainty measure to counterbalance the posterior probability. Discriminative spatio-temporal features are extracted from human skeleton given RGB-D data. Assessment on well-known human daily activity datasets and also using a mobile robot for assisted living were successfully carried out with overall accuracy greater than 90%. A real time application for AR including risk situations was implemented in ROS (Robot Operating System) useful for robot-assisted living.
An overview of the DBMM approach for human daily activity recognition is presented below.
Dynamic Bayesian Mixture Model Approach. (click on the Figure to see the figure with higher resolution)
Future Work
- Extend the activity recognition framework to recognize more than one activity happening in parallel in the robot's field of view (two or more subjects);
- Extend the activity recognition framework for social behavior classification.
Videos
Publications
- D. R. Faria, C. Premebida, U. Nunes. "Dynamic Bayesian Mixture Model: Probabilistic Classification for Human Daily Activity Recognition", submitted to a journal (Under Review).
- Diego R. Faria, Mario Vieira, Cristiano Premebida, Urbano Nunes, "Probabilistic Human Daily Activity Recognition towards Robot-assisted Living". Proceedings of IEEE RO-MAN'15: IEEE International Symposium on Robot and Human Interactive Communication. Kobe, Japan, August, 2015.
- Mario Vieira, Diego R. Faria, Urbano Nunes, "Real-time Application for Monitoring Human Daily Activities and Risk Situations in Robot-assisted Living". Proceedings of Robot'15: 2nd Iberian Robotics Conference. Lisbon, Portugal 2015.
- Diego R. Faria,Cristiano Premebida, Urbano Nunes - "A Probabilistic Approach for Human Everyday Activities Recognition using Body Motion from RGB-D Images". IEEE RO-MAN'14: IEEE International Symposium on Robot and Human Interactive Communication. Edinburgh-Scotland, UK, August 25-29th, 2014. * Finalist for Kazuo Tanie Award (focusing on practical application or one that can be applied to real products).
MSc Thesis
- M. Vieira. ""Recognition of Daily Activities and Risk Situations towards Robot-assisted Living". M.Sc. Thesis. Universidade de Coimbra, Portugal, 2015. Supervised by Dr Diego R. Faria and Professor Urbano Nunes.
Probabilistic Social Behavior Analysis by Exploring Motion-based Features
Objectives: <back to top>
- Develop a hierarchical probabilistic framework for Social Behavior Estimation;
- Extract low level features in both, spatial and frequency domain;
- Develop descriptors using Laban Movement Analysis (LMA) components to parameterize human motion.
Achievements:
An hierarchical model for body motion analysis aiming at social behavior estimation was proposed. The multi-level framework, which estimates various levels of human activities (individual and pair) consists of extraction of low level features in spatial and frequency domain, body motion descriptor using LMA space and inference of social behaviors in a conversation scenario using Bayesian networks. The analysed human activities are categorized in a set of levels, where low-level features are extracted, later on the features set feeds the LMA middle level analysis as a body motion descriptor used as input for different inference levels, allowing individual human movement classification; human-object and human-human interaction inference for behavior analysis. The overall accuracy in classification is above 90%.
Hierarchical framework for Social Behavior Estimation
Publications:
- K Khoshhal Roudposhti, U. Nunes and J. Dias, "Probabilistic Social Behavior Analysis by Exploring Body Motion-based Patterns," in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015.
Child-Robot Interaction
General Objective: Interdisciplinary work involving experts from robotics and psychology applied to the context of child-robot interaction towards assisting the facilitation of adaptive health-related coping and improved quality of life outcomes in pediatric settings (robotherapy).
- Scientific/Technological goals:
- Develop a framework for child emotional state estimation: combining facial expressions and body motion;
- Develop an autonomous system with machine learning techniques in order to exploit different sensory information;
- Develop a module for robot reactions accordingly to the current child's emotional state;
- Develop a Learning by Imitation framework based on Kinesthetic movements.
Achievements: We have started to program the humanoid robot NAO (Aldebaran robotics) to endow the robot with enough skills to interact with children. Initially, we have prepared a script to control the robot action and reaction during the child-robot interaction online by tele-operation, where an expert in robotics selects the appropriate reactions given the inputs and feedback of the child (talks, gestures, etc.). Experiments with six children (e.g. boys and girls between 5 and 8 years old) were carried out. We followed the strategy of covering different types of interaction such as: verbal, gestural and physical interaction. These experiments were followed by a psychologist to assess the child's performance, reactions and acceptance of the child and their parents during the CRI with the NAO robot. Questionnaires were applied to the children’s parents to quantify the aforementioned parameters.
- The robotic psychology perspective:
- Ascertain the emotional reactions of children and parents;
- Assess children's and parents's acceptance about the child-robot interaction;
- Analyze the differences between parents' and childrens' emotional reactions to CRI.
Emotional Reactions to Child-Robot Interaction: An Exploratory Study
Background: Child-Robot Interaction (CRI) has been conceptualized as an intervention context, with a number of potential applications in school (e.g., modeling learning processes, development of perceived self-efficacy), therapy (e.g., facilitation of coping skills), and health promotion settings (e.g., training of health-related skills). Current empirical evidence suggests that the use of robots in therapeutic context is likely to enhance the outcomes of cognitive-behavioral interventions, for example. However, as the utilization of robots for promoting child development and adaptation raises a number of applicability issues and ethical concerns, paired with the fact that mothers assuming the role of their child’s primary caregivers are the most important attachment figures for modeling children’s emotional and behavioral reactions to stressful or strange situations, current empirical evidence is scarce as regards the psychometric assessment of mothers’ and their children’s emotional reactions to CRI.
Objectives: This exploratory was aimed at examining mothers’ and their children’s subjective emotional reactions to a structured playful experience of CRI: first, by identifying the most frequent emotional responses experienced immediately after CRI; and then, by analyzing the differences between mothers’ and their children’s emotional reactions.
Method: The Emotional Assessment Scale (EAS) – a visual-analogical instrument (intensity of emotional responses coded from 0 to 100) designed for assessing the emotional reactions of Surprise, Fear, Anger, Guilt, Anxiety, Sadness, Disgust and Happiness – was administered to a convenience sample of 6 mothers and one of their children (aged between 5 and 8 years old; n = 12), immediately after a structured playful CRI experience (i.e., introducing dialogue, game of recognition of figures, dance and physical exercises). This scale was administered to mothers in both self-report (i.e., assessing their own emotional reactions) and proxy-report (i.e., assessing their children’s emotional reactions) formats. Cronbach’s alphas were computed to assess the instrument’s reliability in this study’s samples, and Wilcoxon Signed-Rank Nonparametric Test was used to detect any differences between the emotional reactivity experienced by mothers and that reported to be experienced by their children.
Preliminary Results: The obtained sample included children of both genders (50% girls, 50% boys), with a mean age of 6,7 years (SD = 1,2). For assessing the global construct of “emotional reactivity”, excellent and acceptable levels of internal consistency were respectively observed for the samples of children (α = .87) and their mothers (α = .68). The most frequent emotional reactions (≥ 60% of reported intensity) experienced by children and their mothers were: Surprise (M = 65,3; DP = 17,4 / M = 64,6; DP = 18,9) and Happiness (M = 74,8; DP = 10,3 / M = 85,4; DP = 3,3). With the exception of Anxiety (M = 29,2; DP = 25,9 / M = 8,9; DP = 12,5), for the remaining emotions of Fear, Anger, Guilt, Sadness and Disgust, a mean of reported intensity inferior to 20% was observed for both children and their mothers. There were no significant differences between mothers and their children in the levels of experiencing Surprise, Anger, Sadness, Disgust and Happiness; however, children were reported to experience more Fear (Z = -2,02; p = .04) and Anxiety (Z = -2,20; p = .03) , and less Guilt (Z = -2,02; p = .04), than their mothers.
Discussion: These preliminary results suggest that CRI may be a context where positive emotions, such as happiness and surprise, tend to be elicited in both children and their mothers. The observed differences in the experience of Anxiety and Fear, may reflect the distinctive adaptive function that those emotions assume when children, at this developmental stage, face new, unknown situations.
Work still under progress. <back to top>
Page updated
Google Sites
Report abuse