Home

workshop on Modeling Socio-Emotional and Cognitive Processes from Multimodal Data in the Wild

COVID-10 ANNOUNCEMENT

Echoing the organizers of the ICMI Conference, we hope you are healthy in these worrying times! We have been assured that accepted papers *will be published* via the Conference Proceedings (opt out version available, see the Submission Section for details). The workshop will take place in a virtual format. We look forward to receiving your amazing submissions!

Workshop Description

This workshop is part of the 22nd ACM International Conference on Multimodal Interaction (ICMI) to be held in Utrecht, the Netherlands.

Multimodal signal processing in HRI and HCI is increasingly entering into a more applied stage. Many systems now aim to provide an engaging interaction experience in contexts of everyday life. At the same time, this means that the variability and complexity of the involved social contexts has dramatically increased. Data-driven system behaviors that that may have been adequately understood, labeled, and trained in one context may perform rather poorly when deployed in the wild. But what are the major roadblocks for multimodal signal processing in the wild – and how can they be overcome?

A central aim of this workshop is to engage in discussion about novel approaches and lessons learned in modeling multimodal data. Beyond this, we also need to do more to anticipate also future challenges in both data modelling and interaction design. Ethical challenges and treatment of highly variable availability of data provided or donated by users are just one example where an exchange on the state of art in multimodal data processing in the wild appears to be urgently needed.

Topics of interest

Multimodal modeling of emotions and social action tendencies
Multimodal engagement, attention, stress, memory, and workload estimation
Modeling and response estimation with biological signals (e.g., EEG, EDA, EMG, HR)
Eye tracking and shared attention
Studies bridging multimodal research between the laboratory and the wild
Case studies involving rich and dynamic signal modeling
Cognition-adaptive human-computer interfaces
Experiment design for cognitive processes
Ethical perspectives on current Human-Computer Interaction (HCI), Human-Robot Interaction (HRI) and applications of Artificial Intelligence (AI)
Interdisciplinary collaborations to understand the underpinnings of multimodal data

invited speaker

Tony belpaeme

Ghent University, Belgium

Participants

"Emotion Recognition using EEG and Physiological Data for Robot-Assisted Rehabilitation Systems" by Elif Gumuslu, Duygun Erol Barkana, and Hatice Kose
"Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition" by Srinivas Parthasarathy, and Shiva Sundaram
"Multimodal Fuzzy Assessment for Robot Behavioral Adaptation in Educational Children-Robot Interaction" by Daniel Tozadore, and Roseli Romero
"Is There ‘ONE way’ of Learning? A data-driven approach" by Jauwairia Nasir, Barbara Bruno, and Pierre Dillenbourg
"Assessment of situation awareness during robotic surgery using multimodal data" by Aurélien Léc, Mathieu Chollet, and Caroline Cao
"Model-based Prediction of Exogenous and Endogenous Attention Shifts During an Everyday Activity" by Felix Putze, Merlin Burri, Lisa-Marie Vortmann, and Tanja Schultz
"Measuring cognitive load: heart-rate variability (HRV) and pupillometry assessment" by Nerea Urrestilla Anguiozar, and David St-Onge
"SmartHelm: Towards Multimodal Detection of Attention in an Outdoor Augmented Reality Biking Scenario" by Sromona Chatterjee, Kevin Scheck, Dennis Küster, Felix Putze, Harish Moturu, Johannes Schering, Jorge Marx Gómez, and Tanja Schultz

Program - CET Time

5:00pm - 5:10pm

Opening remarks by workshop organizers

5:10pm - 5:55pm

Keynote talk by Tony Belpaeme

5:55pm - 6:10pm

Oral paper "Emotion Recognition using EEG and Physiological Data for Robot-Assisted Rehabilitation Systems" by Elif Gumuslu, Duygun Erol Barkana, and Hatice Kose

6:10pm - 6:25pm

Oral paper "Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition" by Srinivas Parthasarathy, and Shiva Sundaram

6:25pm - 6:45pm

Virtual coffee break and social get-together

6:45pm - 7:00 pm

Oral paper "Multimodal Fuzzy Assessment for Robot Behavioral Adaptation in Educational Children-Robot Interaction" by Daniel Tozadore, and Roseli Romero

7:00pm - 7:15pm

Oral paper "Is There ‘ONE way’ of Learning? A data-driven approach" by Jauwairia Nasir, Barbara Bruno, and Pierre Dillenbourg

7:15pm - 7:30pm

Oral paper "Assessment of situation awareness during robotic surgery using multimodal data" by Aurélien Léc, Mathieu Chollet, and Caroline Cao

7:30pm - 7:45pm

Oral paper "Model-based Prediction of Exogenous and Endogenous Attention Shifts During an Everyday Activity" by Felix Putze, Merlin Burri, Lisa-Marie Vortmann, and Tanja Schultz

7:45pm - 8:00pm

Oral paper "Measuring cognitive load: heart-rate variability (HRV) and pupillometry assessment" by Nerea Urrestilla Anguiozar, and David St-Onge

8:00 pm - 8:15 pm

Oral paper "SmartHelm: Towards Multimodal Detection of Attention in an Outdoor Augmented Reality Biking Scenario" by Sromona Chatterjee, Kevin Scheck, Dennis Küster, Felix Putze, Harish Moturu, Johannes Schering, Jorge Marx Gómez, and Tanja Schultz

8:15pm

Closing Remarks by conference organizers

Submission Instructions & Important Dates

Submissions for this workshop are using the following ACM SIG templates:

Full Paper = 8 page limit, excluding references (+optional auxiliary materials)
Short Paper = 4 page limit, with 1 extra page for references and appendices only
Extended Abstract = 2-4 pages, with references included

Links to the ACM SIG templates are available on the ACM website (please use the "sample-sigconf.tex" template). An Overleaf template for the for all three submission formats is directly available here. Word authors can the ACM interim layout template. Papers should be submitted via the Microsoft Conference Management Toolkit.

Each paper will be sent to at least two expert reviewers and will have one of the organizers assigned as editor. Review will be double-blind. A sufficient number of external reviewers from all areas has been identified at the participating institutions and in the network of the participating organizers.

Submission deadline: 31 ~~July 2020~~ ~~20 August 2020~~
Notifications of acceptance: 31 ~~August 2020~~ ~~7 September 2020~~
Camera-ready versions: 28 ~~September 2020~~
Workshop date: 29 October 2020 from 17-20:15pm CET