Social signals induce cognitive conflict in a handover task with a humanoid robot

Key research questions

1. Do irrelevant social signals from a robot hinder performance?

2. Can we measure behavioral, eye movement and electrophysiological correlates of cognitive conflict?

3. Are those findings translated to real collaborative scenarios with a humanoid robot?

METHODOLOGY

In the task, participants had to choose where to place the object the robot would offer depending on the predominant color. So, observers knew in advance where to respond but were only allowed to do it after iCub suggested a place, left or right. We had objects with different color proportions to play with the complexity of the task. We expected that incongruent gaze from the robot would induce conflict evident in response times, accuracy, eye movements and neural correlates.

We opted for a very systematic approach: from online studies to the implementation of the paradigm in a real interaction. The screen-based studies used the same paradigm. We measured behavioral responses online (Study 1), eye movement responses (Study 2), and EEG and behavioral responses (Study 3).

ANALYSES

We analyzed behavioral responses online (Study 1), eye movement responses (Study 2) and EEG and behavioral responses (Study 3).

RESULTS

Study 1

Reaction times were slower and there were more mistakes when the robot looked at the wrong locations. Those are typical markers of cognitive conflict.

Study 2

Similar to Study 1, but with participants in the lab, eye movement responses were slower and there were more errors when the robot looked at the wrong location.

Study 3

Study 3 confirmed that the origin of the performance profile was related to the cognitive conflict. That was observed in the ERPs and in the neural oscillations, indexes of difficulties processing gaze direction and planned motor responses.

The systematic approach paid off. From the behavioral to the EEG study, we observed how artificial agents can influence performance. This work was possible thanks to close collaboration with Aziz.

My contribution was to adapt the paradigm in the three studies, collect the data, help in analyzing the data, and write the paper.

Our work was accepted in the Journal of Cognitive Neuroscience.

Reference: J. Perez-Osorio, A. Abubshait, and A. Wykowska (2021). Irrelevant Robot Signals in a Categorization Task Induce Cognitive Conflict in Performance, Eye Trajectories, the N2 Component of the EEG Signal, and Frontal Theta Oscillations. Journal of Cognitive Neuroscience. doi: https://doi.org/10.1162/jocn_a_01786

Next steps

Implementation of the paradigm in a real interaction with the robot to measure brain activity during a handover task. We also investigate how framing (team vs solo) influences performance and neural correlates.