Abstract: In a flexible collaborative task sharing framework, it is necessary to take situation awareness issues into account. A robot must be able to “understand” the context of the collaboration, especially the difficulty of the task, the willingness of the human to collaborate or his availability. We propose to implement a robotic decision process based on the situation awareness model defined by Endsley. An experiment has been carried out to illustrate an adaptive collaboration in which the robot takes the initiative and fulfill the task that was assigned to the human.

Abstract: Computer vision-based gesture recognition and cog- nitive modeling connectivity can play a key role in developing efficient algorithms and understanding the neural mechanism behind multiple human communication. It leverages to detect, track, and interpret human body movements in intuitive ways so that we can control and interact with robots in a safe manner. In this paper, we present our experience of deploying real-time gesture recognition based on the multiple industrial camera system. A new strategy is presented based on YOLO and OpenPose to reach 13 fps with robustness to detect, track, and decompose movements of multiple humans wearing different clothing. YOLO was used to detect and track the person and the background was eliminated to focus only on the area in the image that represents humans. OpenPose was used to detect the 15 key point information from each human bounding box. This precise key point information was then used to construct the human 2D skeleton for gesture recognition. The approach yields robust and stable results with limitations pertaining to human crossing and human cloth-to-background color matching.

Abstract: To facilitate a common representation for human- robot communication, and to allow robots to interact with changing environments, we propose a cognitive architecture to answer language queries, inspired by the theory of conceptual spaces. We focus on the gap between the human learning of new visual concepts from few examples and the adaptation to new environments, and the current state-of-the-art models which require millions of annotated training examples, and often lack the ability to generalize to unseen environments. We introduce a dataset for learning individual language constructs without bounding boxes. Our approach compares favourably with state-of-the-art approaches designed for similar tasks, with advantages which make it suitable for a video dataset.

Abstract: Advances in robotics and artificial intelligence (AI) have enabled the possibility of human-robot teaming. One potential avenue for collaborative robots is to provide decision-support for human partners in complex decision-making tasks. However, such agents are imperfect in real-world scenarios and may provide incorrect or suboptimal recommendations. Thus, it is imperative for human collaborators to understand when to trust the robot’s suggestions for maximizing task performance. Explainable AI (xAI) attempts to improve user understanding by providing explanations or rationales for agent recommendations. However, constantly providing explanations is unnecessary and can induce cognitive overload among users. In this work, we propose a POMDP framework that allows the robot to infer the users’ latent trust and preferences to provide appropriate and timely explanations for maximizing human- robot team performance in a sequential decision-making game.