HHI-Assist

A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario

Saeed Saadatnejad*, Reyhaneh Hosseininejad*, Jose Barreiros, Katherine M. Tsui, Alexandre Alahi
École Polytechnique Fédérale de Lausanne (EPFL) , Toyota Research Institute (TRI)

Abstract

The increasing labor shortage and aging population underscore the need for assistive robots to support human care recipients. Developing such robots requires accurate human motion prediction to ensure their responsiveness and safety. This task is challenging due to the variability in scenarios and the necessity of modeling interactions between agents. To address these challenges, we present HHI-Assist, a collection of motion capture clips of human-human interaction (HHI) for physical assistance.

Dataset

Physical assistance scenarios showcasing the importance of predicting human interactions for a smooth assistance transfer from a chair and a bed.

More visualisation about the dataset settings

Skeletal structure of the joints

visualization in motion capture software

Animated movement in the Drake simulator

Model

IDD takes the pose sequences of both CG (blue skeleton) and CR (pink skeleton) as input, predicting their corresponding future pose sequences, which are visualized here for three timesteps. The model architecture includes two 1D convolution layers—one serving as an embedding layer for the inputs and another as the decoder—and four Transformer Blocks with skip connections. Each Transformer Block integrates cascaded temporal and spatial multi-head attentions to effectively capture the spatio-temporal features of the data, followed by a 1D convolution layer.

HHI-Assist

data

code

paper

Abstract

Dataset

Model

Results