Home

Paper

Code

Video

Person Re-Identification for Robot Person Following with Online Continual Learning

Hanjing Ye, Jieting Zhao, Yu Zhan, Weinan Chen, Li He and Hong Zhang

Abstract: Robot person following (RPF) is a crucial capability in human-robot interaction (HRI) applications, allowing a robot to persistently follow a designated person. In practical RPF scenarios, the person can often be occluded by other objects or people. Consequently, it is necessary to re-identify the person when he/she reappears within the robot's field of view. Previous person re-identification (ReID) approaches to person following rely on a fixed feature extractor. Such an approach often fails to generalize to different viewpoints and lighting conditions in practical RPF environments. In other words, it suffers from the so-called domain shift problem where it cannot re-identify the person when his re-appearance is out of the domain modeled by the fixed feature extractor. To mitigate this problem, we propose a ReID framework for RPF where we use a feature extractor that is optimized online with both short-term and long-term experiences (i.e., recently and previously observed samples during RPF) using the online continual learning (OCL) framework. The long-term experiences are maintained by a memory manager to enable OCL to update the feature extractor. Our experiments demonstrate that even in the presence of severe appearance changes and distractions from visually similar people, the proposed method can still re-identify the person more accurately than the state-of-the-art methods.

Overview:

Robot person following with online continual learning. To this end, long-term and short-term experiences are utilized to optimize the feature extractor online for representing a discriminative appearance of the target person.

Pipeline:

The top part is the pipeline of our RPF system and the bottom part is the proposed person ReID framework. When the target person is consistently tracked, his and other people's observation is added to the memory manager for memorization. Additionally, these patches are fed into the feature extractor to extract ReID features. These features are utilized by the target classifier to estimate the target confidence. If the target confidence is greater than a threshold, the corresponding position is designated as the target position. In addition to the inference above process, the memory manager simultaneously replays long-term and short-term experiences to train the feature extractor and the target classifier, respectively. If the target person is not found among the tracked individuals, the training process pauses, and all observations become candidates for re-identification. The above training and inference processes are managed by the ReID lifecycle.