RSPT: Reconstruct Surroundings and Predict Trajectories for Generalizable Active Object Tracking

Introduction

Motivation: Active Object Tracking (AOT) aims to maintain a specific relation between the tracker and object(s) by autonomously controlling the motion system of a tracker given observations. It is widely used in various applications such as mobile robots and autonomous driving. However, building a generalizable active tracker that works robustly across various scenarios remains a challenge, particularly in unstructured environments with cluttered obstacles and diverse layouts. To realize this, we argue that the key is to construct a state representation that can model the geometry structure of the surroundings and the dynamics of the target.

Method: To this end, we propose a framework called RSPT to form a structure-aware motion representation by Reconstructing Surroundings and Predicting the target Trajectory.  Moreover, we further enhance the generalization of the policy network by training in the asymmetric dueling mechanism.  

Experiments: Empirical results in virtual environments show that the tracker with RSPT significantly outperforms the existing methods among unseen environments, especially in environments with cluttered obstacles and diverse layouts. We further deploy the RSPT in a real-world scenario, showing good generalization in sim-to-real transfer.