Decentralized Structural-RNN for Robot Crowd Navigation
with Deep Reinforcement Learning
Shuijing Liu*, Peixin Chang*, Weihang Liang†, Neeloy Chakraborty†, and Katherine Driggs-Campbell
University of Illinois, Urbana-Champaign
Published at ICRA 2021
FoV environment
Robot with a limited field of view (FoV)
Group environment
Dense crowd with static and moving humans
Abstract
Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structural-Recurrent Neural Network (DS-RNN), a novel network that reasons about spatial and temporal relationships for robot decision making in crowd navigation. We train our network with model-free deep reinforcement learning without any expert supervision. We demonstrate that our model outperforms previous methods in challenging crowd navigation scenarios. We successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.
Decentrialized Structural-RNN (DS-RNN)
We model the crowd navigation scenario as a spatio-temporal graph (st-graph). Then, we convert the st-graph to a factor graph, which factorizes the robot policy function into three smaller factors.
We derive our network architecture from the factor graph. In our network, we represent each factor with an RNN, where
The spatial edgeRNN Rs captures the spatial interactions between humans the robot;
The temporal edgeRNN RT captures the dynamics of the robot's own trajectory;
The nodeRNN RN combines the robot state and the outputs of previous RNNs to determine the robot actions and state value.
An attention module assigns attention weights to each spatial edge.
Qualitative Results in Simulation
Here are some example episodes of our DS-RNN compared with ORCA (Van Den Berg et. al, 2010) and OM-SARL (C. Chen et. al, 2019):
ORCA
OM-SARL
DS-RNN (Ours)
ORCA
OM-SARL
DS-RNN (Ours)