Abstract

Real-world decision making often requires reasoning in a partially observable environment using information obtained from complex visual observations --- major challenges for deep reinforcement learning. In this paper, we introduce the Discriminative Particle Filter Reinforcement Learning (DPFRL), a reinforcement learning method that encodes a particle filter structure with learned discriminative transition and observation models in a neural network. The particle filter structure allows for reasoning with partial observations, and discriminative parameterization allows modeling only the information in the complex observations that are relevant for decision making. In experiments, we show that in most cases DPFRL outperforms state-of-the-art POMDP RL models in Flickering Atari Games, an existing POMDP RL benchmark, as well as in Natural Flickering Atari Games, a new, more challenging POMDP RL benchmark that we introduce. We also show that DPFRL performs well when applied to a visual navigation domain with real-world data.

Paper

Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee, Nan Ye

Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations

ICLR 2020, [PDF], [Code]

BibTex

@inproceedings{

Ma2020Discriminative,

title={Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations},

author={Xiao Ma and Peter Karkus and David Hsu and Wee Sun Lee and Nan Ye},

booktitle={International Conference on Learning Representations},

year={2020}

}

Related Works

Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee

Particle Filter Recurrent Neural Networks

AAAI 2020, [PDF]

Maximilian Igl, Luisa Zintgraf, Tuan Anh Le, Frank Wood, Shimon Whiteson

Deep Variational Reinforcement Learning for POMDPs

ICML 2018, [PDF]