Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
Xiao Ma^, Peter Karkus^, David Hsu^, Wee Sun Lee^, Nan Ye'
^National University of Singapore, 'University of Queensland
Abstract
Real-world decision making often requires reasoning in a partially observable environment using information obtained from complex visual observations --- major challenges for deep reinforcement learning. In this paper, we introduce the Discriminative Particle Filter Reinforcement Learning (DPFRL), a reinforcement learning method that encodes a particle filter structure with learned discriminative transition and observation models in a neural network. The particle filter structure allows for reasoning with partial observations, and discriminative parameterization allows modeling only the information in the complex observations that are relevant for decision making. In experiments, we show that in most cases DPFRL outperforms state-of-the-art POMDP RL models in Flickering Atari Games, an existing POMDP RL benchmark, as well as in Natural Flickering Atari Games, a new, more challenging POMDP RL benchmark that we introduce. We also show that DPFRL performs well when applied to a visual navigation domain with real-world data.
Paper
Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee, Nan Ye
Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
BibTex
@inproceedings{
Ma2020Discriminative,
title={Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations},
author={Xiao Ma and Peter Karkus and David Hsu and Wee Sun Lee and Nan Ye},
booktitle={International Conference on Learning Representations},
year={2020}
}
Network Overview
