Attend Before You Act

Attend Before you Act

Leveraging human visual attention for continual learning

Motivation: Where do humans look while navigating in a 3D maze environment ? Does foveating around the regions where humans look helps the reinforcement learning process in the context of continual learning ? We hypothesise that knowing where to look in a task aids continual learning across tasks

Where do humans look while navigating a 3D maze? Here we show the saliency maps generated from Saliency Attentive Model overlaid on a sequence of frames encountered by the agent during navigation in a 3D maze. Visualization of these heat maps depicts where humans look in such stimulus.

UNREAL agent and Visually-Attentive UNREAL agent are evaluated once training is stopped for transfer learning. Transfer is evaluated on three variations of training categorized as - Easy: Simple Gaussian noise is added in the original frames, Moderate: Tinting of frames is done by randomly flipping a coin with the same hue of 0.25, and Difficult: At random, some frames are tinted with different amounts of hue ranging from 0 to 1.

Visually-Attentive UNREAL agent navigating the 3D maze

Maze Navigation with tinting of frames at random with same hue

Maze Navigation with tinting of frames at random with different hue

UNREAL agent navigating the 3D maze

Maze Navigation with tinting of frames at random with same hue

Maze Navigation with tinting of frames at random with different hue

Paper The 2nd Lifelong Learning: A Reinforcement Learning Approach (LLARLA) Workshop, Stockholm, Sweden, FAIM 2018. Copyright 2018 by the author(s)

Link to Code

ICML 2018 Spotlight, Oral presentation, *Best Student Paper Award

Google Sites

Report abuse