Self-supervised goal proposal and reaching is a key component for exploration and efficient policy learning algorithms. Such a self-supervised approach without access to any oracle goal sampling distribution requires deep exploration and commitment so that long horizon plans can be efficiently discovered. In this paper, we propose an exploration framework, which learns a dynamics-aware manifold of reachable states.

An overview of LEAF on the Sawyer Push and Reach task. The initial and goal images are encoded by the encoder of the VAE into latent states. Random states are sampled from the currently learned manifold of the VAE latent states, and are used to infer the current frontier. The currently learned deterministic policy is used to reach a state in the frontier from the initial latent state. After that, the currently learned stochastic policy is used to reach the latent goal state. A reconstruction of what the latent state in the frontier decodes to is shown. For clarity, a rendered view of the actual state reached by the agent is shown alongside the reconstruction.

Illustration of the environments used in our experiments. Details about them are in the appendix linked above.