Reinforcement Learning (RL) based methods have been increasingly explored for robot learning. However, RL based methods often suffer from low sampling efficiency in the exploration phase, especially for long-horizon manipulation tasks, and generally neglect the semantic information from the task level, resulted in a delayed convergence or even tasks failure. To address these issues, we develop a Temporal-Logic-guided Hybrid policy framework (HyTL) which exploits three-level decision layers to facilitate robot learning. Specifically, the task specifications are encoded via linear temporal logic (LTL) to improve performance and offer interpretability. And a waypoints planning module is designed with the feedback from the LTL-encoded task level as a high-level policy to improve the exploration efficiency. The middle-level policy chooses which behavior primitives to implement and the low-level policy determines how to interact with the environment. We evaluate HyTL on four challenging manipulation tasks, which demonstrate its effectiveness and interpretability. 

Figure: The framework of HyTL. (a) The architecture of Transformer Encoder for representation LTL instrucuions. (b) The LTL progression for progressing LTL formulas. (d) The challenging manipulation situations for simulation study.

The compositionality scores are shown in Table, in which higher scores reflect better compositionality and more stable performance. As shown in Table, Mapleway and HyTL can select and combine more appropriate behavior primitives by guiding within waypoints, resulting in higher compositionality scores than other methods.