Abstract

A noisy and diverse demonstration set may hinder the performances of an agent aiming to acquire certain skills via imitation learning. However, state-of-the-art imitation learning algorithms often assume the optimality of the given demonstration set. In this paper, we address such optimal assumption by learning only from the most suitable demonstrations in a given set. Suitability of a demonstration is estimated by whether imitating it produce desirable outcomes for achieving the goals of the tasks. For more efficient demonstration suitability assessments, the learning agent should be capable of imitating a demonstration as quick as possible, which shares similar spirit with fast adaptation in the meta-learning regime. Our framework, thus built on top of Model-Agnostic Meta-Learning, evaluates how desirable the imitated outcomes are, after adaptation to each demonstration in the set. The resulting assessments hence enable us to select suitable demonstration subsets for acquiring better imitated skills.

Demonstration Set: a demonstration set with demonstrations of mixed suitability towards a task is given.

Meta Training: learn a set of adaptive parameters.

Meta Testing (Adaptation): the agent tries to adapt to a given demonstration.

Evaluation: evaluate the suitability of demonstrations using task-related heuristics and behavioral changes before and after adaptation.

Selecting Suitable Subset: select the most suitable subset of the demonstration set and learn from the selected subset.