Few-Shot Task Learning Through Inverse Generative Modeling

MoCap

The CMU Motion Capture Database is collected from real humans performing actions.

Training Evaluation

Few-Shot New Concept Learning Evaluation on New Initial States

breaststroke

Compositions of Learned New Concepts and Training Concepts

Learning New Concepts as Single Concepts vs Compositions of 2 Concepts

Driving

In the highway environment, the green vehicle is controlled by the model, blue vehicles are controlled by a separate controller, red indicates collision. In all scenarios the controlled vehicle must maintain a high speed and avoid collisions. In the highway scenarios it must stay on the rightmost lanes, in exit make the exit, in merge allow a vehicle to merge, in intersection make a left turn and in roundabout take the second exit. Evaluation is closed loop.

Training Evaluation

Few-Shot New Concept Learning Evaluation on New Initial States

Goal Oriented Navigation

In the AGENT environment, an agent navigates to one of two targets based on their shape and/or color. Evaluation is closed loop. 

Training Evaluation

target defined by single attribute (shape or color).

Few-Shot New Concept Learning Evaluation on New Initial States

target defined by attribute compositions (shape and color).

Object Rearrangement

In the Object Rearrangement environment, three objects need to be positioned in a certain configuration that satisfies spatial relations between them.

Training Evaluation

single pairwise relation.

Few-Shot Concept Learning

Compositions of Learned New Concept and Training Concepts

generation of a new concept (diagonal) composed with a training pairwise relation.

t-SNE Analysis

New concepts that are not explicit compositions in the natural language symbolic space of training concepts for Object Rearrangement, MoCap and Driving. Hover over data points to view which concept they represent!