MEDAL-ADR - Agent Behaviour

Our cultural transmission agent (MEDAL-ADR, in blue) finds an expert (in red) in a held-out task, follows it on a path through goals while navigating terrain and obstacles, and continues to display the demonstrated trajectory within the same episode after the expert has dropped out. Note that the trails left behind by both avatars are not visible to the agent, they just make these videos easier to follow for human observers.

Human Trajectory Probe Tasks

One example of each type of probe task. All probes aim to cover a wide, representative range of crossings and color combinations (as much possible with only ten probes). In addition, the complex world probes all aim at giving clean demonstrations of jumping and/or crouching behaviors and navigation around vertical obstacles. The human movement pattern in all probes is always goal-directed and near-optimal (it does not incur any score penalties) but clearly different from a scripted bot, taking a time to get situated in the first few seconds and not always taking the exact same path twice. In all videos our agent is marked blue, the expert in red.

Behaviour Over Time During Training

Progression of emergent social-learning behaviors during training in a fixed environment at certain times during training (in billion steps). In all videos our agent is marked blue, the expert in red.

Effects of ADR During an Experiment

Accumulation of navigation skills of social-learning agent under ADR training. From left to right: horizontal obstacles, horizontal obstacles in a larger world, horizontal and vertical obstacles, horizontal and vertical obstacles over bumpy terrain. In all videos our agent is marked blue, the expert in red.

Generalisation: World Space

MEDAL-ADR generalises across world space parameters, demonstrating both following and recall across much of the space. The space of worlds is parameterised by the size and bumpiness of the terrain, and the density of obstacles. To quantify generalisation over this space, we generate tasks with worlds from the Cartesian product of obstacle density and world size, with a perfect expert bot and game uniformly sampled with 2 crossings and 5 spheres. In all videos our agent is marked blue, the expert in red.

Generalisation: Game Space

The space of games is defined by the number of goals in the world as well as the number of crossings contained in the correct navigation path between them. To quantify generalisation over this space, we generate tasks across the range of feasible “N-goal M-crossing” games, with a perfect expert bot, and a flat, empty world of size 20x20. In all videos our agent is marked blue, the expert in red.

Generalisation: Expert Space

The space of experts is defined by the speed and action distribution taken by the expert in the world. Such experts can be either scripted bots, allowing us to precisely control their movement speed and action noise, or human players with more realistic and diverse movement patterns. To quantify generalisation over this space, we generate tasks with experts bots from the Cartesian product of movement speed and action noise. In all videos our agent is marked blue, the expert in red.

Trajectory Plot Examples

Trajectory plots for MEDAL-ADR agent for a single episode. The coloured parts of the lines correspond to the colour of the goal hemisphere the agent and expert have entered and the xs correspond to when the agent entered the incorrect goal. Here, position refers to the agent’s position along the z-axis. In all videos our agent is marked blue, the expert in red.