Learning Object-Centric Abstractions for High-Level Planning

Minecraft Results

Here we show the results for learning a PDDL representation of a complex Minecraft task. We use data collected from 15 episodes, with uniform random option execution. Below we show visualisations for each of the steps along the way to building the model. Note that all of these steps, including generating the problem PDDL definition, are autonomously learned from data.

Partitioning

We have 9 options that are partitioned into 51 approximately subgoal options. We visualise the subgoal options below by considering the agent's POV and inventory and computing the mean image for the set of start and end states for each partitioned option.


WalkToItem1-init

WalkToItem1-effect

WalkToItem2-init

WalkToItem2-effect

WalkToItem3-init

WalkToItem3-effect

AttackItem1-init

AttackItem1-effect

Preconditions

For each partitioned option, we learn a classifier to determine whether an option can be executed given a particular state. We visualise the learned preconditions below by feeding all our data to the classifiers, selecting only those states classified as positive, and then averaging the image of the states to produce a single image. The classifiers are conditioned on a number of objects, so a precondition is visualised showing the state of each relevant object.

AttackItem1 (the agent is in front of the redstone ore, and the redstone ore is whole)

PickupItem1 (the agent is in front of the pickaxe, the pickaxe is present, and the inventory is empty)

ToggleDoor1 (the agent is front of a closed door, and the door is closed)

ToggleDoor2 (the agent is front of an open door, and the door is open)

Effects

For each partitioned option, we fit a density estimator (KDE) over the set of terminating states. We visualise the learned effects below by sampling from these distributions , and then averaging the image of the states to produce a single image. The KDEs are distributions over objects changed by the option, so an effect is visualised showing the state of each object affected.

OpenChest1 (the agent is in front of an open chest, and the chest is open)

AttackItem2 (the agent is in front of a broken gold block, and the gold block is broken)

ToggleDoor8 (the agent is in front of a closed door, and the door is closed)

WalkNorthDoor3 (the agent is in front of a closed door)

Propositional PDDL

We next generate a propositional PDDL. Each proposition is a distribution over a particular object's state. We visualise some of these propositions below

Gold block is whole

Door 2 is closed

Chest is closed

Door 4 is closed

Final Typed PDDL

After determining object types and merging, we produce a sound PDDL with high-level parameterised operators. The domain description file and problem definition are accessible below. Clicking on "Solve" will take you to an online planner than will generate a plan to solve the task

Final Plan

Below is a video of the agent executing the generated plan to solve the task.

Click to see the full plan description

(:action walk-to-partition-1-8a :parameters (agent pickaxe inventory) :precondition (and (notfailed) (symbol_10 agent) (symbol_5 pickaxe) (symbol_1 inventory) (psymbol_5) ) :effect (and (symbol_12 agent) (not (symbol_10 agent) ) (psymbol_6) (not (psymbol_5) ) ) )
 (:action pickup-partition-0-80a :parameters (agent pickaxe inventory) :precondition (and (notfailed) (symbol_12 agent) (symbol_5 pickaxe) (symbol_1 inventory) (psymbol_6) ) :effect (and (symbol_23 inventory) (symbol_24 pickaxe) (symbol_25 agent) (not (symbol_1 inventory) ) (not (symbol_5 pickaxe) ) (not (symbol_12 agent) ) (psymbol_19) (not (psymbol_6) ) ) )
 (:action walk-north-door-partition-2-110a :parameters (agent door1) :precondition (and (notfailed) (symbol_25 agent) (symbol_9 door1) (psymbol_19) ) :effect (and (symbol_37 agent) (not (symbol_25 agent) ) (psymbol_24) (not (psymbol_19) ) ) )
 (:action toggle-door-partition-2-298a :parameters (agent door1) :precondition (and (notfailed) (symbol_37 agent) (symbol_9 door1) (psymbol_24) ) :effect (and (symbol_64 door1) (symbol_65 agent) (not (symbol_9 door1) ) (not (symbol_37 agent) ) ) )
 (:action through-door-partition-3-258a :parameters (agent door1) :precondition (and (notfailed) (symbol_65 agent) (symbol_64 door1) (psymbol_24) ) :effect (and (symbol_50 agent) (not (symbol_65 agent) ) (psymbol_12) (not (psymbol_24) ) ) )
 (:action attack-partition-0-77a :parameters (agent ore) :precondition (and (notfailed) (symbol_50 agent) (symbol_3 ore) (psymbol_12) ) :effect (and (symbol_17 ore) (symbol_18 agent) (not (symbol_3 ore) ) (not (symbol_50 agent) ) (psymbol_13) (not (psymbol_12) ) ) )
 (:action pickup-partition-1-81a :parameters (agent ore inventory) :precondition (and (notfailed) (symbol_18 agent) (symbol_17 ore) (symbol_23 inventory) (psymbol_13) ) :effect (and (symbol_26 inventory) (symbol_27 ore) (symbol_28 agent) (not (symbol_17 ore) ) (not (symbol_18 agent) ) (not (symbol_23 inventory) ) (psymbol_20) (not (psymbol_13) ) ) )
 (:action walk-south-door-partition-1-205a :parameters (agent door1) :precondition (and (notfailed) (symbol_28 agent) (symbol_64 door1) (psymbol_20) ) :effect (and (symbol_44 agent) (not (symbol_28 agent) ) (psymbol_14) (not (psymbol_20) ) ) )
 (:action walk-north-door-partition-5-116a :parameters (agent door2) :precondition (and (notfailed) (symbol_44 agent) (symbol_9 door2) (psymbol_14) ) :effect (and (symbol_37 agent) (not (symbol_44 agent) ) (psymbol_15) (not (psymbol_14) ) ) )
 (:action toggle-door-partition-5-326a :parameters (agent door2) :precondition (and (notfailed) (symbol_37 agent) (symbol_9 door2) (psymbol_15) ) :effect (and (symbol_64 door2) (symbol_65 agent) (not (symbol_9 door2) ) (not (symbol_37 agent) ) ) )
 (:action through-door-partition-5-277a :parameters (agent door2) :precondition (and (notfailed) (symbol_65 agent) (symbol_64 door2) (psymbol_15) ) :effect (and (symbol_52 agent) (not (symbol_65 agent) ) (psymbol_7) (not (psymbol_15) ) ) )
 (:action walk-to-partition-2-39a :parameters (agent chest inventory) :precondition (and (notfailed) (symbol_52 agent) (symbol_4 chest) (symbol_26 inventory) (psymbol_7) ) :effect (and (symbol_13 agent) (not (symbol_52 agent) ) (psymbol_8) (not (psymbol_7) ) ) )
 (:action walk-north-door-partition-7-167a :parameters (agent door3) :precondition (and (notfailed) (symbol_13 agent) (symbol_9 door3) (psymbol_8) ) :effect (and (symbol_41 agent) (not (symbol_13 agent) ) (psymbol_10) (not (psymbol_8) ) ) )
 (:action toggle-door-partition-8-342a :parameters (agent door3) :precondition (and (notfailed) (symbol_41 agent) (symbol_9 door3) (psymbol_10) ) :effect (and (symbol_64 door3) (symbol_73 agent) (not (symbol_9 door3) ) (not (symbol_41 agent) ) ) )
 (:action through-door-partition-7-284a :parameters (agent door3) :precondition (and (notfailed) (symbol_73 agent) (symbol_64 door3) (psymbol_10) ) :effect (and (symbol_54 agent) (not (symbol_73 agent) ) (psymbol_0) (not (psymbol_10) ) ) )
 (:action walk-north-door-partition-0-101b :parameters (agent door4) :precondition (and (notfailed) (symbol_54 agent) (symbol_6 door4) (psymbol_0) ) :effect (and (symbol_35 agent) (not (symbol_54 agent) ) (psymbol_3) (not (psymbol_0) ) ) )
 (:action toggle-door-partition-0-292a :parameters (agent door4) :precondition (and (notfailed) (symbol_35 agent) (symbol_6 door4) (psymbol_3) ) :effect (and (symbol_60 door4) (symbol_61 agent) (not (symbol_6 door4) ) (not (symbol_35 agent) ) ) )
 (:action through-door-partition-0-252a :parameters (agent door4) :precondition (and (notfailed) (symbol_61 agent) (symbol_60 door4) (psymbol_3) ) :effect (and (symbol_47 agent) (not (symbol_61 agent) ) (psymbol_18) (not (psymbol_3) ) ) )
 (:action walk-to-partition-4-75b :parameters (agent gold) :precondition (and (notfailed) (symbol_47 agent) (symbol_2 gold) (psymbol_18) ) :effect (and (symbol_15 agent) (not (symbol_47 agent) ) (psymbol_17) (not (psymbol_18) ) ) )
 (:action attack-partition-1-78a :parameters (agent gold) :precondition (and (notfailed) (symbol_15 agent) (symbol_2 gold) (psymbol_17) ) :effect (and (symbol_19 gold) (symbol_20 agent) (not (symbol_2 gold) ) (not (symbol_15 agent) ) ) )
 (:action pickup-partition-3-89a :parameters (agent gold inventory) :precondition (and (notfailed) (symbol_20 agent) (symbol_19 gold) (symbol_26 inventory) (psymbol_17) ) :effect (and (symbol_29 inventory) (symbol_30 gold) (symbol_31 agent) (not (symbol_19 gold) ) (not (symbol_20 agent) ) (not (symbol_26 inventory) ) (psymbol_23) (not (psymbol_17) ) ) )
 (:action walk-south-door-partition-0-199b :parameters (agent door4) :precondition (and (notfailed) (symbol_31 agent) (symbol_60 door4) (psymbol_23) ) :effect (and (symbol_43 agent) (not (symbol_31 agent) ) (psymbol_16) (not (psymbol_23) ) ) )
 (:action through-door-partition-1-253a :parameters (agent door4) :precondition (and (notfailed) (symbol_43 agent) (symbol_60 door4) (psymbol_16) ) :effect (and (symbol_48 agent) (not (symbol_43 agent) ) (psymbol_2) (not (psymbol_16) ) ) )
 (:action walk-to-partition-0-3b :parameters (agent) :precondition (and (notfailed) (symbol_48 agent) (psymbol_2) ) :effect (and (symbol_11 agent) (not (symbol_48 agent) ) (psymbol_1) (not (psymbol_2) ) ) )
 (:action craft-partition-1-287a :parameters (agent inventory) :precondition (and (notfailed) (symbol_11 agent) (symbol_29 inventory) (psymbol_1) ) :effect (and (symbol_57 inventory) (symbol_11 agent) (not (symbol_29 inventory) ) ) )
 (:action craft-partition-0-286a :parameters (agent inventory) :precondition (and (notfailed) (symbol_11 agent) (symbol_57 inventory) (psymbol_1) ) :effect (and (symbol_55 inventory) (symbol_56 agent) (not (symbol_11 agent) ) (not (symbol_57 inventory) ) ) )
 (:action walk-south-door-partition-5-248a :parameters (agent door3) :precondition (and (notfailed) (symbol_56 agent) (symbol_64 door3) (psymbol_1) ) :effect (and (symbol_46 agent) (not (symbol_56 agent) ) (psymbol_4) (not (psymbol_1) ) ) )
 (:action through-door-partition-8-285a :parameters (agent door3) :precondition (and (notfailed) (symbol_46 agent) (symbol_64 door3) (psymbol_4) ) :effect (and (symbol_53 agent) (not (symbol_46 agent) ) (psymbol_11) (not (psymbol_4) ) ) )
 (:action walk-to-partition-2-48d :parameters (agent chest inventory) :precondition (and (notfailed) (symbol_53 agent) (symbol_4 chest) (symbol_55 inventory) (psymbol_11) ) :effect (and (symbol_13 agent) (not (symbol_53 agent) ) (psymbol_8) (not (psymbol_11) ) ) )
 (:action open-chest-partition-0-291a :parameters (agent chest inventory) :precondition (and (notfailed) (symbol_13 agent) (symbol_4 chest) (symbol_55 inventory) (psymbol_8) ) :effect (and (symbol_58 chest) (symbol_59 agent) (not (symbol_4 chest) ) (not (symbol_13 agent) ) ) )