Blind Pick
Blind Locomotion
Blind and Deaf Piano
Blind and Numb Pen Rotation
Blind and Numb Cube Rotation
Noisy Monkey Bars
Wrist Camera Pick-and-Place
Occluded Pick-and-Place
Visual Pen Rotation
Visual Cube Rotation
Scaffolder learns to do spiral search and robustly picks up the block even with perturbations (shown to the left and above).
Baselines move erratically and frequently push the block far from the workspace or search in unfeasible areas.
Scaffolder bounds to minimize the risk of hitting obstacles and quickly recovers when it does.
Guided Observability moves quickly in unobstructed areas but is frequently stopped or slowed by hurdles.
DreamerV3 moves slowly and is easily caught by obstacles.
Scaffolder plays an imperfect but recognizable rendition of Twinkle Twinkle Little Star
DreamerV3's rendition is unrecognizable.
Scaffolder quickly achieves the desired pen orientation and maintains stability.
DreamerV3 briefly achieves the desired pen orientation but struggles to maintain stability.
Scaffolder quickly rotates the block to achieve the desired orientation and maintains stability once reaching it.
Informed dreamer is able to rotate the block rather quickly but struggles to maintain stability.
Both Scaffolder and Guided Observability learn efficient, swinging brachiating motions that closely mimic that of a Gibbon.
DreamerV3 and Informed Dreamer can swing between the handholds but use inefficient motions that incur large energy costs.
Scaffolder quickly locates the block and goal location with its active-perception wrist camera. DreamerV3 locates the block relatively quickly but struggles to find the goal location.
Scaffolder quickly locates the block with touch sensing and places it on the goal.
DreamerV3+BC locates the block but fails to fully pick it up and even attempts to go to the goal location long after dropping it.
Guided Observability locates the block but fails to fully place it on the goal on its first attempt, as its picking is less robust.
As with blind pen rotation, Scaffolder quickly achieves the desired orientation and maintains stability.
Informed Dreamer initially achieves the desired orientation but struggles to maintain stability.
Both Scaffolder and DreamerV3+BC are able to quickly rotate the block, while Guided Observability fails to rotate the block at all.