Real-World Failure Videos

The most common failure modes are the planner not finding a valid plan due to precondition inaccuracies or exceeding the maximum execution step length. However, these failure modes are not as interesting to visualize, since they just involve the robot trying actions that do not complete the task. Instead, we show less frequent but more visible real-world failure modes below, where inaccuracies in the vision perception module's estimated object and shelf primitive geometries lead to various Pick-and-Place failures (top row) and HFVC failures (bottom row). These inaccuracies only occur in the real world due to noisy depth perception.

All videos are sped up 10x.