"Watch, Learn, Help: Proactive Robot Assistants using Vision-Language Models

A robot that understands user language commands and plans to perform tasks. Hierarchical controller loops let the robot adapt to environments, try again in case of failures, update plans, and more. It can also remember things - it has memory. Built a brain for a robot.

Previous Work and Prototypes