Multi-Modal MPPI and Active Inference for
Reactive Task and Motion Planning
Yuezhe Zhang, Corrado Pezzato, Elia Trevisan, Chadi Salmi, Carlos Hernández Corbato, Javier Alonso-Mora
Abstract
Task and Motion Planning (TAMP) has made strides in complex manipulation tasks, yet the execution robustness of the planned solutions remains overlooked. In this work, we propose a method for reactive TAMP to cope with runtime uncertainties and disturbances. Particularly, we combine an Active Inference planner (AIP) for adaptive high-level action selection, and a novel Multi-Modal Model Predictive Path Integral controller (M3P2I) for low-level control. The AIP generates alternative symbolic plans, each linked to a cost function and samples for M3P2I. M3P2I employs a physics simulator for diverse trajectory rollouts, deriving optimal control by weighing the different samples according to their cost. This idea enables blending different robot skills for fluid and reactive plan execution, accommodating plan adjustments at both the high and low levels to cope for instance with dynamic obstacles or disturbances that invalidate the current plan. We have tested our approach both in simulations and real-world scenarios.
Simulation Results of Push and Pull
Push only using MPPI, middle to corner --- success
Push only using MPPI, corner to corner --- fail
Pull only using MPPI, middle to corner --- fail
Pull only using MPPI, corner to corner --- fail
Multi-modal motion using M3P2I, middle to corner --- success
Multi-modal motion using M3P2I, corner to corner --- success
Simulation Results of Pick and Place
Reactive top pick using MPPI
Multi-modal pick(top pick + side pick) using M3P2I
Multi-modal motion using M3P2I with collision avoidance
Reactive pick and place using RL, oscillation occurs
Real-World Experiments
Reactive pick and place using MPPI
Multi-modal pick using M3P2I