Goal of this project

This study explores approaches that combine model-based methods with reinforcement learning to enable humanoid robots to overcome modeling uncertainties and perform locomotion, loco-manipulation, and contact-rich behaviors in real-world environments. Reinforcement learning has proven effective for replicating robust, human-like walking, though it is less suitable for tasks requiring precise loco-manipulation. Extending to loco-manipulation, joint torque and force references improve the precision of RL-based controllers but require additional planning to manage contact sequences. For more complex contact-rich behaviors, contact-implicit trajectory optimization (CITO) provides dynamically feasible solutions, yet its implementation is hindered by high computational load and sensitivity to initialization. To address these challenges, Future work will explore the seamless integration of trajectory optimization with reinforcement learning, aiming to enhance robustness and scalability in multi-contact scenarios that demand precise and contact-rich motions. 

• Advantages 

1. Capable of controlling walking parameters such as step time and width

2.  Enables prediction of future motions based on planner outputs (optimized foot placement and center of mass)

• Disdvantages

1. contact sequence is requried for asymmetric locomotion

2. Additional rewards are required for human-like Upper-body motions 

• Future work

1. RL for Real-Time Trajectory Optimization: Using reinforcement learning (RL) to provide initialization for trajectory optimization (TO) is expected to yield dynamically feasible solutions in multi-contact scenarios that require high accuracy and complex contact sequences.

2. Trajectory Optimization for RL: Imitating dynamically feasible trajectories, including joint torques and contact forces, is expected to improve RL agents' precision and physical consistency in contact-rich behaviors.