Abstract

Loco-manipulation, physical interaction of various objects that is concurrently coordinated with locomotion, remains a major challenge for legged robots due to the need for both precise end-effector control and robustness to unmodeled dynamics. While model-based controllers provide precise planning via online optimization, they are limited by model inaccuracies. In contrast, learning-based methods offer robustness, but they struggle with precise modulation of interaction forces. We introduce RAMBO, a hybrid framework that integrates model-based whole-body control within a feedback policy trained with reinforcement learning. The model-based module generates feedforward torques by solving a quadratic program, while the policy provides feedback corrective terms to enhance robustness. We validate our framework on a quadruped robot across a diverse set of real-world loco-manipulation tasks, such as pushing a shopping cart, balancing a plate, and holding soft objects, in both quadrupedal and bipedal walking. Our experiments demonstrate that RAMBO enables precise manipulation capabilities while achieving robust and dynamic locomotion.

Abstract

Designing robotic manipulators for generic tasks while meeting specific requirements is a complex, iterative process involving mechanical design, simulation, control, and testing. New computational design tools are needed to simplify and speed up such processes. This work presents an original formulation of the computational design problem, tailored to help design generic manipulators with strong reachability requirements. The primary challenges addressed in this work are twofold. First, the necessity to consider the design of both continuous quantities and discrete components. Second, the ability to guide the design using high-level requirements, like the robot's workspace, without needing a specific manipulation task, unlike other co-design frameworks. These two challenges are addressed by employing a novel kineto-static formulation, resulting in a Mixed Integer Nonlinear Programming problem, which is solved using bi-level optimization. A compelling use case from a real industrial application is presented to highlight the practical effectiveness of the proposed method.

Abstract

This paper presents a bi-level optimization framework to concurrently optimize a quadruped hardware and control policies for achieving dynamic cyclic behaviors. The longterm vision to drive the design of dynamic and efficient robots by means of computational techniques is applied to improve the development of a new quadruped prototype. The scale of the robot and its actuators are optimized for energy efficiency considering a complete model of the motor, that includes friction, torque, and bandwidth limitations. This model is used to optimize the power consumption during bounding and backflip tasks and is validated by tracking the output trajectories on the first prototype iteration. The co-design results show an improvement of up to 87% for a single task optimization. It appears that, for jumping forward, robots with longer thighs perform better, while for backflips, longer shanks are better suited. To understand the trade-off between these different choices, a Pareto set is constructed to guide the design of the next prototype.

Abstract

This paper outlines a bi-level optimization method to concurrently optimize robot hardware parameters and control trajectories that ensure robust performance. The outer loop consists in a genetic algorithm that optimizes the hardware according to its average performance when tracking a locally optimal trajectory in perturbed simulations. The tracking controller exploits the locally optimal feedback gains computed in the inner loop with a Differential Dynamic Programming algorithm, which also finds the optimal reference trajectories. Our simulations feature a complete actuation model, including friction compensation and bandwidth limits. Our method can potentially account for arbitrary perturbations, and it discards hardware designs that cannot robustly track the reference trajectories. Our results show improved performance of the designed platform in realistic application scenarios, autonomously leading to the selection of lightweight and more transparent hardware.

Abstract 

We present a novel approach to quantifying and optimizing stability in robotic systems based on the Lyapunov exponents addressing an open challenge in the field of robot analysis, design, and optimization. Our method leverages differentiable simulation over extended time horizons. The proposed metric offers several properties, including a natural extension to limit cycles commonly encountered in robotics tasks and locomotion. We showcase, with an ad-hoc JAX gradient-based optimization framework, remarkable power, and flexibility in tackling the robustness challenge. The effectiveness of our approach is tested through diverse scenarios of varying complexity, encompassing high-degree-of-freedom systems and contact-rich environments. The positive outcomes across these cases highlight the potential of our method in enhancing system robustness.

Abstract 

This paper presents a computational framework for the design of high-performance legged robotic systems. The framework relies on the concurrent optimization of hardware parameters and control trajectories to find the best robot design for a given task. In particular, we focus on energy efficiency, presenting novel electro-mechanical models to account for the losses of the actuators due to friction and Joule effects. Thanks to a bi-level optimization scheme, featuring a genetic algorithm in the outer loop, our framework can also optimize for the duration of the motion, the actuators, and the size of the robot. We present a novel approach to scale both the actuators and the robot structure in a way that ensures structural integrity by maintaining constant the normalized deflection of the links. We validated our approach by designing a two-joint monoped robot to execute a jumping task. Our results show that our framework can lead to remarkable energy savings (up to 60%) thanks to the concurrent optimization of robot size, motion duration, and actuators.

Abstract 

Reinforcement learning often requires extensive training data. Simulation-to-real transfer offers a promising approach to address this challenge in robotics. While differentiable simulators offer improved sample efficiency through exact gradients, they can be unstable in contact-rich environments and may lead to poor generalization. This paper introduces a novel approach integrating sharpness-aware optimization into gradient-based reinforcement learning algorithms. Our simulation results demonstrate that our method, tested on contact-rich environments, significantly enhances policy robustness to environmental variations and action perturbations while maintaining the sample efficiency of first-order methods. Specifically, our approach improves action noise tolerance compared to standard first-order methods and achieves generalization comparable to zeroth-order methods. This improvement stems from finding flatter minima in the loss landscape, associated with better generalization. Our work offers a promising solution to balance efficient learning and robust sim-to-real transfer in robotics, potentially bridging the gap between simulation and real-world performance.

Abstract

Our ongoing research aims to investigate the potential of integrating robot design optimization with reinforcement learning (RL). In co-design literature, exploiting the ties between design and control is the key to unlocking otherwise unreachable performance. However, the problem of obtaining optimal policies that will adapt to a range of different robots is still open. We reason about the challenges that this problem setting brings. Moreover, we hint at a few possible future research directions that may help advance robot morphology and design-aware control policies while ensuring their optimality with respect to the task.

Abstract

The problem of designing complex robotic hardware using numerical optimization has gotten significant attention in recent years. However, ensuring robustness, which is essential to guarantee the practical applicability of the designed solutions, remains an ongoing challenge in co-design. This problem becomes particularly pronounced when dealing with inherently unstable systems, such as legged robots. In this extended abstract, we want to reason about these challenges and investigate possible solutions to tackle such a problem based on current trends in the community.

Abstract

Co-design is a valuable method for selecting optimized robots for a given task. In past research, this approach has been used to enhance the performance of legged robots. In our latest work, the conception of a quadruped prototype was driven by this approach, in order to achieve energy efficiency in cyclic tasks. The optimization of hardware and control features a complete electro-mechanical model that factors in actuator losses and mechanical design considerations. While this co-design formulation is versatile enough to cope with various tasks, the optimization-to-real gap still needs to be properly assessed and possibly reduced in some specific scenarios, hinting to some of the biggest open challenges in the practical application of co-design.