N-LIMB: Neural Limb Optimization
for Efficient Morphological Design
Charles Schaff Mathew R. Walter
TTI-Chicago TTI-Chicago
Optimized robot for sprinting
Optimized robot for gap crossing
Optimized robot for wall climbing
Abstract
A robot’s ability to complete a task is heavily dependent on its physical design. However, identifying an optimal physical design and its corresponding control policy is inherently challenging. The freedom to choose the number of links, their type, and how they are connected results in a combinatorial design space, and the evaluation of any design in that space requires deriving its optimal controller. In this work, we present N-LIMB, an efficient approach to optimizing the design and control of a robot over large sets of morphologies. Central to our framework is a universal, design-conditioned control policy capable of controlling a diverse sets of designs. This policy greatly improves the sample efficiency of our approach by allowing the transfer of experience across designs and reducing the cost to evaluate new designs. We train this policy to maximize expected return over a distribution of designs, which is simultaneously updated towards higher performing designs under the universal policy. In this way, our approach con- verges towards a design distribution peaked around high-performing designs and a controller that is effectively fine-tuned for those designs. We demonstrate the potential of our approach on a series of locomotion tasks across varying terrains and show the discovery novel and high-performing design-control pairs.
Efficient Optimization of Design and Control
Core to our approach is an efficient co-optimization algorithm that maintains a distribution over physical designs, and trains a design-conditioned control policy to maximize reward in expectation under that distribution. While training this policy, the design distribution is simultaneously updated towards higher performing designs until the approach converges towards a locally optimal design and a control policy that is effectively finetuned for that design.
Optimizing over Morphologies
Our approach is able to optimize over large sets of morphologies, including robots with arbitrary kinematic trees, with discrete and continuous parameters kinematic and dynamic parameters. The picture on the left shows potential quadruped and hexapod robots with various limb and joint configurations.
Graph Grammars for Morphological Design
Context-free graph grammars can be used to procedurally construct arbitrary robot morphologies, and allows users to easily tailor the search space to their needs, such as only searching over designs that can be fabricated or incorporating domain knowledge such as left-right symmetry.
Autoregressive Models for the Design Distribution
The grammar definition allows for a natural, auto-regressive definition of the design distribution. Our approach samples robots by recursively feeding partial robot graphs through a transformer model that produces a distribution over valid expansion rules, sampling an expansion rule, and updating the graph until a complete robot is formed.
Transformers for Design-conditioned Control
Robots of different morphologies have state and action spaces that vary in size based on the number of rigid bodies and controllable degrees of freedom. To train a controller that generalizes across morphologies, an architecture that can process graph structures is needed. Therefore, we design transformer-based actor critic architecture that can efficiently parse robot pose and task information to produce actions and a value estimate.
Experiments
We test our approach by optimizing over quadruped and hexapod robots for locomotion tasks in a variety of terrains, and find that our method discovers novel and high-performing robots for each task. Check out the for video highlights ofthese results, and see the paper for more details!