Accelerated co-design of robots through morphological pretraining

Luke Strgar, Sam Kriegman

Northwestern University

Abstract

The co-design of robot morphology and neural control typically requires using reinforcement learning to approximate a unique control policy gradient for each body plan, demanding massive amounts of training data to measure the performance of each design. Here we show that a universal, morphology-agnostic controller can be rapidly and directly obtained by gradient-based optimization through differentiable simulation. This process of morphological pretraining allows the designer to explore non-differentiable changes to a robot's physical layout (e.g. adding, removing and recombining discrete body parts) and immediately determine which revisions are beneficial and which are deleterious using the pretrained model. We term this process "zero-shot evolution" and compare it with the simultaneous co-optimization of a universal controller alongside an evolving design population. We find the latter results in diversity collapse, a previously unknown pathology whereby the population—and thus the controller's training data—converges to similar designs that are easier to steer with a shared universal controller. We show that zero-shot evolution with a pretrained controller quickly yields a diversity of highly performant designs, and by fine-tuning the pretrained controller on the current population throughout evolution, diversity is not only preserved but significantly increased as superior performance is achieved.

Results: before and after learning

These are four randomly generated designs performing phototaxis before and after training the universal controller. In the first 7 seconds of the video the robots are stationary, unable to move with a randomly initialized controller. After having been empowered with the universal controller, the robots effectively sense and locomote towards the randomly positioned light source.

Results: adaptive phototaxis

These are some of the designs that evolved. In each video, one design is shown in four randomly generated test environments. All of these designs, along with millions of others not shown, use the same universal controller to perform adaptive phototaxis. This morphology agnostic controller was pretrained using analytical gradients from differentiable simulation in minutes and then used as a prior for zero-shot and few-shot morphological evolution.