LTC21: Learning to control

See the final presentation of LTC group

Topic Leaders

Expert Panel

Invited speakers

Topic area tutorials (YouTube playlist)

Plan

  • Week 1 : Tutorials and invited talks on control and robotic. Setup software for projects.

  • Week 2-3: Project work by participants along with invited talks.


We target live demonstration of the project results at the final presentation to accompany the theoretical and experimental results on learned control.

Projects

  1. Open-loop stochastic sampling via MPPI vs MPPI combined with some form of gradient descent - which can find the optimum MPC plan with less computation? (Marcin Paluch, Frederik Heetmeyer, Florian Bolli, Krishna Kidambi)

  2. What is the optimum loss function for training DNN model of cartpole dynamics? (Marcin Paluch and Frederik Heetmeyer)

  3. What is the best lap time that can be achieved in L2RACE car racing simulation with learned car dynamics? (Florian Bolli, Antonio Rios, and Tobi Delbruck)

  4. Can we control a physical cartpole robot with MPC and with MPPI? And once we manage that using the dymamical systems model, can we do it with a model trained from data? (Asude Aydin and Antonio Rios, Marcin and Frederik)

  5. Can we control a wheeled robot with unique morphology (Xiang Deng)

Team

  • Marcin Paluch (PhD ETHZ)

  • Antonio Rios (U Seville postdoc lecturer)

  • Krishna Kidambi (U Maryland postdoc)

  • Nikhil Garg (BITS Pilani/USherbrooke masters student)

  • Ante Maric (U Zagreb masters student)

  • Frederik Heetmeyer (ETHZ masters student)

  • Florian Bolli (ETHZ masters student)

  • Gerald Wunsching (TU Munich)

  • Asude Aydin (Masters student, NSC program, UZH-ETH Zurich)

  • Xiang Deng (UPenn GRASP Lab alumni)


Goal

Our aim is to learn dynamics from data and do optimal control using model prediction, with the goal of eventually using a DNN accelerator like EdgeDRNN for model inference.


Background

There is a lot of interest in applying advances in machine learning to control. The first and second workshops on Learning for Dynamics and Control (L4DC) that took place June 2019/2020 attracted over 400 participants. There was also a recent IFAC workshop on Machine Learning meets Model-based Control.

Controlling AMPRO e-leg with EdgeDRNN from 2019 workshop

Current status

In 2019 and 2020 our Telluride topic areas “CDS19: Controlling Dynamical Systems” and “LTC20: Learning to Control” explored a number of basic ideas and systems. The projects were basic, but resulted in the ICRA 2019 conference paper on controlling Caltech's AMBER lab AMPRO with the EdgeDRNN accelerator.

Since the online 2020 workshop the LTC group has been meeting weekly with the goal of learning dynamics from data and using the learned model with model-predictive control (MPC). We have had more than 25 meetings since the workshop. We are learning a lot about the practicalities of MPC and learning models in RNNs.

L2RACE results from 2020 workshop

Introductory hands-on tutorials

  1. Basics of model predictive control with software exercises, using cart-pole environment.

  2. Tensorflow or PyTorch to train a small RNN for prediction.

  3. The L2RACE environment for moving horizon MPC - vehicle dynamics, PD control, pure-pursuit.

Provided Hardware and Software

  1. L2RACE multiplayer internet car racing simulation. We have continued using it, and in addition,

  2. Cart-pole simulation framework.

  3. A physical cartpole with developed tools to capture training data from it. We will consider setting it up in Zurich for remote use, which is manageable with a local operator to help with data collection.