Hierarchical Meta-learning-based Adaptive Controller
Accepted to ICRA 2024
Fengze Xie*, Guanya Shi**, Michael O'Connell*, Yisong Yue*, and Soon-Jo Chung*
*California Institute of Technology, **Carnegie Mellon University
Abstract
We study how to design learning-based adaptive controllers that enable fast and accurate online adaptation in changing environments. In these settings, learning is typically done during an initial (off-line) design phase, where the vehicle is exposed to different environmental conditions and disturbances (e.g., a drone exposed to different winds) to collect training data. Our work is motivated by the observation that real-world disturbances fall into two categories: 1) those can be directly monitored or controlled during training, which we call "manageable"; and 2) those that cannot be directly measured or controlled (e.g., nominal model mismatch, air plate effects, and unpredictable wind), which we call "latent". Imprecise modeling of these effects can result in degraded control performance, particularly when latent disturbances continuously vary. This paper presents the Hierarchical Meta-learning-based Adaptive Controller (HMAC) to learn and adapt to such multi-source disturbances. Within HMAC, we develop two techniques: 1) Hierarchical Iterative Learning, which jointly trains representations to caption the various sources of disturbances, and 2) Smoothed Streaming Meta-Learning, which learns to capture the evolving structure of latent disturbances over time (in addition to standard meta-learning on the manageable disturbances). Experimental results demonstrate that HMAC exhibits more precise and rapid adaptation to multi-source disturbances than other adaptive controllers.
Experiment Results
Our Experiment Environment is as follows: We place three box fans in one row horizontally as our training and testing environment. In the training phase, we use the Crazyflie with air plate type 1. In the testing phase, we use both Crazyflies with air plate type 1 and type 2.
Crazyflie Configuration 1
Crazyflie Configuration 2
The flight trajectory for each experiment is shown in the figure at left. We employ Crazyflies' integrated PID and INDI controllers, fine-tuning their parameters to enhance performance. We see that the PID controller's performance diminishes in the test environment. Meanwhile, the INDI controller relies on IMU-derived acceleration as a reference, although this value exhibits considerable noise. To ensure system stability, INDI applies a low-pass filter to the IMU acceleration data, resulting in a notably conservative acceleration.
In contrast, NF and HMAC perform considerably well in our testing environment. Analyzing the wave trajectory, we observe that NF encounters challenges in tracking performance along the y-axis. This performance degradation is attributed to the fact that in the proposed DAIML approach, the wind condition remains constant along the y-axis. On the contrary, the wind generated by the box fan exhibits non-linearity and instability, rendering adaptation difficult for NF in this specific scenario. In conclusion, HMAC outperforms NF, including figure-8, wave, and figure-8-transfer trajectories.
Our hardware experiment demo is included in the following youtube video.