Reinforcement learning for bio-inspired robot locomotion

Biological locomotion is the movement of an animal from one location to another, through periodic changes in the shape of the body, along with interaction with the environment. The periodic motion of the shape, which constitutes the building block of locomotion, is called the locomotion gait. Examples of the locomotion gait are legged locomotion, flapping of the wings for flying, or wavelike motion of the fish for swimming.

We construct a bio-inspired  central pattern generator (CPG)-type architecture for learning optimal maneuvering control of the periodic locomotion gait. The CPG circuit is realized as a coupled oscillator feedback particle filter. The objective function is minimized through continuous-time Q-learning with finite approximation of Hamiltonian functions. The architecture is presented here with the aid of two examples involving planar locomotion of coupled rigid body systems.

Publications

T. Wang, A. Taghvaei, and P. G. Mehta, "Bio-inspired learning of sensorimotor control for locomotion," in 2020 American Control Conference (ACC). IEEE, 2020, pp. 2188-2193. [DOI] (Presentation)

T. Wang, A. Taghvaei, and P. G. Mehta, "Q-learning for POMDP: An application to locomotion gaits," in 2019 IEEE 58th Conference on Decision and Control (CDC). IEEE, 2019, pp. 2758-2763. [DOI]