Real-time Trajectory Generation via Dynamic Movement Primitives for Autonomous Racing
Catherine Weaver, Roberto Capobianco, Peter R. Wurman, Peter Stone, and Masayoshi Tomizuka
Sony AI, UC Berkeley, Sapienza University of Rome, UT Austin
arXiv
Improving real-time control of autonomous racecars with high-order motion primitives
Abstract
We employ sequences of high-order motion primitives for efficient online trajectory planning, enabling competitive racecar control even when the car deviates from an offline demonstration. Dynamic Movement Primitives (DMPs) utilize a target-driven non-linear differential equation combined with a set of perturbing weights to model arbitrary motion. The DMP's target-driven system ensures that online trajectories can be generated from the current state, returning to the demonstration. In racing, vehicles often operate at their handling limits, making precise control of acceleration dynamics essential for gaining an advantage in turns. We introduce the Acceleration goal (Acc. goal) DMP, extending the DMP's target system to accommodate accelerating targets. When sequencing DMPs to model long trajectories, our (Acc. goal DMP explicitly models acceleration at the junctions where one DMP meets its successor in the sequence. Applicable to DMP weights learned by any method, the proposed DMP generates trajectories with less aggressive acceleration and jerk during transitions between DMPs compared to second-order DMPs. Our proposed DMP sequencing method can recover from trajectory deviations, achieve competitive lap times, and maintain stable control in autonomous vehicle racing within the high-fidelity racing game Gran Turismo Sport.
Autonomous racing presents a challenging problem, with recent advancements in reinforcement learning (RL) showing great performance in simulated environments. Conversely, structured trajectory planning and control methods are more interpretable and reliable in real racing vehicles. Online trajectory planning, in contrast to offline or static planning, enhances adaptability and performance by dynamically adjusting to system changes, planning errors, or disturbances. Furthermore, control inputs can be optimized using real-time measurements of the system's state so the controller can make the best possible decision at each moment. This paper explores the topic of computationally efficient, online trajectory planning, and consider how this can be applied to the complicated domain of autonomous racing.
Dynamic Movement Primitives
Motion primitives, like Dynamic Movement Primitives (DMPs), simplify trajectory generation in various robotic domains by combining pre-defined basic motion patterns. DMPs consist of a target-driven system for smooth trajectory generation and a perturbing function for flexibility in modeling state variations or system dynamics. We propose an approach that segments a racing demonstration and imitates each segment with a DMP, to create a sequence of DMPs.
To adapt DMPs to the autonomous racing domain, we propose the Acceleration goal (Acc. goal) DMP, designed to combine sequences of third-order DMPs to generate near-optimal and near-feasible trajectories. Typically, DMP sequences only use second-order DMPs, which are not sufficient for modeling the acceleration of a racing trajectory. For instance, in racing, precise control of lateral forces is essential for maintaining vehicle stability at the limits of handling. Vehicles can also gain an advantage in turns by employing a ``slow in, fast out" racing approach, delaying braking until the last moment of entering the turn and accelerating immediately upon exiting. Therefore, trajectories that accurately control the acceleration can result in faster lap times and more stable control. The Acc. goal DMP extends the capabilities of a third-order DMP [11] by including the acceleration of the target state. This improvement allows for explicit modeling of acceleration at the points where a DMP in a sequence meets its successor. It is applicable to DMP weights learned by any method.
Contributions
The proposed Acc. goal DMP extends the target system to handle accelerating targets.
Acc. goal DMP plans trajectories with less aggressive acceleration and jerk when transitioning between DMPs in sequence compared to 2nd-order DMPs.
We generate online DMP trajectories from a sequence of DMPs for control in Gran Turismo Sport.
Our method recovers from starting states where a baseline controller fails and yields less tracking error, less aggressive control, and the lowest lap-time compared to existing, non-RL, methods.
Real Time Control Experiments
Three trials are carried out for each starting point to test how well the vehicle can recover from various parts of the track in Gran Turismo Sport. Blue indicates initial points where all methods recover in all three trials. Green indicates initial points where the MPC recovers using Acc. goal DMP, but Fixed Ref. MPC with offline demonstration does not. Orange indicates that none of the methods recover.
Traversing a corner from a difficult starting position
The vehicle starts near a corner on the track, but it is not near the original demonstration. The online DMP trajectories must plan reference trajectories for the MPC to return to the demonstration in order to navigate through the corner.
Starting near the corner...
The Standard MPC (Fixed Ref) can generate good control commands without the DMP trajectories.
Starting away from the corner...
The DMP trajectories are necessary to allow the car to generate a new reference so that it can traverse the corner.
Comparing the trajectories from a corner in the track, we see that the Acc. Goal DMP is best able to recovery from the difficult starting position.
Standard MPC: a standard model predictive control tracking offline demonstration
The standard MPC fails almost immediately into the corner since it cannot re-calculate an optimal trajectory from its difficult starting position
2nd order DMP + MPC: online trajectories with the third order Vel. Goal DMP
The 2nd order DMP does not model the acceleration of the trajectories. Even though it can occasionally recover, it's trajectories are slow and unstable for racing
Vel. Goal DMP + MPC: online trajectories with the third order Vel. Goal DMP
Since the Vel. Goal DMP does not adequately model the acceleration at the joining points between DMPs in sequence, the trajectories have high acceleration that are difficult to track with the MPC
Proposed Acc. Goal DMP + MPC: online trajectories with our Acc. Goal DMP
The Acc. Goal DMP plans a reasonable trajectory from the difficult starting position, allowing the vehicle to recover and continue at a high speed
Recovering from various starting points around the track
The car starts from various points along the track at the center, left, or right edges of the track. It can start near or far away from the offline demonstration. We record whether the vehicle recovers. A successful recovery is defined as returning to the offline demonstration and remaining near the demonstration for at least 4 seconds.
In all of the trials, our Acc. Goal DMP is the most effective at recovering from the various starting points, particularly as the initial deviation from the offline demonstration increases:
The Acc. Goal DMP is able to recover the best because it has the least tracking error of all of the methods. Using the Acc. Goal DMP reduces the magnitude of the control commands, thereby sending less extreme commands to the vehicle.
