Biased forecasts in a didactic setting
To train and test our model, we created a simulation environment where the ego vehicle always drives at a constant speed on a straight road. It shares the road with a pedestrian that walks in a straight line from a random initial position in a random orientation. The stochastic behavior model of the pedestrian was designed so that their ground-truth traveling distance is given by a bimodal distribution over the prediction horizon of 4.5 seconds.
Pedestrian travel distances
We train a CVAE model as represented below. The encoder takes the past trajectory x, the future trajectory y or the risk-level σ. It produces the parameters of a Gaussian in the latent space. A sample z and the past trajectory x are given to the decoder that predicts y.
At first the model is trained as a CVAE without the risk-level input. Then the decoder is fixed and the encoder is trained to produce a biased distribution such that a batch of biased samples zi and the past trajectory xi are decoded into a batch of future trajectories yi that lead to an average cost that is equal to an unbiased risk estimation with a risk-level σ. There are several ways to achieve this constraint. This is why we train the biased encoder to also minimize the KL-divergence with the latent prior distribution. This constrained optimization is written below:
We measure the distance from the constraint that we want to meet with the squared distance of the two sides of the equality in the constraint minimization problem written above. The graph below shows how the encoder is able to converge towards a small distance from the constraint. This means that the biased prediction distribution allows a good estimation of the risk.
Those modes correspond to the pedestrian walking slower or faster.
We test our model with two settings:
In the first setting, the initial position and orientation of the pedestrian is such that walking slowly is safe, the car would pass in front of them without collision but walking quickly is dangerous and would lead to a collision if the car keeps its pace.
In the second setting, the pedestrian is initialized slightly further on the road such that walking quickly is safe, the pedestrian finishes crossing the road before the car hits them, but walking slowly would be dangerous.
Scenario 1: Safer Slow
This is a representation of the simulated scene. The ego car in red is going from left to right down the road. A pedestrian in blue is walking across the road. Different random samples are drawn from the same initial position. The final positions are represented with a circle. In this plot only the ground truth distribution for the pedestrian behavior is represented with trajectory samples. The trajectories are colored depending on the associated cost. The fast mode in red is more costly than the slow mode in green.
This representation of the same scene shows the model predictions. The yellow lines represent the forecasts for a risk-level set to 0 (unbiased). The red lines represent the forecasts for a risk-level set to 1 (maximum pessimism). The green lines represent trajectories from the ground truth distribution of pedestrian behavior. A perception noise prevents the model from predicting the direction perfectly and does not allow a good prediction for the pedestrian speed. The most pessimistic forecasts in red are faster on average than either the ground truth in green or the prediction with the risk-level set to 0 in orange.
Histogram of the distance traveled along the ground truth simulation of the pedestrians during the 5 second scenario.
Histogram of the distance traveled along the unbiased (risk-level = 0) trajectory forecasts of the pedestrians during the 5 second scenario. Both modes are represented, some mode averaging is visible.
Histogram of the distance traveled along the most pessimistic (risk-level = 1) trajectory forecasts of the pedestrians during the 5 second scenario. Only the most dangerous faster mode is represented. The slower behavior is in the tail of the distribution
Histogram of the cost with the ground truth simulation of the pedestrians.
Histogram of the cost with the unbiased (risk-level = 0) trajectory forecasts of the pedestrians. Both modes are represented.
Histogram of the cost with the most pessimistic (risk-level = 1) trajectory forecasts of the pedestrians. Only the most costly mode is represented.
Scenario 2: Safer Fast
This is a similar scene with the pedestrian starting in the middle of the road, a few meters further along the road crossing than the previous scenario. Now the highest cost is associated with the slow mode (in red) and the lowest cost with the fast mode (in green).
This is the same scene including the forecasts from the model. The most pessimistic forecasts in red are slower on average than either the ground truth in green or the prediction with the risk-level set to 0 in orange.
Histogram of the distance traveled along the ground truth simulation of the pedestrians during the 5 second scenario.
Histogram of the distance traveled along the unbiased (risk-level = 0) trajectory forecasts of the pedestrians during the 5 second scenario. Both modes are represented, some mode averaging is visible.
Histogram of the distance traveled along the most pessimistic (risk-level = 1) trajectory forecasts of the pedestrians during the 5 second scenario. Only the most dangerous slower mode is represented. The faster behavior is in the tail of the distribution
Histogram of the cost with the ground truth simulation of the pedestrians.
Histogram of the cost with the unbiased (risk-level = 0) trajectory forecasts of the pedestrians. Both modes are represented.
Histogram of the cost with the most pessimistic (risk-level = 1) trajectory forecasts of the pedestrians. Only the most costly mode is represented.