Uncertainty Modeling for Stochastic Video Prediction

Extrapolating the present into the future is a task that is essential for evolved AI systems, especially since they are being increasingly deployed to assist with tasks such as autonomous driving, action forecasting, etc.

In this work, we consider the task of stochastic prediction of future frames of a video. Towards this end, we propose a hierarchical, variational, deep neural network that quantifies the predictive uncertainty of a stochastic frame prediction model. We call this Neural Uncertainty Quantifier (NUQ). NUQ features a variance encoder-decoder network that encodes the covariance of the prior distribution on the latent space of the future frame and predicts the frame-level uncertainty, as shown in the figure above.

Code: Link

Publication:

• M. Chatterjee, N. Ahuja, A. Cherian, “A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Prediction”, International Conference on Computer Vision 2021 (ICCV 2021).