Gated Inference Network (GIN 🍸)

Model

Gated Inference Network (GIN) is an efficient approximate Bayesian inference algorithm for state space models (SSMs) with nonlinear state transitions and emissions. GIN disentangles two latent representations: one representing the object derived from a

nonlinear mapping model, and another representing the latent state describing its dynamics. This disentanglement enables direct state estimation and missing data imputation as the world evolves. To infer the latent state, a deep extended Kalman filter (EKF) approach is utilized that integrates a compact RNN structure to compute both the Kalman Gain and smoothing gain, completing the data flow. 

Animated files of image imputation and sequence generation

We showcase the model's capability to independently learn the dynamics model from video data. Subsequently, we employ it for tasks such as imputing missing data and generating sequences. These tasks are carried out across distinct environments, each featuring an irregular polygon with a varying number of edges. 

4 edges

5 edges

6 edges

7 edges