Selected Projects

Reinforcement Learning for Google football

Discretize the whole football field into meshgrid. Used Monte Carlo Search Tree to learn the action strategy of football AI to deal with the state captured by environment. See demo for more information.

Voice informed VAE face generation

Extract latent code from both Speech signal and face image signal. Forced them to distribute close to each other. In testing time, generate talking face from speech data. code

Video prediction from single image

Extract language features from VisualCOMET to predict the context information given single image. Combined with the video generation backbone (Ordinary Differential Equation and 3D convolutional VAE) and Noise Contrastive Estimation (NCE) contrastive loss, we generate sharper and temporally coherent videos. Colab notebook

Generative music

Took advantage of GAN algorithm to learn music style and generate its own music after training. demo