Selected Projects
Reinforcement Learning for Google football
Discretize the whole football field into meshgrid. Used Monte Carlo Search Tree to learn the action strategy of football AI to deal with the state captured by environment. See demo for more information.
Voice informed VAE face generation
Extract latent code from both Speech signal and face image signal. Forced them to distribute close to each other. In testing time, generate talking face from speech data. code
Video prediction from single image
Extract language features from VisualCOMET to predict the context information given single image. Combined with the video generation backbone (Ordinary Differential Equation and 3D convolutional VAE) and Noise Contrastive Estimation (NCE) contrastive loss, we generate sharper and temporally coherent videos. Colab notebook
Generative music
Took advantage of GAN algorithm to learn music style and generate its own music after training. demo