Interplay of Speech and Lip Movements

The Interplay of Speech and Lip Movements

Please find the list of related papers, code and demo videos. If you would like to try out the interactive lip-sync demo, please go to: https://bhaasha.iiit.ac.in/lipsync/

A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild

Prajwal K R*, Rudrabha Mukhopadhyay*, Vinay Namboodiri, CV Jawahar, accepted at ACM Multimedia 2020

Project page: http://cvit.iiit.ac.in/research/projects/cvit-projects/a-lip-sync-expert-is-all-you-need-for-speech-to-lip-generation-in-the-wild

Code and Models: https://github.com/Rudrabha/Wav2Lip

Paper: will be released.

Interactive Demo: https://bhaasha.iiit.ac.in/lipsync/

Note: A preliminary demo video is attached on the left. Please note that these results are from a slightly inferior model. We will release a newer demo video soon with superior results.

Towards Automatic Face-to-Face Translation

Prajwal K R*, Rudrabha Mukhopadhyay*, Jerin Philip, Abhishek Jha, Vinay Namboodiri, CV Jawahar, ACM Multimedia 2019 (Oral)

Code and Models: https://github.com/Rudrabha/LipGAN

Paper: https://dl.acm.org/doi/10.1145/3343031.3351066

Project page: https://cvit.iiit.ac.in/research/projects/cvit-projects/facetoface-translation

Learning Individual Speaking Styles for Accurate Lip-to-Speech Synthesis

Prajwal K R*, Rudrabha Mukhopadhyay*, Vinay Namboodiri, CV Jawahar, CVPR 2020

Code and Models: https://github.com/Rudrabha/Lip2Wav

Paper: https://openaccess.thecvf.com/content_CVPR_2020/html/Prajwal_Learning_Individual_Speaking_Styles_for_Accurate_Lip_to_Speech_Synthesis_CVPR_2020_paper.html

Project page: https://cvit.iiit.ac.in/research/projects/cvit-projects/speaking-by-observing-lip-movements

Page updated

Google Sites

Report abuse