Multimodal_ASR

Multimodal Automatic Speech Recognition

Multimodal Automatic Speech Recognition

     Contents

1. Goal

2. Schematic Diagram

3. Lip-reading Process

4. Application Demos

1. Goal         

• Augment automatic speech recognition system by providing visual (lip-reading) information in noisy environment

2. Schemetic Diagram        

• Merge audio/visual features　　

• Lip detection : Binary image in YCbCr space

• Lip movement tracking : Active shape modeling

3. Lip-reading Process