Facial Expression Recognition and Synthesis

The objective in this project was two fold. In the first phase, we performed recognition of facial expressions in videos. Towards this end, we developed a novel descriptor for effectively encoding motion of the facial region so as to be able to distinguish between various emotions. Our descriptor is constructed using a graph-theoretic approach. The above figure shows an overview of the same. The recognition was performed using a Bag-of-Visual Words scheme.

In the second phase, we were interested in synthesizing facial expressions and imposing them on expression "neutral" faces. In order to do this, we learnt identity independent basis vectors of the deformations of the facial region corresponding to the various expressions. These deformations were then re-targeted on a new "neutral" face. The figure below shows a sample result for this synthesis task.

The project was conducted in a team of two. The results were published as multiple publications in a notable conference.

Publications:

• S. Agarwal, M. Chatterjee, D. P. Mukherjee, “Recognizing Facial Expression Using a NovelShape Motion Descriptor”, Indian Conf. on Computer Vision, Graphics and Image Processing 2012 (ICVGIP 2012).

• S. Agarwal, M. Chatterjee, D. P. Mukherjee, “Synthesis of EmotionalExpressions specific to Facial Structure”, Indian Conf. on Computer Vision, Graphics and Image Processing 2012 (ICVGIP 2012).

Project Funding Support:

We are extremely grateful to the Department of Science and Technology (DST), Govt. of India for their financial support to this project.