Face Tracking and its Applications

Saturday, September 8th, 2018 9:00-12:00 Theresianum 602 (Building 6), TU Munich

Course description:

The tutorial is about monocular face tracking techniques and also discusses the possible applications. Specifically, the tutorial will cover topics such as:

• Input Modalities (RGB and RGB-D sensors)
• Image Formation Models (Camera Models and Light Transport)
• Statistical Face Priors and Blendshapes
• Advanced Face Models and Parametric Rigs
• Optimization-based Face Reconstruction
• Applications of Face Reconstruction
  • Video Editing, Facial Reenactment, Video Dubbing, Facial Projection Mapping
• Deep Learning for Face Reconstruction
• Open Challenges
• Social Implications

Related tutorials and state-of-the-art reports:

• State of the Art on Monocular 3D Face Reconstruction, Tracking and its Applications by Zollhöfer et al.
  • Zollhöfer, M. and Thies, J. and Bradley, D. and Garrido, P. and Beeler, T. and Pérez, P. and Stamminger, M. and Nießner, M. and Theobalt, C., "State of the Art on Monocular 3D Face Reconstruction, Tracking and its Applications", Eurographics 2018
• Image-based 3D face reconstruction survey of Sylianou et al.
  • Stylianou, Georgios and Lanitis, Andreas, “Image Based 3D Face Reconstruction: A Survey”, International Journal of Image and Graphics 2009
• A recent report on capturing appearances of faces, with the goal of re-rendering in CG by Klehm et al.
  • Oliver Klehm and Fabrice Rousselle and Marios Papas and Derek Bradley and Christophe Hery and Bernd Bickel and Wojciech Jarosz and Thabo Beeler, “Recent Advances in Facial Appearance Capture”, Computer Graphics Forum 2015
• Face tracking is related to the problem of non-rigid surface registration, where the goal is to align a specific surface to images or 3D scans, a topic that is covered in detail in recent tutorials of Bouaziz et al.
  • Bouaziz, Sofien and Tagliasacchi, Andrea and Pauly, Mark, “Dynamic 2D/3D Registration”, Eurographics 2014
  • Bouaziz, Sofien and Tagliasacchi, Andrea and Li, Hao and Pauly, Mark, “Modern Techniques and Applications for Real-time Non-rigid Registration”, SIGGRAPH ASIA 2016 Courses
• Excellent surveys exist that discuss facial rigging by Orvalho et al. and blendshape based facial animation by Lewis et al.
  • Verónica Orvalho and Pedro Bastos and Frederic Parke and Bruno Oliveira and Xenxo Alvarez, “A Facial Rigging Survey”, Eurographics State of The Art Reports 2012
  • Lewis, J. P. and Anjyo, Ken and Rhee, Taehyun and Zhang, Mengjie and Pighin, Fred and Deng, Zhigang, “Practice and Theory of Blendshape Facial Models”, Eurographics 2014 - State of the Art Reports