Overview:
Digitizing humans with utmost realism is the holy-grail of Metaverse kind of immersive platform enabling a large set of tele-presence applications, namely, digital gaming, sports analytics, content creation for multimedia/animation, 3D virtual try-on, etc. This is a challenging task as the body shape geometry evolves over time, yielding a large space of complex body poses as well as shape variations. In addition to this, there are several other challenges such as self-occlusions by body parts, obstructions due to free form clothing, background clutter (in-the-wild setup), sparse set of cameras with non-overlapping fields of views (multi-view setup), sensor noise, etc. Traditionally, image-based techniques for 3D human digitization uses stereo/multi-view (including RGB and depth cameras) setup that typically require studio environments with controlled lighting and multiple synchronized and calibrated cameras. With the advent of learning based methods in 3D Computer Vision, in-the-wild human digitization has become possible. The models such as SCAPE and SMPL models the human 3D surface by parameterizing the body shape and the 3D joint locations and orientation. Model based reconstruction techniques fail to capture accurate geometrical information over the body surface (both body parts and garments) is not retained and are typically applicable only for tight clothing scenarios. In this tutorial, we plan to introduce the following fundamentals to the students/researchers working in the field of 3D computer vision, computer graphics, deep learning, etc.
Sensing and 3D representation of the human body
Parametric 3D Human model fitting using images, monocular videos, RGB-D sequence, etc.
3D digitization of clothed human body.
Current Research challenges and applications of 3D human digitzation.
Tentative Schedule:
Introduction to 3D Representation Sensing (30 minutes): Overview of popular acquisition technologies and computational representation of 3D shapes.
Parametric 3D Human Model Fitting (75 minutes): 3D Human body representation models such as SCAPE and SMPL. Algorithms for fitting fit these models for reconstructing 3D humans.
Digitizing Clothed Humans (75 minutes): Birds-eye-view introduction to various learning based solution for in-the-wild clothed human digitization.
