Research

Abbhinav Venkat, Chaitanya Patel, Yudhik Agrawal, Dr. Avinash Sharma, ICCVW '19

In this work, we propose an initial solution that integrates deep learning with the sparse mesh representation in an implicit manner to perform monocular 3D human reconstruction. While several of the existing works formulate it as volumetric or parametric learning with complex and indirect reliance on reprojections of the mesh, we learn an "implicitly structured" point cloud that is regularized by the neighborhood constraint imposed by the mesh topology, thus, ensuring a smooth surface reconstruction.

Neeraj Battan*, Abbhinav Venkat*, Dr. Avinash Sharma, ACPR '19 (Oral)

3D Human Motion Indexing and Retrieval is an interesting problem due to the rise of several data-driven applications aimed at analyzing and/or re-utilizing 3D human skeletal data, such as data-driven animation, analysis of sports bio-mechanics, human surveillance etc. Spatio-temporal articulations of humans, noisy/missing data, different speeds of the same motion etc. make it challenging and several of the existing state of the art methods use hand-craft features along with optimization based or histogram based comparison in order to perform retrieval. In this work, we propose a 3D human motion descriptor learned using a deep network for the task of 3D Human Motion Retrieval. Our learned embedding is generalizable and applicable to real-world data - addressing the aforementioned challenges and further enables sub-motion searching in its embedding space using another network.

Abbhinav Venkat, Sai Sagar Jinka, Dr. Avinash Sharma, BMVC '18

We propose a deep learning based solution for textured 3D reconstruction of human body shapes from a single RGB image. This is achieved by first recovering the volumetric grid of the non-rigid human body given a single RGB image, followed by orthographic texture view synthesis using the respective depth projection of the reconstructed (volumetric) shape and input RGB image. We propose to co-learn the depth information readily available with affordable RGBD sensors (e.g., Kinect) while showing multiple views of the same object during the training phase, thus enabling single view reconstruction at test time.

Abbhinav Venkat, Harit Pandya, Dr. Avinash Sharma (CVIT), Dr. K. Madhava Krishna (RRC)

We propose a novel two stage pipeline to obtain the textured 3D reconstruction of non-rigid human body shapes. The first stage recovers the volumetric representation of human body shapes using a deep learning model trained with multi-view RGBD images. Subsequently, using the proposed depth based visual servoing, we estimate the relative transformation between the virtual and reference Kinect cameras. Using this transformation, we perform back-projection to transfer texture information onto the reconstructed mesh.

Vaishali Pal, Abbhinav Venkat, Dr. Avinash Sharma (CVIT)

We first calculate the sparse correspondences between a pair of human 3D models by examining the principal eigen vector of the affinity matrix to evaluate each assignment, and rejecting those assignments that have a low association. This however leads to incorrect matches for symmetric parts. Using the wavelet kernel descriptor and heat kernel, we successfully distinguish between the symmetric parts, thus leading to an accurate sparse matching, which can be subsequently used as an reliable prior for dense matching.

Govinda Surampudi, Abbhinav Venkat, Dr. Avinash Sharma (CVIT)

My contribution was in implementing the heat kernel based diffusion model (baseline) for predicting the functional (FC) from the structural connectivity (SC), and vice versa.

An extension of this work was later published in NIPS Workshop, 2016, in which a multiple diffusion kernel model (MKL) was proposed to perform a better estimation of FC from SC.