Mesh2Animation: Unsupervised Animating for Quadruped 3D Objects

Animating quadruped 3D objects, such as chairs and tables, typically involves three steps (Rigging, Skinning, and Retargeting) in the traditional computer graphics pipeline. Commonly, prevailing methods for a specific step are conceived in isolation. For rigging and skinning steps, prevailing approaches have typically relied on optimization-based methods, which tend to be slow and susceptible to variations in 3D mesh surfaces. For the retargeting step, the obtained results often fall short of expectations, especially when dealing with dissimilar source and target skeletons, leading to issues like joint twisting. The devised procedure is time-intensive, leading to a final pipeline that is notably intricate. To this end, we present a unified framework, termed Mesh2Animation, providing an end-to-end solution to the above problem. In Mesh2Animation, a learning-based method is proposed for quadruped 3D skeleton estimation. We introduce both skeleton-level and mesh-level loss, so that the aforementioned steps, i.e., rigging, skinning and retargeting, can be optimized simultaneously. Specifically, a general predicted estimation from the rigging step can initialize the skeleton, making the skinning step faster and more accurate, further leading the retargeting step to a better solution. Finally, the rigging, skinning and retargeting process are optimized simultaneously on the static and temporal constraints. Additionally, we can construct a novel animating dataset termed ShapeNet2Animation (SN2Animation ) based on the proposed method, which shows potential application for pose transfer. Qualitative and quantitative results on SN2Animation, ShapeNet, Object3D and ModelNet10 datasets for animation demonstrate that our method achieves competitive performance and shows promising generalization ability on quadruped 3D objects.

Overview. Starting with a quadruped object (taking a chair as an example) and quadruped motion sequences (taking walking action as an example) on the desired skeleton hierarchy, Rigging part predicts the bone hierarchy and joint adjacency through a skeletal point prediction network with the help of symmetry and ratio consistency. Skinning part separates skinning weights into different regions and smooths the corresponding weights respectively. Retargeting part transfers the quadruped motion sequences to produce diverse animations.

We provide animated results for quadruped objects(chairs and tables) and  winged objects(airplanes). Meshes are sampled from ShapeNet, Object3D, ModelNet10. Reference motions include walking, running, jumping, turning around and lying for quadruped objects, flying for winged objects. Notice  that we downsample the video clips and use lower resolutions to ensure faster webpage loading.