Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale

Code coming soon! 

Non-Rigid Structure-from-Motion

(potential to be mapping part of deformable SLAM)

Non-rigid structure-from-motion (NRSfM) addresses the challenge of reconstructing the 3D shapes of deforming objects from multiple calibrated monocular images. It is a fundamental problem in 3D computer vision, with applications ranging from entertainment to modern surgery. The NRSfM algorithm reconstructs 3D shapes in local camera coordinates, inherently intertwining camera motion with object deformations. This concept closely aligns with deformable visual SLAM, which aims to localize a robot and map its environment, even in dynamically deforming scenarios. NRSfM has the potential to play a pivotal role in overcoming the mapping challenges associated with deformable visual SLAM. By integrating tools for robot pose estimation, NRSfM can enhance mapping consistency through information fusion, paving the way for significant advancements in deformable visual SLAM research.

Conclusion

This paper introduces a novel theoretical framework, Con-NRSfM, for addressing conformal NRSfM problems. The framework leverages the rotational invariance of connections to express conformal constraints without relying on assumptions about surface geometry. The proposed formulation is solved using selected image warps, a separable parallel graph optimization strategy, and a self-supervised convolutional network. Our method has been extensively tested on a wide range of synthetic and real datasets featuring diverse baseline viewpoints and deformations. The results demonstrate that Con-NRSfM outperforms state-of-the-art methods in both accuracy and robustness.