SuperPoint-E: Tracking adaptation to improve superpoint for 3D reconstruction in endoscopy

This work was developed during the EndoMapper project, at the University of Zaragoza.

Brief Description

Endoscopy is the gold standard procedure for early detection and treatment of numerous diseases. Obtaining 3D reconstructions from real endoscopic videos would facilitate the development of assistive tools for practitioners, but it is a challenging problem for current Structure From Motion (SfM) methods. This work presents a novel learned model for feature extraction in endoscopy, called SuperPoint-E, which improves upon existing work using recordings from real medical practice. SuperPoint-E is based on the SuperPoint architecture but it is trained with a novel supervision strategy. The supervisory signal used in our work comes from features extracted with existing detectors (SIFT and SuperPoint) that can be successfully tracked and triangulated in short endoscopy clips (building a 3D model using COLMAP). We validate the effectiveness of our model for 3D reconstruction in real endoscopy data. 

• Publication: O. León Barbed, José MM Montiel, Pascal Fua, and Ana C. Murillo. Tracking adaptation to improve superpoint for 3d reconstruction in endoscopy. In International conference on medical image computing and computer-assisted intervention (MICCAI), pp. 583-593. Cham: Springer Nature Switzerland, 2023. https://doi.org/10.1007/978-3-031-43907-0_56 PDF

• Code and model

• Supplementary Information. There is an extended version with more details and extended evaluations at this report.

Preliminary work on this project published at

• Barbed, O.L., Chadebecq, F., Morlana, J., Montiel, J.M.M., Murillo, A.C. (2022). SuperPoint Features in Endoscopy. MICCAI Workshop on Imaging Systems for GI Endoscopy. Lecture Notes in Computer Science, vol 13754. Springer, Cham. https://doi.org/10.1007/978-3-031-21083-9_5     ArXiv