Abstract

Accurate 3D sensing of suturing thread is a challenging  problem in automated surgical suturing because of the high state-space complexity, thinness and deformability of the thread, and possibility of occlusion by the grippers and tissue. In this work we present a method for tracking surgical thread in 3D which is robust to occlusions and complex thread configurations, and apply it to autonomously perform the surgical suture "tail-shortening" task: pulling thread through tissue until a desired "tail" length remains exposed. The method utilizes a learned 2D thread-detection network to segment suturing thread in RGB images, then reconstructs the thread in 3D as a NURBS spline by triangulating the detections from two stereo cameras. Once a 3D thread model is initialized, the method tracks the thread across subsequent frames. Experiments suggest the method achieves a 1.33 pixel average reprojection error on challenging single-frame 3D thread reconstructions, and an 0.84 pixel average reprojection error on two tracking sequences. On the tail-shortening task, it accomplishes a 90% success rate across 20 trials.

Reference

Self-Supervised Learning for Interactive Perception of Surgical Thread for Autonomous Suture Tail Shortening

Vincent Schorp, Will Panitch, Kaushik Shivakumar, Vainavi Viswanath, Justin Kerr, Yahav Avigal,  Danyal Fer, Lionel Ott, Ken Goldberg. 

IEEE International Conference on Automation Science and Engineering (CASE).  Auckland, New Zealand. August 26-30, 2023.

Authors

Vincent Schorp (1, 2), Will Panitch (1), Kaushik Shivakumar (1), Vainavi Viswanath (1), Justin Kerr (1), Yahav Avigal (1), Danyal M Fer (3), Lionel Ott (2), Ken Goldberg (1)  

(1): AUTOLAB at University of California, Berkeley

(2): Autonomous Systems Lab at ETH Zurich

(3): MD, Department of Surgery, University of California San Francisco East Bay

Support or Contact

Please contact Vincent Schorp at vschorp@berkeley.edu