Embedded Files
1st decade of the 21st century - The early stages
INRIA Sophia Antipolis & IRCAD collaboration
Keywords : Laparoscopy, Augmented Reality, Image-guided surgery, Interventional Radiology, Structured Light, in vivo evaluation
Videos
Laparoscopic Augmented Reality
INRIA / IRCAD
A la croisée des cheminsNicholas Ayache biography
Laparoscopic Augmented Reality (VRST 2005)
This paper presents a low cost guidance system for laparoscopic surgery. This system tracks in real-time the laparoscopic instruments and registers at 10 Hz a 3D preoperative model of the patient. The tracking error is close to 1 mm for instrument tips, and under 1 mm for endoscopic superimposition.
Visual evaluation of the reprojection in the endoscopic view. The green ball would be perfectly centered on the square corners if there were no noise. In practice, due to both tracking and endoscopic calibration errors, there is a small discrepancy. One can asses it to 1 mm in the image (size of the square side: 4 mm)
Illustration of the current system interface. Bottom left: the software displays one of the video images on which the phantom CT model and the needle model are superimposed. Right: the system displays the relative position of the needle with respect to the CT model. The point of view and the zoom can be adapted to radiologist requirements. Top left: to make the needle orientation easier for the radiologist, we show a virtual view of a camera located at the needle tip and oriented along its axis.
Structured Light Registration (SPIE 2008)
To register the preoperative image in the camera frame, we firstly reconstruct the patient skin in 3D using the structured light. Then, the surfacic registration between the reconstructed skin and the segmented skin from the preoperative image is performed using the Iterative Closest Point (ICP) algorithm. Ensuring the quality of this registration is the most challenging task of the system. Indeed, a surfacic registration cannot correctly converge if the surfaces to be registered are too smooth. The main contribution of our work is the evaluation on patients of the conditions that can ensure a correct registration of the preoperative skin surface with the reconstructed one. Furthermore, in case of unfavourable conditions, we propose a method to create enough singularities on the patient abdomen so that the convergence is guaranteed. In the coming months, we plan to evaluate the full system during standard needle insertion on patients.
Illustrative example of an augmented reality view of the patient using the registration computed with the ICP algorithm.
Example of radio-opaque marker automatic detection in a video image of a human abdomen. The markers extracted are indicated with black crosses.
Exemple of the external AR view associated to the AV view. Bottom: augmented view of a reference camera. One can see the instrument models superimposed on the real ones. The green and grey instruments are touching two markers. Top: corresponding virtual view from the user point of view. The surgeons evaluated the guiding accuracy of the instruments to 2 mm for skin markers.
Example of a video image of a patient during the intervention. One can see the pattern rigidly attached to the inserted needle. The radio-opaque markers are stuck on the patient skin and removedafter the CT acquisition for sterility reasons. To be able to locate them in the video images after they are retrieved, we print a black dot inside their central hole. Cables are hiding several markers and disturb their automatic extraction.
Example of image obtained when the structured light pattern is projected on the patient skin.
Example of the surface reconstruction provided by the structured light component.
Extraction of the patient skin from the CT acquisition.
Page updated
Google Sites
Report abuse