Meta Quest 3
How is Augment Reality (AR) used?
Adds virtual elements to the real world
A head-mounted display such as the Meta Quest 3 is used to display AR
Can be used to overlay a 3D virtual patient spine model
Augmented Reality Development Process
Create a 3D model to use
Experimental Set Up
To test the effectiveness of our augmented reality application for surgical use, we performed a simple experiment to test its accuracy and success rate. For this experiment, we wanted to examine how often and accurately an average user can hit the red target point shown in this diagram with only the guidance of our AR surgical navigation system. To do so, we used a simple ballistic gel cube containing a 3D printed cylinder. The ballistic gel was used to replicate human tissue and the 3D printed cylinder was used to represent the spine. Inside this ballistic gel cube model is a simple LED feedback circuit which turns the LED on when the needle touches the metal detector connected to the circuit; the metal detector is the target point the user tries to hit using the needle. Using this circuit, we’re able to know whether or not we successfully reached the target point inside the ballistic gel.For the experiment, we tested three injection points on the ballistic gel cube labeled as 1, 2, and 3, and a total of 60 trials were conducted: 30 trials with the AR navigation system and 30 trials without the AR navigation system. To reduce single user bias, two different people performed the experiment. If the needle successfully reached the target point, the LED would turn on and the trial was counted as successful. Otherwise, it was considered unsuccessful.
Experimental Results
Average success rate (%) was significantly higher using the AR navigation system
~20% higher with system than without system
Success rate for each of injection points 2 and 3 were also higher using the AR navigation system
Publication:
J. Hui, S. Lu, E. Lee, D. Tsui, F. E. Talke. “Assessment of an integrated computer vision and augmented reality approach for improved minimally invasive surgeries,” ACM 23rd annual Symposium on Computer Animation (SCA), submitted, 2024.