While our results with a $500 AR system and open-source modeling files are promising, further research is needed to achieve accuracy levels near 100%, as expected with fluoroscopy. One future consideration is the incorporation of convolutional neural networks (CNNs) for the reconstruction of MRI scans into 3D CAD models for AR applications. Research has shown that CNNs can reconstruct 3D models while simultaneously reducing image noise, potentially leading to greater accuracy in real-world scenarios [28], [29]. Additionally, our experiment did not account for real-world variables, such as patient specific factors like breathing movements, sweating, and medical anomalies. Other future goals will involve integrating an accurate spinal model, advanced CAD techniques, and real MRI data to simulate practical procedures. These challenges could be addressed in future designs by reconstructing proper spinal models in CAD using advanced medical imaging techniques to enhance system robustness in operating room environments.
Additionally, the next step would involve incorporating our ArUco tracking system into our AR surgical navigation system. Currently, the AR navigation system uses the provided Unity SDK packages already provided by Meta and does not use the ArUco marker tracking system to track and position a virtual 3D model where the ArUco marker is located due to camera restriction set by the Meta Quest 3. Thus, a future team can develop a more coherent system that combines ArUco tracking with augmented reality when the Meta Quest 3’s camera is accessible in the future. This step would also be more feasible if a future team has the budget to buy the more expensive HoloLens 2 which is friendlier for software development and does not restrict access to its camera.
Another future direction would be to include surgical tool tracking which would enable the surgeon to know the exact position and orientation of the surgical tool they’re using. Our current system lacks real-time needle tracking, so it’s currently difficult to know exactly where the needle is located and how it is oriented in real-time. Thus, incorporating surgical tool tracking would help to create a more robust and useful surgical navigation system. One potential method of incorporating needle tracking involves placing a unique ArUco marker on each face of a multifaceted 3D object such as a dodecahedron and attaching the 3D object to the needle. This approach would allow at least one ArUco marker to always be visible and tracked regardless of the orientation of the 3D object and the needle; a limitation of a single ArUco marker is its limited viewing angle for it to be trackable: a marker rotated past a 180 degrees away from the camera prevents the marker from being tracked.