Embedded Files
Problem space
Problem space
Specialized surgeries are often needed in places where there is no access to surgeons: rural areas, or after catastrophic events. Having timely access to telemedicine will allow specialized surgeons to help novices in need save new lives.
Solution
Solution
We propose to develop a Spatial Computing system to guide surgeons through specialized procedures with MagicLeap (ML)
Our proposed system will provide expert help through:
- holographic representation of experts appearing on the other side of the patient’s bed
- extra pair of hands to provide contextual help through remote gestures;
- specific visual aids (text, images and videos) in the field of view of the novice surgeon;
- telestration of the visual field with sketches, annotations, etc. to guide the surgeon.
Additional depth cameras could be located in the environment (e.g. Intel Real Sense) and integrate with ML’s view to provide context to remote surgeons with third person views.
LeapMotion finger tracking sensors can be deployed to enable accurate tracking of finger movements during surgical procedures.
The combined views allows remote experts to understand specific help needed by novice surgeons and interact with them to provide AR guidance.
Integration of the finger tracking on the remote side will allow very specific hand manipulation to be communicated to the bedside using projected hands holograms.
We are working in collaboration with clinics, operating rooms, simulation training centers, surgeons, and more at the UCSD’s School of Medicine and the Naval Medical Center in San Diego and have access to the environment and expertise necessary to perform this project. In addition, we have an ongoing project exploring the use of HoloLens in combat situation with the Navy.
Report abuse