This site is still in progress. I’m building it to share my thoughts and studies, but for now, it’s mainly used for my MET coursework.
Sean Jeon
Updated 2025.06.29
Sumitted 2025.06.29
4th year medical students face challenges mastering complex musculoskeletal anatomy and pathologies due to limited exposure, lack of targeted practice, and inconsistent feedback opportunities. This product proposes a scaffolded immersive learning tool evolving from a fully analog, hands-on prototype, through interim VR and AR prototypes, to a full-featured AR/MR tool using smart glasses. This approach aligns with evidence-based best practices for immersive medical education and maximizes engagement and learning efficiency.
As part of the design process, I conducted user interviews using ChatGPT to simulate authentic perspectives from key stakeholders. In these sessions, ChatGPT took on the roles of a 4th-year medical student and an experienced orthopedic surgeon. By asking targeted questions and exploring detailed responses from both simulated viewpoints, I was able to gain valuable insights into learners’ challenges, expectations, and desired features, as well as expert opinions on anatomical accuracy, procedural fidelity, and practical feasibility. This approach helped me better understand user needs, refine the learning tool’s features, and anticipate potential barriers before building the prototype.
As an educational tool developer, i want to interview 4th year medical studentto understand their study pttern and life, pain point. Act as a 4th year medical student. This is basic description of persona. You’ve been assigned to support a local physician in the development of a new learning tool for students in medical school. The physician is an orthopedic surgeon and an expert in their field. You’ve learned that the physician’s department just purchased 10 new AR-capable tablets as part of a one-time grant funding opportunity. The department is excited to try using the devices because it's the first time anyone in the department has used immersive technologies like AR to teach. Because of this, the physician is very interested in considering using AR as a platform for the learning experience. After an initial meeting with the physician, you learn a little more about their teaching context and their goals. Learners are in their 4th and final year of medical school, and they have historically had difficulty diagnosing certain musculoskeletal diseases. When curricular leads investigated why it was such a challenge, it was determined that a number of factors contributed to the difficulty: Complex anatomical structures: learners found it difficult to visualise the complicated anatomy and their corresponding pathologies. Test-scores and tutorial session feedback support also supported the presence of an understanding deficit. Lack of practice: learners were only exposed to several orthopaedics case-based learning sessions prior to their 4th year. Learners report having free time in their schedules for more practice but struggle to find targeted resources to use, as many practice cases do exist but they focus more on general clinical cases across all of medicine as opposed to specific orthopaedic subject matter. Lack of feedback: learners wanted more opportunities to ask questions and receive feedback on particular musculoskeletal diseases, especially from an expert in the field. Variable Exposure and Access: learners are distributed across the region, learning in both urban centres and remote rural clinics, so learners have widely variable exposure to musculoskeletal diseases while on their hospital rotations in their 3rd year of study. The diseases they see depend on the patient population that they encounter while shadowing a practising doctor. The physician says learners are interested in immersive learning approaches, but admits that most only have experience with using AR-type immersive on their phones. The physician hopes to be present when learners use the tool, but they also acknowledge that they have a busy schedule, so they would like some sort of automated data collection component that could inform them of learners’ use and learning from the tool. When asked about any staff who could help support the use of the new tool, the physician responded by saying that administrative support is “limited” on a good day. They also mention that this tool could potentially have secondary benefits to patients. They mention how patients often struggle to understand and grasp the nature of their ailments, leading to shortened recovery periods and a higher risk for re-injury.
ChatGPT said:
Of course—let’s do a role-play. I’ll answer as if I’m a 4th-year medical student fitting your description:
Hi! Thanks for meeting with me. I’m Alex Kim, a 4th-year medical student rotating through orthopedics this term. I’d be glad to share how I study and where I struggle.
What does your typical day look like during your ortho rotation?
My days start early, around 6:30–7:00 AM. We do rounds with the team until about 9:00, followed by OR time or clinic until 4:00 or 5:00 PM. In between, if there’s downtime, I try to review anatomy or cases on my phone. Evenings are spent reading or practicing cases—maybe 1–2 hours if I’m not too exhausted. Weekends are the best time for deeper study.
What resources do you currently use to study musculoskeletal diseases?
Mostly textbooks like Hoppenfeld’s Orthopaedic Physical Examination, question banks, and online resources like Orthobullets. For anatomy, I sometimes use 3D apps on my phone, but they’re limited. Videos on YouTube help too. But there’s no single tool that brings it all together in a focused way.
What do you find most challenging about learning orthopedics?
Visualizing complex joints in 3D—like the carpal bones or acetabulum fractures.
Remembering patterns of injury and corresponding physical exam findings.
Linking imaging to real anatomy: X-rays and MRIs still feel abstract.
Feeling confident diagnosing rare conditions because I rarely see them in-person.
How do you feel about the idea of using AR to study?
That sounds amazing, honestly. If I could manipulate a virtual joint, see a fracture appear or rotate it to compare normal vs. abnormal, it would help me remember better. It would also be useful when prepping for OSCEs or oral exams.
How much free time do you realistically have to use a new tool?
I’d say I could fit in 15–30 minutes most days. More on weekends. But it has to be convenient: I’d use it on my tablet or phone during breaks or before bed.
What kind of feedback would help you most?
Immediate feedback on practice cases—like if I diagnose a virtual patient incorrectly, I want to know why and what I missed. Also, an option to ask a question and get a response later from a physician would be great.
How comfortable are you with AR or immersive technology?
I’ve used AR apps on my phone—like Snap filters or Google Translate’s live camera—but never in a learning context. I’d probably need a short tutorial or orientation to get started.
If you had a magic wand, what would your ideal ortho learning tool look like?
Interactive 3D anatomy I can rotate, zoom, and dissect.
Clinical cases with branching scenarios.
Integrated quizzes with explanations.
A way to see imaging (X-ray, CT, MRI) side-by-side with 3D models.
Track my progress and suggest topics I’m weak in.
How do you see this kind of tool helping patients?
If it could show them what’s wrong with their bones or joints in 3D, I think patients would understand injuries or surgeries better. I’ve noticed they often nod but don’t really get what’s happening.
Do you see any barriers to using this tool?
If it’s only available in the hospital, it’ll be hard—I need it on my own device.
If it’s buggy or crashes, I’ll probably give up quickly.
If it takes too long to learn how to use it, I won’t have time.
Would you like me to continue with more details about motivation, emotional struggles, or specifics on how I’d like cases structured?
You said:
“So Alex, can you describe your day-to-day life?”
00:03
ChatGPT said:
Absolutely! So, a typical day for me starts pretty early, around 6:30 or 7:00 in the morning. We begin with rounds, seeing all the patients on the ward with the orthopedic team. That usually takes a couple of hours, and then we might move on to either the operating room or the clinic for the rest of the day. There's a lot of variety, but it keeps me on my toes!
You said:
“How do you currently practice your orthodontic surgery?”
00:03
ChatGPT said:
So, right now, I try to practice in a few different ways. I spend some time reviewing textbooks and watching videos to get a better understanding of procedures and anatomy. Whenever there's an opportunity, I like to observe surgeries and assist in the OR, which is super helpful for hands-on experience. I also run through practice cases and quizzes whenever I can, but I definitely feel like more targeted practice would be great!
You said:
“What do you mean by targeted practice?”
00:02
ChatGPT said:
By targeted practice, I mean having resources that focus specifically on orthopedic cases and the kinds of conditions we actually see in the field. So instead of general medical cases, it would be great to have more specialized scenarios that really help us dive deep into musculoskeletal issues, and get feedback that’s directly related to those conditions.
You said:
“What kind of feedback do you appreciate to study orthopedics?”
00:05
ChatGPT said:
I really appreciate feedback that's immediate and specific. So, for example, if I'm working through a case and I miss something in my diagnosis, I love when the feedback tells me exactly what I overlooked and gives me some guidance on how to think about it differently next time. It’s also super helpful to get feedback from an expert, but even automated feedback that’s detailed and tailored to ortho cases would be fantastic.
You said:
“Here are how X-ray images help you study.”
00:03
ChatGPT said:
X-ray images are really helpful because they give a real-world view of what's happening inside the body. They help me connect what I'm studying in textbooks or 3D models to actual patient cases. When I see an X-ray, it helps me understand the alignment of bones, identify fractures, or spot any abnormalities. It’s like a puzzle piece that helps everything click into place!
You said:
“In your opinion, what is the best way to study the layer hierarchy of the human body?”
00:07
ChatGPT said:
I think one of the best ways is to start with a big-picture overview and then gradually break it down into smaller components. So, beginning with systems—like the skeletal system, muscular system—and then zooming into individual organs, tissues, and cells. Using interactive tools, like AR or 3D models, can make it easier to visualize how everything layers together. And repetition definitely helps—it's all about reinforcing that hierarchy over time!
You said:
“If we have to use AL, how are you going to use the AL?”
00:04
ChatGPT said:
I think using AR would be fantastic for creating interactive, immersive experiences. For example, we could overlay 3D anatomical models onto a physical space, allowing students to walk around and explore structures from every angle. It could also be used to simulate clinical scenarios, like diagnosing a virtual patient who presents with a musculoskeletal issue, and students could practice their diagnostic skills in real-time. It
I think AR could be super helpful for visualizing complex anatomical structures in a really interactive way. For example, I could use AR to overlay a 3D model of a joint or a bone onto a real-life patient or even just onto my textbook, so I can see how everything lines up. It would also be awesome for practicing procedures virtually or running through case scenarios in a more immersive and engaging way.
You said:
“What would be the great pain point when you study those kind of things?”
00:04
ChatGPT said:
I think one of the biggest pain points is definitely the complexity of the material. It can be really challenging to visualize and remember all the intricate details of anatomy and pathologies. Another pain point is just the limited time we have to really dive deep and practice these skills, especially when we’re balancing rotations and studying for exams. Plus, not having enough hands-on practice or immediate feedback can make it harder to feel confident in our understanding.
You said:
“When you study those kind of things, what does space look like?”
00:04
ChatGPT said:
The space where I study is usually pretty quiet and organized. I like to have my desk set up with all my resources, like my textbooks, my tablet, maybe some anatomical models if I have them. I try to keep it well-lit and comfortable, maybe with a bit of background music to help me focus. It's definitely a space that I try to keep as distraction-free as possible so I can really focus on the material.
You said:
“When you're using AR or those kinds of digital tools, do you need hand-free interactions so you can use your hand to interact with the models?”
00:08
ChatGPT said:
Oh, definitely! Hands-free interactions would be super helpful, especially when I'm trying to manipulate 3D models or go through a procedure step-by-step. Being able to use gestures or voice commands to interact with the AR content would make it a lot more intuitive and keep my hands free for taking notes or referencing other materials.
You said:
“이 자료를 통해, 어떤 반응을 찾고 있는지, 그리고 어떤 반응을 찾고 있는지에 대해 설명해 주세요.”
00:04
ChatGPT said:
I’m really looking for feedback that’s immediate and specific to what I’m working on. So if I’m diagnosing a virtual patient or studying a particular anatomical structure, I’d love to know right away if I got something right or if I missed something important. It’s also great to have feedback that includes explanations or suggestions for improvement, so I can learn from my mistakes and get better over time.
You said:
“이 도구를 사용하면서 필요한 추가 정보는 무엇일까요?”
00:05
ChatGPT said:
I think having some context about real-world cases or clinical scenarios would be super helpful. For instance, understanding a patient’s history, their symptoms, and what led to their condition can make the learning experience more comprehensive. It’s also great to have references or links to additional resources, like articles or videos, that I can dive into if I want to learn more about a particular topic. Having that extra layer of information can really help solidify my understanding.
You said:
“When you are assisting your surgeon, what kind of information is more helpful?”
00:06
ChatGPT said:
When I'm assisting a surgeon, the most helpful information is definitely anything that helps me anticipate what comes next. So, knowing the steps of the procedure really well, understanding the instruments that will be needed, and having a clear idea of what the surgeon expects in terms of assistance is crucial. It's also super helpful to have a good grasp of the patient's background and the specifics of their case so I can better understand the context of what we're doing and why.
You said:
“연결되면, 환자와 연결되기 어렵나, 의사와 연결되기 어렵나요?”
00:07
ChatGPT said:
I think it can depend on the situation, but communicating with patients can sometimes be more challenging because you really want to make sure you're explaining things in a way that’s easy to understand, especially if they’re feeling anxious or worried about their condition. With surgeons, the communication is often more fast-paced and precise, especially during procedures, so it requires a different kind of focus. Both have their own unique challenges, but they’re both super important skills to develop.
You said:
“You are studying Non-Digital Ways, How You Study.”
00:03
ChatGPT said:
When I'm studying without digital tools, I usually rely on good old-fashioned textbooks, taking handwritten notes, and maybe using flashcards to quiz myself. I also like to draw out diagrams or sketches to help visualize anatomical structures. Study groups are also really helpful, so discussing cases or quizzing each other with friends can make a big difference.
You said:
“만약에 매직파워를 가지고 있다면, 어떤 툴을 추가하고 싶나요?”
00:05
ChatGPT said:
Oh, that's a fun question! If I had magic powers, I think I'd love to have a tool that could instantly create a 3D, interactive hologram of whatever I’m studying, right in front of me. It would be amazing to just point at something in a textbook and have it come to life so I could rotate it, zoom in, and really explore it from every angle. That would make studying so much more engaging and fun!
You said:
“If the 3D hologram shows the surgical procedure in 3D hologram, so you have the immersive feeling you are in the surgical room, does it help?”
00:10
ChatGPT said:
Oh, absolutely! That would be an incredible way to learn. Having that immersive feeling, like I'm actually in the surgical room, would make it so much easier to understand the flow of the procedure and the spatial relationships between everything. It would help build confidence and muscle memory, and it would definitely make studying and practicing a lot more engaging and effective.
You said:
“How important is the muscle memory and hand haptic feedback?”
00:04
ChatGPT said:
Muscle memory and haptic feedback are super important, especially in something like surgery where precision and technique really matter. When you're practicing procedures, building that muscle memory helps you perform movements more naturally and confidently over time. And haptic feedback, like feeling resistance or textures, can really reinforce that learning because it mimics what you'd experience in real life. So having that physical component can make a huge difference in mastering surgical skills.
You said:
“OK, thank you.”
00:00
ChatGPT said:
You're welcome! I'm glad I could help. If you have any more questions or need anything else, just let me know!
You said:
Now you are the surge. Who want to make this tool lets star the interview
ChatGPT said:
Perfect—switching roles! I’ll answer as the orthopedic surgeon leading this AR learning tool project.
Hi, I’m Dr. Morgan Lee, orthopedic surgeon and faculty member at the medical school. Thanks for taking the time to chat with me about developing this tool. I’d love to share my vision and the realities of teaching orthopedics.
✅ What motivated you to pursue an AR-based learning tool for your students?
I’ve seen students struggle year after year with visualizing complex joint anatomy and rare conditions they don’t get to see on rotation. I believe AR can bridge that gap by letting them interact with 3D models anytime, anywhere—so they’re not limited by the cases they happen to encounter in clinic. It’s an opportunity to bring the anatomy to life.
✅ What are your main goals for this tool?
First, I want it to improve students’ diagnostic skills, especially for challenging musculoskeletal diseases. Second, I’d like it to give them more practice opportunities outside of lectures and rounds. And third, I hope it provides feedback that helps them learn from mistakes without fear of being judged in front of peers or staff.
✅ How do you imagine integrating this into your existing curriculum?
I’d like to start by offering it as a supplemental resource during their ortho rotation, so they can use it during downtime or for self-study. If it works well, we might integrate it into structured case-based tutorials or even assessments.
✅ What specific features do you want the tool to include?
High-resolution, manipulatable 3D models of bones and joints
Layer toggles to show ligaments, tendons, and nerves
Clinical cases with patient history, imaging, and interactive diagnosis
Immediate feedback on case outcomes
Data collection on student progress (like time spent, accuracy) so I can review trends.
✅ What challenges do you anticipate in implementing this tool?
The biggest challenge is my time—my schedule is packed, and I don’t have extra staff to coordinate training or tech support. Also, our learners are spread out regionally, so we need the tool to work well independently without constant supervision. And of course, technical issues or lack of buy-in could slow adoption.
✅ How comfortable are you with AR and immersive technologies?
I’m comfortable with AR on a basic level—like mobile apps—but I’ve never used it professionally. I’d need a partner who can help design and maintain the technical side while I focus on content and clinical accuracy.
✅ How would you like to see feedback presented to students?
I think immediate, case-specific feedback is ideal, but I’d also like a summary dashboard where students can see patterns in their performance. This way they can identify weak areas to revisit. If possible, automated explanations with visuals highlighting mistakes would be powerful.
✅ What about patient education—do you see a role for this tool?
Definitely. Patients often don’t grasp their injuries or the importance of rehab. A simplified version of the tool could help them see what’s happening inside their body and what recovery involves. I think it could improve compliance and outcomes.
✅ What would success look like for you?
If students who use the tool show better understanding and more confidence diagnosing musculoskeletal conditions compared to those who don’t, that’s a win. I’d also like to see fewer mistakes on exams or during patient assessments.
✅ Anything else you’d like developers to know?
Keep it simple to start. Even if it’s just a few high-quality cases and models, we can
You said:
“Hi, Dr. Morgan Lee. I want to know about how you teach your student for this orthopedic surgeon.”
00:08
ChatGPT said:
Hi there! Great question. My approach to teaching orthopedics is very hands-on and case-based. I like to start with foundational knowledge—making sure students understand the anatomy and biomechanics of the musculoskeletal system. From there, we dive into real-life cases, where they get to review patient histories, look at imaging, and come up with differential diagnoses. I also encourage them to spend time in the clinic and the OR, observing and assisting with procedures. And of course, feedback is key—I make sure to give them constructive and immediate feedback so they can continuously improve and gain confidence.
You said:
Transcript Unavailable
00:04
ChatGPT said:
From what I've seen, students tend to get really excited when they have those "aha" moments—like when something complex suddenly clicks for them. They also get really engaged when they get to work with real patients or see the impact of their learning in a clinical setting. And any time we introduce new technology—like AR or innovative tools—they get pretty excited about that as well, because it adds a whole new dimension to their learning experience.
You said:
“Like you mentioned about immersive tool, but to create aha moment, what kind of current tool are you using it?”
00:06
ChatGPT said:
In our current setup, we rely a lot on traditional methods like cadaver labs, which give students a hands-on experience with real anatomy. We also use a lot of imaging tools, like X-rays and MRIs, to help students connect what they see in textbooks to real-life cases. And we do a lot of case-based learning, where students work through real patient scenarios to piece together diagnoses. All of these tools together really help create those aha moments for the students.
You said:
“실제 개바를 통해 접근할 수 없을 경우, 어떤 다른 도구를 사용할 수 있을까요?”
00:05
ChatGPT said:
In cases where students can't access real-life cadavers, we often turn to high-quality anatomical models or even digital dissection tools. There are some really great virtual anatomy platforms that allow students to explore the human body in 3D. These tools can simulate dissections and let students peel back layers to see different structures. Plus, we can use detailed textbooks, atlases, and even augmented reality apps to help bring the anatomy to life in a different way.
You said:
“Could you please give me some example of the tools you are talking about?”
00:04
ChatGPT said:
Sure! One popular tool is something like the Anatomage Table, which is a virtual dissection table that allows students to explore full-body cadaver scans in 3D. Another great resource is Complete Anatomy, which is an app that provides highly detailed 3D models of the human body and allows for interactive learning. There are also platforms like Visible Body, which offer a range of anatomy apps that can be used on tablets or computers. And for augmented reality, apps like Human Anatomy Atlas AR can project 3D models into the real world, making it easier to study anatomy in an interactive way.
You said:
“While you are teaching, is there any language barriers you experience?”
00:03
ChatGPT said:
In some cases, yes, language barriers can come up, especially if some students are non-native speakers of the primary language of instruction. In those situations, I try to use clear, simple language and check in often to make sure everyone understands the material. Visual aids, like diagrams or models, can really help bridge those gaps as well. Ultimately, creating an inclusive and supportive environment is key, so that everyone feels comfortable asking questions and participating.
You said:
“당신이 가르치는 동안 당신의 고통점은 무엇일까요?”
00:02
ChatGPT said:
I think one of the biggest pain points is definitely balancing time and resources. Teaching while also managing a busy surgical schedule means there’s often limited time to spend one-on-one with students. Another challenge is making sure that the material is engaging and accessible to all students, regardless of their learning styles. And of course, keeping up with the latest advancements in both medical knowledge and teaching technology can be a challenge, but it's also part of what makes it exciting!
You said:
“What is your main teaching pedagogy for those topics?”
00:03
ChatGPT said:
My main teaching pedagogy is really centered around active, experiential learning. I believe that students learn best when they can engage directly with the material, whether that’s through hands-on practice, working through real-life cases, or participating in simulations. I also emphasize collaborative learning, encouraging students to work together, discuss, and teach each other, which helps reinforce their understanding. And, of course, I try to integrate technology and innovative tools whenever possible to make learning more dynamic and engaging.
You said:
“What will be your daily life journey?”
00:02
ChatGPT said:
Well, as a surgeon and educator, my daily life is definitely a busy one! My mornings usually start early with rounds, checking in on patients who have had surgery or are preparing for procedures. Then it’s often a mix of time in the operating room, teaching in the classroom or the lab, and mentoring students one-on-one.
You said:
Transcript Unavailable
00:00
ChatGPT said:
Every day is a bit different, but that variety definitely keeps things interesting! And in between all of that, I try to keep up with the latest research and advancements in orthopedics, as well as stay connected with colleagues and students. It's definitely a full schedule, but it’s incredibly rewarding!
You said:
“How many students are you teaching at once?”
00:02
ChatGPT said:
The number of students I'm teaching at once can vary quite a bit depending on the setting. In a lecture or a seminar, I might have a larger group, maybe 20 to 30 students. But in more hands-on settings, like in the OR or small group tutorials, it's usually a smaller group, maybe just a handful of students at a time. This allows for more personalized interaction and direct mentorship, which I think is really valuable.
You said:
“Okay, thank you.”
⚠️ Assumptions
The department’s AR tablets are capable of running SLAM-based AR apps with stable performance.
Students will receive a short onboarding session covering AR controls, best practices, and troubleshooting.
High-quality animated surgical sequences will be created in collaboration with experienced orthopedic surgeons.
Image generated by ChatGPT
Components: 3D printed bones + hand-drawn or digitally printed paper muscle and soft tissue illustrations, cut to anatomical shapes.
Interaction: Learners manually layer muscle anda tissue illustrations onto 3D printed bones to physically construct anatomical relationships, removing and adding layers to explore spatial organization.
Learning Outcome: Build foundational spatial awareness through tactile, analog interaction ideal for low-barrier, device-free practice.
Image generated by ChatGPT
Platform: Oculus Quest 3 VR headsets.
Components:
Three photogrammetry scans of donated cadaver specimens representing superficial, intermediate, and deep anatomical layers with MR annotations, offering high-fidelity, realistic anatomy in VR.
3D puzzle pelvis model designed for VR interaction to test knowledge of complex bony relationships.
Interaction:
Students navigate between anatomical layers and assemble/disassemble the 3D pelvis puzzle, building spatial comprehension through immersive VR exploration.
Purpose: Validate analog interaction concepts in MR, and gather data on students’ ability to manipulate layered, detailed anatomical content.
Photogrammetry Scan Example from VanVR App
Pelvis Puzzle Interaction Example from VanVR App
Platform: Department’s AR-capable tablets (10 units).
Technology: Uses SLAM (Simultaneous Localization and Mapping) to precisely align virtual content with physical 3D printed bones, enabling accurate AR overlays in real-world space.
Interaction:
Students aim AR tablets at 3D printed bones to activate animated surgical procedures, viewing step-by-step operations from multiple angles.
Learners can walk around or reposition themselves to see different perspectives of surgical techniques superimposed on the bones, leveraging SLAM to maintain consistent, stable alignment.
Learning Outcome: Enhance understanding of surgical procedures in context of actual anatomy, build confidence in procedural steps, and bridge the gap between static anatomical knowledge and dynamic surgical execution.
Purpose: Provide a realistic yet accessible experience with real-time, interactive, device-based AR to test engagement, comprehension, and practical skill visualization before final deployment on smart glasses.
Image generated by ChatGPT
Platform: Smart glasses with AR/MR capabilities.
Interaction: Hands-free holographic overlays on real bones, gesture or voice-based controls, and seamless MR exploration.
Analytics Dashboard: Secure cloud-based system tracking student performance and engagement asynchronously for instructors.
Image generated by ChatGPT
The most important thing to consider is that the AR tablets we have now are limited. They don’t allow students to use both hands freely because they must hold the tablet to see the AR content. This means the AR animations on tablets give only a passive experience, and students can’t fully interact with the content. But in the next five years, I believe smart glasses with AR/MR features will become common. These devices will allow students to interact with holograms hands-free. That will make the learning much more active and realistic, like practicing surgery on a real patient.
Another important consideration is that the tool should help students in both cities and rural areas. Many students miss chances to see certain musculoskeletal diseases during their hospital rotations. The tool needs to give all students the same opportunity to practice, no matter where they are placed.
Immersive learning gives students a chance to see complex anatomy in 3D. This helps them understand how bones, muscles, and tissues fit together (Dhar et al., 2021). It also lets them safely practice surgeries many times without risking harm to real patients. Immersive learning makes practice the same for all students, so they get equal training.
But there are risks, too. If the animations or tasks are too hard or confusing, students could feel overloaded and give up (Liao et al., 2024). Technical problems, like tracking issues with SLAM in AR or weak Wi-Fi, could interrupt practice sessions. There’s also a risk that students might focus more on the cool technology than the actual learning goals if the tool isn’t designed well.
My design choices are based on several learning theories. Kolb’s Experiential Learning Theory says students learn best by doing, which supports our analog paper layers and MR puzzle interactions. Vygotsky’s Zone of Proximal Development (ZPD) tells us that learning happens best when students work on tasks just beyond what they already know, which we can support with hints or adjustable difficulty. Cognitive Load Theory reminds us to keep tasks clear and not overwhelm students by breaking complex information into smaller steps. Finally, Constructivism shows us that students learn better when they build their own understanding by exploring models instead of just reading or listening.
Research by Dhar et al. (2021) showed AR helps students learn anatomy by making them more engaged and helping them understand spatial relationships. Aiello et al. (2023) found immersive simulation tools like AR and MR help students learn safely and practice many times, which is perfect for orthopedic surgery training. Liao et al. (2024) reported that AR can make learning faster and help students remember what they learn better, which is great for busy medical students.
I completely agree with the physician’s idea to use immersive learning. The students clearly need more practice and better ways to see complex anatomy. Immersive learning is one of the best ways to fill these gaps. It lets students practice on their own time and get the same quality of practice no matter where they are.
Yes, immersive learning is a perfect fit. Orthopedic anatomy is complex, 3D, and very hands-on. Students can’t just read about it in a book — they need to see and practice with realistic models. Using immersive tools helps students visualize, practice, and repeat important procedures. Photogrammetry scans and animated surgeries make the experience even closer to real-life situations.
I would definitely use AI. AI could look at each student’s performance and give feedback without the teacher needing to watch every session. AI could also make the tool adjust the difficulty of cases based on how well the student is doing. Finally, AI could create easy-to-read reports so teachers know how students are progressing, saving time for busy instructors.
In the future, smart gloves like those developed at MIT could be added to the tool. These gloves can sense the pressure and finger coordination of a surgeon’s hands (Mack, 2024). By comparing the student’s hand pressure and movements to those of an expert surgeon, AI could give students precise feedback on their technique, helping them develop better skills faster.
I would involve orthopedic surgeons to make sure the anatomy and surgical animations are correct. I’d include medical educators to connect the tool with the curriculum. XR developers and UX designers are needed to make the AR and MR interactions work smoothly. Students should help test the prototypes to make sure they’re clear and easy to use. IT staff should also be involved to make sure everything works well in different hospitals and clinics.
Some questions I still have are: How much time can teachers spend checking reports or giving feedback? What lighting or environment is best for SLAM to work accurately with the AR tablets? Will there be money and staff time available to keep the tool updated?
I’m assuming students and teachers will get some training on using the AR and MR devices. I also assume that the department will support keeping the tools updated after the grant ends, and that ethical approval will be given to use scanned images of real anatomical specimens.
How can we ensure that immersive learning tools like AR/MR animations and photogrammetry scans actually improve students’ diagnostic accuracy and procedural skills, rather than just increasing engagement?
What ethical guidelines and consent processes should we establish to responsibly use high-resolution photogrammetry scans of donated bodies for immersive medical education?
How can we streamline the production process and reduce costs of developing and maintaining analog, AR, and MR learning tools so that more medical schools — including those in resource-limited settings — can adopt and benefit from this technology?
Aiello, S., Cochrane, T. & Sevigny, C. (2023). The affordances of clinical simulation immersive technology within healthcare education: a scoping review. Virtual Reality 27, 3485–3503. https://doi.org/10.1007/s10055-022-00745-0
Dhar, P., Rocks, T., Samarasinghe, R. M., Stephenson, G., & Smith, C. (2021). Augmented reality in medical education: Students' experiences and learning outcomes. Medical Education Online, 26(1), 1953953-1953953. https://doi.org/10.1080/10872981.2021.1953953
Liao, S., Shao, S., Gao, S., & Lai, E. C. (2024). Augmented reality enhances medical education and improves efficiency. Medical Education, https://doi.org/10.1111/medu.15340
Mack, E. (2024, February 20). This “smart glove” can teach you to play piano, tie knots, or perform surgery. MIT News. https://news.mit.edu/2024/smart-glove-teaches-new-physical-skills-0220