In 2020 I began working at the University of Wisconsin-Madison's DoIT-WaMS (Web and Mobile Solutions) as a student developer on an augmented reality project. Since then I have been working with AISC (American Institute of Steel Construction) to develop and demonstrate potential AR solutions for industry. Additionally, I have worked with a civil and environmental engineering professor, Hannah Blum, to develop an in class AR activity for students in a steel structures course.

In an augmented reality environment, it is difficult to precisely place holograms. As a solution, we explored the possibility for the user to fine tune the position and orientation of a hologram to the desired precision using a menu interface after placing it with their hand.

We found placing a hologram in an approximate location before adjusting it was a time consuming process that was impractical for our application. As a solution, we explored using QR codes to not only identify a piece, but also to place it accurately enough for a user to determine at a glance if there is a fabrication flaw in the steel.

We attached the QR codes to the provided steel piece so that the HoloLens could place the hologram on top of the piece. To simulate a fabrication error, we were given incorrect models demonstrating common flaws that might appear and we placed them on top of the real steel piece using the QR codes to determine if it was easy to recognize a mismatch.

During development of our in teaching aid application,  I was presented with the task of allowing the user to accurately and intuitively select a group of holograms in close proximity to each other and then move them together. I ended up developing an interface which allowed the user to select or deselect individual pieces of a larger hologram by tapping them with their finger. From there the user was able to the selected pieces along a strict axis by grabbing onto a nearby "handle" and pulling it away from the larger hologram.

This allowed the user to visualize how the pieces of a larger hologram might detach from each other and physically see the result. This was utilized in our teaching aid application for students to visualize different ways for a steel piece to break apart.

The AR application we developed for the classroom had students presented with one of two different steel structures. From there they were walked through the activity through heads up prompts and arrows to point their attention to the relevant area of interest in the case they were looking in the wrong direction.

Students were first asked to identify one of a predetermined set of groups of pieces that may break off from the larger steel structure. From there they were given the opportunity to pull them apart and then were asked to identify what type of fracture pattern they made before finally calculating a "strength" value.

Our application, featured in this article from the College of Engineering, gave students a valuable opportunity to see real world examples in the classroom. Many of the students were very excited to try the new technology, which also gave us a unique perspective of the challenges that augmented reality presents to untrained users and how efficient design can help to improve the experience.