ART 22

Welcome to ART22!

ART 22 - Computer Programming for Arts
Developting Wide-Area Augmented Reality Theater Experience

Disciplines: Mixed Reality, Augmented Reality, Virtual Reality, Human–Computer Interaction

In the future, Augmented Reality (AR) and Virtual Reality (VR) will become as universal and essential as smartphones and personal computers are today. As computers become more deeply enmeshed in human lives, we must be prepared to push the boundaries of computer engineering as we confront looming questions on the extent to which we will allow the integration of technology into our experiences. As the domain gains traction through commercial adaptation, the technology will mature as a portal to connect, feel, and express all within the computational environment. Through Human-Computer Interaction research, the fidelity of the simulated experience will approach mirroring the real-life experience.

As seen in the gaming community, striving for a better experience or more provocative content can conflate the value of virtual and physical though surreal and hyper-reality experience. The lifelike realism of VR and AR presents a host of ethical implications related to identity, privacy, self-image, and the body politic. In this course, students will explore the integration between the virtual world and the physical through hands-on interactive platform development. The course will address fundamental issues in Human-Computer Interaction and perceptual psychology through system-building and design experimentation. Through the use of sensing technologies, new methods of interaction, and inquiry, students will gain fresh insights that will challenge our notions of what human experience with technology can or should be.

HoloLens 2 headset is a self-contained computer with Wi-Fi connectivity, provides standalone HMD immersive mixed reality experience. (Unity Blog)

During the first part of the class, we will discuss the basic concepts of Virtual Reality and some of the main differences between VR and AR technologies. Students will become familiarized with the necessary developing platforms and essential mixed reality tools kits required to rapidly prototype for a designated headset with Unity’s platform dependent compilation. The main focus will be understanding the dependencies of assets, libraries, and MX Unity plugins to jump straight in to making AR Theater application, designing user experience for the Kirby Crossing. The studio portion of the course will allow students to create and experience the virtual environment through the FPS navigator (emulating First-Person Perspective viewport). Furthermore, in this part of the course students will familiarize themselves with the concept of Repository (GitHub), Version Control (Git Guides), Creating Prefabs (Unity Asset). We will also discuss the ethical challenges of virtual reality and the concerns that will shape the VR application and adaptation.

Eye of the Temple (2020) is a VR game that takes full advantage of the room-scale abilities of your VR headset, letting you walk, dodge, and duck your way through a vast temple complex.

In the second part of the class, students will be introduced to the development pipeline along with the locomotion navigation system, a control input system. We will concentrate on understanding what is needed to make 6 minutes AR Theater application; preparing for the soundtracks, virtual environment stages and necessary Unity assets for the project. Afterwards, we will learn to implement Dynamic Theater Scene System to additively load one or more scenes to construct the user experience. By the end of week 4, each team will have tested the merging features in shared repository in provided virtual environment. During this time, each team will choose a framework for project management (Scrum vs Waterfall), communication protocol and documentation style.

In the third part of the class, students will work in a team to refine the game logic implementation. We will cover how carefully arranged digital contents in the simulated environment can enhance the user experience, catering personalized immersive theater experience. Students will evaluate the implementation of their application by conducting a user study. In addition, we will discuss the methods of enhancing user immersion and virtual presence in an interactive virtual environment such as VR locomotion technique. Much of this part of the course will focus on feature implementation and function testing.

In the last part of the class, we will concentrate on refining the user experience and stability of the AR application. through optimization. Much of the last part of the course will focus on optimization; reducing polygon count, limit overdraw, implementing Level of Detail (LOD) meshes, and object pulling for better FPS + smoother transitions. Through this process, we will learn methods to evaluate and improve the system before publishing to a video game digital distribution service such as Steam or Epic. Here students will learn to license content before publishing their application.

The course will end with a final project where each team will showcase their theatre poster, webpage, and Mixed Reality Capture (MRC) of their 6 minutes-play in the Kirby Crossing. Throughout the course, students will be exposed to innovative research and application development cycle happening in the field of Mixed Reality. The peer feedback presentation will train the skills necessary to critique the play for constructive feedback.

In this course we will explore these questions:

How can we translate human interaction behavior within the computational environment?
What can we simulate within the Mixed Reality domain?
What human experience would AR VR technology struggle to emulate?
How to implement interactivity into their AR and VR experiences?
What are the applications/methods of wide-area AR technology for locomotive navigation?

Santa Barbara

Page updated

Google Sites

Report abuse