Schedule

The course schedule is subject to change at the instructor's discretion. Any updates will be announced in the #announcements channel on the class Discord.

 So please check this page regularly. 

Week 01

Week 01: Course Orientation, Narrative Flow & Wide-Area AR -- [📄 Statement of Understanding Due]

• Tue (08/22/2023) -- First Day of TR Class: Lecture -- Wide-area AR vs. traditional fixed frames; user agency, spatial UI, and organic environmental cues for open-world narrative and enterprise training.

• Thur (08/24/2023): Lab Focus -- Unity Hub setup, LTS installation, interface navigation (Hierarchy, Scene, Inspector, Project), and basic game object manipulation.

• Assigned Reading:

  • 2002 | Interactive Theatre Experience in Embodied + Wearable Mixed Reality Space | IEEE ISMAR

  • Professor David Lindlbauer – Carnegie Mellon University – YouTube Video

• 📄 Milestone: Statement of Understanding Due

Week 02

Week 02: Unreal's RealityScan & Photogrammetry 3D Reconstruction -- [📌 Assignment 1 Released]

• Tue (08/29/2023): Lecture -- Principles of spatial environment capture, Digital Twins, and capturing 1,000+ photo datasets under overcast, diffuse lighting.

• Thur (08/31/2023): Lab Focus -- Point-cloud alignment and high-density mesh reconstruction using Epic Games RealityScan / RealityCapture.

• Assigned Reading:

  • 2020 | The Digital Docent: XR storytelling for a Living History Museum | ACM VRST

  • Professor Ryo Suzuki – University of Colorado Boulder – YouTube Video

• 📌 Milestone: Assignment 1 Released

Week 03

Week 03: Unity Engine Foundations & Scripting

• Tue (09/05/2023): Lecture -- Object-oriented programming in C# (Start, Update, Debug.Log); skeletal bone hierarchies and skin weight painting fundamentals.

• Thur (09/07/2023): Lab Focus -- Skeletal rigging in Blender/Maya for non-standard appendages; setting up clean weight painting to prevent mesh collapsing during extreme bends.

• Assigned Reading:

  • 2004 | Theatrical storytelling in a virtual space | ACM SRMC

  • Professor Francisco R. Ortega – Colorado State University – YouTube Video

Week 04

Week 04: Multi-Environmental Physics & Solver Architecture -- [📥 Assignment 1 Due / 📌 Assignment 2 Released]

• Tue (09/12/2023): Lecture -- Technical architecture for complex spatial physics; equations for low-g/zero-g fields, fluid resistance, buoyancy, and atmospheric drag.

• Thur (09/14/2023): Lab Focus -- Coding custom C# Rigidbody physics solvers governing object state transitions across changing mediums (e.g., air to water to zero-g).

• Assigned Reading:

  • 2016 | Designing extreme 3D user interfaces for augmented live performances | IEEE 3DUI

  • Professor Pedro Lopes – University of Chicago – YouTube Video

• 📥 Milestone: Assignment 1 Due

• 📌 Milestone: Assignment 2 Released

Week 05

Week 05: Advanced Mechanics, Coroutines & Limb Rigging

• Tue (09/19/2023): Lecture -- Physics sweep mechanics (Physics.OverlapSphere, Physics.Raycast), Rigidbody rotation constraints, and physics joint boundary tuning.

• Thur (09/21/2023): Lab Focus -- Scripting adaptive C# grasping logic that calculates surface collisions to let limbs wrap around and conform to arbitrary geometry.

• Assigned Reading:

  • 2018 | Holojam in Wonderland: Immersive Mixed Reality Theater | Leonardo

  • Professor Emily Whiting – Boston University – YouTube Video

Week 06

Week 06: Kinematic Rigs & Procedural Motion

• Tue (09/26/2023): Lecture -- Inverse Kinematics (IK) vs. Forward Kinematics (FK); Unity Animation Rigging package, Two-Bone IK, Chain IK, and damped transform constraints.

• Thur (09/28/2023): Lab Focus -- Setting up procedural IK graphs in Unity to drive dynamic limb reaching and real-time target following.

• Assigned Reading:

  • 2010 | The Westwood Experience: Connecting story to locations via Mixed Reality | IEEE ISMAR

  • Professor Misha Sra – University of California, Santa Barbara – YouTube Video

Week 07

Week 07: MRTK Integration & XR Spatial Attention

• Tue (10/03/2023): Lecture -- Directing user Field of View (FOV) in 3D space; UI Canvas render modes (Screen Space vs. World Space), TextMeshPro, and interactive feedback loops.

• Thur (10/05/2023): Lab Focus -- Animator Controller state blending, procedural IK integration, and binding input controls with audio, visual, or haptic feedback.

• Assigned Reading:

  • 2017 | Coordinating attention and cooperation in multi-user virtual reality narratives | IEEE VR

  • Dr. Andy Wilson – Microsoft Research – YouTube Video

Week 08

Week 08: Voice, Avatar & Embodied AI -- [📥 Assignment 2 Due / 📌 Assignment 3 Released]

• Tue (10/10/2023): Lecture -- Nanite mesh inspection in Unreal Engine, texture map remediation (removing pre-baked shadows), and low/mid-poly LOD exports for Unity.

• Thur (10/12/2023): Lab Focus -- Importing photogrammetry FBX/OBJ meshes into Unity HDRP; calibrating real-world bounding scale (1 Unit = 1 meter) and generating mesh colliders.

• Assigned Reading:

  • 2021 | Story CreatAR: a Toolkit for Spatially-Adaptive Augmented Reality Storytelling | IEEE VR

  • Professor Parastoo Abtahi – Princeton University – YouTube Video

• 📥 Milestone: Assignment 2 Due

• 📌 Milestone: Assignment 3 Released

Week of Break

Mid-Semester Recess

• Tue (10/17/2023): No Class (Fall Break)

• Thur (10/19/2023): No Class (Fall Break)

Week 09

Week 09: LLM Conversational VR Pipeline

• Tue (10/24/2023): Lecture -- LLM Conversational VR Pipeline architecture; integrating conversational AI agents into spatial VR environments.

• Thur (10/26/2023): Lab Focus -- Configuring speech-to-text / text-to-speech pipelines and embedding LLM response hooks in Unity VR scenes.

• Assigned Reading:

  • 2022 | Interactive augmented reality storytelling guided by scene semantics | ACM SIGGRAPH

  • Professor Missie Smith – Auburn University – YouTube Video

Week 10

Week 10: NavMesh Navigation & Autonomous AI Crowds -- [📥 Assignment 3 Due / 🚀 Final Project Released]

• Tue (10/31/2023): Lecture -- Pathfinding on uneven terrain; baking NavMesh Surface over photogrammetric geometry and structuring C# AI State Machines / Behavior Trees.

• Thur (11/02/2023): Lab Focus -- Deploying humanoid AI NPCs guided by NavMeshAgent and C# state logic (Idle, Wander, Inspect) across the imported scanned map.

• Assigned Reading:

  • 2023 | Location-Aware Adaptation of Augmented Reality Narratives | ACM CHI

  • Dr. Mar Gonzalez-Franco – Google – YouTube Video 

• 📥 Milestone: Assignment 3 Due

• 🚀 Milestone: Final Project Released

Week 11

Week 11: Spatial Attention, UI/UX, & Multi-User Flow

• Tue (11/07/2023): Lecture -- Wide-area spatial anchoring, World Locking, Spatial Awareness, and anchoring stable physical-digital assets across real-world geography.

• Thur (11/09/2023): Lab Focus -- Profiling frame rate stability in Unity Profiler during dynamic light updates; finalizing shadowless photogrammetry scene builds.

• Assigned Reading:

  • 2023 | Gumball Dreams: Live Theatre in VR | ACM SIGGRAPH

Week 12

Week 12: Final Project Development: Deterministic Narrative Flow

• Tue (11/14/2023): Lecture -- Managing user positioning in open spaces; scene management (SceneManager), trigger volumes, and asynchronous delays (IEnumerator coroutines).

• Thur (11/16/2023): Lab Focus -- Implementing door animations, trigger box level transitions, and executing extreme joint range-of-motion stress tests.

• Assigned Readings:

  • 2023 | Dynamic Theater: Location-Based Immersive Dance Theater | ACM VRST

  • 2023 | The Effects of Spatial Complexity on Narrative Experience in Space-Adaptive AR Storytelling | IEEE TVCG

Week 13

Week 13: Final Project Development: Spatial Audio & Coroutines -- [📌 Final Project Milestone Due]

• Tue (11/21/2023): Lecture -- Simulating compound natural phenomena; fluid particle dynamics, rising hazards (floods, fires, blizzards), and spatial threat volumes.

• Thur (11/23/2023) -- No Class (Thanksgiving Day): Lab Focus (Self-directed) -- Creating dynamic environmental disasters in VFX Graph; setting up spatial physics trigger volumes to track threat severity.

• Assigned Readings:

  • 2024 | Investigating the Design of Augmented Narrative Spaces Through Virtual-Real Connections | ACM CHI

  • 2024 | Audience Amplified: Virtual Audiences in Asynchronously Performed AR Theater | IEEE ISMAR

• 📌 Milestone: Final Project Milestone Due

Week 14

Week 14: Final Project Development: Optimization & Visual Polish

• Tue (11/28/2023): Lecture -- Shader Graph real-time material parameters (wetness, charring, frost buildup); AI threat perception raycasts and dynamic path re-routing.

• Thur (11/30/2023): Lab Focus -- Programming reactive material shaders; scripting AI panic states and dynamic path rerouting away from disaster zones.

• Assigned Reading: None (Dedicated to Final Project Development)

Week 15

Week 15: Code Audits & Peer Dry Runs

• Tue (12/05/2023): Lecture -- Spatial Audio propagation across mediums (underwater muffling, vacuum attenuation), HDRP post-processing, and technical pitch deck design.

• Thur (12/07/2023) -- Last Day of TR Classes: Lab Focus -- Setting up Unity Audio Mixers; optimizing multi-system execution in Unity Profiler and compiling standalone executable builds (.exe / .app).

• Assigned Reading: None (Dedicated to Final Project Development)

Week 16

Week 16: Final Presentation -- [🎓 Final Submission & Live Demo]

• Final Exam Window: Monday (12/11/2023) through Friday (12/15/2023)

• Lecture Session: Final Technical Slide Deck Presentations & Review Panel Evaluation.

• Lab Focus Session: Live 10-minute technical demonstration showcasing multi-environmental physics, spatial simulation mechanics, and Unity Profiler performance logs.

• Assigned Reading: None

• 🎓 Milestone: Final Standalone Build, Pitch Deck, & Live 10-Min Technical Demo Due

© You-Jin Kim
        Nebraska–Lincoln 🌽