Multiplayer

1. Local Mixed Reality (In-Person / Co-Located)

Use Case

In-Person Physical Board Game / Tabletop Simulator

Two players wearing Meta Quest headsets sit at the same physical dining table and play a digital tabletop strategy game together in real time.

• Why it is good:
  Physical alignment is 1:1. When Player A places a virtual chess piece on the table corner, Player B sees it sitting in the exact same physical spot on their shared table. Physical gestures, eye contact, and spatial voice feel natural because physical and digital boundaries overlap seamlessly.

• What it is not good for:
  Remote or hybrid play. If Player B is at home in another city, the physical room geometry breaks down—Player B’s room may not have a table at those coordinates, causing objects to float or clip through physical walls.

2. Remote Online VR (Distance-Agnostic / Cross-Platform)

Use Case

Distributed Enterprise VR Training / Virtual Meeting Rooms

Engineers in London, Tokyo, and New York put on headsets to inspect a full-scale digital jet engine in a shared virtual room.

• Why it is good:
  Location-independent and hardware-flexible. Users do not need to share a physical room or even identical hardware. The app relies on standard virtual world coordinates (0, 0, 0), allowing users across different spaces or hardware platforms to collaborate without setup friction.

• What it is not good for:
  In-person co-location. If two participants happen to stand in the same physical room while in a Remote VR session, they will constantly bump into each other physically while their avatars appear far apart in the virtual world.

Technical Implementation Guide: Meta XR SDK

Implementation A: Making Colocation Work (Shared Spatial Anchors)

In the Meta XR SDK, local co-location relies on Shared Spatial Anchors (SSA). The headset creates a point of reference based on physical room features, uploads it to Meta's servers, and shares a Group UUID across your networking framework (e.g., Netcode for GameObjects or Photon).

Implementation B: Making Remote VR Work (Pure Transform Synchronization)

Remote VR bypasses Meta's spatial anchor services entirely. You set up a standard OVRCameraRig (or BuildingBlocks Camera Rig) and replicate tracking data using networked transforms.

Key Implementation Steps in Meta SDK:

1. Configure OVRManager: In OVRManager, ensure Spatial Anchors and Anchor Support are Disabled (saving system resources).

2. Network Rig Setup: Map Meta's tracking anchors directly to networked transforms.

Instead of relying on vendor-locked tools like the Meta XR SDK, this approach uses open standards: OpenXR, Unity AR Foundation, and the XR Interaction Toolkit (XRI).

1. Local Mixed Reality (In-Person / Co-Located) — Vendor-Agnostic

Use Case

Cross-Device Physical Tabletop Collaboration

Two users—one using a Meta Quest, the other using an Apple Vision Pro or Android/Pico headset—sit at the same physical desk to manipulate a shared 3D model.

• Why it is good:
  Zero Vendor Lock-in. It does not rely on Meta’s Cloud servers or proprietary APIs. By using a printed Image Target / QR Code placed on the table, any device with a camera can align its coordinate system to the exact same physical spot.

• What it is not good for:
  Visual tracking can drift or fail under poor lighting or if a user blocks the camera's view of the printed marker. Native spatial anchors are generally more robust than visual optical tracking.

2. Remote Online VR (Distance-Agnostic / Cross-Platform) — Vendor-Agnostic

Use Case

Universal Multiplayer VR App (SteamVR, Quest, Apple Vision Pro)

A remote multiplayer VR application deployed across diverse headset platforms without maintaining separate platform SDKs or codebases.

• Why it is good:
  100% Code Reusability. You write one C# script using OpenXR and Unity Netcode. The exact same build target runs on Meta Quest, Valve Index, HTC Vive, Pico, and Apple Vision Pro.

• What it is not good for:
  You cannot easily access hardware-specific native features (such as Quest Pro eye-tracking, Vision Pro hand pinch gestures, or Meta-specific passthrough filters) without adding vendor-specific extension packages.

Technical Implementation Guide (Non-SDK / Open Standards)

Implementation A: Non-SDK Colocation (Image Target Marker Alignment via AR Foundation)

In this approach, a physical printed image (or QR code) acts as the shared origin (0, 0, 0). When each headset scans the image using Unity AR Foundation, its local world coordinate system re-aligns to match the marker.

Implementation B: Non-SDK Remote VR (OpenXR + XR Interaction Toolkit)

Remote VR without vendor SDKs relies entirely on the XR Interaction Toolkit (XROrigin) and standard OpenXR Input Actions.

Other Unity MRTK3 Tips

Intro to Mixed Reality Toolkit 3 [1]

HoloLens 2 fundamentals: develop mixed reality applications [1]

Journey into Mixed Reality development for beginners [1]

Solver overview — MRTK3 [1]

Configuring Unity for the XR SDK pipeline [1]

MRTK Standard Shader [1]

Unity development for HoloLens [1]

How to use HoloLens [1]

If you are getting an issue with SDK version in the build settings under Universal Window Platform (UWP) -> you must download the Windows SDK [1]. The troubleshooting UWP and solutions can be found in the Unity forum [1].

AugCog Open scene -> KirbyWLTManager826New (First scene to check out and test running). 

When you get to scene -> navigate in left Hierarchy window:

WLT_Tutorial -> MainContainer -> Kirby -> Tutorial Controller -> Tutorial Controller script

Tutorial Controller functions as a master cue sheet (theatrical) and each clip are like a short play in the tutorial. This is important as it will become main controller for the DynamicAR play. 

Audio (files can be found in): Assets -> Study -> Tutorial -> Audio files 

Directional Indicator Example Scene can be found:  Assets/MRTK/Examples/Demos/Solvers/Scenes/DirectionalIndicatorSolverExample.unity

• Mac users: When you get the error "DllNotFoundException: UnityOpenXR"
Fix: Window -> XR-> OpenXR-> Project Validation [1]

• How to show skybox in HoloLens using MRTK [1]

© You-Jin Kim
        Nebraska–Lincoln 🌽