MR with Unity and Meta Quest 3

What you will do

In this post, you will learn a step-by-step process to setup an MR application with Unity and Meta Quest 3 that uses passthrough over Quest Link in two modes: virtual-first and real-first.

Virtual-First MR Application

First things first, let's define what Virtual-First means. When we say Virtual-First, you should understand that there is going to be a window to the "real world" inside the "virtual world", meaning your view of the "real world" will be limited and your surroundings will be the virtual world you created in Unity, hence Virtual-First.

Pre-requisites

Before starting in Unity, you have to add the Meta XR packages you might use to your assets in the Unity Asset Store. For this tutorial, we will only use one package for simplicity, the Meta XR All-in-One.

To add this package to your assets, you can go to the Unity Asset Store and search for "meta xr all-in-one" or you can follow this link and add the package to your assets.

This tutorial also assumes that you already have connected your Meta Quest 3 to your PC and configured everything that is needed. If you have not, you will have to create a developer account and configure the headset to work with passthrough over link.

Download both the Oculus App and the Meta Quest Developer Hub. Create a developer account in Meta Quest Developer Hub. With the developer account setup, you should have the "Passthrough over Oculus Link" option in the Beta tab on the Settings page. You might also enable the "Developer Runtime Features".

Initializing the Project

After you have added the Meta Quest XR package to your assets, you can create a new project in Unity Hub.

For this project, I am using Unity version 2022.3.13f1 with the 3D High-Definition Render Pipeline (HDRP).

You can name your project however you want. I am going to call it MR with Meta Quest 3.

Installing necessary packages

After you have created the project, you will need to install some packages and plug-ins to get everything going.

First, install the XR Plugin Management package. You can do that either in the Project Settings or in the Package Manager. Go to Edit>Project Settings, click on the XR Plugin Management tab, and install it. You will also need to install the Oculus XR Plugin Provider by clicking on the Oculus option after you have installed the XR Plugin Management tool.

Then, go to Window>Package Manager. Click on the Packages button right next to the "+" button, and select My Assets. You should see the Meta XR package you added to your assets in the Pre-requisites section of this post. Select the Meta XR All-in-One package and install it.

Just a quick note: you will be required to restart the editor along the way, but don't worry about it.

Configuring the Game Objects

Now that you have installed everything that you will need, let's start building!

The first thing you might want to try is to setup the XR Rig. To do that, search for OVRCameraRig and choose "All" on the search tab to search in the entire project. This object comes with the Meta XR All-in-One package that you installed. The OVRCameraRig object contains the OVRManager component, and this script manages all the configurations that you set in your project and interfaces with the headset. Under the OVRManager component, scroll down to the "Passthrough Support" drop-down and select "Required". Also, check the "Insight Passthrough>Enable Passthrough" option. Don't forget to delete the main camera because the OVRCameraRig object already contains a camera object.

After setting up the camera rig, let's create a window to the "real world" using the quad object.

Right-click on the Hierarchy and select 3D Object>Quad to place a quad into the game. Then, add the OVR Passthrough Layer component to the quad. This script is also a part of the Meta XR All-in-One package, so you should have it already.

For the "Project Surface" drop-down, select "User Defined", and drag the Quad Mesh-Filter down to the "Projection Surfaces" list after hitting the "+" button to add a surface. Make sure you have the Compositing Placement option set to "Overlay".

You will also need to create a material to the quad to make it transparent black - RGBA(0, 0, 0, 0). This will tell the headset to render the "real world" within that region. Don't forget to add the material to the quad!

Testing

Now you should have everything configured to see a window to the "real world" within your "virtual world" in Unity! Try hitting the play button and seeing if it works.

Final note: sometimes it does not work if you put on your headset after hitting the play button. Try putting on your headset, activating Quest Link, and then hitting the play button.

This is what it will look like in the Unity Editor. However, in the headset displays, you should see your "real world" environment where the black surface is.

Real-First MR Application

First things first, let's define what Real-First means. When we say Real-First, you should understand that there is going to be a window to the "virtual world" on the real world, meaning your view of the "virtual world" will be limited and your surroundings will be your room, your office or wherever you are at the moment.

Pre-requisites

Even though they are different applications, try doing the previous section in case you didn't, and setting up the Virtual-First MR Application because it will be almost the same thing, but with some slight adjustments.

Configuring Game Objects

There are very few configurations needed once we have the basic setup from the previous section. All you will need to do is two things: add a Passthrough Object, and configure the camera

To add the passthrough object, just right-click on the Hierarchy and add an Empty Object. Then, add the OVR Passthrough Layer component to this object, and set the projection surface to "Reconstructed" and the compositing placement to "Underlay".

To configure the camera, go to the Hierarchy and find OVRCameraRig>TrackingSpace>CenterEyeAnchor. In the camera component, set the Environment Background Type to "Color" and set the color to RGBA(0, 0, 0, 0), which is the same transparent black used in the quad material.

Just one last thing, add a Cube in front of the camera just so you can see something of the "virtual world" in front of you.

You can also delete the quad since we don't need it in the Real-First Application.

Testing

Now you should have everything configured to see a window to the "real world" within your "virtual world" in Unity! Try hitting the play button and seeing if it works.

Final note: sometimes it does not work if you put on your headset after hitting the play button. Try putting on your headset, activating Quest Link, and then hitting the play button.

This is what it will look like in the Unity Editor. However, in the headset displays, you should see your "real world" environment with the cube in front of you.

Page updated

Google Sites

Report abuse