Virtual and Extended Reality (VR/XR)

In this chapter, we will present a set of recipes introducing Virtual Reality (VR) game development in Unity. XR stands for eXtended Reality. This is an umbrella term encompassing both fully immersive VR systems and Augmented Reality (AR) systems that allow users to sense the world around them. Some of the recipes in this chapter cover how to publish projects using the WebXR standards.

VR is about presenting an immersive audio-visual experience to the player, engaging enough for them to lose themselves in exploring and interacting with the game world that has been created. As well as gaming, VR applications offer amazing experiences to help us explore graphics and videos in immersive 3D, such as Google Earth VR and Oculus Quest Wander:

Figure 18.1: The Oculus Wander application showing the Eiffel Tower in Paris, France

From one point of view, VR simply requires two cameras in order to generate the images for each eye to give a...

Setting up the Oculus Quest 2 for Unity development

The current most popular VR headset is probably Meta’s Oculus Quest 2. These are low-cost, walk-around headsets that provide a great introduction to VR for many people. It can be a bit fiddly to set up a Quest for Unity project development, so we’ll go through the steps required in this short recipe. Between this and the next recipe, you’ll have the satisfaction of creating VR projects and loading them onto your headset.

These steps were performed at the time the book was being written. But things change quickly, and it may be that not all the steps/screenshots are the same when you attempt this developer mode setup. However, we have included this recipe since we believe it will still be helpful to those beginning development for the Quest 2 to see the kinds of actions they’ll need to take.

Getting ready

For this recipe, you’ll need the following:

• A mobile phone...

Creating and running a Unity project on a VR headset

VR devices are mobile devices, so if you’ve created some mobile apps and loaded them onto a cell phone you’ll be familiar with the procedure for working with VR headsets. Most VR devices run using the Android mobile operating system (including the Oculus Quest/2/3 and Pico VR headsets). This recipe is based on working with a Quest 2 headset, but the steps are similar for other headsets.

In this recipe, you’ll learn how to create a project containing an empty, all-green scene, and build and run this project on a Quest 2 headset.

This recipe will be the foundation for the other recipes in this chapter.

Getting ready

Before beginning the recipes in this chapter, you should have explored the Android recipes in Chapter 15, Mobile Games and Applications, to be familiar with the procedure for switching build targets to mobile devices and running games on USB-connected devices. Also, since you’...

Beginning with the XR Interaction Toolkit

Having got a project onto a VR headset with the previous recipe, let’s add some interaction with the XR Interaction Toolkit. Player interaction with VR projects involves very different components and user actions than for typical first- or third-person controllers. Players may move via teleportation, so they have to be able to select the destination point they wish to teleport to. Hand controllers and even human hand tracking can be used to allow users to grab, release, and move objects. Alternatively, they may select objects with laser raycasts. Real-world interactable components can be simulated, such as press-in buttons, levers, and switches. The XR Interaction Toolkit provides features for all of these VR interactions, and its samples allow us to get to know, and repurpose, examples of different interactables in a playground VR scene.

In this recipe, you’ll create a scene with an XR Origin from the XR Interaction Toolkit...

Creating a 360-degree video VR project

The prevalence of affordable 360-degree cameras means that it’s easy to create your own, or find free online, 360-degree images and video clips. In this recipe, we’ll learn how to add a 360-degree video clip as a skybox to a VR project.

You will also learn how 360-degree video clips can be played on the surface of 3D objects, including the inside of a sphere – a bit like Google Earth VR mode when you raise the sphere to your head to view its 360-degree image contents:

Figure 18.19: A 360-degree video sphere object in Google Earth VR

Getting ready

This recipe builds on the previous recipe. That project was based on a 3D (Core) Unity project, with the XR Interaction Toolkit package and Starter Assets installed. So, make a copy of that project and work on the copy.

For this recipe, we have provided a short (royalty-free!) video called Snowboarding_Polar.mp4 in the 18_04 folder.

Special thanks...

Exploring and building the Desert WebXR demo project

One likely future of VR is through the web, using the WebXR standard. WebXR uses OpenGL as a way to publish AR and VR applications through websites. At present, WebXR projects can be published as websites, and then VR-enabled browsers running on headsets can be used to enter those interactive 3D scenes. The current free up-to-date WebXR browser is Wolvic from Igalia (see https://wolvic.com/).

Note: The Wolvic project has superseded the previous Firefox Reality browser from Mozilla Corporation.

For several years, the Desert open-source WebXR demo project around which this recipe is based has been maintained for up-to-date versions of Unity by De-Panther – GitHub URL: https://github.com/De-Panther.

In this recipe, you’ll first explore De-Panther’s published version of the WebXR project, then you’ll download the project source and build the website folder on your own computer using...

Using GitHub Pages to publish your WebXR project for free

GitHub offers us a way to publish a free website for each project with a feature called GitHub Pages. The simplest way to do this is to have a docs folder in your GitHub project such as the one we created in the previous recipe.

Figure 18.32: The published build via the free GitHub Pages service

Getting ready

This recipe makes use of the docs folder you built in the previous recipe. This is the content you’ll be hosting for free with GitHub Pages in this recipe.

Since you’ll be creating and making changes to a GitHub project, you will need to have a working GitHub account. If you don’t already have such an account, you can create one for free here: https://github.com/join.

Note: GitHub projects are called “repositories,” often abbreviated to “repos.”

How to do it...

To use GitHub Pages to publish your WebXR project for free, perform...

Creating a simple WebXR scene from scratch

Rather than just exploring the Desert scene from De-Panther’s WebXR project, let’s create and publish a new scene from scratch, making use of the WebXR plugins already added as part of that project.

In this recipe we’ll create a new scene containing a Cube, acting as a table, and two orange Spheres that can be picked up and interacted with.

Getting ready

This recipe builds on the Exploring and building the Desert WebXR demo project recipe. So, make a copy of that project and work on the copy.

How to do it...

To create a simple WebXR scene from scratch, perform the following steps:

1. In Unity create a new Basic (Built-in) scene – this should contain a Main Camera and Directional Light.
2. Delete the Main Camera GameObject from the scene (since we’ll be adding a WebXR camera set next).
3. From the Project panel, drag the WebXRCameraSet asset file into the scene from the folder...

Further reading

The following are some useful sources for the topics that were covered in this chapter.

Here are some online forums for Unity VR developers:

• Unity’s VR forum: https://forum.unity.com/forums/vr.160/
• Meta Unity forum: https://communityforums.atmeta.com/t5/Unity-VR-Development/bd-p/dev-unity/page/41

Rajat Kumar Gupta’s tutorial on creating a 360 video in Unity: https://blog.logrocket.com/make-360-vr-video-unity/

Video tutorial on using the WebXR exporter from De-Panether: https://www.youtube.com/watch?v=e7hVROj9qm4

Video tutorial on XR Interaction Toolkit hand tracking: https://www.youtube.com/watch?v=mJ3fygb9Aw0

Unity has a guide to playing 360 videos as skyboxes: https://learn.unity.com/tutorial/play-360-video-with-a-skybox-in-unity

Although we don’t have the space to explore it in this chapter, Oculus has published an integration package for Unity, which provides several Prefabs and demo scenes especially...