Augmented Reality (AR)

Augmented Reality (AR) involves adding computer-generated content to the user’s real-world experience in real time. As discussed in the previous chapter, the device may be a dedicated AR system, such as Microsoft’s HoloLens, or the user may be using an Apple or Android mobile device (phone or tablet) as the interface between them and the real-world environment they are viewing with augmentations. We’ll look at mobile phone-based AR devices in this chapter, so before beginning the recipes you should have explored the Android and/or Apple iOS recipes in the previous chapter (Chapter 16, Mobile Games and Applications) to be familiar with the procedure for switching build targets to mobile devices, and running games on USB-connected devices. Also, since you’ll be running your AR projects on either an Android or Apple iOS device, ensure you have completed the first and/or second recipe of the previous chapter so that your system is set...

Exploring the Unity AR samples

Unity provides an up-to-date set of sample AR Scenes in a project you can download from the https://github.com/Unity-Technologies/arfoundation-samples website. In this recipe, we’ll download the project, build an AR phone app, and try some of the samples from Unity.

Figure 17.1: Running the face mesh scene, which maps a flag image over your face!

How to do it...

To download and explore the Unity AR samples, do the following:

1. Go to https://github.com/Unity-Technologies/arfoundation-samples and download the files as a ZIP file.

Figure 17.2: Downloading the AR samples from GitHub

2. Unzip the project files and open them in Unity.
3. Open the Build Settings dialog window and switch Build Target to Android (or iOS if appropriate).
4. You’ll see that all the scenes have already been added to the build for this project, with the AR Foundation Menu scene listed first.
5. Connect your computer...

Creating an AR project with the AR (Core) template

Unity have kindly created a project template, which does the required work for AR projects of adding the AR Foundation package, and provides a sample scene that already has the core AR GameObjects needed for an AR project.

In this recipe, you’ll learn how to create a new project from the provided template, open the sample scene and add a 3D object, and then set up the Build Settings config for deployment on an Android device.

Figure 17.8: The AR project we’ll create, with just a Cube overlaid on the device camera view

How to do it...

To create an AR project with the AR (Core) template, do the following:

1. In Unity Hub, click the New Project button.
2. Select your version of Unity, then scroll down to find the AR (Core) template. If you’ve not used this before you may need to first download this template.
3. Provide a project name and destination and click the Create project...

Adding the AR Foundation package and GameObjects to a 3D scene

If you already have a 3D project, then there are some basic steps you need to perform so that your Unity Editor application is set up for AR development. AR projects created with Unity have, at their core, the AR Foundation package and XR Plugin Manager. In the previous recipe, things were set up for us already as part of the Unity AR (Core) template project. In this recipe, we’ll create a new Unity 3D project and then manually add the AR Foundation package to the project. In this recipe, we’ll create the same scene setup, but will be building a scene from scratch to get to know the AR Foundation GameObjects and how to add them to any scene.

This recipe can be used as a guide on how to add AR capabilities to any existing 3D project you might already have. As part of this recipe, we’ll also be adding the appropriate plugin for the Android target device on which the AR project is going to be developed...

Detecting and highlighting planes with AR Foundation

One of the features of AR is its ability to detect flat surfaces through device cameras and other sensors (such as the LIDAR sensors on some high-end iPads and iPhones).

This recipe illustrates how a Unity AR Raycast, combined with some AR Foundation objects and scripts, can work to highlight detected planes (both horizontal and vertical) with a circular highlight:

Figure 17.18: A plane being detected in an indoor scene

Getting ready

This recipe follows the previous one, so make a copy of it and work on that copy.

We also need an image to display on the detected planes. We have provided a suitable circular image called plane_highligher.png in the 17_04 folder.

How to do it...

To create an AR plane highlighter app, do the following:

1. Delete the two cubes in the Hierarchy panel.
2. In the Hierarchy panel, select the XR Origin GameObject. In the Inspector panel, add an AR Plane Manager...

Creating an AR furniture previewer by detecting horizontal planes

As we saw in the previous recipe, one of the features of AR is its ability to detect flat surfaces through device cameras and other sensors (such as the LIDAR sensors on some high-end iPads and iPhones). This recipe will illustrate how a Unity raycast, combined with some AR Foundation objects and scripts, lets us create a basic AR furniture previewer just by tapping on the screen where we want a virtual 3D piece of furniture to be displayed.

Figure 17.22: Adding 3D sofas to a back yard

Getting ready

This recipe builds upon the previous recipe, so make a copy of that project to work on in this recipe.

We’ll be making use of a free model from the Unity Asset Store, which can be found here:

• https://assetstore.unity.com/packages/3d/props/furniture/patio-sofa-set-227952

How to do it...

To create an AR furniture previewer, do the following:

1. In the Hierarchy panel...

Creating a floating 3D model over an image target

A common way to use AR to enhance the world that’s seen through the device’s camera is to recognize a target image and either display a different image or animation in its place, or add a 3D model to the view that tracks the location of the target image.

We’ll do the latter in this recipe, making a 3D model appear over a book’s cover image that we will add to a library.

Figure 17.25: A 3D “car” model displayed over the book cover image in our AR phone app

Getting ready

This recipe builds upon the previous recipe, so make a copy of that project and use the copy.

You’ll also need a target image. Either take a photo of an object in your home/office or use the provided book cover image called tesla_book.jpg in the 17_06 folder. You’ll also need to print the image out or display the image onscreen while running the AR app on your phone.

How to do it...

...

Further reading

Learn about basic AR Scenes and the XR Interaction Toolkit requirements from the Unity documentation:

• Basic AR scene requirements:

https://docs.unity3d.com/Manual/AROverview.html

• Example XR Interaction Toolkit project:

https://github.com/Unity-Technologies/XR-Interaction-Toolkit-Examples

• XR Interaction Toolkit Blog:

https://forum.unity.com/threads/xr-interaction-toolkit-1-0-0-pre-2-pre-release-is-available.1046092/

Learn about the AR Foundation package:

• https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/index.html
• https://unity.com/mobile-solution-guide
• http://virtualxdesign.mit.edu/blog/2019/6/22/viewing-your-models-in-ar

Learn about Unity and developing for the Microsoft HoloLens AR headset here:

• Video tutorial on setting up development for HoloLens

https://www.youtube.com/watch?v=dOsYerpKloY

• Microsoft recommend Unity...

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask questions to the author, and learn about new releases – follow the QR code below:

https://packt.link/unitydev