Getting Started with React VR

Chapter 1. What is Virtual Reality, Really?

You are reading this book to learn to make Virtual Reality (VR), but what is Virtual Reality?

It seems like a simple enough question, but the answer is all over the map. Most people think VR means virtually real or alternate reality.

That is not what Virtual Reality is.

I think this is because the word virtual can mean several different things. To a computer scientist, the word virtual means something that simulates the thing it virtualizes. In other words, a virtual hard drive pretends to be a hard drive.

The virtual object acts like it is real, but it isn't--frequently, its more flexible and easier to control, modify, and support than a physical object. In many ways, it is better than the physical object. A virtual disk, for example, acts just like a computer disk. It can store data. Yet that data could be on a physical spinning disk, in a solid-state drive, or even in memory. The virtual disk can be resized, whereas a physical disk can only be copied to a larger (or smaller) disk. A virtual disk is more flexible.

Some people think virtual means almost. If a Tesla drives by, they might say, "That's virtually noiseless!" People know it's not really noiseless, but it is much quieter than a big V8 driving by. Or, that person is a virtual saint about a person they like. In this case, it means nearly or in all but name.

Virtual can also mean someone with virtue. A person who behaves ethically is virtual, although this is not the normal usage of the word (virtuous would be). This is where the word came from; in Latin virtualis means strength or virtue. Yet, in our case, we mean something that seems real, but isn't.

I think this is the misconception about Virtual Reality. People think it means almost real. Many people think VR isn't there yet because it doesn't look nearly like the real world. It will be quite some time before the view through a VR headsets looks just like the real world; other senses, especially touch and taste may take quite a while until they can be simulated.

Yet, this is not the point; the point with Virtual Reality isn't that it's nearly real. The point is, when you are in it, it seems real, even if it looks nothing like reality.

I'll say this again as it's an important distinction. Virtual Reality, or for that matter Augmented Reality, doesn't need to be nearly real, but it will seem real when you are in it (even if it doesn't remotely look real).

By the end of this chapter, you'll learn:

What Virtual Reality is and how it works
Some of the history of VR -it's not new, the technology is over 50 years old!
User agency - interacting with the world through controllers
Rendering hardware
How to view VR
Types of headsets

What Virtual Reality is and how it works

We have many senses. To make us feel an alternative reality is real, we need to involve these senses to fool the brain. Most VR systems make use of two: sight and sound; touch is also used but not in a full reach-out-and-touch-someone sense (although people are working on it!).

Tor Nørretranders compiled data about the senses and their relative bandwidth, in computer terms. This is a bit like comparing apples and motor oil, although useful to see how it applies to VR.

So, we can see if we make you see something that is virtually real, we may be able to convince the brain it is real. Yet, simply putting a vision screen in front of us is not the complete answer.

Giving someone depth perception is most of the answer.

This is a fairly complicated topic, but the main method of showing depth of objects is stereoscopic depth perception. Remember those ViewMaster toys? Here is an example of one:

You put in a disk that had a left eye and a right eye image. The images on the left and right nearly look identical, but they represent what the right and left eyes would see if you were standing at that location; each slightly different due to parallax. From the disk, here, we can see each of the left and right eye images. The lenses in the preceding View-Master focuses your eyes on the images.

Your brain would look at these two images and fuse the images into something that looked real. This uses a depth perception technique called stereoscopic depth perception.

Yes, the View-Master was an early Virtual Reality viewing device!

Now, what is really going on here? How does Stereo work?

When you look at something, perspective and separation between your eyes will make you focus your eyes differently at something closer as opposed to something far away. In this diagram, the yellow lines show our sight lines to a near object, and the orange lines show the sight lines to a distant object. Note that the angle between the yellow lines is larger than the narrow angle of the orange lines:

A friendly robot loaned us the lower half of her eyes to make this image (that's why it shows circuit boards). Your real eye is constructed somewhat similar; I omitted the light rays and where they fall on the back of the eye for illustration's sake.

Your brain will automatically figure out if your eyes are pointed at a close or far object by the difference in angles between the yellow and the orange lines.

This is just one method our brain uses to distinguish depth. Another that is also vital to Virtual Reality is the use of parallax.

Parallax refers to the way that, not only do the left and right android eyes point differently (as would your eyes, when they are attached to your head), but that each eye sees a slightly different view of the same objects. This will work even with one eye if you move your head to the left and right, and is how people with mono-vision perceive depth (among other ways).

This is how your left eye sees the scene:

This is how the right eye sees the same object:

Parallax refers to the way that an object that is more distant will be less to the right/left than a nearby object, when viewed with the other eye, OR (an extension) when moving your head left to right. Our brain (as well as the brains of animals) will instinctively see these as closer/further.

The red cube is either next to the blue cube or the green cube, depending on what eye sees the image. Your brain will integrate this, coupled with how the cubes move if you move your eye from side to side, to also give you a sense of depth.

Note

Don't despair if you are in the percentage of the population that do not perceive 3D movies. They strictly rely on stereoscopic depth perception and do not take parallax effects into account; they are pre-recorded. With true VR (computer generated or light field based 360 video), if you move your head, you will see the parallax effect and the VR can seem real just like someone with stereoscopic depth perception sees.

I have mono-vision because I have a nearsighted eye and a farsighted eye, and VR works great for me. Your mileage may vary, but if you don't like 3D movies, give VR a try (then again, I really like 3D movies).

Parallax depth perception will work even if you have one eye, when you move your head right to left.

There is one additional way that your brain will use to determine depth of an object - focusing. (Actually, there are many ways other than those listed, such as blue shifting of objects in the far distance, like mountains, and other effects). Focusing on an object in the real world will make that object and other objects at roughly the same distance appear in focus, and objects both further and closer will appear blurry. Sort of like this:

Current HMD's cannot accurately show focus as an effect. You are looking at a small screen that generally has a fixed focus of about 5 feet in front of you. All objects, close and far, will appear to be focused the same as they are actually just being shown on the screen. This can cause a mild VR discomfort, called the accommodation-vergence conflict. Basically, if you focus on the far focus cube (the salmon colored one), your eyes will still focus as if the salmon cube was located where the red cube is; your eyeballs will, however, aim stereoscopically as if it was located where it should be. This effect is most pronounced with very close objects.

Note

The accommodation-vergence conflict is most severe with close objects - so try not to have anything, such as a GUI, located too close to the user's location. You will reduce sickness this way. This means you may need to float GUI elements out into the room instead of having them very close. This may cause overlapping UI elements.

VR design is challenging. I'm looking forward to what you design!

Use of stereo and parallax in Virtual Reality

As far back as 1968, Ivan E. Sutherland first observed that objects with stereoscopic depth perception, and that seemed to be positioned in space when the users head moved (motion parallax), seemed real.

The system that he and Bob Sproul developed, often referred to as the The Sword of Damocles, just displayed a handful of glowing lines in the air, yet:

"Even with this relatively crude system, the three dimensional illusion was real."

-Ivan E. Sutherland, AFIPS '68 (Fall, part I) Proceedings of the December 9-11, 1968, fall joint computer conference, part I: http://bit.ly/2urAV5e

Real in this instance meant that despite a total lack of realistic rendering - just a glowing cube - people thought of it as real. This is due to the stereoscopic rendering and the parallax effect. People could turn their heads and move a little bit from side to side.

They invented the first VR Headset, or Head Mounted Display (HMD).

The person widely regarded as creating the term Virtual Reality, Jaron Lanier, said:

“It's a very interesting kind of reality. It's absolutely as shared as the physical world. Some people say that, well, the physical world isn't all that real. It's a consensus world. But the thing is, however real the physical world is – which we never can really know – the virtual world is exactly as real, and achieves the same status. But at the same time it also has this infinity of possibility that you don't have in the physical world: in the physical world, you can't suddenly turn this building into a tulip; it's just impossible. But in the virtual world you can …. [Virtual Reality] gives us this sense of being able to be who we are without limitation; for our imagination to become objective and shared with other people.”

- Jaron Lanier, SIGGRAPH Panel 1989, Virtual Environments and Interactivity: Windows to the Future. http://bit.ly/2uIl0ib

A researcher named Mel Slater has performed further work on this concept, coining the further terms presence and plausibility. Some people call all of this Immersion. 3D images on a screen are not as compelling as when you have an HMD on and your only sight is of the constructed 3D world. You feel a sense of presence due to the audio and visual cues, even if the rendering isn't just like the real world. Plausibility means that what you see has rules and works even if it's not exactly what you see in the real world.

The combination of being limited to seeing just what is in the HMD, along with the parallax and sterescopic views, and any audio (sounds are very important, if done well), will immerse you in the VR world. With all of these things, even if the graphics are not real, you will feel immersed, and it becomes real. For more academic detail see http://bit.ly/2vGFso0, although I'll explain more about this in this section.

This really works.

VR does not have to look anything like reality, but it will seem as if it is real. For example, take a look at the game Quell4D:

The graphics are block like, the images look nothing like reality. Yet, when the Ancient Triple Trunked Elephant Necro Shaman come at you, you will be scared. They seem real. To you, when you play the game, they are literally deadly real, meaning that your (in-game) person will die if you don't take them seriously.

VR simulations of fire will send about 10% of the people out of the room in a panic, even though the flames don't look anything like real flames.

VR is here. We do not have to wait until the graphics get better. Many people say this about VR, but it is because they haven't tried it yet and are making assumptions about what it has to be.

Dive in, the water is fine!

Note

So, Virtual Reality is something that will seem real, not something that necessarily looks real (but it helps if it does!). You do not have to wait until better graphics come around.

Why does Virtual Reality work if it doesn't look 100% real?

Our eyes are possibly our most important senses for showing us what the world consists of. If we substitute images and immerse someone with these images, they will start to seem real. When you first get into VR, your initial reaction is, "that doesn't really look real", but with a good VR setup, you will get to a point where you think "whoa, that was real" even though you know you are looking at what is basically a computer game.

Note

A fast frame rate (speed of display) and enough resolution will trick your brain into thinking what it sees visually really does exist. This is a powerful effect that most, but not all, people will have when immersed in such images (not everyone with normal eyesight sees 3D movies either).

In fact, the sense of reality is so good that people can get sick by watching VR. This happens because your eyes may say that's real, but your other senses, like your inner ear, says we aren't jumping 10 feet in the air. If your eyes think you are bouncing up in the air and your leg muscles (the sense of proprioception) say you're on the ground, your skin says you don't feel the wind and your inner ear says you are not tilting as you fly forward, your mind will be confused at a very deep level.

When your senses disagree powerfully, your body has a defense mechanism. It thinks you've been poisoned; as a result, your body will feel nausea and possibly even get sick. Your body is concerned that your eyes don't see what the rest of your body feels, so it may try to get rid of everything in your stomach, just in case something you ate had poisoned you.

Yes, it's no fun; different people will react differently.

However, not all VR does this! Generally speaking, poorly constructed VR will give you this feeling. Academic papers have been written on this effect. This book will summarize these discussions into a few simple rules that will make your VR much more comfortable for people.

Another important aspect of Virtual Reality is that it is something that you can interact with (the reality itself). This brings up mechanical difficulties; not everyone owns 3D controllers. We cover this in the section: User Agency - interacting with the world through controllers. True VR can be interacted with even if it is something as simple as gaze detection - look at something (gaze) and things happen - movement occurs, you're teleported, an animation plays.

Other types of VR; AR, XR, SR/FR

There is another type of imagery which is sometimes called VR, which is 360 Video. There are special video players that record in all directions. Sophisticated software will stitch the different camera inputs together to make a video stream that playback software will project to be all around you. When you turn your head, you seem to change your point of view inside the filmed world. It's as if you're in the real world, looking around at whatever you want to see.

The 360 Video looks possibly better than most computer graphic generated VR, but to me it's not reality because, at best, you are a disembodied ghost. Sure, the world looks great, but you can't reach out and touch things, because it's been filmed. The 360 Video and systems like it are beyond the scope of this book. Having said this, I do think the 360 Video is certainly a valid art form and something to pursue--just not covered by this book.

Please understand - I don't mean this dismissively of 360 video, just because it's not real VR. (Pronouce that second to last word like it had air quotes around it). 360 Video can be very heart warming, intense, emotional dramas. You do get a hint of presense, and the visuals are astounding. It is an area where we should see amazing art being made as more people are familiar with it and work out the details.

I am proposing a new term for 360 Video; Filmed Reality (FR) or Surround Reality (SR). (Although no one uses actual film for these, the phrase 'filmed' still means to record something through a lens, but maybe SR is better. You choose!)

There are other types of VR. So many that some people use the phrase XR which means (Anything) Reality; mainly to mean AR and VR. What is AR?

The HMD, which consists of some small displays and sophisticated optics, will allow you to see a stereo 3D image when you put the headset on. Most VR headsets intentionally block out the rest of the world while you are in them, to further immerse you in VR. This is an important component of VR, although there is a type of VR called Augmented Reality (AR) where the VR items are projected into the real world by wearing a type of HMD that is transparent. There are a number of manufacturers, although the Microsoft Hololens may be the best well known. There is also the game Pokemon Go, which is a type of AR. People hold up their cell phone, which shows images layered on top of reality. This is not a headset, but is still AR. Reality has been augmented with the Pokemon world.

VR systems can also be windows on world systems, although this is not commonly called VR today. In other words, a real, persistent 3D world that you sit at your keyboard and view through your screen. During the last wave of VR, years ago, this was referred to as VR, although today it's commonplace enough that people do not call it VR. You may have heard of World of Warcraft.

This is a type of Virtual Reality; although it's not (usually) in 3D, it is a persistent world in an alternate reality. It is also a full 3D world that you can see by looking at your screen; the screen transports you to a virtual reality so it's similar to a Windows on World system (although not head tracked).

Watching movies can be viewed as a valid form of VR; you are transported to another world, and, for a short time, feel as if you are immersed in the story. Television is a type of VR.

The first use of the term VR in fact referred to the theater. While many people today would say that's not VR, they spend much of their lives watching other realities and not paying attention to the people sitting next to them. How is this not Virtual Reality? You are pretty immersed with Dancing with the Stars, but do you know any of them? They are virtually real.

Still, this is not what most people think. This book will use the modern (2014+) interpretation of Virtual Reality as being something viewed through a VR headset or HMD of some type. Today, the term VR usually implies a headset or HMD, and is fairly often coupled with some form of hand controllers. Good, effective HMDs are all commercially available now to consumers. It is a great time to be interested in VR.

The nice thing about WebVR though, is that we can still see these VR worlds through the browser without an HMD; this is great for testing and for people without the hardware.

WebVR is very inclusive.

User agency - interacting with the world through controllers

The HMD is not everything, although it is certainly the most important part. Being able to see a VR world is great, but at some point you want to be able to interact with it. If the world was static, you would feel like a disembodied ghost. It is Virtual Reality when you can interact with the world.

Eventually, something like full suit haptics (physical feedback) and body tracking, along with sophisticated software, will allow us to reach out and touch the virtual world. This is something to look forward to in the future.

For now, the way that we usually interact with the world is through various hand-held controllers. Different controllers have radically different capabilities and requirements. Controllers for high-end (but still consumer available) VR setups, like the Rift and Vive, work considerably differently from mobile VR controllers. We'll discuss the higher-end systems first, then discuss mobile VR controllers.

High-end controllers for PC, Mac, and Linux

With PC VR, such as the HTC Vive or Oculus Rift, controllers give a very important ability to interact with the VR world. These controllers are tracked in 3D space so that the software knows where they are. Developers can code these to look like hands, guns, and so on. This allows you to reach out and touch the world around you--very important to make the Virtual Reality you are interacting with actually something you can interact with.

To do this, both the Oculus and Vive controllers require external tracking hardware. With the Vive, these are lighthouses or VR base stations that are placed at the corners of the VR area. (There is a diagram here, and it is available at http://bit.ly/VIVEManual). These small unobtrusive cubes send out IR tracking signals that the controllers and headsets pick up and use to accurately position them in the real 3D world. With the Rift, there are two or three sensors that also track the devices to give them a real-world position:

The base stations and tracking hardware are really important for the HMD itself as well.

This tracking of your real world position (your actual head/hands) is what makes movement, turning your head, moving your hands/controllers seem real, because the position, orientation, and movement of the headsets and controllers is so precisely tracked in real 3D space, any head movement seems real once the software displays the VR world to the user.

In practice, this means that the PC controllers will seem right where you see them. My first experience with an HTC Vive at a tech demo was amazing--I'd put on the HTC Vive headset, and, in the virtual world, saw the controllers in front of me. I expected to sort of fumble around until the controllers were where I thought they would be. I reached out, and my fingers felt the controllers exactly where my eyes saw them--through the HMD.

I was hooked! The virtual world really was Virtual Reality! The phantom controllers I was looking at were real even though I knew I was looking at a small screen in front of my face.

How do they work?

HTC Vive

The HTC Vive uses two small square cubes, called base stations or Lighthouses, at opposite ends of the area they are covering. These send out beams of infrared light over 120 degrees; this means if they are in a corner, they can be a few inches away from the corner and still cover the walls (otherwise, you'd have to dig a hole in your wall to put the lighthouses at the right spot!)

Usually, you mount the two base stations at opposite sides of a room, up to about 16 feet or 5 meters apart, and above head height, around 2 meters or 6 feet 6 inches. It's okay if you're taller--mount them higher!

The base stations can also be mounted on microphone stands with the right adapters or with custom stands. Not everyone has a large living room, so these arrangements may help fit it in.

The Vive can also be used in a seated configuration, although the real point is what is called Room Scale.

Room Scale VR means that you can walk around in the VR world, as if you are walking around the real world. No teleporting or other tricks needed. Of course, the area needs to be clear of furniture, which is one of the issues with VR in general; not everyone has a big room they can clear.

The Vive will keep you safe by showing bounds or guards at the edges of your space, if you get too close.

Note

Make your room bounds slightly less than the actual room, if it's a wall or other area. If it's a couch or chair, you can go right to the end of the chair.We do this so you don't hit your arm on the walls. This is easy to do if you are standing up against the wall, but still in the virtual world and therefore can't see the wall, and you swing your arm, your hand won't go through the real wall!

Going to the edge of a sofa is nice, as your shins will run into the sofa before your hands hit the wall. In practice it's not that much of a problem, as you'll see the guards before you get close. Pay attention to the Vive/Steam VR tutorial!

The HTC Vive works by having some Inertial Measurement Units (IMUs), which detect where the HMD as well as the controllers are located. These IMUs drift, so the base stations have an infrared beam that sweeps over the room. When the controllers, trackers, or HMD detects these beams, they recenter themselves. This re-centering is completely undetectable. The advantage of such a system is that, even if a controller goes out of sight from one of the base stations or Lighthouses, the VR system still knows where that item is and where it is pointed.

The overall effect is precision and presence, although the main effect is stability. If you cross your hands and a controller briefly goes out of sight of the base station, the controller won't lose lock.

Note

Try not to put your VR space in an area with a lot of windows or mirrors. The Infrared beams can reflect off of them, leading to instability.

Oculus Rift

The Rift first came out just as a headset, without controllers. It's initial base stations are two cameras that you place on the left and right of a desk; they point at the HMD and are used to position it in the world.

Soon after, Rift added the capability of third camera; with three cameras, you can do room-scale VR. The positioning of them is slightly different than the Vive; take a look at the Rift documentation for the best positioning.

Note

Be careful with cables. As I wrote this book, the Rift is cabled directly into the back of the PC. If you trip over the cables, you could yank them out of the PC fairly hard, leading to damage. The Vive has a breakout box, so if you trip over a cable, you'll hopefully pull it out of the box. Don't trip over the cables.

The purpose of this book is not to be an analysis of why the Vive or Rift is better or worse than another; both of them work roughly the same way with the base stations/cameras helping the controllers and HMD to keep track of their position and rotations. Here is a typical setup:

In it, the Vive base stations are mounted on the walls; we have a desktop PC and a VR user viewing a 3D model as if it was real. The VR user is holding two Vive controllers; the virtual image is holding a Xbox-style game controller.

This image also shows an Oculus Rift 3 camera tracker system. They are the light-gray items sitting to the left and right of the screen, and on the credenza on the back side of the couch (right in front of us).

That's right, the cybernetic robot is the user. She doesn't need an HMD; the system pumps in the video straight to her eyes. The virtual object is the human seeming to sit at the desk.

The preceding figure is the view that a third person in front of the couch might see of the scene.

What the lighthouses see is actually a little different, but interesting. They actually have a couple of infrared bars that sweep across the view and the controllers see these lines tracking across. When they do, the controllers (and HMD) will resync their inertially tracked positioning. This means that even if a controller is out of view of a base station, it still keeps tracking, although you don't want to hide a controller for very long. Inertially tracked systems will drift. The visual issue with drift is that your arm would seem to slowly move away from your body - which is obviously highly disconcerting. The Vive lighthouses and Rift cameras keep the drift from happening. The Vive angle of view that a lighthouse projects is around 120 degrees. This is what the rear, rightmost lighthouse would see if there were a camera with this field of view at the lighthouse:

You can see both controllers and the HMD through this Lighthouse. There is, however, an issue. Notice the red circles--the large mirror on the left is actually a gigantic TV, but it's shiny. As a result, the Lighthouse IR beams will bounce off of it and the controllers will sense two beams: one directly and one reflection.

This may cause the HMD and your point of view to jump or your controllers to move about inexplicably.

Note

Avoid shiny objects, mirrors, and windows in your VR room. You might need to draw drapes, or even throw sheets over TV's, glass china cabinets, and the like. Art requires sacrifice!

From the other lighthouse, one of the controllers is blocked, but it is still tracked 100% through its internal inertial tracking and the other lighthouse.

Mobile VR

For mobile VR, there are also the Google Daydream and Samsung Gear VR controllers. Due to the simpler hardware used, which keeps the price more reasonable, these are not fully 3D tracked.

With mobile VR, due to the absence of room tracking external sensors, that the Vive and Oculus both have, the VR controllers are not so precisely tracked. In practice they will seem just as real, but will periodically drift. It's as if your hand slowly moved off to the right without your control. Mobile VR thus has a reset controller button that will move the controller to a predefined position, such as near your hip. Your hand might be held straight out, but if you hit the Home button the VR display will show your hand now at your hip.

This might take some getting used to. There are advantages to this setup; it's cheaper, requires less external hardware, and there are many more such systems out in the world. However, the PC hardware does give a better VR experience.

The other wrinkle with the mobile controllers is that there are only three Degrees of Freedom (DOF). This means they track tilt, yaw, and roll, but not position; if you move the controller flat to your left, in the game your controller hasn't moved at all. This is why you can't grab things with a mobile controller. The Vive and Rift both have 6 DOF controllers, so you can move them around and grab things.

Rendering hardware

To avoid VR sickness, you need a fast frame rate. What is frame rate? This is how fast your computer can generate the images on the screen. A lot depends on the complexity of the scene, of course; showing a cube and a box is a lot faster than showing the city of Los Angeles with all of its buildings.

You can control this, of course, when you design the VR world you will implement.

Each image has to be generated, in real time. Most VR headsets try for 90 Hertz. Hertz refers to the frequency - in cycles per second, or in this case, frames per second.

The difficulty of VR is that nothing can slow down this frame rate. If something has to load, or a web page fetched, if you slow down the frame rate slightly, people get woozy.

There are two ways to speed up the frame rate. One is to have less scene complexity, the other is to have a fast computer.

A classic line in the Movie Jaws is when they discover that the shark is much bigger than they expect and it tears up their boat. Roy Scheider says, "You're going to need a bigger boat."

To view VR, you're going to need a bigger computer.

Fortunately, computers keep get faster and faster. By computers, we also mean high-end smart cell phones. For the worlds we build here, a reasonably fast smart phone should be OK.

Scene complexity is a bit of a dilemma; you want a rich, detailed virtual world, but you also want that virtual world to render quickly. By render quickly, we mean 90 frames (updates) per second, as discussed previously. You also need to know your target audience in terms of hardware support. Are they all on high-end PCs with pairs of thousand dollar video cards? (somewhat an overkill; I'm making a point here.) Or are they on last year's cell phone models with a $10 cardboard box and some lenses? If you know your potential target audience, you can develop a VR application that works well with their system.

The United States Marine Corps has a saying: "Train as you would fight." During World War Two they practiced the combat operations of amphibious landings off the coast of Southern California. When they had to do this during the War in the Pacific, they hadn't planned for coral reefs. As a result, they developed a doctrine that you should train people in the same, or reasonably similar, environment that they were expected to fight in.

While a good VR experience is (hopefully) not life or death, this is still valuable advice. If you think most of your customers or consumers of your VR app will be on last year's cell phone, then test with last year's cell phone. If you think they will be on high-end PCs, test with a high-end PC.

Don't assume, if your VR app is slow, that customers will have much better computers and everything will be okay. Get something similar to what they use, and then you will suffer through the nausea and vertigo before your customers will, and then recode or simplify your scenes to be fast enough.

How much hardware is enough? For that, you should consult the minimum specifications of the headsets you plan to target. As this can change, I won't summarize it in this book, but the guidelines that different VR manufacturers give is good advice.

You may need a bigger PC (or cell phone); this is the price you pay to be an early adopter!

How to view VR?

To view VR, you need some type of headset or HMD. In the old days VR was also characterized by 3D images on 2D screens. In effect, VR back then meant any 3D program viewed with any device — basically sitting in front of your PC like normal, but this is not truly immersive. Today, VR means with an HMD/headset; so to view one, you need a headset.

Ironically, React VR works fine in a browser as a 3D world as well, and could be used to make parallax–capable web pages, although that's a bit overkill.

VR can be dangerous

You probably think, this is pretty safe. However, one VR headset came with 33 pages of warnings. Read them.

Most of the warnings are common sense, for example, don't wave your hands around if you are next to things or people. With what are essentially blinders on, you can really smack your hand. Philosophically, I don't believe in the nanny state, but you can really get hurt with VR. Imagine if someone gave you a blindfold, told you to put it on, and then wander around your house. You might be a little uncomfortable.

That's about what we're going to do in this book, except with the added wrinkle that you'll be wandering around in a state of wonder and excitement. There are plenty of YouTube videos where people smash walls, run into them, knock over lamps, and so on. They look silly, but when you have an HMD on, you are totally immersed in the virtual world and wouldn't think of pulling your punch. So, make sure you clear out of the room and warn friends not to come in.

This includes your furry friends. It's more difficult to keep out pets of your VR area, but a good idea as they won't understand that HMD has distracted you, and you can't see them. If you can find a way, it would be a good idea to prevent them from getting underfoot or you may step on them inadvertently.

VR is safe; use responsibly.

VR Headset options

With WebVR, there are a few options. I'll keep this to the simplest, most available headsets. You can certainly use Open Source Virtual Reality (OSVR), which is actually a hardware platform, but you'll need to figure out what browser to use, and so on. Some of the terms, such as Gaze, will be covered later in the UI section. For now, Gaze movement means you need to stare at something for the UI to move you there or for you to select an object, typically.

Here are the various mainstream WebVR options (you can read about this at https://webvr.info/):

Type	Controls	Movement	Cost
Gear VR (Mobile)	1 handheld, HMD	Gaze/Touchpad	Medium
Daydream VR (Mobile)	1 handheld	Gaze/Touchpad	Medium
Cardboard/other headsets	None (click possible)	Gaze selection	Low
HTC Vive	Tracked, 2 controllers	Walk around	High
Oculus Rift 2 Camera	Keyboard/Joystick	Gaze Selection	High
Oculus Rift 3 Camera	Tracked, 2 controllers	Walk around	High

Types of headsets

Broadly characterized, they can be a PC-connected headset or a mobile headset. Some types of stand alone headsets, such as the Hololens or Vive stand–alone VR headset, include a fully working PC, so they are really more like a mobile headset, but don't require a PC.

Mobile headsets

Mobile headsets really just use your cell phone to display their data and put you into the VR world. As such, the performance is entirely based on what your cell phone can do.

This is a time when bigger really is better.

There is a limit though; there are some headsets that use tablets, but they are extremely heavy and don't really offer any advantages over a smaller mobile device.

While you use a mobile headset, you'll have battery life, weight, and control issues. There are various VR controllers in the market, as well as bundled options such as the Samsung Gear VR and Google Daydream, which include a controller as well as a case you put your phone into.

The nice thing about these bundles is that the phones are generally certified to work correctly and the software is easy to use. You can build your own VR headset/controller combination.

Mobile headsets can also be as simple as a box with some lenses in them, although there is actually a lot of math in the sizes and specifics of the optics. The most commonly referred to one is Google Cardboard; Google doesn't sell them directly but companies can implement Cardboard viewers. There is also the Unofficial Cardboard as well as a number of reasonably priced better built holders that you can put a cell phone into.

Generally speaking, most of them do not have sensors. Some have a small lever that will touch the screen, allowing some controls other than movement.

You can also purchase separate Bluetooth controllers, although they will most likely not have three–dimensional positioning. We cover different types of controllers in Chapter 11, Take a Walk on the Wild Side.

Some VR headsets work with glasses and some do not--a lot depends on your face size, the size of the glasses you use, and your exact vision issues. I have both a nearsighted eye and a farsighted eye, and do not need glasses (for either eye!), but your mileage may vary. I highly recommend trying out a headset before purchasing, or purchasing from a source that has a good return policy.

At the high end (of mobile headsets), there is the Samsung Gear VR and Google Daydream. These offer a well–constructed headset, which you again put a cell phone into, as well as a separate controller.

The controller is the most important part of this (although the headset is also worth it as they are much more comfortable than even the best Cardboard viewers). The controller in both the Daydream and Gear VR bundles are Bluetooth, meaning wireless, and are sort of tracked.

They have sensors in them that will detect movement, but are not precisely positioned in space. As a result, they have center buttons on them. This is because the type of three-dimensional positional sensors in these units will drift over time. In the VR world, it will seem as if your hand/control/gun (or whatever visual representation the controller has) will drift away from you or even move behind you! This can be very disconcerting. If/when this happens, merely use the appropriate button to recenter your controller.

Higher–end PC setups have different types of tracking, and re-centering is not usually required. However, note that they need initial calibration/setup and can have tracking issues as well.

Note

Notes on GearVR A few things with the GearVR I did a little wrong. There's an extra elastic strap that I thought was for slack or something; its not. They tell you to put the straps on, but neglect to mention this extra strap is to hold the controller. Flip ahead to the controller part before putting the straps on the headset. The controller should actually be the part you fiddle with first. You need to pair it and carry out some downloads, and that can't really be done when the headset is on, so do this part first.

PC, Mac, and Linux headsets

Most people think choosing a PC headset will be between the HTC Vive or the Oculus Rift, but there are dozens or hundreds of PC type headsets.

All of them will perform based on how well your PC performs. Folks, this is where a Mac has a bit of a disadvantage; you need a fast video card, and Mac's typically are fast enough for graphics and some game play, but not for VR. However, Apple has introduced VR ready PCs. When you decide what platform to do React VR with, consider this.

As this is written, Mac support for the Oculus Rift or HTC Vive is experimental at best, so the steps and examples will assume you are using a PC. Linux support was promised for several of the headsets, but as this book is written it is experimental at best. If you are using Linux, you will need to check the documentation and/or follow along the Windows examples as best you can.

Most of the React VR demos have simpler geometry than a lot of VR worlds, so they will run on comparatively smaller hardware. Check with the manufacturer of your headset for minimums; don't think you can get by with less than the minimum. You'll get sick or have a less than comfortable experience.

Of the whole marketplace, there are really just two headsets we will cover in this book; the HTC Vive and the Oculus Rift. If you have a different headset, the samples should work OK, but you may need to fiddle with them slightly.

Generally speaking, PC, Mac, and Linux headsets will work with Firefox or an experimental browser, Servo.org. An experimental version of Chrome (Chromium) may also work to view WebVR. See the complete up to date list at webvr.info.