What do you get with eBook?

Instant access to your Digital eBook purchase

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Buy Now

Product Details

Publication date : Apr 28, 2017

Length 376 pages

Edition : 1st Edition

Language : English

ISBN-13 : 9781786464729

Vendor :

Microsoft

Category :

Mobile

Languages :

C# (Beginner)

Concepts :

Augmented Reality

Tools :

Universal Development Platform (Beginner) , .NET (Beginner)

HoloLens Beginner's Guide

Chapter 1. Welcome to the New World

We live in very exciting times. Technology is changing at a pace so rapid that it is becoming near impossible to keep up with these new frontiers as they arrive; they seem to arrive on a daily basis now. Moore's Law continues to stand, meaning that technology is getting smaller and more powerful at a constant rate. As I said, very exciting.

One of these new emerging technologies that finally is reaching a place more material than science fiction stories is Augmented or Mixed Reality. Imagine a world where our communication and entertainment devices are worn, and the digital tools we use, as well as the games we play, are holographic projections in the world around us. These holograms know how to interact with our world and change themselves to fit our needs. Microsoft has led the charge by releasing such a device: the HoloLens.

The Microsoft HoloLens changes the paradigm of what we know as personal computing--we can now have our Word window up on the wall (this is how I am typing right now), we can have research material floating around it, and we can have our communication tools, such as Gmail and Skype, in the area as well. We are finally no longer trapped by a virtual desktop, on a screen, sitting at a physical desktop; we aren't even trapped in the confines of a room anymore.

What exactly is the HoloLens?

The HoloLens is a first of its kind head-worn, standalone computer with a sensor array, which includes microphones and multiple types of camera, a spatial sound speaker array, a light projector, and an optical waveguide.

The HoloLens is not only a wearable computer, it is also a complete replacement for the standard 2D display. It has the capability of using holographic projection to create multiple screens throughout an environment and fully 3D-rendered objects. With the HoloLens sensor array, these holograms can fully interact with the environment you are in.

The sensor array allows the HoloLens to see the world around it, to see the input from the user's hands, and for it to hear voice commands. Although Microsoft has been very quiet about what the entire sensor array includes, we have a good general idea about the components used in the sensor array; let's take a look at them:

One IMU: The Inertia Measurement Unit (IMU) is a sensor array that includes an accelerometer, a gyroscope, and a magnetometer. This unit handles head orientation tracking and compensates for the drift that comes from the Gyroscope's eventual lack of precision.
Four environment understanding sensors: These together form the spatial mapping that the HoloLens uses to create a mesh of the world around the user.
One depth camera: This is also known as a structured light 3D scanner. This device is used for measuring the 3D shape of an object using projected light patterns and a camera system. Microsoft first used this type of camera inside the Kinect for the Xbox 360 and Xbox One.
One ambient light sensor:Ambient light sensors or photosensors are used for ambient light sensing and proximity detection.
2 MP photo/HD video camera: This is used for taking pictures and video.
Four-microphone array: These do a great job of listening to the user and not the sounds around them. Voice is one of the primary input types with HoloLens.

Putting all of these elements together forms a Holographic computer that allows the user to see, hear, and interact with the world around them in new and unique ways:

What you need to develop for the HoloLens

The HoloLens development environment breaks down into two primary tools, Unity and Visual Studio. Unity is the 3D environment that we will do most of our work in; this includes adding holograms, creating user interface elements, adding sound and particle systems, and other things that bring a 3D program to life.

Visual Studio on the other hand is the glue that makes everything work. Here, we write scripts or machine code to make our 3D creations come to life and add a level of control and immersion that Unity cannot produce on its own.

Unity

Unity is a software framework designed to speed up the creation of games and 3D-based software. Generally speaking, Unity is known as a game engine but the more apparent the holographic world becomes, the more we will use such a development environment for different kinds of applications.

Unity is an application that allows us to take 3D models, 2D graphics, particle systems, and sound to make them interact with each other and our user. Many elements are dragged and dropped and plugged and played; what you see is what you get. This can simplify the iteration and testing process. As developers, we most likely do not want to build and compile little changes we make in the development process forever. This allows us to see the changes in context to ensure that they work; then once we hit a group of changes, we can test them on the HoloLens ourselves. This does not work for every aspect of HoloLens-Unity development, but it does work for a good 80%-90%.

Visual Studio community

Microsoft Visual Studio Community is a great, free Integrated Development Environment (IDE). Here, we use programming languages, such as C# or JavaScript, to code changes in the behavior of objects, and generally make things happen inside of our programs.

HoloToolkit - Unity

The HoloToolkit-Unity is a repository of samples, scripts, and components to help speed up the process of development. This covers a large selection of areas in HoloLens development, such as the following:

Input:Gaze, gesture, and voice are the primary ways in which we interact with the HoloLens.

Sharing:The sharing repository helps allow users to share holographic spaces and connect to each other via the network.

Spatial Mapping:This is how the HoloLens sees our world. A large 3D mesh of our space is generated and give our holograms something to interact with or bounce off of.

Spatial Sound:The speaker array inside the HoloLens does an amazing job of giving the illusion of space. Objects behind us seem like they are behind us.

HoloLens emulator

The HoloLens emulator is an extension to Visual Studio that will simulate how a program will run on the HoloLens. This is great for those who want to get started with HoloLens development but do not have an actual HoloLens, yet. This software does require the use of Microsoft Hyper-V, a feature only available inside the Windows 10 Pro operating system. Hyper-V is a virtualization environment, which allows the creation of a virtual machine. This virtual machine emulates the specific hardware, so one can test without the actual hardware.

Visual Studio tools for Unity

This collection of tools adds IntelliSense and debugging features to Visual Studio; if you use Visual Studio and Unity, these are a must have:

IntelliSense:An intelligent code completion tool for Microsoft Visual Studio. This is designed to speed up many processes when writing code. The version that comes with Visual Studios tools for Unity has Unity-specific updates.
Debugging: Before this extension existed, debugging Unity apps proved to be a little tedious. With this tool, we can now debug Unity applications inside Visual Studio, thus speeding the bug squashing process considerably.

Other useful tools

The following are some the useful tools that are required:

Image editor: Photoshop and Gimp both are good examples of programs that allow us to create 2D UI elements and textures for objects in our apps.
3D modeling software: 3D Studio Max, Maya, and Blender are all programs that allow us to make 3D objects that can be imported in Unity.
Sound editing software: There are a few resources for free sounds on the Web. With that in mind, Sound Forge is a great tool for editing those sounds, layering sounds together to create new sounds.

Installing the software

Now, we will get our development environment installed and set up so that we can start building the next killer app for HoloLens. We will start with our main tools, Visual Studio and Unity HoloLens technical preview, and then add the extensions that will help speed things along.

Installing Visual Studio

Microsoft has made the installation process for Visual Studio very simple. Just be warned that it can take a while to download and install it:

Navigate to http://www.visualstudio.com/en-us/visual-studio-homepage-vs.aspx.

Click on Download Community 2015; this is a free version of Visual Studio that works well with Unity and the HoloLens environment. By default, you should be installing at least Visual Studios 2015 Update 3.

Run vs_community. exe and complete the installation process.
Create a Microsoft Developers account (this is a requirement for the HoloLens development).
The first time you run VS2015, you will be prompted to input your account credentials.

Unity HoloLens technical preview

Now, we will install Unity HoloLens technical preview. At the time of writing this, version 5.4. 0f 3 is the most recent and stable version. This is a custom version of Unity made to work specifically with HoloLens. As a result, it does automate a few processes for HoloLens developers:

To install Unity HoloLens technical preview, we will need to perform the following steps:

Go to https://unity3d.com/partners/windows/hololens.
Scroll down to the bottom of the page.
Click on the Unity Download Assistant.
Follow the instructions to complete the installation process.
You will need to create an account with Unity.

Visual Studio 2015 tools for Unity

In your web browser, go to https://visualstudiogallery.msdn.microsoft.com/8d26236e-4a64-4d64-8486-7df95156aba9. Later, click on the Download button and run vstu2015.exe:

HoloLens emulator

To install the HoloLens emulator, perform the following steps:

Go to http://go.microsoft.com/fwlink/?LinkID=823018.
Run EmulatorSetup.exe once it finishes downloading.
Install the emulator.

Note

This will not install if you do not have Windows 10 Pro installed on your development PC.

This installation process can take a while.

HoloToolkit-Unity

To install HoloToolkit-Unity, perform the following steps:

Enter https://github.com/Microsoft/HoloToolkit-Unity in your browser.
Click on Clone or download.
Click on Download ZIP.
Once the ZIP file is downloaded, unzip the file to your desktop; we will copy this into our project later.

Description

HoloLens revolutionizes the way we work and interact with the virtual world. HoloLens brings you the amazing world of augmented reality and provides an opportunity to explore it like never before. This is the best book for developers who want to start creating interactive and intuitive augmented reality apps for the HoloLens platform. You will start with a walkthrough of the HoloLens hardware before creating your first app. Next you will be introduced to the various HoloLens sensors and find out how to program them efficiently so that they can interact with the real world seamlessly. Moving on, you will learn how to create smart animations and add video overlay that implements real-time tracking and motion-sensing abilities to your HoloLens app. Finally, you will learn how to test your app effectively.

What do you get with eBook?