What do you get with eBook?

Instant access to your Digital eBook purchase

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Buy Now

Product Details

Publication date : Jun 20, 2017

Length 336 pages

Edition : 1st Edition

Language : English

ISBN-13 : 9781787281943

Vendor :

Microsoft

Category :

Mobile

Languages :

C# (Intermediate)

Concepts :

Augmented Reality

Tools :

Microsoft HoloLens (Beginner) , Unity (Intermediate)

HoloLens Blueprints

Chapter 1. Digital Reality - Under the Hood

Welcome to the world of Digital Reality. The purpose of Digital Reality is to bring immersive experiences, such as taking or transporting you to different world or places, make you interact within those immersive, mix digital experiences with reality, and ultimately open new horizons to make you more productive. Applications of Digital Reality are advancing day by day; some of them are in the field of gaming, education, defense, tourism, aerospace, corporate productivity, enterprise applications, and so on.

The spectrum and scenarios of Digital Reality are huge. In order to understand them better, they are broken down into three different categories:

Virtual Reality (VR): It is where you are disconnected from the real world and experience the virtual world. Devices available on the market for VR are Oculus Rift, Google VR, and so on. VR is the common abbreviation of Virtual Reality.
Augmented Reality (AR): It is where digital data is overlaid over the real world. Pokemon GO, one of the very famous games, is an example of this globally. A device available on the market, which falls under this category, is Google Glass.Augmented Reality is abbreviated to AR.
Mixed Reality (MR): It spreads across the boundary of the real environment and VR. Using MR, you can have a seamless and immersive integration of the virtual and the real world. Mixed Reality is abbreviated to MR.

This book is mainly focused on developing MR applications using Microsoft HoloLens devices.

Although these technologies look similar in the way they are used, and sometimes the difference is confusing to understand, there is a very clear boundary that distinguishes these technologies from each other. As you can see in the following diagram, there is a very clear distinction between AR and VR. However, MR has a spectrum, which overlaps across all three boundaries of real world, AR, and MR.

Digital Reality Spectrum

The following table describes the differences between the three. Later in the chapter, we will go through the details of each one of them.

Virtual Reality	Augmented Reality	Mixed Reality
Complete Virtual World User is completely isolated from the real world Device examples: Oculus Rift and Google VR	Overlays Data over the real world Often used for mobile devices Device example: Google Glass Application example: Pokemon GO	Seamless integration of the real and virtual world Virtual world interacts with real world Natural interactions Device examples: HoloLens and Meta

Let's explore VR, AR, and MR in more detail.

Virtual Reality - what it is?

The dictionary definition of VR reads, "

a realistic and immersive simulation of a three-dimensional environment, created using interactive software and hardware, and experienced or controlled by movement of the body

". However, we will say that there is one and only one primary purpose of the VR, that is, to make you believe that you are somewhere else, for example, in a gaming environment, a war zone, or touring some city.

Virtual Reality illustration

Realizing Virtual Reality

A person wearing a VR device/headset will be able to view the virtual environment, move around wearing it, and interact with items within that virtual environment. Most Virtual Realities are created using a head-mounted helmet or a set of goggles, which are given the generic name: head-mounted display (HMD). HMD is usually connected to a computer or a smartphone, which does all the 3D rendering work, and HMD is just used to display that rendered 3D content. HMD completely covers user's view, by covering both eyes. As a result, the user is completely cut off from the outside world and completely focused on the virtual or digital world.

HMDs basically consists of stereoscopic displays and motion tracking hardware. The way HMDs implement stereoscopic displays is by generating different image for each eye, which results in generating the illusion of depth. Within HMDs, motion tracking hardware mostly consists of a gyroscope and accelerometer to measure motion/position changes. This helps in simulating real-world experiences.

Virtual Reality in the field

There are various implementations of VR in diverse fields currently, and numerous new ones are coming up every day. It will be a very interesting domain for the next few years, where we will globally see lots of new fields picking up VR and merging it with other upcoming technologies. One such technology, which we can think of, is Artificial Intelligence (AI); Are we talking about the movie "Matrix" here?

While it's too early to predict possibilities in the future, let's explore some of the current fields where VR is being implemented. They are as follows:

Gaming: Gaming was one of the first fields to pick up, start using, and commercialize VR. Apart from single-user VR games, multi-user online VR games are also available today. With the availability of low-cost devices, such as Google VR and smartphones, VR games/applications are within the reach of everyone.
Tourism: Tourism is another field that has picked up VR. VR is used for generating stereoscopic 360-degree panoramic views, where the user interact with the virtual place and explores it. So, no more traveling or air-travel; just wear your VR device and explore new places every day.
Education: Distant learning, visualization, and interactive teaching a few global examples of VR implementations within the education field. Earlier, teachers used to describe the scene and students used to imagine the possibilities. Now, they just visualize new possibilities and interact with them, and spend more time in thinking what's next.
Architecture: Architecture is a field where VR has made life easier for the architects by making it easy to create prototypes. Now, they do not have to create cardboard prototypes or sketches to convey their thoughts and ideas. VR has made it possible for them to share ideas early in the cycle of development and receive early feedback. This saves lot of time in rework.
Enterprise productivity: Especially, for enterprise training scenarios, VR is used very frequently. Scenarios in consideration can be simulating war situations for the armed forces, flight simulation for pilots, medical diagnostics simulations for doctors, and many others.
Web content: With the support of WebGL technology by most of the latest browsers available in the market, rendering Virtual 3D content has become very easy. A lot of commercial websites for automobile companies, for example, have started publishing their automobile features in the form of 3D content, which has made it very interactive for end users to visualize the content.

Augmented Reality - what it is?

AR is all about bringing digital information and overlaying it over the real environment. The only difference from VR is that it creates a totally artificial environment around you, whereas AR uses the environment around you and overlays the digital information over it. For VR, you will require a VR device, but for AR it can be achieved by simply using a smartphone, tablet, or dedicated VR devices.

Visualization techniques for Augmented Reality

AR can be categorized into three different types based on the display types:

Screen-based
HMD-based
Projection-based

Screen-based

A common example of screen-based AR is overlaying digital information over smartphones or tablet camera displays. For example, you switch on and point your smartphone/tablet camera over an object, and the application recognizes that object and overlays that object information, such as the price or description, as digital information over the object image.

Another example of screen-based AR is the video game Pokemon GO, in which, based on the user's location and direction, digital characters are overlaid over video images.

User is viewing augmented object using smartphone

In the preceding figure, a user is viewing an augmented object, that is, the elephant's digital object is overlaid over the video frame. This is an example of screen-based AR.

Augmented Reality head-mounted globally displays

Using AR HMD devices, digital information is overlaid directly over the user's view of the real world. So, there is no need to hold any screen to view the digital information. Digital information is directly rendered over the view area of the user's eyes.

HMD with information overlaid over the real view

In the preceding image, the user is using a head mounted AR device to view the physical element in front of him. Within the view of the user, the AR device embeds information about the physical object and provides them with a more immersive experience.

Projection-based

User projection-based AR, a projection is rendered on the target surface itself. This target surface could be anything, such as building, person, room, and so on. To render this kind of projection, the system needs to know the exact dimensions of the target surface, and then using single/multiple projectors, it renders the projection on the target:

Phone dialer pad projection over the hand

In the preceding image, the user is viewing a projection of the phone dialer on the palm of their hand.

Augmented Reality in the field

Applying AR is quite different from applying VR. AR applications are more focused on integration scenarios with the real world. Some are as follows:

Gaming: With the release of the Pokemon GO game by Niantic, the demand for AR games has picked up drastically in the market. A lot of companies are coming up with AR games and launching them.
Architecture/construction/archaeology: AR can be used to visualize completed buildings for in-progress or upcoming building construction. Digital architecture/building images can be overlaid over the real view of the property or ground.
E-Commerce: Businesses can't reach each customer with demo-able physical product, and opening showrooms is every city is a costly business, especially for start-ups and newcomers in the market. So, reaching out to customers through AR, online furniture retail companies, for example, allows users to consume AR through smartphones, let them visualize their furniture products, and enable them to design their house interiors.
Education: Traditional education systems require hands-on instructions and real/prototype equipment to explain it better to students. With AR, teachers and students can visualize the same equipment in virtual mode, with very similar training instructions as with an actual device. Another implementation of AR in the education system is distance learning, where students and a teacher very far away, can use VR devices for interactions and virtual classrooms.
Medical: In the medical field, there are different imaging techniques for various requirements, such as X-ray, ultrasound, and magnetic resonance imaging (MRI), but there is no consolidated view for medical practitioners. AR could be used for these scenarios, where the output from different imaging techniques could be overlaid over the patient and give a consolidated view to medical practitioners.
Industrial design: AR is being used for designing, sharing ideas, and brainstorming design views among different designers and architects and supplies quick feedback and brainstorming cycles. Earlier, the same process used to take a long time, as designers used to create physical prototypes and then discuss them.
Travel/navigation/tourism: AR is also used for developing navigational applications; for example, travel-related digital information is overlaid on the vehicle windscreen, which helps the driver in real-time navigation without looking at any other device, such as a GPS or smartphone. AR is also used to develop travel-related applications, which help the user with location-specific information of the place where the user is currently placed, such as historical information about tourist places, or information about nearby restaurants and cuisines.

Mixed Reality - what it is?

We talked about VR and AR. Now, think of MR as a hybrid of VR and AR. Using MR, a user can view the real world just like in AR, but can also view virtual objects as in VR. The difference with MR is that those virtual objects are referenced or anchored with the real space. That means that the virtual object is interacting within the real world.

For example, suppose you have a physically real table in front of you, and now using MR you are viewing a virtual ball; if you drop a virtual ball object on a table, in MR, that virtual ball will interact with the physical table. This means that it will bounce on the physical table, and roll off from the table to the ground.

The immersive experience of the virtual world with the real world is so strong in MR that it is very hard to break that illusion.

A user viewing a MR object in real space

At the time of writing this book, there are very few MR devices available on the market; Microsoft HoloLens is one of them. This book is focused on developing the MR experience, using HoloLens.

Mixed Reality in the field

MR is still evolving, and lot of different industries are still exploring possibilities with it. The following are some of them, which represent early progress:

Education and training: MR has already started transforming the way education and industrial training are delivered. Companies have started using MR as a medium for delivering readiness/training content to participants.
Exploring new markets: MR is giving an opportunity to enterprises to rethink their marketing strategy and target new customers. Few popular automobile companies have started using MR device in their specialized car showrooms. With this, they can target consumers early in the sales cycle, even before the first model of the car is available.
Healthcare: The possibilities for MR in the healthcare industry are enormous. Physicians can use MR devices to visualize consolidated reports, and these devices can also be used as a guidance device during operations or diagnosis.
Gaming: The gaming industry may become one of the biggest consumers of MR devices. There are so many possibilities for MR games that can interact with the real world.

Description

Do you want to create stunning applications with HoloLens? Are you a developer who is fascinated with Microsoft HoloLens and its capabilities? If so, this is the book for you. This book introduces and demystifies the HoloLens platform and shows you different ways of interaction with computers (mixed-reality). You will start your mixed-reality journey by understanding different types of digital reality. You will learn to build your first holographic app. Also, you will understand holographic application integration possibilities within Line of Business Applications using Azure. Moving ahead, you will create Integrated Solutions using IoT with HoloLens. Gradually you'll learn how to create and deploy apps on a device. You will learn to publish application to the store; if you are an enterprise developer, you will also manage and distribute applications for enterprise-enabled or domain-joined HoloLens. Finally, you will develop an end-to-end realistic holographic app, ranging from scenario identification to sketching, development, deployment, and, finally, production.

What you will learn

Interact with holograms using different interaction models Develop your first holographic app Integrate holographic applications with cloud systems Visualize data feeds coming from the cloud through holograms Manage the application distribution of enterprise-enabled HoloLens Integrate HoloLens applications with services deployed on Azure Identify and create 3D Assets and Scenes Use HoloLens to explore the Internet of Things

What do you get with eBook?