Home Mobile HoloLens Blueprints

HoloLens Blueprints

By Abhijit Jana , Manish Sharma , Mallikarjuna Rao
books-svg-icon Book
eBook $39.99 $27.98
Print $48.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $39.99 $27.98
Print $48.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Digital Reality - Under the Hood
About this book
Do you want to create stunning applications with HoloLens? Are you a developer who is fascinated with Microsoft HoloLens and its capabilities? If so, this is the book for you. This book introduces and demystifies the HoloLens platform and shows you different ways of interaction with computers (mixed-reality). You will start your mixed-reality journey by understanding different types of digital reality. You will learn to build your first holographic app. Also, you will understand holographic application integration possibilities within Line of Business Applications using Azure. Moving ahead, you will create Integrated Solutions using IoT with HoloLens. Gradually you'll learn how to create and deploy apps on a device. You will learn to publish application to the store; if you are an enterprise developer, you will also manage and distribute applications for enterprise-enabled or domain-joined HoloLens. Finally, you will develop an end-to-end realistic holographic app, ranging from scenario identification to sketching, development, deployment, and, finally, production.
Publication date:
June 2017
Publisher
Packt
Pages
336
ISBN
9781787281943

 

Chapter 1. Digital Reality - Under the Hood

Welcome to the world of Digital Reality. The purpose of Digital Reality is to bring immersive experiences, such as taking or transporting you to different world or places, make you interact within those immersive, mix digital experiences with reality, and ultimately open new horizons to make you more productive. Applications of Digital Reality are advancing day by day; some of them are in the field of gaming, education, defense, tourism, aerospace, corporate productivity, enterprise applications, and so on.

The spectrum and scenarios of Digital Reality are huge. In order to understand them better, they are broken down into three different categories:

  • Virtual Reality (VR): It is where you are disconnected from the real world and experience the virtual world. Devices available on the market for VR are Oculus Rift, Google VR, and so on. VR is the common abbreviation of Virtual Reality.
  • Augmented Reality (AR): It is where digital data is overlaid over the real world. Pokemon GO, one of the very famous games, is an example of this globally. A device available on the market, which falls under this category, is Google Glass.Augmented Reality is abbreviated to AR.
  • Mixed Reality (MR): It spreads across the boundary of the real environment and VR. Using MR, you can have a seamless and immersive integration of the virtual and the real world. Mixed Reality is abbreviated to MR.

This book is mainly focused on developing MR applications using Microsoft HoloLens devices.

Although these technologies look similar in the way they are used, and sometimes the difference is confusing to understand, there is a very clear boundary that distinguishes these technologies from each other. As you can see in the following diagram, there is a very clear distinction between AR and VR. However, MR has a spectrum, which overlaps across all three boundaries of real world, AR, and MR.

Digital Reality Spectrum

The following table describes the differences between the three. Later in the chapter, we will go through the details of each one of them.

Virtual Reality

Augmented Reality

Mixed Reality

  • Complete Virtual World
  • User is completely isolated from the real world
  • Device examples: Oculus Rift and Google VR
  • Overlays Data over the real world
  • Often used for mobile devices
  • Device example: Google Glass
  • Application example: Pokemon GO
  • Seamless integration of the real and virtual world
  • Virtual world interacts with real world
  • Natural interactions
  • Device examples: HoloLens and Meta

Let's explore VR, AR, and MR in more detail.

 

Virtual Reality - what it is?


The dictionary definition of VR reads, "

a realistic and immersive simulation of a three-dimensional environment, created using interactive software and hardware, and experienced or controlled by movement of the body

". However, we will say that there is one and only one primary purpose of the VR, that is, to make you believe that you are somewhere else, for example, in a gaming environment, a war zone, or touring some city.

Virtual Reality illustration

Realizing Virtual Reality

A person wearing a VR device/headset will be able to view the virtual environment, move around wearing it, and interact with items within that virtual environment. Most Virtual Realities are created using a head-mounted helmet or a set of goggles, which are given the generic name: head-mounted display (HMD). HMD is usually connected to a computer or a smartphone, which does all the 3D rendering work, and HMD is just used to display that rendered 3D content. HMD completely covers user's view, by covering both eyes. As a result, the user is completely cut off from the outside world and completely focused on the virtual or digital world.

HMDs basically consists of stereoscopic displays and motion tracking hardware. The way HMDs implement stereoscopic displays is by generating different image for each eye, which results in generating the illusion of depth. Within HMDs, motion tracking hardware mostly consists of a gyroscope and accelerometer to measure motion/position changes. This helps in simulating real-world experiences.

Virtual Reality in the field

There are various implementations of VR in diverse fields currently, and numerous new ones are coming up every day. It will be a very interesting domain for the next few years, where we will globally see lots of new fields picking up VR and merging it with other upcoming technologies. One such technology, which we can think of, is Artificial Intelligence (AI); Are we talking about the movie "Matrix" here?

While it's too early to predict possibilities in the future, let's explore some of the current fields where VR is being implemented. They are as follows:

  • Gaming: Gaming was one of the first fields to pick up, start using, and commercialize VR. Apart from single-user VR games, multi-user online VR games are also available today. With the availability of low-cost devices, such as Google VR and smartphones, VR games/applications are within the reach of everyone.
  • Tourism: Tourism is another field that has picked up VR. VR is used for generating stereoscopic 360-degree panoramic views, where the user interact with the virtual place and explores it. So, no more traveling or air-travel; just wear your VR device and explore new places every day.
  • Education: Distant learning, visualization, and interactive teaching a few global examples of VR implementations within the education field. Earlier, teachers used to describe the scene and students used to imagine the possibilities. Now, they just visualize new possibilities and interact with them, and spend more time in thinking what's next.
  • Architecture: Architecture is a field where VR has made life easier for the architects by making it easy to create prototypes. Now, they do not have to create cardboard prototypes or sketches to convey their thoughts and ideas. VR has made it possible for them to share ideas early in the cycle of development and receive early feedback. This saves lot of time in rework.
  • Enterprise productivity: Especially, for enterprise training scenarios, VR is used very frequently. Scenarios in consideration can be simulating war situations for the armed forces, flight simulation for pilots, medical diagnostics simulations for doctors, and many others.
  • Web content: With the support of WebGL technology by most of the latest browsers available in the market, rendering Virtual 3D content has become very easy. A lot of commercial websites for automobile companies, for example, have started publishing their automobile features in the form of 3D content, which has made it very interactive for end users to visualize the content.
 

Augmented Reality - what it is?


AR is all about bringing digital information and overlaying it over the real environment. The only difference from VR is that it creates a totally artificial environment around you, whereas AR uses the environment around you and overlays the digital information over it. For VR, you will require a VR device, but for AR it can be achieved by simply using a smartphone, tablet, or dedicated VR devices.

Visualization techniques for Augmented Reality

AR can be categorized into three different types based on the display types:

  • Screen-based
  • HMD-based
  • Projection-based

Screen-based

A common example of screen-based AR is overlaying digital information over smartphones or tablet camera displays. For example, you switch on and point your smartphone/tablet camera over an object, and the application recognizes that object and overlays that object information, such as the price or description, as digital information over the object image.

Another example of screen-based AR is the video game Pokemon GO, in which, based on the user's location and direction, digital characters are overlaid over video images.

User is viewing augmented object using smartphone

In the preceding figure, a user is viewing an augmented object, that is, the elephant's digital object is overlaid over the video frame. This is an example of screen-based AR.

Augmented Reality head-mounted globally displays

Using AR HMD devices, digital information is overlaid directly over the user's view of the real world. So, there is no need to hold any screen to view the digital information. Digital information is directly rendered over the view area of the user's eyes.

HMD with information overlaid over the real view

In the preceding image, the user is using a head mounted AR device to view the physical element in front of him. Within the view of the user, the AR device embeds information about the physical object and provides them with a more immersive experience.

Projection-based

User projection-based AR, a projection is rendered on the target surface itself. This target surface could be anything, such as building, person, room, and so on. To render this kind of projection, the system needs to know the exact dimensions of the target surface, and then using single/multiple projectors, it renders the projection on the target:

Phone dialer pad projection over the hand

In the preceding image, the user is viewing a projection of the phone dialer on the palm of their hand.

Augmented Reality in the field

Applying AR is quite different from applying VR. AR applications are more focused on integration scenarios with the real world. Some are as follows:

  • Gaming: With the release of the Pokemon GO game by Niantic, the demand for AR games has picked up drastically in the market. A lot of companies are coming up with AR games and launching them.
  • Architecture/construction/archaeology: AR can be used to visualize completed buildings for in-progress or upcoming building construction. Digital architecture/building images can be overlaid over the real view of the property or ground.
  • E-Commerce: Businesses can't reach each customer with demo-able physical product, and opening showrooms is every city is a costly business, especially for start-ups and newcomers in the market. So, reaching out to customers through AR, online furniture retail companies, for example, allows users to consume AR through smartphones, let them visualize their furniture products, and enable them to design their house interiors.
  • Education: Traditional education systems require hands-on instructions and real/prototype equipment to explain it better to students. With AR, teachers and students can visualize the same equipment in virtual mode, with very similar training instructions as with an actual device. Another implementation of AR in the education system is distance learning, where students and a teacher very far away, can use VR devices for interactions and virtual classrooms.
  • Medical: In the medical field, there are different imaging techniques for various requirements, such as X-ray, ultrasound, and magnetic resonance imaging (MRI), but there is no consolidated view for medical practitioners. AR could be used for these scenarios, where the output from different imaging techniques could be overlaid over the patient and give a consolidated view to medical practitioners.
  • Industrial design: AR is being used for designing, sharing ideas, and brainstorming design views among different designers and architects and supplies quick feedback and brainstorming cycles. Earlier, the same process used to take a long time, as designers used to create physical prototypes and then discuss them.
  • Travel/navigation/tourism: AR is also used for developing navigational applications; for example, travel-related digital information is overlaid on the vehicle windscreen, which helps the driver in real-time navigation without looking at any other device, such as a GPS or smartphone. AR is also used to develop travel-related applications, which help the user with location-specific information of the place where the user is currently placed, such as historical information about tourist places, or information about nearby restaurants and cuisines.
 

Mixed Reality - what it is?


We talked about VR and AR. Now, think of MR as a hybrid of VR and AR. Using MR, a user can view the real world just like in AR, but can also view virtual objects as in VR. The difference with MR is that those virtual objects are referenced or anchored with the real space. That means that the virtual object is interacting within the real world.

For example, suppose you have a physically real table in front of you, and now using MR you are viewing a virtual ball; if you drop a virtual ball object on a table, in MR, that virtual ball will interact with the physical table. This means that it will bounce on the physical table, and roll off from the table to the ground.

The immersive experience of the virtual world with the real world is so strong in MR that it is very hard to break that illusion.

A user viewing a MR object in real space

At the time of writing this book, there are very few MR devices available on the market; Microsoft HoloLens is one of them. This book is focused on developing the MR experience, using HoloLens.

Mixed Reality in the field

MR is still evolving, and lot of different industries are still exploring possibilities with it. The following are some of them, which represent early progress:

  • Education and training: MR has already started transforming the way education and industrial training are delivered. Companies have started using MR as a medium for delivering readiness/training content to participants.
  • Exploring new markets: MR is giving an opportunity to enterprises to rethink their marketing strategy and target new customers. Few popular automobile companies have started using MR device in their specialized car showrooms. With this, they can target consumers early in the sales cycle, even before the first model of the car is available.
  • Healthcare: The possibilities for MR in the healthcare industry are enormous. Physicians can use MR devices to visualize consolidated reports, and these devices can also be used as a guidance device during operations or diagnosis.
  • Gaming: The gaming industry may become one of the biggest consumers of MR devices. There are so many possibilities for MR games that can interact with the real world.
 

Digital Reality and the current market place


We learned about VR, AR, and MR; let's see what devices are currently available on the market to explore all these different experiences:

Digital Reality device family

  • Oculus: Owned by Facebook, the first in the industry to launch a VR device
  • HTC Vive: A VR device launched by the Taiwanese phone manufacturer HTC
  • Sony PlayStation VR: A VR device that works with Sony PlayStation, but also works with any personal computer
  • Google VR: This is a Google VR device, and is used along with Android smartphones
  • Gear VR: This is a Samsung VR device, and is used along with Galaxy smartphones
  • OSVR: An open source VR device, started with the support of companies such as Intel and Razer
  • Google Glass: An AR device launched by Google
  • HoloLens: Microsoft MR wireless and wearable device
  • Met: Mixed or Augmented Reality device, which projects virtual images in the wearer's field of vision
  • Magic Leap: MR device, still under development
 

Summary


We have seen in this chapter the different types of reality solutions available on the market and their possible usage. We have learned about how these digital realities are similar to each other and how they differ. We have also explored several field applications of these technologies. We have seen the different sets of devices available in the current market place. The knowledge gained from this chapter will help you grasp the subject discussed in subsequent chapters.

Further on within this book, we will focus on MR application development using the HoloLens device. In the next chapter, we will focus on HoloLens as a device. Later, our focus will be on application development, taking you through the step-by-step process of developing a new holographic application and deploying it on the HoloLens device.

About the Authors
  • Abhijit Jana

    Abhijit Jana is a technology leader, strategist, solution architect, development consultant, and trusted technology advisor with 14 years of experience in the IT industry, with expertise in development, architecting, engineering, consulting, services delivery, leadership, and mentoring. He is currently associated with West Pharmaceutical Services as the director of software engineering and is responsible for building, managing, and leading the overall application development and software engineering team. Before West, he was associated with Microsoft and is a former MVP. There, he managed several complex projects and delivered solutions to enterprise customers worldwide. He has been a speaker at various technology conferences and several IITs across India.

    Browse publications by this author
  • Manish Sharma

    Manish Sharma works with Microsoft as a solution architect as part of Microsoft Services. As, a solution architect, he is responsible for leading large enterprise-transformational engagements defining technology/solution roadmaps, conducting architecture and technical evaluations of enterprise engagements, and architecting multimillion-dollar solution developments, and maintenance of outsourcing management. He is also a technology evangelist and speaker at prestigious events, such as Microsoft TechEd on the latest and cutting edge technologies, such as HoloLens, Internet of Things, connected cars, and cloud technologies such as Azure.

    Browse publications by this author
  • Mallikarjuna Rao

    Mallikarjuna Rao works with Microsoft as UX Designer/3D mixed reality artist as part of Microsoft Services. He has 15 years of experience in the animation industry, with around 12 years' international exposure. He has worked as a senior 3D asset supervisor for feature films, television series, commercials, and game projects. Mallikarjuna has worked on pivotal projects for big names in the industry, such as Walt Disney Pictures, Disney Junior, Warner Bros Pictures, Yash Raj Films, and UTV Motion Pictures, which put him under the limelight at the international level. He has an in-depth knowledge of the 3D Asset production pipeline and UX design principles.

    Browse publications by this author
HoloLens Blueprints
Unlock this book and the full library FREE for 7 days
Start now