Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Learn ARCore - Fundamentals of Google ARCore
Learn ARCore - Fundamentals of Google ARCore

Learn ARCore - Fundamentals of Google ARCore: Learn to build augmented reality apps for Android, Unity, and the web with Google ARCore 1.0

Arrow left icon
Profile Icon Micheal Lanham
Arrow right icon
$24.99 $35.99
Book Mar 2018 274 pages 1st Edition
eBook
$24.99 $35.99
Print
$43.99
Subscription
Free Trial
Renews at $19.99p/m
Arrow left icon
Profile Icon Micheal Lanham
Arrow right icon
$24.99 $35.99
Book Mar 2018 274 pages 1st Edition
eBook
$24.99 $35.99
Print
$43.99
Subscription
Free Trial
Renews at $19.99p/m
eBook
$24.99 $35.99
Print
$43.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Table of content icon View table of contents Preview book icon Preview Book

Learn ARCore - Fundamentals of Google ARCore

Chapter 1. Getting Started

Welcome to the world of immersive computing and augmented reality with Google ARCore. In this book, we will start with the basics. First, we will cover the basics of augmented reality (AR) on some important core concepts. From there, we will cover the installation and basics of the three development platforms (Android, web, and Unity) that we will use throughout the book. Next, we will take a more in-depth look at the technical challenges faced by AR developers, including various solutions techniques and for solving them. In the final chapters of the book, we will expand on those skills by developing three example AR and mixed reality (MR) apps, where we will build a Machine Learning object recognizer, an AR Designer app, and an app that transitions from AR to MR.

Note

We decided to omitthe Unreal platform from this book, not because it is an inferior platform, but quite the opposite. Unreal is a proven and cutting-edge game engine that is well suited for experienced graphic and game developers. However, Unreal and Unity are essentially on par for development features. Therefore, it made more sense to focus on Unity, which is far better suited for learning game and graphic development.

In this chapter, we will begin by quickly covering the fundamental concepts of immersive computing and augmented reality. Then, we will look at the core problems ARCore is designed to address (motion tracking, environmental understanding, and light estimation). Here's a quick look at the topics we will cover in this chapter:

  • Immersive computing
  • ARCore and AR
    • Motion tracking
    • Environmental understanding
    • Light estimation
  • The road ahead

Note

This book was written with a beta version of ARCore. If you find something different or something that needs to be changed, contact Packt with your errata.

Immersive computing


Immersive computing is a new term used to describe applications that provide an immersive experience for the user. This may come in the form of an augmented or virtual reality experience. While our attention in this book will be primarily focused on building an augmented reality experience, we will highlight techniques that can be used for VR as well. In order to better understand the spectrum of immersive computing, let's take a look at this diagram:

The Immersive Computing Spectrum

The preceding diagram illustrates how the level of immersion affects the user experience, with the left-hand side of the diagram representing more traditional applications with little or no immersion, and the right representing fully immersive virtual reality applications. For us, we will stay in the middle sweet spot and work on developing augmented reality applications. In the next section, we will be introduced to AR and ARCore in more detail.

AR and ARCore


Augmented reality applications are unique in that they annotate or augment the reality of the user. This is typically done visually by having the AR app overlay a view of the real world with computer graphics. ARCore is designed primarily for providing this type of visual annotation for the user. An example of a demo ARCore application is shown here:

Google ARCore demo application; the dog is real

The screenshot is even more impressive when you realize that it was rendered real time on a mobile device. It isn't the result of painstaking hours of using Photoshop or other media effects libraries. What you see in that image is the entire superposition of a virtual object, the lion, into the user's reality. More impressive still is the quality of immersion. Note the details, such as the lighting and shadows on the lion, the shadows on the ground, and the way the object maintains position in reality even though it isn't really there. Without those visual enhancements, all you would see is a floating lion superimposed on the screen. It is those visual details that provide the immersion. Google developed ARCore as a way to help developers incorporate those visual enhancements in building AR applications.

Note

Google developed ARCore for Android as a way to compete against Apple's ARKit for iOS. The fact that two of the biggest tech giants today are vying for position in AR indicates the push to build new and innovative immersive applications.

ARCore has its origins in Tango, which is/was a more advanced AR toolkit that used special sensors built into the device. In order to make AR more accessible and mainstream, Google developed ARCore as an AR toolkit designed for Android devices not equipped with any special sensors. Where Tango depended on special sensors, ARCore uses software to try and accomplish the same core enhancements. For ARCore, Google has identified three core areas to address with this toolkit, and they are as follows:

  • Motion tracking
  • Environmental understanding
  • Light estimation

In the next three sections, we will go through each of those core areas in more detail and understand how they enhance the user experience.

Motion tracking

Tracking a user's motion and ultimately their position in 2D and 3D space is fundamental to any AR application. ARCore allows us to track position changes by identifying and tracking visual feature points from the device's camera image. An example of how this works is shown in this figure:

Feature point tracking in ARCore

In the figure, we can see how the user's position is tracked in relation to the feature points identified on the real couch. Previously, in order to successfully track motion (position), we needed to pre-register or pre-train our feature points. If you have ever used the Vuforia AR tools, you will be very familiar with having to train images or target markers. Now, ARCore does all this automatically for us, in real time, without any training. However, this tracking technology is very new and has several limitations. In the later part of the book, and specifically in Chapter 5, Real-World Motion Tracking, we will add a feature to our AR assistant that allows us to track multiple objects' positions from multiple devices in real time using GPS. Then, in Chapter 10Mixing in Mixed Reality, we will extend our tracking to include augmented maps.

Environmental understanding

The better an AR application understands the user's reality or the environment around them, the more successful the immersion. We already saw how ARCore uses feature identification in order to track a user's motion. Yet, tracking motion is only the first part. What we need is a way to identify physical objects or surfaces in the user's reality. ARCore does this using a technique called meshing.

We will cover more details about meshing in later chapters, but, for now, take a look at the following figure from Google that shows this meshing operation in action:

Google image showing meshing in action

What we see happening in the preceding image is an AR application that has identified a real-world surface through meshing. The plane is identified by the white dots. In the background, we can see how the user has already placed various virtual objects on the surface. Environmental understanding and meshing are essential for creating the illusion of blended realities. Where motion tracking uses identified features to track the user's position, environmental understanding uses meshing to track the virtual objects in the user's reality. In Chapter 8, Recognizing the Environment, we will look at how to train our own machine learning object identifier, which will allow us to extend our meshing to include automatically recognizable objects or areas of an environment.

Light estimation

Magicians work to be masters of trickery and visual illusion. They understand that perspective and good lighting are everything in a great illusion, and, with developing great AR apps, this is no exception. Take a second and flip back to the scene with the virtual lion. Note the lighting and detail in the shadows on the lion and ground. Did you note that the lion is casting a shadow on the ground, even though it's not really there? That extra level of lighting detail is only made possible by combining the tracking of the user's position with the environmental understanding of the virtual object's position and a way to read light levels. Fortunately, ARCore provides us with a way to read or estimate the light in a scene. We can then use this lighting information in order to light and shadow virtual AR objects. Here's an image of an ARCore demo app showing subdued lighting on an AR object:

Google image of demo ARCore app showing off subdued lighting

The effects of lighting, or lack thereof, will become more obvious as we start developing our startup applications. Later, in Chapter 9Blending Light for Architectural Design, we will go into far more detail about 3D lighting and even build some simple shader effects.

 

In this chapter, we didn't go into any extensive details; we will get to that later, but you should now have a good grasp of the core elements ARCore was developed to address. In the next section, we will take a closer look at how best to use the material in this book.

The road ahead


We will take a very hands-on approach for the rest of this book. After all, there is no better way to learn than by doing. While the book is meant to be read in its entirety, not all readers have the time or a need to do this. Therefore, provided in the following table is a quick summary of the platforms, tools, techniques, and difficulty level of each chapter left in the book:

 

Chapter

Focus

Difficulty

Platform

Tools and techniques

Chapter 2,

ARCore on Android

Basics of Android

Basic

Android (Java)

Installation of tools and environment for Android.

Chapter 3,

ARCore on Unity

Basics of Unity

Basic

Android/Unity (C#)

Installation, setup, and deployment of the Unity sample.

Chapter 4,

ARCore on the Web

Building ARCore web apps

Medium

Web (JavaScript)

Installation and setup of tools to support web development and hosting.

Chapter 5, Real-World Motion Tracking

3D spatial audio and Firebase

Medium

Web (JavaScript)

Introduce motion tracking with a mobile device with audio, integrate with Google Firebase, and track multiple objects and/or users in AR.

Chapter 6, Understanding the Environment

Introduction to EU and meshing

Medium

Android (Java)

Learning the ARCore API for Java as well as creating a new ARCore Android project, meshing an environment, and interacting with objects using OpenGL ES.

Chapter 7,

Light Estimation

Introduction to light estimation and lighting in Unity

Advanced

Unity (C#, Cg/HLSL)

Understand the importance of lighting and how it can be used to make AR objects appear more realistic.

Chapter 8,

Recognizing the Environment

Introduction to Machine Learning (ML) for AR and how it can be used.

Advanced

Android (Java), Unity (C#)

Look at various ML platforms in order to better understand how it can be used in AR applications.

Chapter 9,

Blending Light for Architectural Design

3D lighting and shaders

Advanced

Unity (C#)

An advanced introduction to lighting and shaders in Unity, including writing HLSL/ Cg shader code.

Chapter 10,

Mixing in Mixed Reality

Combine all elements together.

Advanced+

Unity (C#), Android (Java)

We will extend the ARCore platform by introducing mixed reality and allowing the app to transition from AR to MR.

Chapter 11,

Performance and Troubleshooting

Performance and troubleshooting tips

Basic

All

Provides some helpful tips on performance, with a section dedicated to addressing the possible issues you may have while working on the samples.

 

 

Also, Chapter 10, Mixing in Mixed Reality, is intended to be used after the reader has reviewed all the previous chapters.

While some readers may prefer to only explore a single ARCore platform by sticking to those specific chapters, you are strongly encouraged to work through all the samples in this book. Given that the ARCore API is so similar across platforms, transferring the techniques you learn for one should translate well to another. Also, don't be intimidated by a different platform or programming language. If you have a good base of knowledge in one C language, learning any other language from the rest of the family takes only minimal effort. Developer, programmer, software engineer, or whatever you want to call yourself, you can always benefit from learning another programming language.

Summary


In this chapter, we took a very quick look at what immersive computing and AR is all about. We learned that augmented reality covers the middle ground of the immersive computing spectrum, that AR is just a careful blend of illusions used to trick the user into believing that their reality has been combined with a virtual one. After all, Google developed ARCore as a way to provide a better set of tools for constructing those illusions and to keep Android competitive in the AR market. After that, we learned the core concepts ARCore was designed to address and looked at each: motion tracking, environmental understanding, and light estimation, in a little more detail. Finally, we finished with a helpful roadmap for users looking to get the most out of this book in the shortest amount of time.

In the next chapter, we begin to dive in and get our hands dirty by getting the sample Android project set up and tweaked for our needs.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Harness the power of the Google’s new augmented reality (AR) platform ARCore to build cutting-edge Augmented reality apps
  • Learn core concepts of Environmental Understanding, Immersive Computing, and Motion Tracking with ARCore
  • Extend your application by combining ARCore with OpenGL, Machine Learning and more.

Description

Are you a mobile developer or web developer who wants to create immersive and cool Augmented Reality apps with the latest Google ARCore platform? If so, this book will help you jump right into developing with ARCore and will help you create a step by step AR app easily. This book will teach you how to implement the core features of ARCore starting from the fundamentals of 3D rendering to more advanced concepts such as lighting, shaders, Machine Learning, and others. We’ll begin with the basics of building a project on three platforms: web, Android, and Unity. Next, we’ll go through the ARCore concepts of motion tracking, environmental understanding, and light estimation. For each core concept, you’ll work on a practical project to use and extend the ARCore feature, from learning the basics of 3D rendering and lighting to exploring more advanced concepts. You’ll write custom shaders to light virtual objects in AR, then build a neural network to recognize the environment and explore even grander applications by using ARCore in mixed reality. At the end of the book, you’ll see how to implement motion tracking and environment learning, create animations and sounds, generate virtual characters, and simulate them on your screen.

What you will learn

  • [*] Build and deploy your Augmented Reality app to the Android, Web, and Unity platforms
  • [*] Implement ARCore to identify and visualize objects as point clouds, planes, surfaces, and/or meshes
  • [*] Explore advanced concepts of environmental understanding using Google ARCore and OpenGL ES with Java
  • [*] Create light levels from ARCore and create a C# script to watch and propagate lighting changes in a scene
  • [*] Develop graphics shaders that react to changes in lighting and map the environment to place objects in Unity/C#
  • [*] Integrate motion tracking with the Web ARCore API and Google Street View to create a combined AR/VR experience

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Mar 30, 2018
Length 274 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781788830409
Category :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want

Product Details

Publication date : Mar 30, 2018
Length 274 pages
Edition : 1st Edition
Language : English
ISBN-13 : 9781788830409
Category :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together

Stars icon
Total $ 82.96 119.97 37.01 saved
Learn ARCore - Fundamentals of Google ARCore
$24.99 $35.99
Augmented Reality for Developers
$29.99 $43.99
Unity 2018 Augmented Reality Projects
$27.98 $39.99
=
Book stack Total $ 82.96 119.97 37.01 saved Stars icon

Table of Contents

17 Chapters
Title Page Chevron down icon Chevron up icon
Packt Upsell Chevron down icon Chevron up icon
Contributors Chevron down icon Chevron up icon
Preface Chevron down icon Chevron up icon
1. Getting Started Chevron down icon Chevron up icon
2. ARCore on Android Chevron down icon Chevron up icon
3. ARCore on Unity Chevron down icon Chevron up icon
4. ARCore on the Web Chevron down icon Chevron up icon
5. Real-World Motion Tracking Chevron down icon Chevron up icon
6. Understanding the Environment Chevron down icon Chevron up icon
7. Light Estimation Chevron down icon Chevron up icon
8. Recognizing the Environment Chevron down icon Chevron up icon
9. Blending Light for Architectural Design Chevron down icon Chevron up icon
10. Mixing in Mixed Reality Chevron down icon Chevron up icon
11. Performance Tips and Troubleshooting Chevron down icon Chevron up icon
1. Other Books You May Enjoy Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.