Preface

Machine learning (ML) has been described as the next technological wave to hit humankind, akin to that of electricity. While this is a big claim, we can make certain analogies between the two technologies. For one, you really don't need to understand the inner workings of electricity to use it, and in some ways that applies to ML and many of the more advanced concepts. If you wire up a light the wrong way, it won't work, or you could hurt yourself, and the same analogy applies to machine learning. You still need enough knowledge to call yourself an MLtician or ML practitioner (if you will), and it is the goal of this book to give you that depth of knowledge. Now, the area of ML is broad, so our focus in this book will be to use deep reinforcement learning (DRL) in the form of Unity ML-Agents. DRL is currently a hot topic for developing robotic and simulation agents in many areas, and it is certainly a great addition to the Unity platform.

Who this book is for

This book is for anyone who wants a good practical introduction to some specific ML technologies that work and are very fun to play with. While this book covers some very advanced topics, anyone with a high-school level of math, patience, and understanding of C# will be able to work through all the exercises. We do feature example Python code and use Python for most of the training, but only a superficial knowledge of the language is required.

What this book covers

Chapter 1, Introducing Machine Learning and ML-Agents, covers the basics of machine learning and introduces the ML-Agents framework within Unity. This is basically just a setup chapter, but it's essential to anyone new to Unity and/or ML-Agents.

Chapter 2, The Bandit and Reinforcement Learning, introduces many of the basic problems and solutions used to teach reinforcement learning, from the multiarm and contextual bandit problems to a newly-derived connected bandit problem.

Chapter 3, Deep Reinforcement Learning with Python, explores the Python toolset available for your system and explains how to install and set up those tools. Then, we will cover the basics of neural networks and deep learning before coding up a simple reinforcement learning example.

Chapter 4, Going Deeper with Deep Learning, sets up ML-Agents to use the external Python trainers to create some fun but powerful agents that learn to explore and solve problems.

Chapter 5, Playing the Game, explains that ML-Agents is all about creating games and simulation in Unity. So, in this chapter, we will focus on various play strategies for training and interacting with agents in a real game or simulation.

Chapter 6, Terrarium Revisited and a Multi-Agent Ecosystem, revisits a coding game developed previously called Terrarium as a way to build self-learning agents who live in a little ecosystem. We learn how game rules can be applied to building a game or simulation with multiple agents that interact together.

To get the most out of this book

The following is a short list of the tools and attributes that may make you more successful as you explore this book:

• Computer: A desktop computer capable of running Unity, but all the samples are basic enough that even a low-end machine should be sufficient. Check the Unity documentation for the minimum requirements to run Unity.
• Patience: You may need to train agents for several hours, so expect to wait. Just remember that your patience will be rewarded (Alton Brown). The better your machine, the less you wait, so there's also that.
• GPU: Don't fret if your computer does not have a support GPU to run TensorFlow; you can run the samples without it. It is nice to have, though.
• High-school math: If you need to brush up, basic statistics, algebra, and geometry should be sufficient. Developing your own apps will certainly benefit from a better understanding of the mathematics.
• Programming: A basic understanding of C# is required. You will find it helpful if you also know Unity and Python, but this is not required to run the exercises.

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

1. Log in or register at www.packtpub.com.
2. Select the SUPPORT tab.
3. Click on Code Downloads & Errata.
4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

• WinRAR/7-Zip for Windows
• Zipeg/iZip/UnRarX for Mac
• 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Learn-Unity-ML-Agents-Fundamentals-of-Unity-Machine-Learning. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/LearnUnityMLAgentsFundamentalsofUnityMachineLearning_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: " This command will create a new 3.5 Python environment named [-n] mlagents. "

A block of code is set as follows:

 void Defend()
      {
        currentAction = "Defend";
        nextAction = Time.timeSinceLevelLoad + (25 / MaxSpeed);
      }

Any command-line input or output is written as follows:

 conda activate ml-agents

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [e-mail] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [e-mail].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.