Preface

With conventional deep learning methods almost hitting a wall in terms of their capability, more and more researchers have started looking for alternative approaches to train artificial neural networks.

Deep machine learning is extremely effective for pattern recognition, but fails in tasks that require an understanding of context or previously unseen data. Many researchers, including Geoff Hinton, the father of the modern incarnation of deep machine learning, agree that the current approach to designing artificial intelligence systems is no longer able to cope with the challenges currently being faced.

In this book, we discuss a viable alternative to traditional deep machine learning methods—neuroevolution algorithms. Neuroevolution is a family of machine learning methods that use evolutionary algorithms to ease the solving of complex tasks such as games, robotics, and the simulation of natural processes. Neuroevolution algorithms are inspired by the process of natural selection. Very simple artificial neural networks can evolve to become very complex. The ultimate result of neuroevolution is the optimal topology of a network, which makes the model more energy-efficient and more convenient to analyze.

Throughout this book, you will learn about various neuroevolution algorithms and get practical skills in using them to solve different computer science problems—from classic reinforcement learning to building agents for autonomous navigation through a labyrinth. Also, you will learn how neuroevolution can be used to train deep neural networks to create an agent that can play classic Atari games.

This book aims to give you a solid understanding of neuroevolution methods by implementing various experiments using step-by-step guidance. It covers practical examples in areas such as games, robotics, and the simulation of natural processes, using real-world examples and datasets to help you better understand the concepts explored. After reading this book, you will have everything you need to apply neuroevolution methods to other tasks similar to the experiments presented.

In writing this book, my goal is to provide you with knowledge of cutting-edge technology that is a vital alternative to traditional deep learning. I hope that the application of neuroevolution algorithms in your projects will allow you to solve your currently intractable problems in an elegant and energy-efficient way.

Who this book is for

This book is for machine learning practitioners, deep learning researchers, and AI enthusiasts who are looking to implement neuroevolution algorithms from scratch. You will learn how to apply these algorithms to various sets of real-world problems. You will learn how neuroevolution methods can optimize the process of training artificial neural networks. You will become familiar with the core concepts of neuroevolution and get the necessary practical skills to use it in your work and experiments. A working knowledge of Python and deep learning and neural network basics is mandatory.

What this book covers

Chapter 1, Overview of Neuroevolution Methods, introduces the core concepts of genetic algorithms, such as genetic operators and genome encoding schemes.

Chapter 2, Python Libraries and Environment Setup, discusses the practical aspects of neuroevolution methods. This chapter provides the pros and cons of popular Python libraries that provide implementations of the NEAT algorithm and its extensions.

Chapter 3, Using NEAT for XOR Solver Optimization, is where you start experimenting with the NEAT algorithm by implementing a solver for a classical computer science problem.

Chapter 4, Pole-Balancing Experiments, is where you continue with experiments related to the classic problems of computer science in the field of reinforcement learning.

Chapter 5, Autonomous Maze Navigation, is where you continue your experiments with neuroevolution through an attempt to create a solver that can find an exit from a maze. You will learn how to implement a simulation of a robot that has an array of sensors to detect obstacles and monitor its position within the maze.

Chapter 6, Novelty Search Optimization Method, is where you use the practical experience gained during the creation of a maze solver in the previous chapter to embark on the path of creating a more advanced solver.

Chapter 7, Hypercube-Based NEAT for Visual Discrimination, introduces you to advanced neuroevolution methods. You'll learn about the indirect genome encoding scheme, which uses Compositional Pattern Producing Networks (CPPNs) to aid with the encoding of large-phenotype ANN topologies.

Chapter 8, ES-HyperNEAT and the Retina Problem, is where you will learn how to select the substrate configuration that is best suited for a specific problem space.

Chapter 9, Co-Evolution and the SAFE Method, is where we discuss how a co-evolution strategy is widely found in nature and could be transferred into the realm of the neuroevolution.

Chapter 10, Deep Neuroevolution, presents you with the concept of Deep Neuroevolution, which can be used to train Deep Artificial Neural Networks (DNNs).

Chapter 11, Best Practices, Tips, and Tricks, teaches you how to start working with whatever problem is at hand, how to tune the hyperparameters of a neuroevolution algorithm, how to use advanced visualization tools, and what metrics can be used for the analysis of algorithm performance.

Chapter 12, Concluding Remarks, summarizes everything you have learned in this book and provides further directions for you to continue your self-education.

To get the most out of this book

A practical knowledge of the Python programming language is essential to work with the examples presented in this book. For better source code understanding, it is preferable to use an IDE that supports Python syntax highlighting and code reference location. If you don't have one installed, you can use Microsoft Visual Studio Code. It is free and cross-platform, and you can download it here: https://code.visualstudio.com.

Python and most of the libraries we discuss in this book are cross-platform, and compatible with Windows, Linux, and macOS. All experiments described in the book are executed from the command line, so make yourself familiar with the terminal console application installed on the OS of your choice.

To complete the experiment described in Chapter 10, Deep Neuroevolution, you need to have access to a modern PC with Nvidia graphics accelerator GeForce GTX 1080Ti or better. This experiment is also better to run in an Ubuntu Linux environment. Ubuntu is a modern Linux-based OS that is free and powerful. Making yourself familiar with it will help you a lot.

Download the example code files

You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

1. Log in or register at www.packt.com.
2. Select the Support tab.
3. Click on Code Downloads.
4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

• WinRAR/7-Zip for Windows
• Zipeg/iZip/UnRarX for Mac
• 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Hands-on-Neuroevolution-with-Python. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781838824914_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "You can start an experiment from the Chapter10 directory by executing the following command."

A block of code is set as follows:

if indices is None:
            indices = np.arange(self.batch_size)

Any command-line input or output is written as follows:

$ conda create -n deep_ne python=3.5

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [e-mail].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.