Preface

Most programmers and data scientists struggle with mathematics, either having overlooked or forgotten core mathematical concepts. This book helps you understand the math that's required to understand how various neural networks work so that you can go on to building better deep learning (DL) models.

You'll begin by learning about the core mathematical and modern computational techniques used to design and implement DL algorithms. This book will cover essential topics, such as linear algebra, eigenvalues and eigenvectors, the singular value decomposition concept, and gradient algorithms, to help you understand how to train deep neural networks. Later chapters focus on important neural networks, such as the linear neural network, multilayer perceptrons, and radial basis function networks, with a primary focus on helping you learn how each model works. As you advance, you will delve into the math used for normalization, multi-layered DL, forward propagation, optimization, and backpropagation techniques to understand what it takes to build full-fledged DL models. Finally, you'll explore convolutional neural network (CNN), recurrent neural network (RNN), and generative adversarial network (GAN) models and their implementation.

By the end of this book, you'll have built a strong foundation in neural networks and DL mathematical concepts, which will help you to confidently research and build custom DL models.

Who this book is for

This book is for data scientists, machine learning developers, aspiring DL developers, and anyone who wants to gain a deeper understanding of how DL algorithms work by learning the mathematical concepts that form their foundation. Having knowledge of the basics of machine learning is required to gain a better understanding of the topics covered in the book. Having a strong mathematical background is not essential but would be quite helpful in grasping some of the concepts.

What this book covers

Chapter 1, Linear Algebra, will give you an understanding of the inner workings of linear algebra, which is essential for understanding how deep neural networks work. In particular, you will learn about multi-dimensional linear equations, how matrices are multiplied together, and various methods of decomposing/factorizing matrices. These concepts will be critical for developing an intuition for how forward propagation works in neural networks.

Chapter 2, Vector Calculus, will cover all the main concepts of calculus, where you will start by learning the fundamentals of single variable calculus and build toward an understanding of multi-variable and ultimately vector calculus. The concepts of this chapter will help you better understand the math that underlies the training process of neural networks, particularly how backpropagation works.

Chapter 3, Probability and Statistics, will teach you the essentials of both probability and statistics and how they are related to each other. In particular, the focus will be on understanding different types of distributions, the importance of the central limit theorem, and how estimations are made. This chapter is critical to developing an understanding of what exactly it is that neural networks are learning.

Chapter 4, Optimization, will explain what exactly optimization is and several methods of it that are used in practice, such as least squares, gradient descent, Newton's method, and genetic algorithms. The methods covered in this chapter are essential to understanding how neural networks learn during their training phase.

Chapter 5, Graph Theory, will teach you about graph theory, which is used to model relationships between objects, and will also help in your understanding of the different types of neural network architectures. Later in the book, the concepts from this chapter will be very useful for understanding how graph neural networks work.

Chapter 6, Linear Neural Networks, will cover the most basic type of neural network and teach you how a model learns to find linear relationships from data through regression. You will also learn that this type of model has limitations, which is where the need for neural networks arises.

Chapter 7, Feedforward Neural Networks, will show you how all the concepts covered in the previous chapters are brought together to form modern-day neural networks, including coverage of how they are structured, how and what they learn, and what makes them so powerful.

Chapter 8, Regularization, will show you the various methods of regularization, such as dropout and norm penalties, that are used extensively in practice to help our models to generalize to test data so that they work well once deployed.

Chapter 9, Convolutional Neural Networks, will explain CNNs, which are a variant of feedforward neural networks and are particularly effective for tasks related to computer vision, as well as time series analysis.

Chapter 10, Recurrent Neural Networks, will explain RNNs, which are another variant of feedforward neural networks that have recurrent connections, which gives them the ability to learn relationships in sequences such as those in time series and language.

Chapter 11, Attention Mechanisms, will show a relatively recent breakthrough in deep learning known as attention. This has led to the creation of transformer models, which have resulted in phenomenal results in tasks related to natural language processing.

Chapter 12, Generative Models, is where the focus will be switched from neural networks that learn to predict classes given data to models that learn to synthetically create data. You will learn about various models, such as autoencoders, GANs, and flow-based networks.

Chapter 13, Transfer and Meta Learning, will teach you about two separate but related concepts known as transfer learning and meta learning. Their goals respectively are to transfer what one model has learned to another to help it work on a similar task and to create networks that can use existing knowledge to learn new tasks or learn how to learn.

Chapter 14, Geometric Deep Learning, will explain another relatively new concept in DL, which extends the power of deep neural networks from the Euclidean domain to the non-Euclidean domain.

To get the most out of this book

It is expected that most of you have had prior experience with implementing machine learning models and have at least a basic understanding of how they work. It is also assumed that many of you have some prior experience with calculus, linear algebra, probability, and statistics; having this prior experience will help you get the most out of this book.

For those of you who do have prior experience with the mathematics covered in the first five chapters and have a background in machine learning, you are welcome to skip ahead to the content from Chapter 7, Feedforward Neural Networks, onward and keep with the flow of the book from there.

However, for the reader who lacks the aforementioned experience, it is recommended that you stay with the flow and order of the book and pay particular attention to understanding the concepts covered in the first five chapters, moving on to the next chapter or section only when you feel comfortable with what you have learned. It is important that you do not rush or be hasty, as DL is a vast and complex field that should not be taken lightly.

Lastly, to become a very good DL practitioner, it is important that you keep learning and going over the fundamental concepts, as these can often be forgotten quite easily. After having gone through all the chapters in the book and through all the chapters, I recommend trying to read the code for and/or implementing a few architectures and trying to recall what you have learned in this book because doing so will help ground your concepts even further and help you to learn much faster.

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781838647292_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "This is known as the antiderivative, and we define it formally as a function."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [e-mail].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.