Search icon CANCEL
Subscription
0
Cart icon
Cart
Close icon
You have no products in your basket yet
Save more on your purchases!
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition
Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

Deep Learning with TensorFlow and Keras – 3rd edition: Build and deploy supervised, unsupervised, deep, and reinforcement learning models, Third Edition

eBook
$39.99 $27.98
Print
$49.99
Subscription
$15.99 Monthly

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon AI Assistant (beta) to help accelerate your learning
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now
Table of content icon View table of contents Preview book icon Preview Book

Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

Neural Network Foundations with TF

In this chapter, we learn the basics of TensorFlow, an open-source library developed by Google for machine learning and deep learning. In addition, we introduce the basics of neural networks and deep learning, two areas of machine learning that have had incredible Cambrian growth during the last few years. The idea behind this chapter is to provide all the tools needed to do basic but fully hands-on deep learning.

We will learn:

  • What TensorFlow and Keras are
  • An introduction to neural networks
  • What the perceptron and multi-layer perceptron are
  • A real example: recognizing handwritten digits

All the code files for this chapter can be found at https://packt.link/dltfchp1.

Let’s begin!

What is TensorFlow (TF)?

TensorFlow is a powerful open-source software library developed by the Google Brain Team for deep neural networks, the topic covered in this book. It was first made available under the Apache 2.0 License in November 2015 and has since grown rapidly; as of May 2022, its GitHub repository (https://github.com/tensorflow/tensorflow) has more than 129,000 commits, with roughly 3,100 contributors. This in itself provides a measure of the popularity of TensorFlow.

Let us first learn what exactly TensorFlow is and why it is so popular among deep neural network researchers and engineers. Google calls it “an open-source software library for machine intelligence,” but since there are so many other deep learning libraries like PyTorch (https://pytorch.org/), Caffe (https://caffe.berkeleyvision.org/), and MXNet (https://mxnet.apache.org/), what makes TensorFlow special? Most other deep learning libraries, like TensorFlow, have auto-differentiation (a...

What is Keras?

Keras is a beautiful API for composing building blocks to create and train deep learning models. Keras can be integrated with multiple deep learning engines including Google TensorFlow, Microsoft CNTK, Amazon MXNet, and Theano. Starting with TensorFlow 2.0, Keras, the API developed by François Chollet, has been adopted as the standard high-level API, largely simplifying coding and making programming more intuitive.

Introduction to neural networks

Artificial neural networks (briefly, “nets” or ANNs) represent a class of machine learning models loosely inspired by studies about the central nervous systems of mammals. Each ANN is made up of several interconnected “neurons,” organized in “layers.” Neurons in one layer pass messages to neurons in the next layer (they “fire,” in jargon terms) and this is how the network computes things. Initial studies were started in the early 1950s with the introduction of the “perceptron” [1], a two-layer network used for simple operations, and further expanded in the late 1960s with the introduction of the “back-propagation” algorithm used for efficient multi-layer network training (according to [2] and [3]). Some studies argue that these techniques have roots dating further back than normally cited [4].

Neural networks were a topic of intensive academic studies up until the...

Perceptron

The “perceptron” is a simple algorithm that, given an input vector x of m values (x1, x2,..., and xm), often called input features or simply features, outputs either a 1 (“yes”) or a 0 (“no”). Mathematically, we define a function:

Where w is a vector of weights, is the dot product , and b is the bias. If you remember elementary geometry, wx + b defines a boundary hyperplane that changes position according to the values assigned to w and b. Note that a hyperplane is a subspace whose dimension is one fewer than that of its ambient space. See (Figure 1.2) for an example:

Figure 1.2: An example of a hyperplane

In other words, this is a very simple but effective algorithm! For example, given three input features, the amounts of red, green, and blue in a color, the perceptron could try to decide whether the color is “white” or not.

Note that the perceptron cannot express a “maybe”...

Multi-layer perceptron: our first example of a network

In this chapter, we present our first example of a network with multiple dense layers. Historically, “perceptron” was the name given to the model having one single linear layer, and as a consequence, if it has multiple layers, we call it a Multi-Layer Perceptron (MLP). Note that the input and the output layers are visible from the outside, while all the other layers in the middle are hidden – hence the name hidden layers. In this context, a single layer is simply a linear function and the MLP is therefore obtained by stacking multiple single layers one after the other:

Diagram  Description automatically generated

Figure 1.3: An example of multiple layer perceptron

In Figure 1.3 each node in the first hidden layer receives an input and “fires” (0,1) according to the values of the associated linear function. Then the output of the first hidden layer is passed to the second layer where another linear function is applied, the results...

A real example: recognizing handwritten digits

In this section we will build a network that can recognize handwritten numbers. To achieve this goal, we use MNIST (http://yann.lecun.com/exdb/mnist/), a database of handwritten digits made up of a training set of 60,000 examples, and a test set of 10,000 examples. The training examples are annotated by humans with the correct answer. For instance, if the handwritten digit is the number “3,” then 3 is simply the label associated with that example.

In machine learning, when a dataset with correct answers is available, we say that we can perform a form of supervised learning. In this case we can use training examples for improving our net. Testing examples also have the correct answer associated with each digit. In this case, however, the idea is to pretend that the label is unknown, let the network do the prediction, and then later on reconsider the label to evaluate how well our neural network has learned to recognize...

Regularization

In this section we will review a few best practices for improving the training phase. In particular, regularization and batch normalization will be discussed.

Adopting regularization to avoid overfitting

Intuitively, a good machine learning model should achieve low error on training data. Mathematically this is equivalent to minimizing the loss function on the training data given the model:

However, this might not be enough. A model can become excessively complex in order to capture all the relations inherently expressed by the training data. This increase in complexity might have two negative consequences. First, a complex model might require a significant amount of time to be executed. Second, a complex model might achieve very good performance on training data but perform quite badly on validation data. This is because the model is able to contrive relationships between many parameters in the specific training context, but these relationships in...

Playing with Google Colab: CPUs, GPUs, and TPUs

Google offers a truly intuitive tool for training neural networks and for playing with TensorFlow at no cost. You can find an actual Colab, which can be freely accessed, at https://colab.research.google.com/ and if you are familiar with Jupyter notebooks you will find a very familiar web-based environment here. Colab stands for Colaboratory and is a Google research project created to help disseminate machine learning education and research. We will see the difference between CPUs, GPUs, and TPUs in Chapter 15, Tensor Processing Unit.

For now, it’s important to know that CPUs are generic processing units, while GPUs and TPUs are accelerators, specific processing units suitable for deep learning. Let’s see how it works, starting with the screenshot shown in Figure 1.23:

Figure 1.23: An example of notebooks in Colab

By accessing Colab, we can either check a listing of notebooks generated in the past or we...

Sentiment analysis

What is the code we used to test Colab? It is an example of sentiment analysis developed on top of the IMDB dataset. The IMDB dataset contains the text of 50,000 movie reviews from the Internet Movie Database. Each review is either positive or negative (for example, thumbs up or thumbs down). The dataset is split into 25,000 reviews for training and 25,000 reviews for testing. Our goal is to build a classifier that can predict the binary judgment given the text. We can easily load IMDB via tf.keras and the sequences of words in the reviews have been converted to sequences of integers, where each integer represents a specific word in a dictionary. We also have a convenient way of padding sentences to max_len, so that we can use all sentences, whether short or long, as inputs to a neural network with an input vector of fixed size:

import tensorflow as tf
from tensorflow.keras import datasets, layers, models, preprocessing
import tensorflow_datasets as tfds
max_len...

Predicting output

Once a net is trained, it can of course be used for making predictions. In TensorFlow, this is very simple. We can use this method:

# Making predictions.
predictions = model.predict(X)

For a given input, several types of output can be computed including a method model.evaluate() used to compute the loss values, a method model.predict_classes() used to compute category outputs, and a method model.predict_proba() used to compute class probabilities.

A practical overview of backpropagation

Multi-layer perceptrons learn from training data through a process called backpropagation. In this paragraph we will give an intuition while more details are in Chapter 14, The Math Behind Deep Learning. The process can be described as a way of progressively correcting mistakes as soon as they are detected. Let’s see how this works.

Remember that each neural network layer has an associated set of weights that determine the output values for a given set of inputs. Additionally, remember that a neural network can have multiple hidden layers.

At the beginning, all the weights have some random assignment. Then the neural network is activated for each input in the training set: values are propagated forward from the input stage through the hidden stages to the output stage where a prediction is made.

Note that we keep Figure 1.27 below simple by only representing a few values with green dotted lines but in reality, all the values...

What have we learned so far?

In this chapter, we have learned the basics of neural networks. More specifically, we have learned what a perceptron is and what a multi-layer perceptron is, how to define neural networks in TensorFlow, how to progressively improve metrics once a good baseline is established, and how to fine-tune the hyperparameter space. In addition to that, we also have a good idea of useful activation functions (sigmoid and ReLU) available, and how to train a network with backpropagation algorithms based on either GD, SGD, or more sophisticated approaches, such as Adam and RMSProp.

Toward a deep learning approach

While playing with handwritten digit recognition, we came to the conclusion that the closer we get to an accuracy of 99%, the more difficult it is to improve. If we want more improvement, we definitely need a new idea. What are we missing? Think about it.

The fundamental intuition is that in our examples so far, we are not making use of the local spatial structure of images, which means we will use the fact that an image can be described as a matrix with data locality. In particular, this piece of code transforms the bitmap representing each written digit into a flat vector where the local spatial structure (the fact that some pixels are closer to each other) is gone:

# X_train is 60000 rows of 28x28 values; we  --> reshape it as in 60000 x 784.
X_train = X_train.reshape(60000, 784)
X_test = X_test.reshape(10000, 784)

However, this is not how our brain works. Remember that our vision is based on multiple cortex levels, each one recognizing...

Summary

In this chapter we learned what TensorFlow and Keras are and introduced neural networks with the perceptron and the multi-layer perceptron. Then, we saw a real example of recognizing handwritten digits with several optimizations.

The next chapter is devoted to regression and classification.

References

  1. Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev, vol. 65, pp. 386–408.
  2. Werbos, P. J. (1990). Backpropagation through time: what it does and how to do it. Proc. IEEE, vol. 78, pp. 1550–1560.
  3. Hinton, G. E., Osindero, S., and Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Comput, vol. 18, pp. 1527–1554.
  4. Schmidhuber, J. (2015). Deep learning in neural networks: an overview. Neural Networks : Off. J. Int. Neural Netw. Soc., vol. 61, pp. 85–117.
  5. Leven, S. (1996). The roots of backpropagation: From ordered derivatives to neural networks and political forecasting. Neural Networks, vol. 9.
  6. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, vol. 323.
  7. Herculano-Houzel, S. (2009). The human brain in numbers: a linearly scaled-up primate...
Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Understand the fundamentals of deep learning and machine learning through clear explanations and extensive code samples
  • Implement graph neural networks, transformers using Hugging Face and TensorFlow Hub, and joint and contrastive learning
  • Learn cutting-edge machine and deep learning techniques

Description

Deep Learning with TensorFlow and Keras teaches you neural networks and deep learning techniques using TensorFlow (TF) and Keras. You'll learn how to write deep learning applications in the most powerful, popular, and scalable machine learning stack available. TensorFlow 2.x focuses on simplicity and ease of use, with updates like eager execution, intuitive higher-level APIs based on Keras, and flexible model building on any platform. This book uses the latest TF 2.0 features and libraries to present an overview of supervised and unsupervised machine learning models and provides a comprehensive analysis of deep learning and reinforcement learning models using practical examples for the cloud, mobile, and large production environments. This book also shows you how to create neural networks with TensorFlow, runs through popular algorithms (regression, convolutional neural networks (CNNs), transformers, generative adversarial networks (GANs), recurrent neural networks (RNNs), natural language processing (NLP), and graph neural networks (GNNs)), covers working example apps, and then dives into TF in production, TF mobile, and TensorFlow with AutoML.

What you will learn

Learn how to use the popular GNNs with TensorFlow to carry out graph mining tasks Discover the world of transformers, from pretraining to fine-tuning to evaluating them Apply self-supervised learning to natural language processing, computer vision, and audio signal processing Combine probabilistic and deep learning models using TensorFlow Probability Train your models on the cloud and put TF to work in real environments Build machine learning and deep learning systems with TensorFlow 2.x and the Keras API

Product Details

Country selected

Publication date : Oct 6, 2022
Length 698 pages
Edition : 3rd Edition
Language : English
ISBN-13 : 9781803232911
Category :
Concepts :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon AI Assistant (beta) to help accelerate your learning
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now

Product Details


Publication date : Oct 6, 2022
Length 698 pages
Edition : 3rd Edition
Language : English
ISBN-13 : 9781803232911
Category :
Concepts :

Table of Contents

23 Chapters
Preface Chevron down icon Chevron up icon
1. Neural Network Foundations with TF Chevron down icon Chevron up icon
2. Regression and Classification Chevron down icon Chevron up icon
3. Convolutional Neural Networks Chevron down icon Chevron up icon
4. Word Embeddings Chevron down icon Chevron up icon
5. Recurrent Neural Networks Chevron down icon Chevron up icon
6. Transformers Chevron down icon Chevron up icon
7. Unsupervised Learning Chevron down icon Chevron up icon
8. Autoencoders Chevron down icon Chevron up icon
9. Generative Models Chevron down icon Chevron up icon
10. Self-Supervised Learning Chevron down icon Chevron up icon
11. Reinforcement Learning Chevron down icon Chevron up icon
12. Probabilistic TensorFlow Chevron down icon Chevron up icon
13. An Introduction to AutoML Chevron down icon Chevron up icon
14. The Math Behind Deep Learning Chevron down icon Chevron up icon
15. Tensor Processing Unit Chevron down icon Chevron up icon
16. Other Useful Deep Learning Libraries Chevron down icon Chevron up icon
17. Graph Neural Networks Chevron down icon Chevron up icon
18. Machine Learning Best Practices Chevron down icon Chevron up icon
19. TensorFlow 2 Ecosystem Chevron down icon Chevron up icon
20. Advanced Convolutional Neural Networks Chevron down icon Chevron up icon
21. Other Books You May Enjoy Chevron down icon Chevron up icon
22. Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Full star icon 5
(1 Ratings)
5 star 100%
4 star 0%
3 star 0%
2 star 0%
1 star 0%
Filter icon Filter
Top Reviews

Filter reviews by


Carlo Estopia Feb 18, 2024
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Feefo Verified review Feefo image
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.