Overview

By the end of this chapter, you will confidently build and tune neural networks using the Sequential deep learning algorithm provided by Keras (TensorFlow). In particular, you will apply deep learning to make meaningful predictions from tabular numerical datasets, in addition to image-based datasets. You will compare the sequential deep learning algorithm to standard machine learning algorithms using regression and classification. You will tune Keras models by modifying Dense layers, Hidden layers, Dropout nodes, and Early Stopping to optimize your neural networks. Finally, you will learn how to classify images by building convolutional neural networks, which are some of the strongest machine learning algorithms in the world today.

Introduction

Deep learning is a specific branch of machine learning modeled after the human brain, commonly referred to as neural networks.

The human brain works by transferring external stimuli through a vast network of neurons. These neurons work together to produce a desired output. For instance, when you are driving a car and your eye detects a red light, the neurons in your brain work together to rapidly output a request to stop the car. This request is based on optimizing past data that your brain has received.

According to the National Library of Medicine (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2776484/), the most advanced human brains contain approximately 100 billion neurons. This is a very deep network. The general idea behind deep learning is to emulate the human brain by creating a deep algorithmic network that responds to incoming data.

Machine learning started with neural networks when Frank Rosenblatt’s 1958 Perceptron demonstrated 100% efficiency...

Technical requirements

You can find the code files for this chapter on GitHub at https://github.com/PacktPublishing/The-Python-Workshop-Second-Edition/tree/main/Chapter12, and within the following Colab notebook: https://colab.research.google.com/drive/14FUXbsuRvz3jO6bzAm1Mgas6faJ0G61-?usp=sharing.

The technical requirements are different for Colab notebooks and Jupyter Notebook. You will need to install Keras and TensorFlow for Jupyter Notebook, whereas they are included with Colab notebooks in advance.

Colab notebooks

In this chapter, I recommend using an online version of Jupyter Notebook, called Colab notebooks (short for Google Colaboratory Notebooks) for the following reasons:

• Colab notebooks allow you to use graphical processing units (GPUs), which will greatly speed up computations for high-demand processing. This is particularly beneficial for neural networks, which can be time-consuming when combining large datasets with deep networks. We will be using them...

Introduction to deep learning

The neurons in a human brain are analogously referred to as nodes in deep learning algorithms. Individual nodes may be thought of as computational units. Although they may stand alone, they are more powerful when connected to one another.

As a visual, here is the Boston Housing DataFrame from Chapter 11, Machine Learning. Each column in the following DataFrame can be represented as a node, as can each entry:

Figure 12.1 – Sample from the Boston Housing dataset

In linear regression, using the standard machine learning algorithm introduced in Chapter 11, Machine Learning, each column, or node, is multiplied by a constant, called a weight, and the individual weights are summed together to make a prediction, as in the following diagram:

Figure 12.2 – Linear regression diagram from Berkeley Coding Academy, created by the author Corey Wade

The process is linear because it uses the simple technique...

Your first deep learning model

Let’s use deep learning to predict the median house values in Boston to compare our results to the standard machine learning algorithms used in Chapter 11, Machine Learning.

First deep learning libraries

Before building your first deep learning model, let’s take a brief look at the libraries that we will import and use:

• pandas: We need data to build a deep learning model, and pandas, Python’s data analytics library, will remain our standard from Chapter 10, Data Analytics with pandas and NumPy, and Chapter 11, Machine Learning, to read and view data.
• train_test_split: We will use train_test_split as in Chapter 11, Machine Learning, to split the data into a training set and a test set.
• TensorFlow: TensorFlow has become the gold standard in deep learning. Created by Google in 2015, TensorFlow is a free, open source library developed by Google Brain. TensorFlow works on its own, but it is also the backend for keras...

Additional regularization technique – Dropout

Regularization is built into the Early Stopping monitor because a validation test is used during each epoch to score against the training set. The idea is that even if the training set continues to improve, the model will stop building after the validation ceases to improve within the callback patience.

It’s important to examine additional regularization techniques so that you can build even larger neural networks without overfitting the data.

Another very popular regularization technique widely used in neural networks is called the Dropout. Given multiple nodes in multiple layers result in thousands or millions of weights, neural networks can easily overfit the training set.

The idea behind Dropout is to randomly drop some nodes altogether. In densely connected networks, since all nodes in one layer are connected to all nodes in the next layer, any node may be eliminated except the last.

Dropout works in code...

Building neural networks for classification

In the previous examples, the final output could have been any given number, so we were dealing with regression. But in many cases, the final output may be 0 or 1, “yes” or “no,” or a range of distinct colors. In each of these cases, the type of machine learning algorithms that we are looking for fall under the general heading of classification.

In neural networks, one primary difference between regression and classification is the loss functions and scoring metrics. For classification, loss functions and scoring metrics are usually based on some kind of percentage of accuracy. It’s standard to use binary_crossentropy as the loss function for classification and to include an accuracy metric, which is the percentage of cases the model predicts correctly.

Another important difference when building a classification model is the final node itself. In regression, we used a Dense layer with one node only...

Convolutional neural networks

Although deep learning performs well on tabular regression and classification datasets, deep learning has a bigger advantage when making predictions from unstructured data such as images or text.

When it comes to classifying images, deep learning shines by analyzing data not one-dimensionally, but two-dimensionally, using convolutional neural networks, or CNNs for short.

Convolutional neural networks are among the strongest machine learning algorithms in the world today for classifying images. In this section, you will learn the basic theory behind convolutions before building your own CNN.

MNIST

MNIST is the name of a famous dataset of handwritten digits from 1998 that has been widely used in computer vision. The dataset consists of 60K training images and 10K test images.

Google Colab includes a smaller sample of 20K training images, along with the 10K test images, that may be directly accessed in a Colab notebook and prepared for machine...

Summary

In this chapter, you learned how to build neural networks using Keras, one of the best deep learning libraries in the world. You built Sequential dense models with a variety of hidden layers and nodes using the ReLU activation function and the Adam optimizer. You used Early Stopping to find an ideal number of epochs, and you used Dropout to help prevent overfitting. Furthermore, you trained both regressors and classifiers, making sure to use binary_crossentropy as the classification loss function and the sigmoid activation function. Additionally, you learned about the foundations behind convolutions and built convolutional neural networks to classify handwritten digits with over 98% accuracy.

Congratulations on completing your deep learning journey.

The next chapter is the final chapter of the second edition of the Python Workshop, New Features in Python, which includes updates from Python 3.7 to Python 3.11.