Chapter 5. Deep Learning

In this chapter, we'll cover some of the basics of deep learning. Deep learning refers to neural networks with lots of layers. It's kind of a buzzword, but the technology behind it is real and quite sophisticated.

The term has been rising in popularity along with machine learning and artificial intelligence, as shown in this Google trend chart:

As stated by some of the inventors of deep learning methods, the primary advantage of deep learning is that adding more data and more computing power often produces more accurate results, without the significant effort required for engineering.

In this chapter, we are going to be looking at the following:

• Deep learning methods
• Identifying handwritten mathematical symbols with CNNs
• Revisiting the bird species identifier to use images

Deep learning methods

Deep learning refers to several methods which may be used in a particular application. These methods include convolutional layers and pooling. Simpler and faster activation functions, such as ReLU, return the neuron's weighted sum if it's positive and zero if negative. Regularization techniques, such as dropout, randomly ignore weights during the weight update base to prevent overfitting. GPUs are used for faster training with the order that is 50 times faster. This is because they're optimized for matrix calculations that are used extensively in neural networks and memory units for applications such as speech recognition.

Several factors have contributed to deep learning's dramatic growth in the last five years. Large public datasets, such as ImageNet, that holds millions of labeled images covering a thousand categories and Mozilla's Common Voice Project, that contain speech samples are now available. Such datasets have satisfied the basic requirement for deep learning...

Identifying handwritten mathematical symbols with CNNs

This sections deals with building a CNN to identify handwritten mathematical symbols. We're going to use the HASYv2 dataset. This contains 168,000 images from 369 different classes where each represents a different symbol. This dataset is a more complex analog compared to the popular MNIST dataset, which contains handwritten numbers.

The following diagram depicts the kind of images that are available in this dataset:

 

And here, we can see a graph showing how many symbols have different numbers of images:

It is observed that many symbols have few images and there are a few that have lots of images. The code to import any image is as follows:

 

We begin by importing the Image class from the IPython library. This allows us to show images inside Jupyter Notebook. Here's one image from the dataset:

This is an image of the alphabet A. Each image is 30 x 30 pixels. This image is in the RGB format even though it doesn't really need to be RGB. The...

Revisiting the bird species identifier to use images

In this section, we're going to revisit the bird species identifier from before. This time, we're going to update it to use neural networks and deep learning. Can you recall the birds dataset? It has 200 different species of birds across 12,000 images. Unlike last time, we won't be using the human-labeled attributes, and instead we'll use the actual images without any pre-processing. In our first attempt, we're going to build a custom convolutional neural network, just like we did for the mathematical symbols classifier.

Let's go to the code. We will start with the typical imports:

We'll make some convenience variables, the rows and columns of the image, the width and height, and the number of channels, RGB, though every bird image will be equal. Even though they're not all necessarily the same size, we're going to resize them to this size so that they're all consistent:

Now, this project introduces an interesting feature on Keras called...

Summary

In this chapter, we discussed deep learning and CNNs. We practiced with convolutional neural networks and deep learning with two projects. First, we built a system that can read handwritten mathematical symbols and then revisited the bird species identifier form and changed the implementation to use a deep convolutional neural network that is significantly more accurate. This concludes the Python AI projects for beginners.