Deep Learning and Convolutional Neural Networks

Deep learning (DL) is a machine learning (ML) technology based on multilayer artificial neural networks (ANNs) that has allowed many applications to reach a high degree of accuracy. Deep NNs (DNNs) are capable of modeling and have the ability to capture intricate connections between input and output information. Among the highly effective uses, computer vision (CV) stands out, encompassing activities such as categorization, image regression, and the identification of objects. As an illustration, an advanced NN can produce a stratified portrayal of entities, wherein each entity is recognized by a collection of features taking the shape of visual basics, such as specific contours, directed lines, surface details, and repetitive designs. Convolutional networks (CNNs) are characterized by convolutional layers, which use filters to analyze data in a local region and produce an activation map. These activation maps are then processed by pooling...

Technical requirements

In this chapter, we will introduce ML basic concepts. To understand these topics, a basic knowledge of algebra and mathematical modeling is needed. A working knowledge of the MATLAB environment is also required.

To work with the MATLAB code in this chapter, you need the following files (available on GitHub at https://github.com/PacktPublishing/MATLAB-for-Machine-Learning-second-edition):

• CNNPistachioClassification.m
• PistachioShort.zip

Understanding DL basic concepts

DL is a branch of ML based on using algorithms to model high-level abstractions about data. This discipline is part of a range of approaches that aim to learn methods for representing data. For example, an observation such as an image can be described in different ways: as a vector of intensity values for each pixel, or more abstractly as a set of edges or regions that have shapes or significant features. Some of these possible representations may prove more effective than others in facilitating the process of training another ML system.

For automatically identifying and extracting relevant features from raw data, we can use automated feature extraction to eliminate the need for manual feature engineering (FE). This process streamlines ML tasks and improves model performance.

Automated feature extraction

In this context, one of the central aspects of DL is the development of learning algorithms that specialize in automatically extracting significant...

Exploring DL models

Various types of DL architectures and techniques have been developed to tackle different tasks and challenges. Here are some examples:

• CNNs: Mainly used for image and video analysis, CNNs are designed to learn spatial hierarchies of features automatically and adaptively from input data. They have been highly successful in tasks such as image classification, object detection, and image segmentation.
• Recurrent NNs (RNNs): RNNs are well suited for tasks involving sequences, such as natural language processing (NLP) and speech recognition. They have an internal memory that allows them to maintain information about previous inputs, making them effective for handling sequential data.
• Long short-term memory (LSTM) networks: A type of RNN, LSTMs are designed to overcome the vanishing gradient problem in training deep networks. They are particularly useful for capturing long-range dependencies in sequential data, making them popular for tasks such as language...

Approaching CNNs

As outlined in Chapter 5, Introducing Artificial Neural Networks Modeling, ANNs draw their inspiration from biological NNs. These ANNs aim to replicate human cognitive processes by emulating the mechanisms observed in natural NNs. They serve the purpose of estimating or approximating functions that might rely on numerous inputs, many of which could be unfamiliar. ANNs are typically conceptualized as networks of interconnected neurons, facilitating the exchange of messages. Each connection possesses an associated weight, the value of which can be adjusted through learning from experience. This adaptive characteristic empowers NNs to accommodate diverse input types and facilitates their capacity to learn (Figure 6.1):

Figure 6.1 – ANN architecture with hidden layers

ANNs define a neuron as the central processing unit (CPU) that executes a mathematical operation to produce a single output from a set of input values. The neuron’...

Building a CNN in MATLAB

In this section, we will see how to train a CNN for image classification, starting from the images as input layers. The overall architecture of a CNN typically comprises a sequence of convolutional layers, interspersed with ReLU layers and, when appropriate, standardization and pooling layers. Ultimately, the network concludes with a series of FC layers leading to the output layer.

CNNs consist of two primary types of neurons:

• Processing neurons: These neurons undertake the responsibility of processing specific sections of the input image through convolution functions. Their primary role involves extracting distinctive features from the input data.
• Aggregation or pooling neurons: These neurons aggregate the input data and reduce its dimensions through subsampling, enhancing efficiency for subsequent layers’ processing.

By assembling the output values from a given layer, it becomes possible to reconstruct an intermediate image that...

Exploring the model’s results

Evaluating results is an essential part of any CNN implementation process. This, of course, is true for any algorithm based on ML. Evaluation metrics are quantitative measures used to assess the performance and quality of a model, algorithm, or system in various tasks, such as ML, data analysis, and optimization. These metrics provide a way to objectively quantify how well a model is performing and to compare different models or approaches.

The type of metric to adopt obviously depends on the type of algorithm we are implementing; in the previous section, we implemented a CNN for the classification of the pistachio species. So, let’s take a look at the metrics available for this type of algorithm.

For a classification task, we can use the following metrics:

• Accuracy: The proportion of correctly classified instances out of the total instances
• Precision: The ratio of true positive predictions to the total number of positive...

Discovering DL architectures

DL models are essentially multi-layered NNs, which refers to NNs that comprise multiple hidden layers (at least two) structured hierarchically. This hierarchical arrangement facilitates the sharing and reuse of information. Across this hierarchy, one can pinpoint features while disregarding unnecessary intricacies, thereby enhancing invariance. Within the realm of multi-level ML, deeper tiers acquire inputs from the outputs of prior layers and execute more complex transformations and abstractions. This layering approach to learning draws inspiration from the information processing and learning methods of mammalian brains, enabling them to react to external stimuli.

DL architectures are the fundamental blueprints that underlie the construction of DNNs, enabling them to effectively learn and represent complex patterns and features from data. These architectures define the layout, connections, and flow of information within the network, determining how...

Summary

In this chapter, we learned the basic concepts of DL and discovered how to implement a CNN algorithm in the MATLAB environment. First, we looked at how DL enables automated feature extraction, then we looked at how to train a deep network, and then we got a taste of the most popular DL architectures.

We then focused on CNN to analyze it in detail. We learned about the different layers that make up this network and what functions these layers perform. We then saw in practice how to implement a CNN in the MATLAB environment for image classification of pistachio nuts. We learned how to correctly import the image database, how to draw the architecture of the network with the different layers one after the other, and how to set the network parameters. Finally, we saw how to use evaluation metrics for the correct interpretation of the results.

In the last section, we introduced some of the most used networks, which will be the subject of more detailed study and application...