Neural Networks for Image Classification

In recent years, we have seen a huge interest in neural networks, which are successfully used in various areas—business, medicine, technology, geology, physics, and so on. Neural networks have come into practice wherever it is necessary to solve problems of forecasting, classification, or control. This approach is attractive from an intuitive point of view because it is based on a simplified biological model of the human nervous system. It arose from research in the field of artificial intelligence, namely, from attempts to reproduce the ability of biological nervous systems to learn and correct mistakes by modeling the low-level structure of the brain. Neural networks are compelling modeling methods that allow us to reproduce extremely complex dependencies because they are non-linear. Neural networks also cope better with the curse...

Technical requirements

You will need the following technical requirements to complete this chapter:

• Dlib library
• Shogun library
• Shark-ML library
• PyTorch library
• Modern C++ compiler with C++17 support
• CMake build system version >= 3.8

An overview of neural networks

In this section, we will discuss what artificial neural networks are and their building blocks. We will learn how artificial neurons work and how they relate to their biological analogs. We will also discuss how to train neural networks with the backpropagation method, as well as how to deal with the overfitting problem.

A neural network is a sequence of neurons interconnected by synapses. The structure of the neural network came into the world of programming directly from biology. Thanks to this structure, the computer has the ability to analyze and even remember information. In other words, neural networks are based on the human brain, which contains millions of neurons that transmit information in the form of electrical impulses.

Artificial neural networks are inspired by biology because they are composed of elements with similar functionalities...

Delving into convolutional networks

The MLP is the most powerful feedforward neural network. It consists of several layers, where each neuron receives its copy of all the output from the previous layer of neurons. This model is ideal for certain types of tasks, for example, training on a limited number of more or less unstructured parameters.

Nevertheless, let's see what happens to the number of parameters (weights) in such a model when raw data is used as input. For example, the CIFAR-10 dataset contains 32 x 32 x 3 color images, and if we consider each channel of each pixel as an independent input parameter for MLP, each neuron in the first hidden layer adds about 3,000 new parameters to the model! With the increase in image size, the situation quickly gets out of hand, producing images that users can't use for real applications.

One popular solution is to lower the...

What is deep learning?

Most often, the term deep learning is used to describe artificial neural networks that were designed to work with large amounts of data and use complex algorithms to train the model. Algorithms for deep learning can use both supervised and unsupervised algorithms (reinforcement learning). The learning process is deep because, over time, the neural network covers an increasing number of levels. The deeper the network is (that is, it has more hidden layers, filters, and levels of feature abstraction it has), the higher the network's performance. On large datasets, deep learning shows better accuracy than traditional machine learning algorithms.

The real breakthrough that led to the current resurgence of interest in deep neural networks occurred in 2012, after the publication of the article ImageNet classification with deep convolutional neural networks...

Examples of using C++ libraries to create neural networks

Many machine learning libraries have an API for creating and working with neural networks. All the libraries we used in the previous chapters—Shogun, Dlib, and Shark-ML—are supported by neural networks. But there are also specialized frameworks for neural networks; for example, one popular one is the PyTorch framework. The difference between a specialized library and the common purpose libraries is that the specialized one supports more configurable options and supports different network types, layers, and loss functions. Also, specialized libraries usually have more modern instruments, and these instruments are introduced to their APIs more quickly.

In this section, we'll create a simple MLP for a regression task with the Shogun, Dlib, and Shark-ML libraries. We'll also use the PyTorch C++ API to...

Understanding image classification using the LeNet architecture

In this section, we'll implement a convolutional neural network for image classification. We are going to use the famous dataset of handwritten digits called the Modified National Institute of Standards and Technology (MNIST), which can be found at http://yann.lecun.com/exdb/mnist/. The dataset is a standard that was proposed by the US National Institute of Standards and Technology to calibrate and compare image recognition methods using machine learning, primarily based on neural networks.

The creators of the dataset used a set of samples from the US Census Bureau, with some samples written by students of American universities added later. All the samples are normalized, anti-aliased grayscale images of 28 x 28 pixels. The MNIST database contains 60,000 images for training and 10,000 images for testing. There...

Summary

In this chapter, we looked at what artificial neural networks are, looked at their history, and examined the reasons for their appearance, rise, and fall and why they have become one of the most actively developed machine learning approaches today. We looked at the difference between biological and artificial neurons before learning the basics of the perceptron concept, which was created by Frank Rosenblatt. Then, we discussed the internal features of artificial neurons and networks, such as activation functions and their characteristics, network topology, and convolution layer concepts. We also learned how to train artificial neural networks with the error backpropagation method. We saw how to choose the right loss function for different types of tasks. Then, we discussed the regularization methods that are used to combat overfitting during training.

Finally, we implemented...

Further reading

• Loss Functions for Deep Neural Networks in Classification: https://arxiv.org/pdf/1702.05659.pdf.
• Neural Networks and Deep Learning, by Michael Nielsen: http://neuralnetworksanddeeplearning.com/.
• Principles of Neurodynamics, Rosenblatt, Frank (1962), Washington, DC: Spartan Books.
• Perceptrons, Minsky M. L. and Papert S. A. 1969. Cambridge, MA: MIT Press.
• Neural Networks and Learning Machines, Simon O. Haykin 2008
• Deep Learning, Ian Goodfellow, Yoshua Bengio, Aaron Courville 2016
• The PyTorch GitHub page: https://github.com/pytorch/.
• The PyTorch documentation site: https://pytorch.org/docs/.
• The LibPyTorch (C++) documentation site: https://pytorch.org/cppdocs/.