Chapter 9: Convolutional Neural Networks for Image Classification

In the previous chapters, we talked about Recurrent Neural Networks (RNNs) and how they can be applied to different types of sequential data and use cases. In this chapter, we want to talk about another family of neural networks, called Convolutional Neural Networks (CNNs). CNNs are especially powerful when used on data with grid-like topology and spatial dependencies, such as images or videos.

We will start with a general introduction to CNNs, explaining the basic idea behind a convolution layer and introducing some related terminology such as padding, pooling, filters, and stride.

Afterward, we will build and train a CNN for image classification from scratch. We will cover all required steps: from reading and preprocessing of the images to defining, training, and applying the CNN.

To train a neural network from scratch, a huge amount of labeled data is usually required. For some specific domains, such as...

Introduction to CNNs

CNNs are commonly used in image processing and have been the winning models in several image-processing competitions. They are often used, for example, for image classification, object detection, and semantic segmentation.

Sometimes, CNNs are also used for non-image-related tasks, such as recommendation systems, videos, or time-series analysis. Indeed, CNNs are not only applied to two-dimensional data with a grid structure but can also work when applied to one- or three-dimensional data. In this chapter, however, we focus on the most common CNN application area: image processing.

A CNN is a neural network with at least one convolution layer. As the name states, convolution layers perform a convolution mathematical transformation on the input data. Through such a mathematical transformation, convolution layers acquire the ability to detect and extract a number of features from an image, such as edges, corners, and shapes. Combinations of such extracted features...

Classifying Images with CNNs

In this section, we will see how to build and train from scratch a CNN for image classification.

The goal is to classify handwritten digits between 0 and 9 with the data from the MNIST database, a large database of handwritten digits commonly used for training various image-processing applications. The MNIST database contains 60,000 training images and 10,000 testing images of handwritten digits and can be downloaded from this website: http://yann.lecun.com/exdb/mnist/.

To read and preprocess images, KNIME Analytics Platform offers a set of dedicated nodes and components, available after installing the KNIME Image Processing Extension.

Tip

The KNIME Image Processing Extension (https://www.knime.com/community/image-processing) allows you to read in more than 140 different format types of images (thanks to the Bio-Formats Application Processing Interface (API)). In addition, it can be used to apply well-known image-processing techniques such as...

Introduction to transfer learning

The general idea of transfer learning is to reuse the knowledge gained by a network trained for task A on another related task B. For example, if we train a network to recognize sailing boats (task A), we can use this network as a starting point to train a new model to recognize motorboats (task B). In this case, task A is called the source task and task B the target task.

Reusing a trained network as the starting point to train a new network is different from the traditional way of training networks, whereby neural networks are trained on their own for specific tasks on specific datasets. Figure 9.19 here visualizes the traditional way of network training, whereby different systems are trained for different tasks and domains:

Figure 9.19 – Traditional way of training machine learning models and neural networks

But why should we use transfer learning instead of training models in the traditional, isolated way...

Applying Transfer Learning for Cancer Type Prediction

We will introduce here a new (and final) case study. We will start from the state-of-the-art VGG16 network as a source network to train a new target network on a dataset of images describing three different subtypes of lymphoma, which are chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL).

A typical task for a pathologist in a hospital is to look at histopathology slide images and make a decision about the type of lymphoma. Even for experienced pathologists this is a difficult task and, in many cases, follow-up tests are required to confirm the diagnosis. An assistive technology that can guide pathologists and speed up their job would be of great value.

VGG16 is one of the winner models on the ImageNet Challenge from 2014. It is a stacked CNN network, using kernels of size with an increasing depth—that is, with an increasing number of filters. The original network was trained...

Summary

In this chapter, we explored CNNs, focusing on image data.

We started with an introduction to convolution layers, which motivates the name of this new family of neural networks. In this introduction, we explained why CNNs are so commonly used for image data, how convolutional networks work, and the impact of the many setting options. Next, we discussed pooling layers, commonly used in CNNs to efficiently downsample the data.

Finally, we put all this knowledge to work by building and training from scratch a CNN to classify images of digits between 0 and 9 from the MNIST dataset. Afterward, we discussed the concept of transfer learning, introduced four scenarios in which transfer learning can be applied, and showed how we can use transfer learning in the field of neural networks.

In the last section, we applied transfer learning to train a CNN to classify histopathology slide images. Instead of training it from scratch, this time we reused the convolutional layers of...

Questions and Exercises

1. What is the kernel size in a convolutional layer?

   a) The area summarized by a statistical value

   b) The size of the matrix moving across an image

   c) The number of pixels to shift the matrix

   d) The size of the area used by a layer

2. What is a pooling layer?

   a) A pooling layer is a commonly used layer in RNNs

   b) A pooling layer summarizes an area with a statistical value

   c) A pooling layer is a commonly used layer in feedforward networks

   d) A pooling layer can be used to upsample images

3. When is transfer learning helpful?

   a) To transfer data to another system

   b) If no model is available

   c) If not enough labeled data is available

   d) To compare different models