Image Classification for Small Data Using Transfer Learning

In the previous chapters, we developed deep learning networks and explored various application examples related to image data. One major difference compared to what we will be discussing in this chapter is that, in the previous chapters, we developed models from scratch.

Transfer learning can be defined as an approach where we reuse what a trained deep network has learned to solve a new but related problem. For example, we may be able to reuse a deep learning network that's been developed to classify thousands of different fashion items to develop a deep network to classify three different types of dresses. This approach is similar to what we can observe in real life, where a teacher transfers knowledge or learning gained over the years to students or a coach passes on learning or experience to new players. Another...

Using a pretrained model to identify an image

Before we proceed, let's load three packages that we'll need in this section:

# Libraries used
library(keras)
library(EBImage)
library(tensorflow)

The Keras and TensorFlow libraries will be used for developing the pretrained image classification model, while the EBImage library will be used for processing and visualizing image data.

In Keras, the following pretrained image classification models are available:

• Xception
• VGG16
• VGG19
• ResNet50
• InceptionV3
• InceptionResNetV2
• MobileNet
• MobileNetV2
• DenseNet
• NASNet

These pretrained models are trained on images from ImageNet (http://www.image-net.org/). ImageNet is a huge image database that contains several million images.

We will start by using a pretrained model known as resnet50 to identify an image. The following is the code we can use to utilize...

Working with the CIFAR10 dataset

For illustrating the use of pretrained models with new data, we will make use of the CIFAR10 dataset. CIFAR stands for Canadian Institute For Advanced Research, and 10 refers to the 10 categories of images that are contained in the data. The CIFAR10 dataset is part of the Keras library and the code for obtaining it is as follows:

# CIFAR10 data
data <- dataset_cifar10()
str(data)
OUTPUT
List of 2
 $ train:List of 2
  ..$ x: int [1:50000, 1:32, 1:32, 1:3] 59 154 255 28 170 159 164 28 134 125 ...
  ..$ y: int [1:50000, 1] 6 9 9 4 1 1 2 7 8 3 ...
 $ test :List of 2
  ..$ x: int [1:10000, 1:32, 1:32, 1:3] 158 235 158 155 65 179 160 83 23 217 ...
  ..$ y: num [1:10000, 1] 3 8 8 0 6 6 1 6 3 1 ...

In the preceding code, we can observe the following:

• We can read the dataset using the dataset_cifar10() function.
• The structure of the data shows that there are...

Image classification with CNN

In this section, we will use a subset of the CIFAR10 dataset to develop a convolutional neural network-based image classification model and assess its classification performance.

Data preparation

We will keep the data size smaller by using only the first 2,000 images in the training and test data from CIFAR10. This will allow the image classification model to be run on a regular computer or laptop. We will also resize the training and test images from 32 x 32 dimensions to 224 x 224 dimensions to be able to compare classification performance with the pretrained model. The following code includes the necessary preprocessing that we went over earlier in this chapter:

# Selecting first 2000 images...

Classifying images using the pretrained RESNET50 model

In this section, we will make use of the pretrained RESNET50 model to develop an image classification model. We will use the same training and test data that we used in the previous section to make comparing classification performance easier.

Model architecture

We will upload the RESNET50 model without including the top layer. This will help us customize the pretrained model for use with CIFAR10 data. Since the RESNET50 model is trained with the help of over 1 million images, it captures useful features and representations of images that can be reused with new but similar and smaller data. This reusability aspect of pretrained models not only helps to reduce the time and...

Model evaluation and prediction

Now, we will evaluate the performance of this model for the training and test data. Calculations relating to the loss, accuracy, and confusion matrix will be carried out so that we can evaluate the model image's classification performance. We will also obtain the accuracy for each of the 10 categories.

Loss, accuracy, and confusion matrix with the training data

The code for obtaining the loss, accuracy, and confusion matrix for the training data is as follows:

# Loss and accuracy
model %>% evaluate(trainx, trainy)
$loss
[1] 1.954347
$acc
[1] 0.8785

# Confusion matrix
pred <- model %>%   predict_classes(trainx)
table(Predicted=pred, Actual=data$train$y[1:2000,])
         Actual
Predicted...

Performance optimization tips and best practices

To explore further image classification improvement, in this section, we will try three experiments. In the first experiment, we will mainly use the adam optimizer when compiling the model. In the second experiment, we will carry out hyperparameter tuning by varying the number of units in the dense layer, the dropout percentage in the dropout layer, and the batch size when fitting the model. Finally, in the third experiment, we will work with another pretrained network called VGG16.

Experimenting with the adam optimizer

In this first experiment, we will use the adam optimizer when compiling the model. At the time of training the model, we will also increase the number of epochs...

Summary

In this chapter, we illustrated the use of pretrained deep neural networks for developing image classification models. Such pretrained networks, which are trained using over 1 million images, capture reusable features that can be applied to similar but new data. This aspect becomes valuable when developing image classification models with relatively smaller datasets. In addition, they provide savings in terms of the use of computational resources and time. We started by making use of the RESNET50 pretrained network to identify an image of a Norwich terrier dog. Subsequently, we made use of 2,000 images from the CIFAR10 dataset to illustrate the usefulness of applying pretrained networks to a relatively smaller dataset. The initial convolutional neural networks model that we built from scratch suffered from overfitting and did not yield useful results.

Next, we used the...