Image Classification Using Convolutional Neural Networks

Convolutional neural networks (CNNs) are popular deep neural networks and are considered to be the gold standard for large-scale image classification tasks. Applications involving CNNs include image recognition and classification, natural language processing, medical image classification, and many others. In this chapter, we will continue with supervised learning situations where a response variable exists. This chapter provides steps for applying image classification and recognition using convolutional neural networks with an easy-to-follow practical example involving fashion-related Modified National Institute of Standards and Technology (MNIST) data. We also make use of images of fashion items downloaded from the internet to explore the generalization potential of the classification model that we develop.

More specifically...

Data preparation

In this chapter, we will make use of the Keras and EBImage libraries:

# Libraries
library(keras)
library(EBImage)

Let's get started by looking at some images downloaded from the internet. There are 20 images that include fashion articles such as shirts, bags, sandals, dresses, and others. These images were obtained using a Google search. We will try to develop an image recognition and classification model that recognizes these images and classifies them in appropriate categories. And to develop such a model, we will make use of the fashion-MNIST database of fashion articles:

# Read data
setwd("~/Desktop/image20")
temp = list.files(pattern = "*.jpg")
mypic <- list()
for (i in 1:length(temp))  {mypic[[i]] <- readImage(temp[[i]])}
par(mfrow = c(5,4))
for (i in 1:length(temp)) plot(mypic[[i]])

The 20 images of fashion items downloaded from the...

Layers in the convolutional neural networks

In this section, we will develop model architecture and then compile the model. We will also carry out calculations to compare the convolutional network with a fully connected network. Let's get started by specifying the model architecture.

Model architecture and related calculations

We start by creating a model using the keras_model_sequential function. The codes used for the model architecture are given as follows:

# Model architecture
model <- keras_model_sequential() 
model %>% 
         layer_conv_2d(filters = 32, 
                        kernel_size = c(3,3), 
                        activation = 'relu', 
                        input_shape = c(28,28,1)) %>...

Fitting the model

For fit the model, we will continue with the format that we have been using in the earlier chapters. The following code is used to fit the model:

# Fit model
model_one <- model %>% fit(trainx, 
                         trainy, 
                         epochs = 15, 
                         batch_size = 128, 
                         validation_split = 0.2)
plot(model_one)

Here, we are using 20 epochs, a batch size of 128, and 20% of the training data is reserved for validation. Since the neural network used here is more complex than the previous chapters, each run is likely to take relatively more time.

Accuracy and loss

After fitting the model, accuracy and loss values for the 15 epochs are plotted as follows...

Model evaluation and prediction

After fitting the model, we will evaluate its performance in terms of loss and accuracy. We will also create a confusion matrix to assess classification performance across all 10 types of fashion items. We will perform a model evaluation and prediction for both train and test data. We will also obtain images of fashion items that do not belong to the MNIST fashion data and explore how well the performance of the model can be generalized to new images.

Training data

Loss and accuracy based on training data are obtained as 0.115 and 0.960, respectively, as shown in the following code:

# Model evaluation
model %>% evaluate(trainx, trainy)

$loss  0.1151372
$acc  0.9603167

Next, we create a confusion...

Performance optimization tips and best practices

In any data analysis task, it is important to understand how the data was collected. With the model that we developed in the previous section, the accuracy dropped from over 90% for the test data to 50% for the 20 fashion item images that were downloaded from the internet. If this difference is not addressed, it will be difficult for this model to generalize well with any fashion items that are not part of the training or test data, and therefore will not be of much practical use. In this section, we will explore improvements to the classification performance of the model.

Image modification

Looking at the 64 images at the beginning of this chapter reveals some clues as to what...

Summary

In this chapter, we showed how to use a convolutional neural network (CNN) deep learning model for image recognition and classification. We made use of the popular fashion-MNIST data for training and testing the image classification model. We also went over calculations involving a number of parameters, and were able to contrast this with the number of parameters that would have been needed by a densely connected neural network. CNN models help to significantly reduce the number of parameters needed and thus result in significant savings in computing time and resources. We also used images of fashion items downloaded from the internet to see whether a classification model based on fashion-MNIST data can be generalized to similar items. We did notice that it is important to maintain consistency in the way images are laid out in the training data. Additionally, we also showed...