Creating New Images Using Generative Adversarial Networks

This chapter illustrates the application of generative adversarial networks (GANs) for generating new images using a practical example. So far in this book, using image data, we have illustrated the use of deep networks for image classification tasks. However, in this chapter, we will explore an interesting and popular approach that helps create new images. Generative adversarial networks have been applied for generating new images, improving image quality, and generating new text and new music. Another interesting application of GANs is in the area of anomaly detection. Here, a GAN is trained to generate data that is considered normal. When this network is used for reconstructing data that is considered not normal or anomalous, the differences in results can help us detect the presence of an anomaly. We will look at an...

Generative adversarial network overview

GANs make use of two networks:

• Generator network
• Discriminator network

For the generator network, noisy data, which is usually random numbers that have been generated from a standard normal distribution are provided as input. A flow chart showing an overview of a generative adversarial network is as follows:

As indicated in the preceding flowchart, the generator network uses noisy data as input and tries to create an image that we can label as fake. These fake images, along with the labels representing them as fake, are provided as input to the discriminator network. Along with the labeled fake images, we can also provide real images with labels as input to the discriminator network.

During the training process, the discriminator network tries to differentiate between a fake image created by the generator network and a real image. While...

Processing MNIST image data

In this section, will use the Keras library, which also includes MNIST data. We will also make use of the EBImage library, which is useful for processing image data. MNIST data contains handwritten images from 0 to 9. Let's take a look at the following code to understand this data:

# Libraries and MNIST data
library(keras)
library(EBImage)
mnist <- dataset_mnist()
str(mnist)
List of 2
 $ train:List of 2
 ..$ x: int [1:60000, 1:28, 1:28] 0 0 0 0 0 0 0 0 0 0 ...
 ..$ y: int [1:60000(1d)] 5 0 4 1 9 2 1 3 1 4 ...
 $ test :List of 2
 ..$ x: int [1:10000, 1:28, 1:28] 0 0 0 0 0 0 0 0 0 0 ...
 ..$ y: int [1:10000(1d)] 7 2 1 0 4 1 4 9 5 9 ...

From the preceding code, we can make the following observations:

• Looking at the structure of this data, we can see that there are 60,000 images in the training data and 10,000 images in the test data.
• These handwritten images...

Developing the generator network

The generator network will be used for generating fake images from data that's provided in the form of noise. In this section, we will develop the architecture of the generator network and look at the parameters that are involved by summarizing the network.

Network architecture

Let's take a look at the code for developing the generator network architecture:

# Generator network
h <- 28; w <- 28; c <- 1; l <- 28  
gi <- layer_input(shape = l)
go <- gi %>% layer_dense(units = 32 * 14 * 14) %>%
         layer_activation_leaky_relu() %>% 
         layer_reshape(target_shape = c(14, 14, 32)) %>% 
         layer_conv_2d(filters = 32, 
                       kernel_size...

Developing the discriminator network

The discriminator network will be used for classifying fake and real images. The architecture and summary of the network will be discussed in this section.

Architecture

The code that's used for developing the discriminator network architecture is as follows:

# Discriminator network
di <- layer_input(shape = c(h, w, c))
do <- di %>% 
         layer_conv_2d(filters = 64, kernel_size = 4) %>% 
         layer_activation_leaky_relu() %>% 
         layer_flatten() %>%
         layer_dropout(rate = 0.3) %>%  
         layer_dense(units = 1, activation = "sigmoid")
d <- keras_model(di, do)

From the preceding code, we can observe the following:

• We provided an input...

Training the network

In this section, we will out training of the network. While training the network, we will save fake images and store loss values to review the training progress. They will help us assess the effectiveness of the network when creating realistic fake images.

Initial setup for saving fake images and loss values

We will start by specifying a few things that we will need for the training process. Let's take a look at the following code:

# Initial settings
b <- 50  
setwd("~/Desktop/")
dir <- "FakeImages"
dir.create(dir)
start <- 1; dloss <- NULL; gloss <- NULL

From the preceding code, we can observe the following:

• We will use a batch size (b) of 50.
• We will save fake images...

Reviewing results

In this section, we will review the network losses that were obtained from 100 iterations. We will also take a look at the progress of using fake images from iteration 1 to 100.

Discriminator and GAN losses

The discriminator and GAN loss values that were obtained from our 100 iterations can be plotted as follows. The discriminator loss is based on the loss values for the fake and real images:

From the preceding plot, we can make the following observations:

• The loss values for the discriminator network and the GAN show high variability during the first 20 iterations. This variability is an outcome of the learning process.
• The discriminator and generator networks are competing against each other and trying...

Performance optimization tips and best practices

In this section, we will carry out an experiment by inserting an additional convolutional layer into the generator network, as well as in the discriminator network. Through this experiment, we will convey performance optimization tips and best practices.

Changes in the generator and discriminator network

The changes in the generator network are shown in the following code:

# Generator network
gi <- layer_input(shape = l)
go <- gi %>% layer_dense(units = 32 * 14 * 14) %>%
         layer_activation_leaky_relu() %>% 
         layer_reshape(target_shape = c(14, 14, 32)) %>% 
         layer_conv_2d(filters = 32, 
                       kernel_size = 5,
               ...

Summary

In this chapter, we used a generative adversarial network to illustrate how to generate images of a single handwritten digit. Generative adversarial networks make use of two networks: generator and discriminator networks. Generator networks create fake images from data containing random noise, while discriminator networks are trained to differentiate between fake images and real images. These two networks compete against each other so that realistic-looking fake images can be created. Although in this chapter we provided an example of using a generative adversarial network to generate new images, these networks are also known to have applications in generating new text or new music, as well as in anomaly detection.

In this section, we went over various deep learning networks that are useful for dealing with image data. In the next section, we will go over deep learning...