Using SRGANs to Generate Photo-Realistic Images

Super-Resolution Generative Adversarial Network, or SRGAN, is a Generative Adversarial Network (GAN) that can generate super-resolution images from low-resolution images, with finer details and higher quality. CNNs were earlier used to produce high-resolution images that train quicker and achieve high-level accuracy. However, in some cases, they are incapable of recovering finer details and often generate blurry images. In this chapter, we will implement an SRGAN network in the Keras framework that will be capable of generating high-resolution images. SRGANs were introduced in the paper titled, Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, by Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, and others, which is available at the following link: https://arxiv.org...

Introducing SRGANs

Like any other GAN, SRGANs contain a generator network and a discriminator network. Both networks are deep. The functionality of both of these networks is specified as follows:

• The generator: The generator network takes a low-resolution image of a dimension of 64x64x3, and, after a series of convolution and upsampling layers, generates a super-resolution image of a shape of 256x256x3
• The discriminator: The discriminator network takes a high-resolution image and tries to identify whether the given image is real (from the real data samples) or fake (generated by the generator)

The architecture of SRGANs

In SRGANs, both of the networks are deep convolution neural networks. They contain convolution layers...

Setting up the project

If you haven't already cloned the repository with the complete code for all the chapters, clone the repository now. The downloaded code has a directory called Chapter05, which contains the entire code for this chapter. Execute the following commands to set up the project:

1. Start by navigating to the parent directory, as follows:

cd Generative-Adversarial-Networks-Projects

2. Now change the directory from the current directory to Chapter05:

cd Chapter05

3. Next, create a Python virtual environment for this project:

virtualenv venv
virtualenv venv -p python3 # Create a virtual environment using 
           python3 interpreter
virtualenv venv -p python2 # Create a virtual environment using 
           python2 interpreter

We will be using this newly created virtual environment for this project. Each chapter has its own separate virtual environment.

4. Next, activate...

Downloading the CelebA dataset

For this chapter, we will use the large-scale CelebFaces Attributes (CelebA) dataset, which is available at http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html. The dataset contains 202, 599 face images of celebrities.

The dataset is available for non-commercial research purposes only and can't be used for commercial purposes. If you intend to use the dataset for commercial purposes, seek permissions from the owners of the images.

We will use the CelebA dataset to train our SRGAN network. Perform the following steps to download and extract the dataset:

1. Download the dataset from the following link:

https://www.dropbox.com/sh/8oqt9vytwxb3s4r/AAB06FXaQRUNtjW9ntaoPGvCa?dl=0

2. Extract images from the downloaded img_align_celeba.zip by executing the following command:

unzip img_align_celeba.zip

We have now downloaded and extracted the dataset. We can...

The Keras implementation of SRGAN

As we discussed, SRGAN has three neural networks, a generator, a discriminator, and a pre-trained VGG19 network on the Imagenet dataset. In this section, we will write the implementation for all the networks. Let's start by implementing the generator network.

Before starting to write the implementations, create a Python file called main.py and import the essential modules, as follows:

import glob
import os

import numpy as np
import tensorflow as tf
from keras import Input
from keras.applications import VGG19
from keras.callbacks import TensorBoard
from keras.layers import BatchNormalization, Activation, LeakyReLU, Add, Dense, PReLU, Flatten
from keras.layers.convolutional import Conv2D, UpSampling2D
from keras.models import Model
from keras.optimizers import Adam
from keras_preprocessing.image import img_to_array, load_img
from scipy.misc import imsave...

Training the SRGAN

Training the SRGAN network is a two-step process. In the first step, we train the discriminator network. In the second step, we train the adversarial network, which eventually trains the generator network. Let's start training the network.

Perform the following steps to train the SRGAN network:

1. Start by defining the hyperparameters required for the training:

# Define hyperparameters
data_dir = "Paht/to/the/dataset/img_align_celeba/*.*"
epochs = 20000
batch_size = 1

# Shape of low-resolution and high-resolution images
low_resolution_shape = (64, 64, 3)
high_resolution_shape = (256, 256, 3)

2. Next, define the training optimizer. For all networks, we will use Adam optimizer with the learning rate equal to 0.0002 and beta_1 equal to 0.5:

# Common optimizer for all networks
common_optimizer = Adam(0.0002, 0.5)

...

Practical applications of SRGANs

Now, let's look at the practical applications of SRGANs:

• Recovery of old photographs
• Industry applications, such as automatically increasing the resolution of logos, banners, and pamphlets
• Automatically increasing the resolution of social media images for users
• Automatically enhancing pictures on cameras while capturing them
• Increasing the resolution of medical images

Summary

In this chapter, we began by introducing SRGANs. Then, we looked at the architecture of the generator and discriminator networks. Later, we carried out the required setup for the project. Then, we gathered and explored the dataset. After that, we implemented the project in Keras before training the SRGAN, evaluating the trained SRGAN network, and optimizing the trained model using hyperparameter optimization techniques. Finally, we took a brief look at some different applications of SRGANs.

In the next chapter, we will be going through StackGAN and its different applications.