Removing Noise from Images Using Autoencoders

In this chapter, we will study a class of neural networks known as autoencoders, which have gained traction in recent years. In particular, the ability of autoencoders to remove noise from images has been greatly studied. In this chapter, we will build and train an autoencoder that is able to denoise and restore corrupted images.

In this chapter, we'll cover the following topics:

• What are autoencoders?
• Unsupervised learning
• Types of autoencoders—basic autoencoders, deep autoencoders and convolutional autoencoders
• Autoencoders for image compression
• Autoencoders for image denoising
• Step-by-step guide to build and train an autoencoder in Keras
• Analysis of our results

Technical requirements

The Python libraries required for this chapter are:

• matplotlib 3.0.2
• Keras 2.2.4
• Numpy 1.15.2
• PIL 5.4.1

The code and dataset for this chapter can be found in the GitHub repository for the book at https://github.com/PacktPublishing/Neural-Network-Projects-with-Python:

To download the code into your computer, you may run the following git clone command:

$ git clone https://github.com/PacktPublishing/Neural-Network-Projects-with-Python.git    

After the process is complete, there will be a folder titled Neural-Network-Projects-with-Python. Enter the folder by running:

$ cd Neural-Network-Projects-with-Python

To install the required Python libraries in a virtual environment, run the following command:

$ conda env create -f environment.yml

Note that you should have installed Anaconda in your computer first, before running this command. To enter the virtual...

What are autoencoders?

So far in this book, we have looked at the applications of neural networks for supervised learning. Specifically, in each project, we have a labeled dataset (that is, features x and label y) and our goal is to train a neural network using this dataset, so that the neural network is able to predict label y from any new instance x.

A typical feedforward neural network is shown in the following diagram:

In this chapter, we will study a different class of neural networks, known as autoencoders. Autoencoders represent a paradigm shift from the conventional neural networks we have seen so far. The goal of autoencoders is to learn a Latent Representation of the input. This representation is usually a compressed representation of the original input.

All autoencoders have an Encoder and a Decoder. The role of the encoder is to encode the input to a learned, compressed...

Latent representation

At this point, you might wonder what is the purpose of autoencoders. Why do we bother learning a representation of the original input, only to reconstruct a similar output? The answer lies in the learned representation of the input. By forcing the learned representation to be compressed (that is, having smaller dimensions compared to the input), we essentially force the neural network to learn the most salient representation of the input. This ensures that the learned representation only captures the most relevant characteristics of the input, known as the latent representation.

As a concrete example of latent representations, take, for example, an autoencoder trained on the cats and dogs dataset, as shown in the following diagram:

An autoencoder trained on this dataset will eventually learn that the salient characteristics of cats and dogs are the the shape...

Autoencoders for data compression

So far, we have seen how autoencoders are able to learn a reduced representation of the input data. It is natural to think that autoencoders can do a good job at generalized data compression. However, that is not the case. Autoencoders are poor at generalized data compression, such as image compression (that is, JPEG) and audio compression (that is, MP3), because the learned latent representation only represents the data on which it was trained. In other words, autoencoders only work well for images similar to those on which it was trained.

Furthermore, autoencoders are a "lossy" form of data compression, which means that the output from autoencoders will have less information when compared to the original input. These characteristics mean that autoencoders are poor at being generalized data compression techniques. Other forms of data...

The MNIST handwritten digits dataset

One of the datasets that we'll use for this chapter is the MNIST handwritten digits dataset. The MNIST dataset contains 70,000 samples of handwritten digits, each of size 28 x 28 pixels. Each sample contains only one digit within the image, and all samples are labeled.

The MNIST dataset is provided directly in Keras, and we can import it by simply running the following code:

from keras.datasets import mnist

training_set, testing_set = mnist.load_data()
X_train, y_train = training_set
X_test, y_test = testing_set

Let's plot out each of the digits to better visualize our data. The following code snippet uses matplotlib to plot the data:

from matplotlib import pyplot as plt
fig, ((ax1, ax2, ax3, ax4, ax5), (ax6, ax7, ax8, ax9, ax10)) = plt.subplots(2, 5, figsize=(10,5))

for idx, ax in enumerate([ax1,ax2,ax3,ax4,ax5, ax6,ax7,ax8,ax9,ax10...

Building a simple autoencoder

To cement our understanding, let's start off by building the most basic autoencoder, as shown in the following diagram:

So far, we have emphasized that the hidden layer (Latent Representation) should be of a smaller dimension than the input data. This ensures that the latent representation is a compressed representation of the salient features of the input. But how small should it be?

Ideally, the size of the hidden layer should balance between being:

• Sufficiently small enough to represent a compressed representation of the input features
• Sufficiently large enough for the decoder to reconstruct the original input without too much loss

In other words, the size of the hidden layer is a hyperparameter that we need to select carefully to obtain the best results. We shall see how we can define the size of the hidden layer in Keras.

...

Denoising autoencoders

Another interesting application of autoencoders is image denoising. Image noise is defined as a random variations of brightness in an image. Image noise may originate from the sensors of digital cameras. Although digital cameras these days are capable of capturing high quality images, image noise may still occur, especially in low light conditions.

Denoising images has been a challenge for researchers for many years. Early methods include applying some sort of image filter (that is, mean averaging filter, where the pixel value is replaced with the average pixel value of its neighbors) over the image. However, such methods can sometimes fall short and the effects can be less than ideal.

A few years ago, researchers discovered that we can train autoencoders for image denoising. The idea is simple. Instead of using the same input and output when training conventional...

Denoising documents with autoencoders

So far, we have applied our denoising autoencoder on the MNIST dataset, which is a pretty simple dataset. Let's take a look now at a more complicated dataset, which better represents the challenges of denoising documents in real life.

The dataset that we will be using is provided for free by the University of California Irvine (UCI). For more information on the dataset, you can visit UCI's website at https://archive.ics.uci.edu/ml/datasets/NoisyOffice.

The dataset can be found in the accompanying GitHub repository for this book. For more information on downloading the code and dataset for this chapter from the GitHub repository, please refer to the Technical requirements section earlier in the chapter.

The dataset consists of 216 different noisy images. The noisy images are scanned office documents that are tainted by coffee stains...

Summary

In this chapter, we looked at autoencoders, a class of neural networks that learn the latent representation of input images. We saw that all autoencoders have an encoder and decoder component. The role of the encoder is to encode the input to a learned, compressed representation and the role of the decoder is to reconstruct the original input using the compressed representation.

We first looked at autoencoders for image compression. By training an autoencoder with identical input and output, the autoencoder learns the most salient features of the input. Using MNIST images, we constructed an autoencoder with a 24.5 times compression rate. Using this learned 24.5x compressed representation, the autoencoder is able to successfully reconstruct the original input.

Next, we looked at denoising autoencoders. By training an autoencoder with noisy images as input and clean images...

Questions

1. How are autoencoders different from a conventional feed forward neural network?

Autoencoders are neural networks that learn a compressed representation of the input, known as the latent representation. They are different from conventional feed forward neural networks because their structure consists of an encoder and a decoder component, which is not present in CNNs.

2. What happens when the latent representation of the autoencoder is too small?

The size of the latent representation should be sufficiently small enough to represent a compressed representation of the input, and also be sufficiently large enough for the decoder to reconstruct the original image without too much loss.

3. What are the input and output when training a denoising autoencoder?

The input to a denoising autoencoder should be a noisy image and the output should be a reference clean image. During...