Generative Adversarial Networks

While I've spent much of this book talking about networks that classify or estimate, in this chapter I get to show you some deep neural networks that have the ability to create. The Generative Adversarial Network (GAN), learns to do this through a sort of internal competition between two deep networks, which we will talk about next. In the case of Deep Convolutional General Adversarial Networks (DCGAN), which is the type of GAN I'm going to focus on in this chapter, the network learns to create images that resemble the images in the training dataset.

We will cover the following topics in this chapter:

• An overview of the GAN
• Deep Convolutional GAN architecture
• How GANs can fail
• Safe choices for a GAN
• Generating MNIST images using a Keras GAN
• Generating CIFAR-10 images using a Keras GAN

An overview of the GAN

Generative Adversarial Networks are all about generating new content. GANs are capable of learning some distribution and creating a new sample from that distribution. That sample might just be a new point on a line that isn't present in our training data, but it could also be a new point in a very complex dataset. GANs have been used to generate new music, sounds, and images. According to Yann LeCun, adversarial training is the coolest thing since sliced bread (https://www.quora.com/session/Yann-LeCun/1). I'm not sure that sliced bread is especially cool, but Yann LeCun is a very cool guy so I'll take his word for it. Regardless, GANs are incredibly popular and while perhaps not as practical as some of the other topics we've covered in a business setting yet, they deserve some consideration in our survey of deep learning techniques.

In...

Deep Convolutional GAN architecture

There are many papers on GANs, each proposing new novel architectures and tweaks; however, most of them are at least somewhat based on the Deep Convolutional GAN (DCGAN). For the rest of the chapter, we will be focusing on this model because this knowledge will hopefully serve you well as you take on new and exciting GAN architectures that aren't covered here, such as the Conditional GAN (cGAN), the Stack GAN, the InfoGAN, or the Wasserstein GAN, or possibly some other new variant that you might choose to look at next.

The DCGAN was introduced by Alex Radford, Luke Metz, and Soumith Chintala in the paper Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (https://arxiv.org/pdf/1511.06434.pdf).

Lets take a look at the overall architecture of the DCGAN next.

...

How GANs can fail

Training a GAN is a tricky thing, to say the least. There are an amazing number of ways one fail at training a GAN. In fact, in writing this chapter, I found myself expanding the vocabulary of my profanity vector significantly while also spending a small fortune on cloud GPU time! Before I show you two working GANs later in the chapter, let's consider what could break and how we might be able to fix those things.

Stability

Training a GAN requires a careful balancing act between the discriminator and generator. The discriminator and generator are both fighting against each other for deep network supremacy. On the other hand, they also need each other to learn and grow. In order for this to work, neither...

Safe choices for GAN

I've previously mentioned Soumith Chintala's GAN hacks Git (https://github.com/soumith/ganhacks), which is an excellent place to start when you're trying to make your GAN stable. Now that we've talked about how difficult it can be to train a stable GAN, let's talk about some of the safe choices that will likely help you succeed that you can find there. While there are quite a few hacks out there, here are my top recommendations that haven't been covered already in the chapter:

• Batch norm: When using batch normalization, construct different minibatches for both real and fake data and make the updates separately.
• Leaky ReLU: Leaky ReLU is a variation of the ReLU activation function. Recall the the ReLU function is .

Leaky ReLU, however, is formulated as:

Leaky ReLU allows very small, non-zero gradients when the unit isn&apos...

Generating MNIST images using a Keras GAN

We've worked with MNIST before, but this time we will be generating new MNIST like images with a GAN. It can take a very long time to train a GAN; however, this problem is small enough that it can be run on most laptops in a few hours, which makes it a great example. Later we will expand this example to CIFAR-10 images.

The network architecture that I'm using here has been found by, and optimized by, many folks, including the authors of the DCGAN paper and people like Erik Linder-Norén, who's excellent collection of GAN implementations called Keras GAN (https://github.com/eriklindernoren/Keras-GAN) served as the basis of the code I used here. If you're wondering how I came to the architecture choices I used here, these are the giants whose shoulders I'm attempting to stand upon.

...

Generating CIFAR-10 images using a Keras GAN

While the network architecture remains for the most part unchanged I felt it necessary to show you an example that uses color images, as well as providing the example in Git, so that you had some place to start if you wanted to apply a GAN to your own data.

The CIFAR-10 is a famous dataset comprised of 60,000 32 x 32 x 3 RGB color images, distributed across 10 categories. Those categories are airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. Hopefully, when you see the generated images later, you might see something that you can imagine looks like those objects.

Loading CIFAR-10

Loading the dataset is almost exactly the same, as Keras also provides a loader...

Summary

In this chapter, we looked at GANs and how they can be used to generate new images. We learned a few rules for building GANs well, and we even learned to simulate MNIST and CIFAR-10 images. There is no doubt that you've probably seen some amazing images, created by GANs, in the media. After reading this chapter and working through these examples, you have the tools to do the same. I hope that you can take these ideas and adapt them. The only limitations left are your own imagination, your data, and your GPU budget.

In this book we covered a great many applications of deep learning, from simple regression to Generative Adversarial Networks. My greatest hope for this book is that it might help you make practical use of deep learning techniques, many of which have existed in the domain of academia and research, outside the reach of the practicing data scientist or machine...