An overview of GANs

Before we move into the more advanced concepts of GANs, let's start by going over GANs, and introducing the underlying concepts of them. GANs are very powerful; this simple statement is proven by the fact that they can generate new celebrity faces that are not of real people by performing latent space interpolations.

A great example of the advanced features of GANs [4] can be seen with this YouTube video (https://youtu.be/G06dEcZ-QTg). The video, which shows how GANs can be utilized to produce realistic faces just shows how powerful they can be. This topic is much more advanced than anything we've looked at before in this book. For example, the above video is something that can't be accomplished easily by autoencoders, which we covered in Chapter 3, Autoencoders.

GANs are able to learn how to model the input distribution by training two competing (and cooperating) networks referred to as generator and discriminator (sometimes known as critic). The role of the generator...

Principles of GANs

As shown in Figure 4.1.1 a GAN is analogous to a counterfeiter (generator) - police (discriminator) scenario. At the academy, the police are taught how to determine if a dollar bill is either genuine or fake. Samples of real dollar bills from the bank and fake money from the counterfeiter are used to train the police. However, from time to time, the counterfeiter will attempt to pretend that he printed real dollar bills. Initially, the police will not be fooled and will tell the counterfeiter why the money is fake. Taking into consideration this feedback, the counterfeiter hones his skills again and attempts to produce new fake dollar bills. As expected the police will be able to both spot the money as fake and justify why the dollar bills are fake.

Figure 4.1.1: The generator and discriminator of GANs are analogous to the counterfeiter and the police. The goal of the counterfeiter is to fool the police into believing that the dollar bill is real.

This scenario continues...

GAN implementation in Keras

In the previous section, we learned that the principles behind GANs are straightforward. We also learned how GANs could be implemented by familiar network layers such as CNNs and RNNs. What differentiates GANs from other networks is they are notoriously difficult to train. Something as simple as a minor change in the layers can drive the network to training instability.

In this section, we'll examine one of the early successful implementations of GANs using deep CNNs. It is called DCGAN [3].

Figure 4.2.1 shows DCGAN that is used to generate fake MNIST images. DCGAN recommends the following design principles:

• Use of strides > 1 convolution instead of MaxPooling2D or UpSampling2D. With strides > 1, the CNN learns how to resize the feature maps.

• Avoid using Dense layers. Use CNN in all layers. The Dense layer is utilized only as the first layer of the generator to accept the z-vector. The output of the Dense layer is resized and becomes the input of the succeeding...

Conditional GAN

In the previous section, the fake images generated by the DCGAN are random. There is no control over which specific digits will be produced by the generator. There is no mechanism for how to request a particular digit from the generator. This problem can be addressed by a variation of GAN called Conditional GAN (CGAN) [4].

Using the same GAN, a condition is imposed on both the generator and discriminator inputs. The condition is in the form of a one-hot vector version of the digit. This is associated with the image to produce (generator) or classified as real or fake (discriminator). The CGAN model is shown in Figure 4.3.1.

CGAN is similar to DCGAN except for the additional one-hot vector input. For the generator, the one-hot label is concatenated with the latent vector before the Dense layer. For the discriminator, a new Dense layer is added. The new layer is used to process the one-hot vector and reshape it so that it is suitable for concatenation to the other input of the...

Conclusion

This chapter discussed the general principles behind GANs, to give you a foundation to the more advanced topics we'll now move on to, including Improved GANs, Disentangled Representations GANs, and Cross-Doman GANs. We started this chapter by understanding how GANs are made up of two networks called generator and discriminator. The role of the discriminator is to discriminate between real and fake signals. The aim of the generator is to fool the discriminator. The generator is normally combined with the discriminator to form an adversarial network. It is through training the adversarial network that the generator learns how to produce fake signals that can trick the discriminator.

We also learned how GANs are easy to build but notoriously difficult to train. Two example implementations in Keras were presented. DCGAN demonstrated that it is possible to train GANs to generate fake images using deep CNNs. The fake images are MNIST digits. However, the DCGAN generator has no control...

References

1. Ian Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv preprint arXiv:1701.00160, 2016 (https://arxiv.org/pdf/1701.00160.pdf).

2. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv preprint arXiv:1511.06434, 2015 (https://arxiv.org/pdf/1511.06434.pdf).

3. Mehdi Mirza and Simon Osindero. Conditional Generative Adversarial Nets. arXiv preprint arXiv:1411.1784, 2014 (https://arxiv.org/pdf/1411.1784.pdf).

4. Tero Karras and others. Progressive Growing of GANs for Improved Quality, Stability, and Variation. ICLR, 2018 (https://arxiv.org/pdf/1710.10196.pdf).

References

1. Ian Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv preprint arXiv:1701.00160, 2016 (https://arxiv.org/pdf/1701.00160.pdf).
2. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv preprint arXiv:1511.06434, 2015 (https://arxiv.org/pdf/1511.06434.pdf).
3. Mehdi Mirza and Simon Osindero. Conditional Generative Adversarial Nets. arXiv preprint arXiv:1411.1784, 2014 (https://arxiv.org/pdf/1411.1784.pdf).
4. Tero Karras and others. Progressive Growing of GANs for Improved Quality, Stability, and Variation. ICLR, 2018 (https://arxiv.org/pdf/1710.10196.pdf).