Summary
We have definitely learned a lot in this chapter. We started by learning about the theory and loss functions of GANs, and how to translate the mathematical value function into the code implementation of binary cross-entropy loss. We implemented DCGAN with convolutional layers, batch normalization layers, and leaky ReLU to make the networks go deeper. However, there are still challenges in training GANs, which include instability and being prone to mode collapse due to Jensen-Shannon divergence.
Many of these problems were solved by WGAN with Wasserstein distance, weight clipping, and the removal of the sigmoid at the critic's output. Finally, WGAN-GP introduces gradient penalty to properly enforce the 1-Lipztschitz constraint and give us a framework for stable GAN training. We then replaced batch normalization with layer normalization to train on the CelebA dataset successfully to generate a good variety of faces.
This concludes part 1 of the book. Well done to...
