CycleGAN is one of the most well-known architectures in the GAN community for good reason. It doesn't require paired training data to produce stunning style transfer results. As you'll see in this chapter, we're going to go over the basic structure of the model and the results you can expect when you use it.
This recipe will focus on dissecting the internal pieces of the CycleGAN paper (https://arxiv.org/pdf/1703.10593.pdf)—the structure they propose, simple tips they suggest throughout their development, and any potential metrics that we may want to use in our development for this chapter.
For this recipe, you will simply need to create a folder for this chapter's code in your home directory. As a reminder, ensure that you've completed all of the prerequisite installation steps such as installing Docker, Nvidia-Docker, and the Nvidia drivers. Last, grab the CycleGAN paper (https://arxiv.org/pdf/1703.10593.pdf) and make sure to read it before you go on to the next section.
You'll get tired of hearing how important data is to us—but honestly, it can make or break your development. In our case, we are going to simply use the same datasets that the original CycleGAN authors used in their development. This has two use cases: we can compare our results to theirs and we can take advantage of their small curated datasets.
So far, we've focused on just reviewing the structure of how we will solve the problem. As with every one of these chapters, we need to spend a few minutes collecting training data for our experiments. Replicate the directory structure with files, as seen as follows:
├── data │ ├── ├── docker │ ├── build.sh │ ├── clean.sh │ └── Dockerfile ├── README.md ├── run.sh ├── scripts │ └── create_data.sh ├── src │ ├──
We'll go and introduce the files you'll need to build so you can have a development environment and data to work with on CycleGAN.
It might seem obvious by now but each of the generators we've built until this point has been an incremental improvement on the last GAN to Deep Convolutional Generative Adversarial Network (DCGAN) to CycleGAN will represent a similar incremental change in the generator code. In this case, we'll downsample for a few blocks then upsample. We'll also introduce a new layer called InstanceNormalization that the authors used to enforce better training for style transfer.
Every recipe is going to demonstrate the structure that you should have in your directory. This ensures that you've got the right files at each step of the way:
├── data │ ├── ├── docker │ ├── build.sh │ ├── clean.sh │ └── Dockerfile ├── README.md ├── run.sh ├── scripts │ └── create_data.sh ├── src │ ├── generator.py
Discriminators are the bread and butter of the discriminative modeling world—it's funny that we use them in such a unique way. Each discriminator that're designed is built to understand the difference between real and fake data but not too well. Why? If the discriminator could always tell the difference between the two types of data then the generator would never improve consistently. The next discriminator, based on the CycleGAN paper, will use a structure heavily based on their original implementation.
Your directory structure should look like the following tree:
├── data │ ├── ├── docker │ ├── build.sh │ ├── clean.sh │ └── Dockerfile ├── README.md ├── run.sh ├── scripts │ └── create_data.sh ├── src │ ├── discriminator.py │ ├── generator.py
Building the GAN is a core step with every one of these architectures—we have to be somewhat careful with CycleGAN because it's one of the first times we are going to develop a multilevel model. The GAN model will have six models in adversarial training mode—let's build it!
Every recipe is going to demonstrate the structure that you should have in your directory. This ensures that you've got the right files at each step of the way:
├── data │ ├── ├── docker │ ├── build.sh │ ├── clean.sh │ └── Dockerfile ├── README.md ├── run.sh ├── scripts │ └── create_data.sh ├── src │ ├── generator.py │ ├── discriminator.py │ ├── gan.py
The code is quite simple but the power of Keras really shines here—we are able to place six separate models into adversarial training in under 50 lines of code.
These are the steps for this:
- Make sure to get your imports for the implementation phase of the code:
#!/usr/bin/env python3 import sys import numpy...
Here we are again—our ole friend training. Training for CycleGAN has its own idiosyncratic components but you'll notice quite a bit of similarities with our previous chapters. You should be on the lookout for additional training steps—because we are training multiple generators and discriminators, we are increasing the time per batch and consequently per epoch significantly. The only advantage is that our batch in this base is only a single image.
Your directory should match the following tree—if you don't have the Python files beneath src, simply make sure to add the blank files for run.py and train.py and we will fill in the code throughout this recipe:
├── data │ ├── ├── docker │ ├── build.sh │ ├── clean.sh │ └── Dockerfile ├── README.md ├── run.sh ├── scripts │ └── create_data.sh ├── src │ ├── discriminator.py │ ├── gan.py │ ├── generator.py │ ├── run.py │ ├── save_to_npy.py │ └── train.py
Training can be broken into a few key components...
© 2018 Packt Publishing Limited All Rights ReservedAuthor (1)
