3D-GAN - Generating Shapes Using GANs

A 3D-GAN is a GAN architecture for 3D shape generation. 3D shape generation is typically a complex problem, due to the complexities involved in processing 3D images. A 3D-GAN is a solution that can generate realistic and varied 3D shapes and was introduced by Jiajun Wu, Chengkai Zhang, Tianfan Xue, and others in the paper titled Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. This paper is available at http://3dgan.csail.mit.edu/papers/3dgan_nips.pdf. In this chapter, we will implement a 3D-GAN using the Keras framework.

We will cover the following topics:

• Introduction to the basics of 3D-GANs
• Setting up the project
• Preparing the data
• A Keras implementation of a 3D-GAN
• Training the 3D-GAN
• Hyperparameter optimization
• Practical applications of 3D-GANs

...

Introduction to 3D-GANs

3D Generative Adversarial Networks (3D-GANs) is a variant of GANs, just like StackGANs, CycleGANs, and Super-Resolution Generative Adversarial Networks (SRGANs). Similar to a vanilla GAN, it has a generator and a discriminator model. Both of the networks use 3D convolutional layers, instead of using 2D convolutions. If provided with enough data, it can learn to generate 3D shapes with good visual quality.

Let's understand 3D convolutions before looking closer at the 3D-GAN network.

3D convolutions

In short, 3D convolution operations apply a 3D filter to the input data along the three directions, which are x, y, and z. This operation creates a stacked list of 3D feature maps. The shape of the output...

Setting up a project

The source code for the project is available on GitHub at the following link: https://github.com/PacktPublishing/Generative-Adversarial-Networks-Projects.

Run the following commands to set up the project:

1. Start by navigating to the parent directory, as follows:

cd Generative-Adversarial-Networks-Projects

2. Next, change directory from the current directory to the Chapter02 directory:

cd Chapter02

3. Next, create a Python virtual environment for this project:

virtualenv venv

4. After that, activate the virtual environment:

source venv/bin/activate

5. Finally, install all the requirements that are indicated in the requirements.txt file:

pip install -r requirements.txt

We have now successfully set up the project. For more information, refer to the README.md file included with the code repository.

Preparing the data

In this chapter, we will use the 3D ShapeNets dataset, available at http://3dshapenets.cs.princeton.edu/3DShapeNetsCode.zip. It was released by Wu and Song et al. and consists of properly annotated 3D shapes for 40 object categories. We will use the volumetric data available in the directory, which we will discuss in more detail later on in this chapter. In the next few sections, we will download, extract, and explore the dataset.

The 3D ShapeNets dataset is for academic use only. If you intend to use the dataset for commercial purposes, request permission from the authors of the paper, who can be reached at the following email address: [e-mail].

Download and extract the dataset

Run the following...

A Keras implementation of a 3D-GAN

In this section, we will implement the generator network and the discriminator network in the Keras framework. We need to create two Keras models. Both of the networks will have their own separate weights values. Let's start with the generator network.

The generator network

To implement the generator network, we need to create a Keras model and add the neural network layers. The steps required to implement the generator network are as follows:

1. Start by specifying the values for different hyperparameters:

z_size = 200
gen_filters = [512, 256, 128, 64, 1]
gen_kernel_sizes = [4, 4, 4, 4, 4]
gen_strides = [1, 2, 2, 2, 2]
gen_input_shape = (1, 1, 1, z_size)
gen_activations = ['relu&apos...

Training a 3D-GAN

Training a 3D-GAN is similar to training a vanilla GAN. We first train the discriminator network on both the generated images and the real images but freeze the generator network. Then, we train the generator network but freeze the discriminator network. We repeat this process for a specified number of epochs. During one iteration, we train both of the networks in a sequence. Training a 3D-GAN is an end-to-end training process. Let's work on these steps one by one.

Training the networks

To train the 3D-GAN, perform the following steps:

1. Start by specifying the values for the different hyperparameters required for the training, shown as follows:

gen_learning_rate = 0.0025
dis_learning_rate = 0.00001
beta...

Hyperparameter optimization

The model that we trained might not be a perfect model, but we can optimize the hyperparameters to improve it. There are many hyperparameters in a 3D-GAN that can be optimized. These include the following:

• Batch size: Experiment with values of 8, 16, 32, 54, or 128 for the batch size.
• The number of epochs: Experiment with 100 epochs and gradually increase it to 1,000-5,000.
• Learning rate: This is the most important hyperparameter. Experiment with 0.1, 0.001, 0.0001, and other small learning rates.
• Activation functions in different layers of the generator and the discriminator network: Experiment with sigmoid, tanh, ReLU, LeakyReLU, ELU, SeLU, and other activation functions.
• The optimization algorithm: Experiment with Adam, SGD, Adadelta, RMSProp, and other optimizers available in the Keras framework.
• Loss functions: Binary cross entropy is the loss...

Practical applications of 3D-GANs

3D-GANs can potentially be used in a wide variety of industries, as follows:

• Manufacturing: 3D-GANs can be a creative tool to help create prototypes quickly. They can come up with creative ideas and can help in simulating and visualizing 3D models.
• 3D printing: 3D images generated by 3D-GANs can be used to print objects in 3D printing. The manual process of creating 3D models is very lengthy.

• Design processes: 3D generated models can provide a good estimate of the eventual outcome of a particular process. They can show us what is going to get built.
• New samples: Similar to other GANs, 3D-GANs can generate images to train a supervised model.

Summary

In this chapter, we have explored 3D-GANs. We started with an introduction to a 3D-GAN and covered the architecture and the configurations of the generator and the discriminator. Then, we went through the different steps required to set up a project. We also looked at how to prepare the dataset. Finally, we implemented a 3D-GAN in the Keras framework and trained it on our dataset. We also explored different hyperparameter options. We concluded the chapter by exploring the practical applications of 3D-GANs.

In the next chapter, we will learn how to perform face aging using Conditional Generative Adversarial Networks (cGANs).