Experimental setup

In this section, we will start by implementing methods that are necessary for preparing and loading our data during training. Next, we will learn to implement auxiliary functions that are required for the Wasserstein GAN with Gradient Penalty. Last, we are going to write code to set up training and the training loop itself.

Data

We will focus on the 1-Billion Word dataset that was proposed in 2013 in the paper One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling by Ciprian Chelba et al. This dataset can be downloaded from http://www.statmt.org/lm-benchmark/.

We will start with the imports, which include the libraries and functions in this subsection:

from collections import...