Steps 7b-8: Importing and defining the model

We will now activate the interaction with the model with interactive_conditional_samples.py.

We need to import three modules that are also in /content/gpt-2/src:

import model, sample, encoder

The three programs are:

• model.py defines the model’s structure: the hyperparameters, the multi-attention tf.matmul operations, the activation functions, and all the other properties.
• sample.py processes the interaction and controls the sample that will be generated. It makes sure that the tokens are more meaningful.

  Softmax values can sometimes be blurry, like looking at an image in low definition. sample.py contains a variable named temperature that will make the values sharper, increasing the higher probabilities and softening the lower ones.

  sample.py can activate Top-k sampling. Top-k sampling sorts the probability distribution of a predicted sequence. The higher probability values of the head of...