Step 5: Downloading the 345M-parameter GPT-2 model

We will now download the trained 345M-parameter GPT-2 model:

#@title Step 5: Downloading the 345M parameter GPT-2 Model
# run code and send argument
import os # after runtime is restarted
os.chdir("/content/gpt-2")
!python3 download_model.py '345M'

The path to the model directory is:

/content/gpt-2/models/345M

It contains the information we need to run the model:

Figure III.4: The GPT-2 Python files of the 345M-parameter model

The hparams.json file contains the definition of the GPT-2 model:

• "n_vocab": 50257, the size of the vocabulary of the model
• "n_ctx": 1024, the context size
• "n_embd": 1024, the embedding size
• "n_head": 16, the number of heads
• "n_layer": 24, the number of layers

encoder.json and vacab.bpe contain the tokenized vocabulary and the BPE word pairs. If necessary, take a few...