Step 2: Cloning the OpenAI GPT-2 repository

OpenAI still lets us download GPT-2 for now. This may be discontinued in the future, or maybe we will get access to more resources. At this point, the evolution of transformers and their usage moves so fast that nobody can foresee how the market will evolve, even the major research labs themselves.

We will clone OpenAI’s GitHub directory on our VM:

#@title Step 2: Cloning the OpenAI GPT-2 Repository
!git clone https://github.com/openai/gpt-2.git

When the cloning is over, you should see the repository appear in the file manager:

Figure III.2: Cloned GPT-2 repository

Click on src, and you will see that the Python files we need from OpenAI to run our model are installed:

Figure III.3: The GPT-2 Python files to run a model

You can see that we do not have the Python training files we need. We will install them when we train the GPT-2 model in the Training a GPT-2 language model section of Appendix...