Alternative gpt2 weight download

[rasbt]

In case the official weights from OpenAI ever become unavailable, I wanted to share a few tips for working around the issue.

 

1) Connection checks

It could be a temporary internet connection issue or a problem with your IP address. I recommend double-checking using a different machine (e.g., Google Colab).

For your convenience, use the following code to download the download script and the GPT-2 weights:

# Download the download file

import urllib.request
url = (
    "https://raw.githubusercontent.com/rasbt/"
    "LLMs-from-scratch/main/ch05/"
    "01_main-chapter-code/gpt_download.py"
)
filename = url.split("/")[-1]
urllib.request.urlretrieve(url, filename)

from gpt_download import download_and_load_gpt2

settings, params = download_and_load_gpt2(model_size="124M", models_dir="gpt2")
# settings, params = download_and_load_gpt2(model_size="355M", models_dir="gpt2")
# settings, params = download_and_load_gpt2(model_size="774M", models_dir="gpt2")
# settings, params = download_and_load_gpt2(model_size="1558M", models_dir="gpt2")

 

2) Alternative download

In case the above doesn't work, you can download the weights from this alternative link here.

Click on "Download" to download the respective weights.

Then,

1. Create a folder called gpt2 in the location where you are planning to execute the code for loading the weights
2. Move the downloaded zip file into this gpt2 folder
3. Load the weights using the code shared in section 1 above

 

3) Alternative weight loading

In case you are still having problems, for example, because of a TensorFlow incompatibility (the original weights were stored as TensorFlow tensors. Hence, we need TensorFlow for the weight conversion to PyTorch), consider this alternative way of loading the weights from the Hugging Face Hub.

[jms-dipadua]

thank you for providing this alternative method. the option within the book wasn't working but Option 1 above did the trick :)

[rasbt]

glad it helped!