Loading GPT Model Weights

Loading pretrained weights into a GPTModel is a silent operation. The call either finishes or raises an error, but nothing on the screen tells you whether the model now behaves the way the trained weights should make it behave. Before you fine-tune anything, you need a quick way to confirm that the loaded model is in a usable state.

The check used here is practical. After the weights are loaded, the text generation utility from chapters 4 and 5 is reused to ask the model to continue a short prompt. If the continuation reads as coherent text, that is taken as evidence that the weights were loaded correctly. The test is a summary of the whole loading step: it does not inspect the weights directly, it only observes the behavior they produce.

The test is a round trip. The prompt starts as a string. It is converted into token IDs by text_to_token_ids, and those IDs are passed to generate_text_simple together with the model, a max_new_tokens limit of 15, and a context_size taken from the configuration. The function returns a new list of token IDs, and token_ids_to_text converts that list back into a string that can be printed.

Notice that the model never sees the prompt as text. It sees a list of integers. The text is converted into IDs on the way in, the model extends the IDs, and the IDs are converted back into text on the way out. Each stage has its own inputs and outputs, so you can reason about each one independently.

The result you check is the printed text. A continuation is judged coherent when the added words follow naturally from the prompt and form grammatical text. That judgment is the entire basis of the test, so it is worth practicing on a real output before you rely on it.

When the printed text is not coherent, the round trip gives you a way to narrow the search. The test does not name the faulty stage for you, but because it has distinct stages, you can ask a sequence of questions. Each answer rules out part of the pipeline. Work through the questions in order, as the flowchart shows.

Before any GPT-2 weights can be loaded, the files must be on your own machine. The function urllib.request.urlretrieve takes two arguments. The first is the web address of the file to fetch. The second is the name the file should have once it is saved locally. The call fetches the contents at the address and writes them to a file in the current working directory of your Python session.

In the GPT-2 download code, the filename is not hard-coded. The code takes the URL, splits it at each forward slash, and keeps the last piece. Because the last piece of a file address is normally the file's own name, the local file ends up with the same name it has on the server.

Inspecting the downloaded file matters because a download can appear to finish without the file being saved correctly. The course recommends checking that the file exists and contains valid Python code. A quick way to do this is to look at the first line of the saved file.

Once the helper module is saved locally, its download_and_load_gpt2 function can be imported like any other function. Calling it with model_size and models_dir downloads the model files and loads two things into your session: the GPT-2 architecture settings, held in settings, and the weight parameters, held in params.

Running the loader call downloads seven separate files associated with the 124M parameter GPT-2 model. Each one is needed for the model to be reconstructed, so it is useful to recognize them by name. The progress lines printed during the call show each file being fetched along with its size and transfer rate. Seeing one line per file is a useful check that all seven files are arriving.

The download code is mostly boilerplate, so when it fails the cause is usually outside your own logic. The course notes list three likely causes. The first is an intermittent internet connection. The second is a problem on the server that hosts the files. The third is a change in how the weights of the open-source GPT-2 model are shared.