Pre-Trained Models

This article works in the opposite order from most explanations. Instead of starting with fine-tuning in the abstract, we start with a short piece of code that prepares text for a pre-trained GPT model and checks what that preparation produces. Once you can read that output, the idea of changing only some layers of the model becomes much easier to follow.

The starting point is a model that was already built and loaded in an earlier step. A new output layer has been added to it, and certain layers have been marked as trainable or nontrainable. Even after those changes, the model can still be used the same way as before: you feed it a text example and it processes that text. So the first question is simply what the text looks like after it has been prepared for the model.

The conversion from text to model input happens in two steps. First, the tokenizer turns the text into a sequence of integer token IDs. Second, torch.tensor wraps that sequence in a tensor, and unsqueeze(0) inserts a new dimension at the front. The print statements in the snippet report both the values and the dimensions, so you can check each step separately.

Keeping these two steps apart matters when something goes wrong. The tokenizer decides how many tokens the text produces. The tensor conversion decides how those tokens are arranged in dimensions. A problem in either step will show up in the shape, so the shape is the first thing to inspect.

The shape is written as (batch_size, num_tokens). The first dimension counts how many input sequences are processed together. The second dimension counts the tokens in each sequence. Reading the shape this way tells you immediately whether the text was encoded the way you intended.

Fine-tuning adjusts a pre-trained model's weights to fit a specific task. The model arrives with weights that were learned earlier, and fine-tuning continues the training so that those weights move toward values that suit the new task. Fine-tuning does not have to touch every part of the model, and the choice of which parts to adjust is the core decision in the process.

Why can this improve predictive performance? The source describes fine-tuning the entire model as an option that can improve predictive performance, after which you assess how much that change helps. Keeping the pre-trained layers nontrainable is a way of preserving what the model already learned, while letting the trainable layers adapt that knowledge toward the new task. The pre-trained structure is the general starting point, and the trainable part is where task-specific adaptation happens.

There are two options the source describes. In full fine-tuning, the entire model is fine-tuned, so every layer's weights can be adjusted. In partial fine-tuning, only some layers are made trainable while the rest are kept nontrainable. The table below compares which parameters receive updates in each case.

In the GPT example, the model contains 12 repeated transformer blocks. The new output layer is trainable, and so are the final LayerNorm and the last transformer block. The remaining 11 transformer blocks and the embedding layers are kept nontrainable. This means most of the pre-trained structure is preserved, and only the parts closest to the output are adapted to the new task.

A useful way to read the diagram is to ask which parts of the model sit closest to the output. Those parts are the only ones the source marks as trainable, and that is where the task-specific adaptation takes place. Everything nearer the input keeps the weights it was loaded with.

When a fine-tuning run does not behave as expected, the cause usually sits at one of three points: the encoded input, the trainable-layer selection, or the way the model is then used. Checking those points in order narrows the failure quickly. The flowchart below places those checkpoints inside the workflow.

Work through the checkpoints in order. If the encoded input shape is wrong, the problem is already present before the trainable layers matter, so there is no point inspecting the layer selection first. Only after the input shape checks out should you move to the trainable-layer selection, and only then to how the model is used.