Pre-Trained Language Models

Imagine you want a program that can write fluent sentences. You could try to write rules for grammar and word choice by hand, but language is too varied for that to work. A different approach is to let a model learn patterns from a large body of text and then use what it learned to generate new text. The process of building that learned starting point is called pretraining. It is the stage where a model is first trained on text before anyone asks it to do a specific job.

A pre-trained language model already knows a great deal about language, but it has not yet learned to follow the kind of instructions you care about. Fine-tuning closes that gap by adjusting the pre-trained model so that it fits a specific task or dataset. In this article the task is concrete: the model is trained on instruction data until it can convert an active sentence into its passive voice counterpart. The article follows one path, from the foundations that come before pretraining, through the cost decision of training from scratch versus loading open weights, to the fine-tuning call and the way you judge its result.

In this course, pretraining is treated as the second major stage of work. The first stage prepares the ingredients: it covers the data preprocessing steps and the attention mechanism that sits at the core of modern LLMs. The second stage puts those ingredients together, showing how to code and pretrain a GPT-like model that can generate new text, and then how to evaluate the result. Keeping these stages in mind helps you see why each topic exists.

Read the diagram from left to right. The first two steps belong to stage 1 and are about preparing data and understanding attention. The pretraining step belongs to stage 2. The branch for loading open model weights sits beside pretraining because it is an alternative way to reach a trained model without running the full training yourself. Both routes end at evaluation, because a model needs to be measured whichever way it was obtained.

Stage 1 supplies two foundations that pretraining depends on. Each is worth understanding before you move on to the model itself, because the later steps assume they are already in place.

Pretraining an LLM from scratch is a significant undertaking. The source states that it demands thousands to millions of dollars in computing costs for GPT-like models. That range is wide, and the bottom and top ends differ enormously, so the practical lesson is that full-scale pretraining is a major commitment rather than a routine exercise.

This cost is the reason the course narrows its goal. Instead of pretraining at full scale, stage 2 focuses on implementing training for educational purposes with a small dataset. Loading openly available model weights is the other route. The table compares the two options so you can see what each one is for.

By the time training starts, the model and its data loaders are already prepared. What remains is a single training call that wires everything together. Because that call is where fine-tuning actually happens, it is the best place to begin. Treat it as a checklist of decisions: which optimizer updates the weights, how many passes over the data to make, how often to check progress, and what to ask the model to generate along the way.

Start by reading the optimizer line. It creates an AdamW optimizer over the model's parameters, which means the optimizer is the component that will update the model during training. Two of its settings are stated explicitly: the learning rate is 0.00005 and the weight decay is 0.1. Those two numbers, together with the optimizer choice, make up the optimizer configuration for this fine-tuning run.

The remaining arguments decide the shape of the run. The number of epochs is set to 2. The evaluation frequency is 5, and the evaluation iteration count is also 5, so the run checks its progress at regular intervals rather than only at the end. The call also receives a starting context, built by formatting the first validation instruction, which the model uses to generate a response while training is underway. The tokenizer is passed in so that text can be encoded and decoded during this process.

The call also returns three things you will read afterward: a list of training losses, a list of validation losses, and a count of tokens seen. The script records the start and end time around the call and reports the elapsed minutes. Timing does not change what the model learns, but it is useful when you compare runs.

Two kinds of loss are tracked. The training loss is computed on the data the model is learning from. The validation loss is computed on held-out instruction examples. Both matter. A model whose training loss falls while its validation loss does not is not doing what you want, so the two must be read together.

Before you look at the output of any run, decide what a healthy result would look like. You have the configuration in front of you: two epochs, evaluation every five iterations, and two loss curves that the run will produce. Commit to an answer first, because the reasoning you do now is what makes the output meaningful when it arrives.

Once the run has finished, the losses are your main evidence. The source treats a steady decrease as the signal of improvement. A single lower number at one checkpoint is weaker evidence than a sequence of values that keeps falling as training continues. Look at the whole run, and then check the responses.

The training call sends a starting context built from the first validation instruction, and the responses generated at the end of each epoch let you see the model's behavior directly. This is the most readable check in the whole run, because you can compare the model's answer against what the task asked for. For this task, that means asking whether the answer is the passive voice counterpart of the active sentence.

The configuration uses exactly two epochs, and the source explains why that choice is reasonable. Continuing beyond that point is not essential and may even hurt, since it can lead to overfitting, where the model fits the training examples so closely that it generalizes less well. Decide epoch counts from the results rather than by habit.

Sometimes a run departs from expectation. The useful habit is to check in a fixed order: the configuration first, then the metrics, then the generated outputs. Each check rules out a stage. If the configuration does not match what you intended, the problem is in the setup. If the configuration is right but the losses do not behave as described, the problem is in the training itself. If the losses look right but the responses are wrong, the problem is in what the model has learned to produce.

The order matters because each check narrows where the fault lies. Reading the outputs first, before the configuration and metrics, tells you that something is wrong but not which stage caused it. Working in order turns a vague sense that the run failed into a specific stage you can correct.

The third topic in stage 2 is evaluating LLMs. The source describes this as essential for developing capable NLP systems. A trained model is a set of learned values, and those values alone do not tell you whether the model produces good text. Evaluation is the step that measures what the model does, so that you can compare versions and decide what to change.

Evaluation applies to both routes. A model you pretrain on a small dataset needs to be evaluated, and so does a model whose weights you loaded from an open source. Because the weights in the second case came from elsewhere, you cannot assume they behave as you expect in your own setting. Measuring is how you find out.