Training Loops and Model Evaluation

By the time you reach this point in building a language model, a great deal of groundwork is already in place. The architecture has been written, a pretrained model has been produced, and pretrained weights from an external source have been loaded into that architecture. Yet a general-purpose language model is not aimed at any particular job. It has learned broadly, but nothing in it yet says, for example, that your task is sorting text messages into spam and not spam.

The step that points the model at a job is fine-tuning: continuing to train the model on a dataset that represents the specific task you care about. The running task in this material is classifying text messages as 'spam' or 'not spam'. Keep that single task in mind as you read. At each step, ask two questions: what should the model output for a given message, and how would anyone know whether it got the answer right?

Building a working LLM application is easier to follow as a sequence of stages. The first stage covers constructing the model. The second covers a general pretrained model and its later adaptation toward assistant-style use. The third stage is where a pretrained model is adapted into a classifier. This article concentrates on that third stage.

Imagine a small collection of text messages. A person has marked each one as spam or not spam. Fine-tuning for classification means training the pretrained model on these pairs of message and label. Once trained, when the model reads a new message it produces the label it has learned to associate with messages like it.

Training works as a loop over the labeled examples. In each pass, the model reads a message and produces a prediction. That prediction is compared with the true label, and the mismatch is used to adjust the model's weights. The cycle then repeats across the examples. Each repetition nudges the weights and the predictions, which is why one pass through the data is rarely the end of training. The diagram below shows a single pass and how the cycle returns to the start.

The source names two main ways of fine-tuning an LLM, labeled step 8 and step 9. They begin from different kinds of data, and that difference determines which one applies. Fine-tuning for classification uses a dataset in which each example already carries a class label. Fine-tuning to follow instructions uses an instruction dataset, in which the model is shown a request and the response it should give.

Once fine-tuning finishes, the model is supposed to be a classifier, but that is a claim that needs checking. Evaluating the accuracy of a fine-tuned LLM classifier means measuring how often its predicted labels match the true labels on examples it did not learn from. The question is simple to state: when the classifier is given messages whose labels are known, how often does its predicted label match the true label?

In its simplest form, accuracy is the share of examples the classifier labels correctly. A classifier that gets most spam messages and most legitimate messages right has high accuracy. A classifier that often confuses the two groups has low accuracy. The number is only as trustworthy as the examples used to compute it, so the choice of evaluation messages matters as much as the calculation itself.