Fine-Tuning Large Language Models

Imagine you have finished building a language model that has already gone through the earlier work of this course. It can produce fluent text, but it has not been shaped for any particular job. Ask it to sort customer emails into categories, or to answer a specific question about a product, and it has no particular reason to do either task well. This article follows one practical question: once a pretrained model exists, how do you turn it into one that performs a specific job?

The course builds an LLM in stages, and the third stage is the one this article covers. In that stage, the course takes a pretrained LLM and fine-tunes it. Read that sentence as a sequence: the pretrained model comes first, and fine-tuning is applied to it afterward. Fine-tuning is the step that comes after pretraining, and it is where the model starts to be aimed at real work.

The diagram shows the order as a flow. Fine-tuning does not begin from an empty model. It begins from a pretrained one, so every fine-tuning step inherits whatever the pretrained model already knows how to do.

Notice the two halves of this definition. The first half names the starting object and the action performed on it. The second half names the purpose. A learner who keeps both halves in mind is less likely to confuse fine-tuning with building a model from nothing. Fine-tuning changes an existing model; it does not create one from an empty starting point.

Setting the two models side by side makes the difference concrete. Both are LLMs, and both are built on the same foundation, but they differ in where they begin and what they are for.

The table shows what changes between the two models. The starting point stays the same kind of object: a pretrained LLM. What changes is the adjustment applied to it and the purpose that adjustment serves.

The source names two example tasks for a fine-tuned model: answering queries and classifying texts. It explains why they matter: they are among the most common tasks in real-world applications and in research. Each task asks something different of the model. Answering a query asks the model to produce a response to a question. Classifying a text asks the model to assign an input to a category. Both are ways of following an instruction, which is the behaviour the fine-tuning stage aims to build.

To match a practical problem to a family, ask what the model must produce. If the output is a response to a question, the problem belongs under answering queries. If the output is a choice of category for an input, it belongs under classifying texts. The two examples in the diagram are generated illustrations of these families, not tasks taken from the source.

Once a classification model has been trained, the next question is whether it works. A classifier takes an input and assigns it to a category, such as spam or not spam, or a diagnosis label. Each assignment is either right or wrong, and the simplest way to summarize that behaviour is to count how often the model is right. This summary is called classification accuracy. Evaluating a classification model involves calculating the classification accuracy.

The formula is the definition written as a ratio. Count the predictions the model got right, divide by the total number of predictions it made, and the result is a value between 0 and 1. Multiply by 100 if you prefer a percentage. A result of 0.8 and a result of 80 percent describe the same accuracy.

Consider a model that sorts ten emails as spam or not spam. Each email has a true label and a predicted label. Accuracy depends on lining up each prediction with the true label for the same email, then counting the rows where the two agree. In the diagram, each node is one email, its sublabel gives the true and predicted labels, and the emails the model got wrong are marked as changed.

Accuracy is a single summary number, so it does not reveal which predictions were wrong or what kinds of errors the model made. In the email example, the calculation never asked which two emails were wrong. The number 80 percent would be the same whether the two errors were harmless messages or dangerous ones. That is the central limitation of accuracy, and it is why you should always ask what the errors were, not only how many there were.

Read a high value as evidence of frequent correct answers, not as proof that the model is good in every case that matters. Read a low value as evidence of frequent wrong answers, not as a diagnosis of what went wrong. In both cases the next step is to examine the predictions that were wrong.