Evaluating Fine-Tuned Classifiers

When a team decides to adapt a large language model, the first question is rarely which training recipe to use. The more useful question is what the adapted model must do. Some projects need one model that can take many different kinds of requests and respond sensibly to each. Other projects need a model that sorts incoming text into a fixed set of labels and nothing else. The two fine-tuning approaches covered here are built around those two goals, and the choice between them shapes the data you collect, the hardware you need, and the evaluation you run afterward.

The two approaches sit on opposite sides of a trade-off. Instruction fine-tuning is the more versatile of the two. It aims at flexibility and interaction quality across many complex user instructions, but it demands larger datasets and greater computational resources in order to produce a model that is proficient across many tasks. Classification fine-tuning asks less of both data and compute. The cost of that economy is scope.

Read the diagram as two columns that answer the same four questions. The left column describes a broad, general-purpose goal that is paid for with large data and heavy compute. The right column describes a narrow goal that is cheaper to reach but cannot leave its predefined classes. Neither column is better in general. The right choice depends on whether the project needs one model that does many things or a model that does one sorting job well.

Classification fine-tuning suits precise categorization into predefined classes, such as sentiment analysis or spam detection. That is the setting this article follows. The running task is classifying text messages as 'spam' or 'not spam'.

By the time fine-tuning begins, a lot of groundwork is already in place. The architecture has been written, a pretrained model has been produced, and pretrained weights from an external source have been loaded into that architecture. A general-purpose language model, however, is not yet aimed at any particular job. The step that points it at a job is fine-tuning: continuing to train the model on a dataset that represents the specific task you care about.

The difference between the two fine-tuning approaches comes down to the training data. The source names two main ways of fine-tuning an LLM, labeled step 8 and step 9. Fine-tuning for classification (step 8) uses a dataset in which each example already carries a class label. Fine-tuning to follow instructions (step 9) uses an instruction dataset, where the model is shown a request and the response it should give. The two categories are distinct, and the training data is what determines which one applies.

Imagine a small labeled collection of text messages. Each message has been marked by a person as spam or not spam. Fine-tuning for classification means training the pretrained model on pairs of message and label. Follow one pair through the process. The message goes in as input, the label is the answer the model should produce, and a difference between what the model produced and that label is what the training adjusts.

Once classification fine-tuning has been chosen, the work proceeds in three stages. Stage 1 is dataset preparation. Stage 2 is model setup. Stage 3 covers fine-tuning the model and evaluating it. The diagram lays out the stages as a pipeline, so you can see what each one produces for the next.

The stages are sequential, and each one hands a product to the next. Stage 1 turns raw material into data the model can consume. Stage 2 turns a general model into one that is ready to be trained on that data. Stage 3 runs the training, checks the result, and then puts the finished classifier to use. Because the stages depend on one another, a problem found during evaluation usually sends you back to revisit earlier choices rather than being patched at the end.

Dataset preparation begins with downloading the dataset, followed by inspecting and preprocessing it before creating data loaders. The text message setting makes the goal intuitive. Each message needs one of two labels, and the dataset contains both spam and non-spam messages. Inspection is where you confirm what those messages actually look like before any training begins. Preprocessing is where the raw text is put into a consistent form the model can read. Data loaders are what feed the prepared examples to the model in batches during training.

Stage 2 modifies the model so it suits fine-tuning. The purpose is to make the model's output match the task. A general language model produces a set of scores over a large vocabulary, which is a broad answer to the question what comes next. A classifier needs something narrower. It needs a score for each class label it was trained on, so that the label with the highest score becomes the prediction.

Read the diagram as a change in what the output means, not only in its size. Before modification, each score answers a question about which token could come next. After modification, each score answers a question about which class the message belongs to. Once the output is framed this way, the fine-tuning and evaluation stages can judge the model against the labels in the dataset.

Once fine-tuning finishes, the model is supposed to be a classifier, but that is a claim that needs checking. The source treats evaluating the accuracy of a fine-tuned LLM classifier as an important step in its own right. The question evaluation answers is simple to state. When the classifier is given messages with known labels, how often does its predicted label match the true label? Evaluation measures that agreement on examples the model did not learn from.

In its simplest form, accuracy is the share of examples the classifier labels correctly. A classifier that gets most spam messages and most legitimate messages right has high accuracy. A classifier that often confuses the two groups has low accuracy. The number alone is only as trustworthy as the examples used to compute it, which is why the choice of evaluation messages matters as much as the calculation itself.

After fine-tuning and evaluation, the classifier can be used on data it has never seen. In practice, a new message is passed to the model, and the model returns one of the labels it was trained on. Nothing about this step retrains the model. It only reads the message and reports a label. Trace a new message through the process. It enters as raw text, the classifier reads it, and the output is one of the two labels, spam or not spam.

Reasoning about the result is the final part of this step. A single label on a single message is a prediction, not a guarantee. Its reliability is judged by the held-out evaluation described earlier. If the classifier's accuracy on labeled, held-out examples is high, a label it returns for a new message is more trustworthy. If accuracy was low, the label should be treated with caution, and the earlier stages should be revisited.