Classification Fine-tuning

Pretraining contributes broad language understanding. It is learned from large, varied text using next-word prediction as the objective. Fine-tuning then adds task-specific behavior on top of that foundation.

Programming

Why a General Model Needs More Training

Imagine a model that has read a very large amount of varied text. It has absorbed a broad sense of how language works, but it was not built for any one job. You might want it to sort your inbox into spam and not spam, or to follow instructions and answer in a particular format. A model that only learned general language does not automatically do these specific things well. The step that bridges that gap is called fine-tuning, and it builds on a stage that comes before it.

What do you think happens?

What would you change about a general-purpose model if you wanted it to do one specific job reliably?

Reveal answer

Answer:

You would train it further on a narrower, labeled dataset that matches the specific task, starting from the model's existing broad language knowledge.

Fine-tuning does not start from nothing. It takes a model that already understands language in general and adapts it using focused, labeled examples. The sections below trace how that happens step by step.

Following One Model Through Two Stages

The general process of creating an LLM has two stages. The first is pretraining, and the second is fine-tuning. The word 'pre' in pretraining points to the fact that it comes first. In the first stage, a model is trained on a large and diverse dataset so that it develops a broad understanding of language. The result is a pretrained model, which then serves as a foundation for the second stage.

What happens to the model after pretraining on a large, diverse corpus, and how does the pretrained model flow into the fine-tuning stage?

Read the diagram from left to right. The pretrained model is not thrown away after the first stage. It is the starting point for the second. Fine-tuning does not begin from nothing; it begins from a model that already has broad language knowledge and then adapts that model using a more focused dataset.

Pretraining as Next-Word Prediction

Pretraining is described in terms of a training objective. The LLM is trained on large text datasets with the goal of predicting the next word in a text. Learning to predict the next word over a huge and varied body of text is what builds the broad understanding of language that the pretrained model carries forward.

Key point

What Fine-tuning Adds to the Data

The key contrast between the two stages is in the kind of data. Pretraining works with large text datasets, and its objective is next-word prediction. Fine-tuning works with labeled data, meaning examples that come with the information the model should learn to produce or recognize for a specific purpose. Fine-tuning trains the pretrained model further on a narrower, labeled dataset specific to particular tasks or domains.

Comparing the Two Labeled Datasets

How does the labeled dataset look different for instruction fine-tuning, which uses prompt-response pairs, compared with classification fine-tuning, which uses text with a single class label?

Notice that the two categories do not differ in how much data they use or in the stage they occur in. They differ in the shape of each labeled example. An instruction example has a request and a response. A classification example has a text and one class label attached to it.

Choosing the Right Category for a Dataset

When you have a labeled dataset for a job, the first question is what each example looks like. The flowchart below turns that question into a decision path.

Given a dataset's label format and the task goal, which fine-tuning category applies?

Worked example

Classifying an inbox dataset

You have a dataset of emails. Each email is paired with one label: spam or not spam. Which fine-tuning category fits, and why?

1
Identify the example shape
Each example is a piece of text (an email) with one associated label (spam or not spam).
2
Match the shape to a category
Texts with associated class labels are the defining form of classification fine-tuning.
3
Rule out the other category
The examples are not instruction and answer pairs, so instruction fine-tuning does not match this dataset.
4
Confirm the task goal
The goal is to sort new emails into one of a few classes, which is the job classification fine-tuning prepares the model for.

Result

Classification fine-tuning fits this dataset. The labeled examples are texts with class labels, and the task is to assign one class to each text.

Mistakes Beginners Make with the Two Categories

Common mistake
Believing fine-tuning replaces pretraining
What it looks like
Starting a new model from random weights and calling it fine-tuning.
Why it fails
Fine-tuning begins from the pretrained model. The pretrained model is the starting point, not something discarded.
Do this instead
Treat pretraining as the foundation and fine-tuning as the refinement that follows it.
Common mistake
Using pretraining data for fine-tuning
What it looks like
Feeding a large pile of unlabeled web text into a classification fine-tuning run.
Why it fails
Pretraining uses large text datasets with next-word prediction. Fine-tuning uses labeled data tied to a specific task.
Do this instead
Prepare a narrower labeled dataset that matches the job before fine-tuning.
Common mistake
Choosing a category by topic instead of label format
What it looks like
Picking classification because the emails are about money, even though each example is an instruction and answer pair.
Why it fails
The category is determined by what each labeled example looks like, not by the subject matter of the text.
Do this instead
Inspect the shape of each example first: text with one class label points to classification, instruction and answer pairs point to instruction fine-tuning.
Common mistake
Assuming the two categories need the same dataset format

Practice Matching Datasets to Categories

EASY

A team has a collection of customer support messages. Each message is stored as text with one label from a fixed list of categories such as billing, technical, or other. Which fine-tuning category fits this dataset, and what makes each training example look the way it does?

Hints

Ask what each labeled example contains: a text with one label, or an instruction with an answer?
The label is a single category chosen from a fixed set.

Your attempt (not saved)

What do you think happens?

If the same team instead wrote each support message as a request paired with a written reply, which category would fit?

Classification fine-tuning
Instruction fine-tuning