Natural Language Prompts and How Models Read Them

Instruction fine-tuning trains a language model on a set of tasks, with each task presented alongside a specific instruction.

Programming

Same Text, Different Request

Imagine sending the same short message to a language model twice. In one case you ask whether the message looks like spam. In the other you ask it to render the message in another language. The text is identical in both cases, yet the useful response depends entirely on the written request attached to it. That gap between the text and the request is the central idea behind instruction fine-tuning.

This article follows that idea step by step. First you will see what a training process looks like when every example carries an instruction. Then you will trace one example through the model. Finally you will check your understanding with a prediction and a practice problem.

Tracing the Spam and Translation Paths

The source illustrates the technique with two scenarios side by side. In the first, the model must decide whether a given text is spam. In the second, the model receives an English sentence along with an instruction to translate it into German. Both paths share one structure: material plus instruction enters the model, and an output comes out. The diagram lays out both paths so you can compare where the instruction enters and where the answer comes out.

What path do the instruction and the input take through the model to become the output, in the spam scenario and in the translation scenario?

Read the spam path from top to bottom. The message is the material, and the instruction tells the model how to format its answer: respond with yes or no. The language model takes both pieces together and produces a single answer. The translation path has the same shape. The English sentence is the material, the instruction names the target language, and the model's output is the German rendering. The structure is identical even though the tasks differ, which is exactly why one training approach can cover both.

What a Training Example Contains

Instruction fine-tuning trains a language model on a set of tasks, with each task presented alongside a specific instruction. A training example therefore has two parts that work together: the material to act on, such as a message or a sentence, and the instruction that says what to do with it. Training on many such pairs is meant to improve how well the model reads and follows those directions.

What fields does a single training example contain, and how do they relate to one another?

The key point is what the model is learning. It is not only learning to continue text or to recognize a pattern in isolation. It is learning that a written direction describes what should be done with the material that follows it. The same material can therefore call for a different response when the instruction changes.

Worked Example: Spam Check Versus Translation

Worked example

One message, two instructions

Suppose the same English sentence is attached to two different instructions: first, decide whether the text is spam, answering yes or no; second, translate it into German. Trace what enters the model and what comes out in each case.

1
Identify the material
In both cases the material is the same English sentence. The text itself does not change between the two requests.
2
Identify the instruction
The spam request says to respond with yes or no. The translation request names German as the target language.
3
Pass both pieces into the model
The material and the instruction enter the model together, as shown in the data flow above.
4
Read the output
The spam request yields a single yes or no answer. The translation request yields a German rendering of the sentence.

Result

The material was identical, but the instruction determined the task and therefore the output. The useful response depends on the written request attached to the text.

Mistakes When Reading the Technique

Common mistake
Thinking the instruction is just extra decoration on the input
Why it fails
The source describes the instruction as the part that says what should be done with the material. It is not optional commentary.
Do this instead
Treat the instruction as a required part of each training example that determines the task.
Common mistake
Assuming the input alone determines the output
What it looks like
The spam and translation scenarios use the same English sentence and still need different answers.
Why it fails
The text is identical in both cases, so the request is what separates the useful responses.
Do this instead
When reasoning about the output, always ask what the instruction asks for, not only what the text says.
Common mistake
Treating instruction fine-tuning as a fixed label set
Why it fails
The spam example has yes or no answers, but the translation example produces a German sentence. The source presents the approach as covering tasks described in natural language, not one fixed set of labels.
Do this instead
Think of the training as covering a set of tasks, each defined by its own instruction.
Common mistake
Equating instruction fine-tuning with ordinary prompting
Why it fails
The source frames instruction fine-tuning as training the model on tasks paired with instructions, which is a training process rather than a single request typed at use time.
Do this instead
Keep the two ideas separate: fine-tuning changes how the model is trained on many instruction-task pairs.
Common mistake
The source does not describe classification fine-tuning in detail, so do not attribute its mechanics to instruction fine-tuning or the reverse.

Setting It Beside Classification Fine-Tuning

The source names instruction fine-tuning and classification fine-tuning as the two most common ways to fine-tune language models, and it asks that they be studied as distinct approaches. This article concentrates on the first. The source does not describe the second in detail, so treat classification fine-tuning here only as the other widely used method you will meet alongside this one.

How do the two methods differ in what the training examples look like, what the model is trained to output, and how it is used afterward? The source details only the instruction side.

Predicting the Effect of a Changed Instruction

What do you think happens?

The English sentence stays exactly the same, but the instruction changes from deciding whether it is spam to translating it into German. What changes in the model's job?

Reveal answer

Answer:

The instruction changes, so the model is asked to perform a different task on the same material. The output changes from a yes or no answer to a German rendering of the sentence.

The material is unchanged, so the difference comes entirely from the written request attached to it. This is the central idea of instruction fine-tuning.

MEDIUM

Write two instructions that could be attached to the same short message, one asking for a classification-style answer and one asking for a rewrite in another language. For each, name the material, the instruction, and the kind of output the model would produce. Then explain in one sentence why the message alone does not determine the output.

Hints

Keep the material identical between your two instructions.
The instruction should say what to do with the material, as the source describes.

Your attempt (not saved)

Key Takeaways

Instruction fine-tuning trains a language model on a set of tasks, with each task presented alongside a specific instruction.
Each training example has two working parts: the material to act on and the written instruction that says what to do with it.
In the spam and German translation scenarios the material and instruction enter the model together, and an output comes out; the structure is the same even though the tasks differ.
The same text can call for different responses when the instruction changes, so the instruction determines the task.
Instruction fine-tuning and classification fine-tuning are the two common methods named in the source and should be studied as distinct approaches.

More concepts starting with N