Chat Templates and Model-Specific Formats

Data for finetuning must be in the format the target model expects, which includes the tokenizer and the chat template.

Programming

The tempting shortcut

Suppose a carefully written prompt helps a base model perform a task. It includes a task description and several worked examples, and the model now responds reasonably well. When you decide to finetune, reusing that whole prompt can seem like the obvious next step. But prompting and finetuning put the teaching in different places: a prompt can explain the task and demonstrate it at inference time, while finetuning examples teach the behavior through the training data.

What do you think happens?

Should you usually copy a prompt-engineering instruction, including its task description and demonstrations, unchanged into finetuning data?

Yes, because the task is the same.
Usually not; finetuning examples can teach the behavior, so the instruction typically does not need the same description and demonstrations.

Reveal answer

Answer:

Usually not; finetuning instructions typically do not need the task description or examples included in a prompt.

A prompt often carries the explanation and demonstrations the model needs at inference time. In finetuning, the training examples themselves can teach the behavior, so the instruction may be different.

From cleaned records to model input

Formatting comes after deduplication and cleaning. At that point, the examples may be clean and free of duplicates, but they still need to be arranged in the form the target model expects. The chat template organizes the example, and the tokenizer determines how its text is split into the units the model reads.

What happens between cleaning examples and representing them as units the target model reads?

What the target format controls

The format is model-specific, not just task-specific. The target model has a tokenizer and expects a chat template. The tokenizer determines how text is split into the units the model reads; the chat template determines how the data is laid out around those units. A supervised finetuning example is typically an instruction paired with a response. The instruction can itself be divided into a system prompt and a user prompt, and the template organizes this structure.

Which parts make up an instruction-and-response example, and which parts belong to the instruction?

Where few-shot examples go

A few-shot prompt may put a task description and several demonstrations together in one instruction. For finetuning, the demonstrations do not have to remain embedded in that instruction. Each example in the prompt can be converted into a training example, with an instruction paired with its response.

How can three demonstrations in one food-classification prompt become separate finetuning records?

Worked example

Three demonstrations become three records

A food-classification prompt contains a task description and three demonstrations. Each demonstration pairs an input with its correct label. How can those demonstrations be prepared as finetuning examples?

1
Separate the demonstrations
Treat each demonstrated input and its correct label as one example rather than keeping all three demonstrations together inside a single prompt.
2
Pair instruction and response
For each example, use the task input as the instruction-side content and the correct label as the response.
3
Format for the target
Arrange the resulting examples using the target model’s expected chat template and tokenizer.

Result

The three demonstrations can become three instruction-and-response training examples. Their exact formatted representation depends on the target model.

Why templates cannot be swapped casually

The same conversation content does not, by itself, determine the format used for training. The target model’s template determines how its data is laid out, and its tokenizer determines how text is split into the units it reads. If the prepared data does not match what the selected model expects, it is not in the required format for that model. Do not assume a layout prepared for one target will suit another.

Why should the same user message and response be prepared according to the selected target model’s own template?

Prompt engineering	Finetuning
The instruction often includes a task description and demonstrations.	The instruction typically does not need the task description or demonstrations.
The prompt carries guidance and examples for use at inference time.	The training examples can teach the behavior.
Demonstrations appear together in the prompt.	Each demonstration can become a training example.

The task may be the same, but the examples play a different role.

Mistakes in preparing the data

Common mistake
Copying the entire prompt-engineering instruction unchanged into finetuning data.
What it looks like
Keeping the task description and all demonstrations inside every training instruction.
Why it fails
Finetuning examples can teach the behavior, so instructions typically do not need the same task description and examples as a prompt.
Do this instead
Use the demonstrations as training examples and prepare the instructions for the finetuning task.
Common mistake
Treating formatting as a generic final polish.
What it looks like
Preparing data without checking the target model’s tokenizer and chat template.
Why it fails
The required format is model-specific and includes both the tokenizer and the chat template.
Do this instead
Format the cleaned examples for the model that will be finetuned.
Common mistake
Leaving several demonstrations together as though they were one training example.
What it looks like
Keeping a three-shot prompt as one undivided instruction instead of separating its demonstrations.
Why it fails
Each example in a few-shot prompt can be converted into a finetuning example.
Do this instead
Separate the demonstrations into individual instruction-and-response examples.

Check the conversion

MEDIUM

A prompt contains a task description and three demonstrations. Each demonstration contains an input and its correct response. Describe how you would turn the demonstrations into finetuning examples, and name the two target-specific formatting requirements to check before training.

Hints

Count the demonstrations and consider whether each can become its own example.
Remember that a supervised finetuning example pairs an instruction with a response.
Check both the target model’s chat template and its tokenizer.

Your attempt (not saved)

Key Takeaways

Formatting is the final preparation stage after deduplication and cleaning.
Finetuning data must match the target model’s expected chat template and tokenizer.
Prompt-engineering instructions typically do not need to be copied unchanged because training examples can teach the behavior.
Each example in a few-shot prompt can be converted into its own instruction-and-response training example.

More concepts starting with C