Understanding and Fine-Tuning Large Language Models
A large language model produces its output in small units called tokens, and it does not hand you a complete reply in a single step.
A Long Reply Is Not Written in One Go
When you read a long answer from a chatbot, it can look as if the whole response was written in one go. The underlying process is more specific than that. A large language model produces its output in small units called tokens, and it does not hand you a complete reply in a single step. This one fact changes how you think about everything the model writes, from a single word to a multi-paragraph explanation.
What do you think happens?
How many output steps does the model need for a 50-token reply?
Reveal answer
Fifty steps, one token per step.
The source states that LLMs output one token at a time. Because each output is a single token, a reply made of many tokens cannot come from a single output. It is built across multiple steps, and counting in tokens (not sentences) is the right way to reason about how much output the model produces.
Tracing a Reply Token by Token
Think of generation as a loop. In each step the model outputs exactly one token. That token is added to the text, and the next step continues from the text built so far. The loop repeats, with each step contributing one more token, until the reply is finished. The diagram below follows this loop for a reply of several tokens.
Tracing a Four-Token Reply
Suppose a reply is made of four tokens, called t1, t2, t3, and t4. Trace which steps produce which parts of the reply, and identify when the reply is complete.
- 1Step 1
The model outputs t1. The text now contains one token, so it is not yet a complete reply.
- 2Step 2
The model outputs t2 and adds it after t1. Each step contributes one more token to the text.
- 3Step 3
The model outputs t3 and adds it to the end of the text built so far.
- 4Step 4
The model outputs t4, the final token. Only now does the complete four-token reply exist.
Four output steps, one token each. No single step produced the whole reply, and the complete reply was available only after the final token was output.
What a Single Output Step Produces
Each output step produces a single token, so a reply of many tokens is built across many steps. This is the rule in its simplest form. Its consequence is that the unit you should use when reasoning about how much output a model produces is the token, not the whole sentence. A short sentence and a long paragraph are both just sequences of tokens, and the model's output grows one token at a time as it runs.
A complete reply exists only after the final token has been output. Before that point there is only a partial reply.
Counting in tokens, not in whole sentences, is the right way to reason about how much output a model produces.
Misconceptions About When Text Is Produced
Learners often picture the process wrongly. The table compares common assumptions with what the one-token-at-a-time rule says about how text is produced.
| Common assumption | What the source supports |
|---|---|
| The model writes the whole reply in one step | Output is one token per step, so a multi-token reply needs multiple steps. |
| The reply exists in full while the model is still working | A complete reply exists only after the final token has been output. |
| The reply is a finished text handed over at once | The text appears as a sequence of token outputs, even when it looks like one go to the reader. |
| The unit of output is the whole sentence | The right unit for reasoning about output size is the token. |
Assumptions about LLM output compared with the one-token-at-a-time rule
Common mistake Assuming a long answer is one unit of output
What it looks likeBelieving a five-paragraph explanation was produced in a single step.
Why it failsEach output step produces a single token, so a reply of many tokens is built across many steps.
Do this insteadReason about the reply as many token outputs added in sequence.
Common mistake Treating a partial reply as a complete one
What it looks likeReading the text after the first few steps as the final answer.
Why it failsA complete reply exists only after the final token has been output.
Do this insteadTreat the text as complete only when generation has finished.
Common mistake Measuring output in sentences
What it looks likeEstimating a reply's size by counting sentences.
Why it failsThe model produces output in tokens, so token count is the relevant measure.
Do this insteadCount tokens when reasoning about how much output is produced.
Where Fine-Tuning Fits in Building an LLM
The course builds an LLM in stages. The third stage is the one about fine-tuning. In that stage, the course takes a pretrained LLM and fine-tunes it. Read that sentence as a sequence: the pretrained model comes first, and fine-tuning is applied to it afterward. The diagram shows that order as a flow from the pretrained model, through the fine-tuning step, to a model ready for specific tasks.
Keep the starting point in mind: fine-tuning does not begin from an empty model. It begins from a pretrained one that can already produce fluent text but has not been shaped for any particular job.
What Fine-Tuning Changes
Fine-tuning adjusts the pretrained model so it follows instructions for named tasks.
The definition has two halves. The first half names the starting object, a pretrained LLM, and the action performed on it, which is adjustment. The second half names the purpose, which is specific tasks. Keeping both halves in mind helps you avoid confusing fine-tuning with building a model from nothing.
The Two Fine-Tuning Task Families
The source names two example tasks for a fine-tuned model: answering queries and classifying texts. It also explains that these are among the most common tasks in many real-world applications and in research. Both are ways of following an instruction, which is the behavior the fine-tuning stage aims to build. The two families differ in what they ask of the model, as the two subsections below explain.
Matching Problems to Task Families
A support team wants a model that reads an incoming message and decides whether it is a billing complaint, a shipping question, or general feedback. Another team wants a model that answers a specific question about one product. Which problem belongs to answering queries, and which belongs to classifying texts? Explain your choice using the input and the output.
Hints
- Ask what the output looks like: a category, or a response to a question.
- The message-sorting problem assigns an input to one of a set of categories.
Key Takeaways
- An LLM outputs text one token at a time, so a longer reply is built across many output steps, one token per step.
- A complete reply exists only after the final token has been output, and token count is the right unit for reasoning about output size.
- Fine-tuning is the stage that comes after pretraining: it adjusts a pretrained LLM so it follows instructions for named tasks, starting from the pretrained model rather than an empty one.
- The two example task families for fine-tuned models are answering queries, which produces a response to a question, and classifying texts, which assigns an input to a category.