Stage 3: Fine-Tuning the Pretrained LLM
Stage 3 of the course takes a pretrained LLM and fine-tunes it for specific work.
A Fluent Model Without a Job
Imagine you have finished building a language model that has already gone through the earlier work of this course. It can produce fluent text, but it has not been shaped for any particular job. This article asks a practical question: once a pretrained model exists, how do you turn it into one that performs a specific job?
What do you think happens?
Suppose this model is asked to sort customer emails into categories, or to answer a specific question about a product. How well does it do either task, given that it has only been pretrained?
Reveal answer
It has no particular reason to do either task well, because it has not been shaped for any particular job.
A pretrained model can produce fluent text, but fluency alone does not mean it has been adjusted to follow instructions for a named task. That adjustment is what fine-tuning provides.
Tracing the Pretrained Model's State
Start by tracking what the model is at this point in the course. It is a pretrained LLM. It begins from the earlier work of the course and can generate fluent text. What it lacks is a purpose: nothing has yet aimed it at a particular kind of work.
Keep this state in mind. Every later step in this article is about changing it, and the change always starts from this pretrained model rather than from an empty one.
Where Fine-Tuning Sits in the Build
The course builds an LLM in stages. The third stage is the one this article covers. In that stage, the course takes a pretrained LLM and fine-tunes it. Read that sentence as a sequence: the pretrained model comes first, and fine-tuning is applied to it afterward.
Fine-tuning adjusts the pretrained model so that it follows instructions for named tasks.
Notice the two halves of this definition. The first half names the starting object, a pretrained LLM, and the action performed on it, which is adjustment. The second half names the purpose, which is specific tasks. Keeping both halves in mind helps you avoid confusing fine-tuning with building a model from nothing.
Comparing the Two Task Families
The source names two example tasks for a fine-tuned model: answering queries and classifying texts. It explains that these are among the most common tasks in many real-world applications and in research. Each task asks something different of the model, but both are ways of following an instruction, which is the behavior the fine-tuning stage aims to build.
Tracing the State After Fine-Tuning
Now trace the same model after the fine-tuning step. Its starting point has not changed: it still begins from a pretrained LLM. What changes is its purpose. Instead of producing fluent text with no assigned job, it is adjusted to follow instructions for named tasks such as answering queries or classifying texts.
Matching Problems to Task Families
Once you know the two task families, the next skill is matching a practical problem to the right one. Look at what the problem asks the model to produce. If it asks for a response to a question, it belongs to answering queries. If it asks the model to assign an input to a category, it belongs to classifying texts.
Sorting Two Practical Problems
A support team has two needs for a fine-tuned model: (a) sort incoming customer emails into categories, and (b) answer a specific question a customer asks about a product. Which task family fits each need?
- 1Identify the output of need (a)
Sorting emails into categories means assigning each input to a category. That matches the description of classifying texts.
- 2Identify the output of need (b)
Answering a specific question about a product means producing a response to a question. That matches the description of answering queries.
- 3Name the cue for each choice
The cue for classifying is the word category. The cue for answering queries is a question that needs a response.
Need (a) belongs to classifying texts. Need (b) belongs to answering queries.
Mistakes Beginners Make With Fine-Tuning
Common mistake Treating fine-tuning as building a model from nothing
What it looks likeThinking the third stage starts with an empty model and creates its abilities from scratch.
Why it failsFine-tuning begins from the pretrained model rather than from an empty one.
Do this insteadStart from the pretrained LLM and remember that fine-tuning adjusts it for named tasks.
Common mistake Assuming fluent text means the model is ready for a job
What it looks likeExpecting a pretrained model to sort customer emails well simply because it writes smooth sentences.
Why it failsA pretrained model can produce fluent text but has not been shaped for any particular job, so it has no particular reason to do a specific task well.
Do this insteadRecognize that the purpose of fine-tuning is to aim the model at specific tasks.
Common mistake Matching a task by the topic of the text rather than the requested output
What it looks likeLooking at a customer email and deciding the task is answering queries just because the email mentions a product.
Why it failsThe family is determined by what the model must produce: a response to a question, or a category assignment.
Do this insteadAsk whether the problem wants a response to a question or an input assigned to a category.
Practice Matching Tasks to Purposes
A team has a pretrained LLM and wants it to be useful for two jobs: deciding which of several categories each incoming support ticket belongs to, and responding to a specific question a user types about an account. For each job, name the task family and explain which output the model must produce. Then state what stays the same about the model's starting point after fine-tuning.
Hints
- Ask whether the job assigns an input to a category or produces a response to a question.
- Remember that fine-tuning begins from the pretrained model in both cases.
Key Takeaways
- Stage 3 of the course takes a pretrained LLM and fine-tunes it for specific work.
- Fine-tuning adjusts the pretrained model so that it follows instructions for named tasks, and it begins from the pretrained model rather than from an empty one.
- The two example task families are answering queries, which asks for a response to a question, and classifying texts, which asks for an input to be assigned to a category.
- These two tasks are described as among the most common tasks in many real-world applications and in research.
- To match a practical problem to a task family, check whether it asks for a response to a question or an assignment to a category.