Pre-trained and post-trained models
The more refined a model is, and the more relevant its knowledge is to the task, the less adaptation work the application developer has to do. This relationship between refinement and effort is what drives the choice of starting point.
Predicting Where Application Developers Start
When people hear that a model has been finetuned, they often imagine one step: take a model, train it a little more, and it is ready. In practice, finetuning is a set of decisions rather than a single action. Someone has to decide who performs the adjustment, which model is used as the starting point, and whether the cost of adjusting the model is justified compared with simpler options. This article follows those decisions in order, tracing one model from its origin to the point where an application developer chooses how to adapt it.
What do you think happens?
When an application developer builds a product on top of an existing model, which kind of model do they most often start from?
Reveal answer
A post-trained model that a model developer has already refined
The source states that application developers might finetune a pre-trained model, but most often they finetune a post-trained one. The more refined a model is, and the more relevant its knowledge is to the task, the less adaptation work the application developer has to do.
Two Groups That Finetune
Finetuning is performed by both model developers, who post-train models before release, and application developers, who adapt existing models to their own tasks or datasets.
Model developers finetune as part of building a model. They post-train it with different finetuning techniques before it is released. Application developers finetune as part of building a product on top of a model that already exists. Both groups use finetuning, but they begin from different places and do different kinds of work.
The two groups also work at different levels. A model developer may release several versions of one model, each finetuned to a different extent. Application developers can then pick the version whose behavior best matches what their application needs, instead of adjusting a single fixed model themselves. The choice of version is therefore part of the application developer's work.
A family of Code Llama models was produced by applying different finetuning techniques, so that each member of the family was tuned for a somewhat different purpose. The family shows that finetuning is a method a model developer can use to create a range of models from a shared starting point. Each member can then serve as a different starting option for someone building an application.
Following One Model Through Its Lifecycle
Follow one model through the process. It begins as a pre-trained model. This model has learned general patterns from a large body of data, but it has not been refined for a particular style of use. A model developer then post-trains it with finetuning techniques, and it may be released in more than one version, each refined to a different degree.
An application developer then chooses one of the released versions and, if needed, finetunes it further on data for a specific task. At each stage the model is the same underlying system, but the amount of task-specific adjustment it has received is different. The choice of starting point determines how much of that adjustment remains for the application developer to do.
The diagram shows the path the source describes. Version B is a released alternative that the application developer could have chosen instead; the arrow marks the version that was picked. The important point is the ordering: the pre-trained model comes first, post-training by a model developer comes second, and task-specific adaptation by an application developer comes last.
Pre-trained Versus Post-trained Starting Points
An application developer can start from two kinds of model, and the difference between them is how much refinement has already been done. The table below compares the two starting points using only what the source establishes.
| Feature | Pre-trained model | Post-trained model |
|---|---|---|
| Who prepared it | Learned general patterns from a large body of data | A model developer refined it with finetuning techniques before release |
| Degree of refinement for a particular style of use | Not refined for a particular style of use | Already refined before release |
| Adaptation work left for the application developer | More | Less, especially when its knowledge is relevant to the task |
| How often application developers start here | Less often | Most often |
Two starting points for an application developer
The more refined a model is, and the more relevant its knowledge is to the task, the less adaptation work the application developer has to do. This relationship between refinement and effort is what drives the choice of starting point.
Deciding Between Finetuning and Prompting
The source insists that the decision to finetune comes before any technique is chosen. Compared with prompt-based methods, finetuning requires significantly more resources. The cost is not only data and hardware but also ML talent. Because of this difference, finetuning is generally attempted only after extensive experiments with prompt-based methods have been tried.
Read the flowchart as a sequence of questions rather than a fixed rule. Prompt-based experiments are the first step. If they meet the need, the cost of finetuning is avoided. If they do not, the resource cost of finetuning becomes the central question: finetuning is worth pursuing only when the extra data, hardware, and ML talent are justified compared with the simpler options already tried. The source does not give a numeric threshold, so the judgment is a comparison rather than a formula.
Combining Finetuning and Prompting in One System
The two approaches are not competitors. The source states that finetuning and prompting are not mutually exclusive, and that real-world problems often require both. A system can use prompting for some parts of the task and finetuning for others. So the question is rarely which one to choose forever. It is which one to try first, and whether the prompt-based results justify the extra cost of finetuning.
A support-ticket assistant built in stages
A small team wants a model to sort incoming support tickets and draft replies. They must decide whether to finetune, and in what order to try their options. This scenario is hypothetical and written for teaching.
- 1Choose the starting model
The team looks at released models and picks a post-trained version, since application developers most often start from one and it reduces the adaptation work they must do.
- 2Run prompt-based experiments first
Before any finetuning, the team writes prompts for ticket sorting and reply drafting and evaluates the results extensively, following the order the source describes.
- 3Compare the results with the need
Suppose sorting works well with prompts, but the drafted replies do not match the company's tone. The team now has a specific gap rather than a vague wish to adapt the model.
- 4Weigh the cost of finetuning
The team checks whether it has the data, hardware, and ML talent that finetuning requires, and whether that cost is justified compared with the simpler prompt-based option that already works.
- 5Combine the two where it makes sense
If the team finetunes, it can keep prompting for sorting and use the finetuned model for reply drafting, so each approach handles the job it suits best.
The team does not treat finetuning as the first move. It runs prompt experiments, identifies the one gap prompting cannot close, and uses finetuning only for that part, with prompting still handling the rest.
Mistakes Beginners Make About Finetuning
Common mistake Treating finetuning as the first thing to try
What it looks likeA team starts adjusting a model before writing a single prompt for its task.
Why it failsFinetuning needs significantly more data, hardware, and ML talent than prompt-based methods, so it is generally attempted only after extensive prompt-based experiments.
Do this insteadRun prompt-based experiments first, and move to finetuning only when those results do not justify staying with prompting.
Common mistake Assuming finetuning and prompting compete
What it looks likeA team chooses one approach permanently and never considers using both.
Why it failsThe source states that the two are not mutually exclusive and that real-world problems often use both.
Do this insteadAsk which approach to try first and which parts of the task each should handle.
Common mistake Assuming application developers always start from a pre-trained model
What it looks likeA developer downloads a raw pre-trained model and begins adapting it, when a released post-trained version would need less adaptation work.
Why it failsApplication developers most often start from a post-trained model, because the more refined the model, the less adaptation work is required.
Do this insteadCheck whether a post-trained version suited to the task is available before choosing the starting point.
Common mistake Believing finetuning is one step rather than a set of decisions
Practice Choosing a Starting Point
A company wants a model that answers questions about its product manuals. Its engineers have not yet tried writing any prompts. Using the lifecycle and the experimentation order from this article, describe the first two steps the engineers should take, and explain why they should not begin with finetuning.
Hints
- Think about the order in which the source says prompting and finetuning should be tried.
- Consider which kind of starting model an application developer most often uses, and why refinement reduces adaptation work.
- Recall the resource difference between finetuning and prompt-based methods.
Key Takeaways
- Finetuning is done by two groups: model developers, who post-train models before release, and application developers, who adapt existing models to their own tasks or datasets.
- A model moves from a pre-trained state to a post-trained state through a model developer, and then to a task-adapted state when an application developer finetunes it further.
- Application developers most often start from a post-trained model, because the more refined the model and the more relevant its knowledge, the less adaptation work is required.
- Finetuning needs significantly more data, hardware, and ML talent than prompt-based methods, so it is generally attempted only after extensive prompt-based experiments.
- Finetuning and prompting are not mutually exclusive, and real-world problems often use both, so the real question is which to try first.