Reasons for Finetuning

Imagine two models. One is very large and performs well across many public benchmarks. The other is much smaller and, straight out of the box, performs worse on general tests. You are building a product that needs only one narrow task. Which model should you use?

Many people would reach for the large model because it looks stronger. This article shows why that instinct can be wrong for a narrow job. Finetuning, and a technique called distillation, can make the smaller model the better choice for that task. Teams also tend to prefer small models once they move systems into production, and the reasons for that preference are practical rather than cosmetic.

Keep the question in mind as you read. By the end you should be able to say when a small finetuned model is likely to beat a large general one, and explain the mechanism that makes that possible.

A model that scores well on a broad range of benchmarks may still handle your particular work poorly. The reason is that the training data it saw did not cover your task well. Benchmark strength and task strength are different things, and finetuning is the tool for closing the gap between them.

Finetuning is also commonly used to make a model produce outputs in a specific structure, such as JSON or YAML. If your application expects a certain format every time, a model that drifts from that format is a problem even when its answers are otherwise reasonable. A correct answer in the wrong shape can still break the software that consumes it.

The diagram below traces what happens to a base model when it is finetuned on task-specific examples. The starting model handles generic requests, but its behavior on your task is weak. After finetuning on examples that show the output you want, the same model's behavior moves toward your task.

Notice what changes and what does not. The model's underlying starting point is the same pre-trained model. Finetuning moves its behavior toward the examples you supply, so the output on your task becomes more like the output you want.

A useful way to decide whether finetuning will help is to look for a gap between what a model does well and what your task needs. A familiar example is converting natural language into SQL. An out-of-the-box model may convert requests into standard SQL well but fail on a less common SQL dialect. In that case, finetuning the model on data that contains that dialect should help.

A related case is a model that handles standard SQL for common queries but often fails on queries specific to one customer. Finetuning on customer-specific queries may help there as well. The pattern is the same in both cases: the base model rarely sees your pattern, and the finetuning data supplies it.

Finetuning is also a practical route to bias mitigation, which is one especially interesting use of the technique. The reasoning starts from a known problem. If a base model carries biases from its training data, it may repeat those biases in its outputs. The remedy is to expose the model to carefully curated data during finetuning, which can counteract the biases it learned.

Take a model that consistently assigns male-sounding names to CEOs. Finetuning it on a dataset containing many female CEOs can mitigate that pattern. The important word is curated. The goal is not simply more data. It is data chosen so that it pushes against the specific bias you have observed.

Research by Garimella and colleagues, reported in 2022, illustrates how targeted the choice of text can be. They found that finetuning BERT-like language models on text written by women can reduce those models' gender biases, and that finetuning them on texts by African authors can reduce racial biases. The choice of finetuning text can therefore aim at particular kinds of bias.

It is possible to finetune a very large model to make it even better. In practice, finetuning smaller models is much more common. The reasons are practical, and each one affects a different stage of a project.

Read the comparison as a set of trade-offs. The large model's breadth is real, but it comes with higher memory, cost, and latency. The small finetuned model gives up breadth in exchange for efficiency and, when finetuned on the right data, strong results on the one task you actually need.

Distillation is a way to pass the useful behavior of a larger model to a smaller one. The common approach is to finetune a small model so that it imitates the behavior of a larger model, using data that the large model generated. The name comes from this idea: the larger model's knowledge is distilled into the smaller model.

Data synthesis techniques, which include this approach, are covered elsewhere in the course. For this article, the important part is the direction of the flow.