Parameter-Efficient Finetuning in Practice

Many people who first hear about finetuning come away with a tidy picture: take a model that has already learned a great deal, run some additional training on a task you care about, and the model is now customized. Because the training step is often described as a single command, it is easy to assume the whole job is routine. This article tests that assumption. The goal is to build a clear picture of what finetuning is, why it became harder as models grew, and why the difficulty is not located where many beginners expect.

A pre-trained model starts from a general state. It has already learned a great deal from earlier training, and its weights encode that knowledge. Finetuning does not build a model from nothing. Instead, it nudges those existing weights so the model's behavior shifts toward the particular task or dataset a practitioner cares about. This is why finetuning is a form of adaptation: the model is adjusted from a general starting point toward a specific target.

The idea of reusing knowledge from one training effort in another is older than the modern vocabulary. Transfer learning is the older name for the general strategy of carrying what a model learned in one setting into a related one. Finetuning is one of the practical forms that strategy takes. Its purpose is to shift a general model toward the particular behavior a practitioner needs, rather than to construct a new model from scratch.

In its early days, finetuning looked a lot like pre-training. Both involved updating the model's entire set of weights. That approach worked when models were smaller, but as models grew, updating every weight became impractical for most practitioners. The rest of this article follows that problem and the responses to it.

The core relationship is simple: the more parameters you update during finetuning, the more memory the finetuning process requires. Full finetuning updates every weight, so its memory demand grows with the size of the model. The diagram below shows the main things that full finetuning keeps in memory at the same time: the weights themselves, the gradients computed during training, and the optimizer states. Each added parameter adds to all three, which is why the total footprint multiplies as models grow.

Memory is only part of the constraint. Most practitioners do not have access to enough hardware, time, and data to perform full finetuning on foundation models. These three limits interact. A setup that could fit the memory might still be too slow to be practical, and a setup with enough compute might still lack the data needed to make the result useful. Any response to the memory problem therefore has to be judged against all three limits, not memory alone.

Two families of techniques respond directly to the memory problem. They approach it from different angles, and it helps to keep those angles separate before looking at any detail. One family reduces how many values are trained. The other reduces how many bits each stored value uses. The subsections below take them one at a time.

The source zooms into LoRA, which stands for Low-Rank Adaptation. LoRA is a technique used for finetuning that is both parameter-efficient and data-efficient. It is also modular, which makes it much easier to serve and to combine multiple LoRA models. Low-rank factorization is the underlying idea the source names as the basis for this family of methods, and it is one reason LoRA appears throughout the finetuning literature.

At a high level, the update to a frozen weight matrix is represented as the product of two small low-rank matrices. The frozen matrix is left alone. The two small matrices are multiplied together to form an update, and that update is added to the frozen weights. Only the two small matrices are trained, so the number of trainable values is much smaller than the number in the full weight matrix. The diagram below shows how these pieces relate.

The idea of combining finetuned models leads to model merging. Its goal is to combine multiple models into one model that works better than the individual models do separately. The source lists several use cases, including on-device deployment and model upscaling, and it also covers general approaches to merging. Merging responds to the same pressure from a different direction: rather than making each finetuning run cheaper, it aims to get more value out of the finetuned models that already exist.

Returning to the prediction from the opening: the training loop is only one stage in a finetuning project. Data has to be curated, formatted into the form the training procedure expects, and then evaluated to check whether the adapted model does what the task needs. The diagram below lays these stages out in order.

The source's own framing points the same way. The hardest part of finetuning is often located in the surrounding context and data rather than in the training procedure. A well-run training step on poorly chosen or badly formatted data can still produce a model that does not serve the task, so the earlier stages carry much of the weight of the outcome.

The source frames a practical question many practitioners face: whether to finetune a model at all. The chapter it comes from begins with the reasons for finetuning and the reasons for not finetuning. It also discusses when to finetune versus when to use retrieval-augmented generation, often called RAG. Treating finetuning as one option among several, rather than as the default, is part of working with it well.