Retrieval-Augmented Generation Versus Finetuning

Many people who first hear about finetuning come away with a tidy picture. Take a model that has already learned a great deal, run some additional training on a task you care about, and the model is now customized. Because the training step is often described as a single command, it is easy to assume the whole job is routine. This article tests that assumption. The goal is to build a clear picture of what finetuning is, why it became harder as models grew, and why the difficulty is not located where many beginners expect.

The idea of reusing knowledge from one training effort in another is older than the modern vocabulary. Transfer learning is the older name for the general strategy of carrying what a model learned in one setting into a related one. Finetuning is one of the practical forms that strategy takes. Its purpose is to shift a general model toward the particular behavior a practitioner needs, rather than to build a model from nothing.

Read the definition slowly. The starting point is a pre-trained model, a model whose weights already encode a large amount of learned structure. Finetuning does not throw those weights away. It continues the learning process from where pre-training left off, but with data that represents the task you care about. Each update nudges the weights so that the model's behavior moves toward that task. This is why finetuning is described as a form of adaptation: the model is the same underlying model, but its behavior has been reshaped by additional training.

It helps to separate two ideas that can look similar. Pre-training builds the general capability. Finetuning starts from that capability and specializes it. The second step is usually much smaller in scope than the first, but the source is clear that it still changes weights, and changing weights is where the cost of the process comes from.

In its early days, finetuning looked a lot like pre-training. Both involved updating the model's entire set of weights. That approach worked when models were smaller, but as models grew, updating every weight became impractical for most practitioners. The core relationship is simple: the more parameters you update during finetuning, the more memory the finetuning process requires.

Follow what the process has to hold in memory for a single trainable value. The stored weight is one part. Training also keeps track of information about how that weight should change, and an optimizer keeps its own bookkeeping for each trained value. The diagram below traces these three kinds of storage for one trainable parameter. Multiply that by every parameter you choose to update, and the total grows with the size of the model.

Memory is only part of the constraint. Most practitioners do not have access to enough hardware, time, and data to perform full finetuning on foundation models. These three limits interact. A setup that could fit the memory might still be too slow to be practical, and a setup with enough compute might still lack the data needed to make the result useful. Full finetuning therefore fails for several reasons at once, which is why researchers developed approaches that attack the memory problem from different directions.

Two families of techniques respond directly to the memory problem. They approach it from different angles, and it helps to keep those angles separate before looking at any detail. One family reduces how many values are trained. The other reduces how many bits each value occupies. The sections that follow take each in turn, then show how model merging builds on finetuning results.

Quantized training takes a different route. Where PEFT reduces how many values are trained, quantized training reduces the number of bits used to represent each value. Fewer bits per value means each parameter takes up less storage, so the same model needs less memory. The trade-off is that a value stored with fewer bits can only be represented approximately. Its precision drops, and that loss is what makes quantization a deliberate compromise rather than a free saving.

The idea of combining finetuned models leads to model merging. Its goal is to combine multiple models into one model that works better than the individual models do separately. The source lists several use cases, including on-device deployment and model upscaling, and it also covers general approaches to merging. The flow below shows the basic shape of that combination: separately finetuned models contribute their results, and those results flow together into one combined model.

Model merging is worth noticing because it treats finetuning results as reusable pieces. A model finetuned for one purpose does not have to be discarded when another is needed. Its contribution can be combined with others, which is one reason modular techniques such as LoRA matter in practice: a small set of adapted values is easier to keep, serve, and combine than a full copy of a changed model.

The source frames a practical question that many practitioners face: whether to finetune a model at all. The chapter it comes from begins with the reasons for finetuning and the reasons for not finetuning, and it also discusses when to finetune versus when to use retrieval-augmented generation, often called RAG. The scenario below is an illustrative one, not a case from the source, meant to show how the earlier ideas combine in a decision.

Return to the prediction you made at the start. The training step is often the smallest visible piece of finetuning, and the reason is that the work around it is hard to automate and easy to underestimate. Raw data does not arrive ready to train on. It must be collected from its sources, cleaned of errors and irrelevant material, formatted into the shape the training process expects, and then evaluated so that you can tell whether the finetuned model actually improved on your task.

Each of these stages can determine the final result more than the training procedure does. A well-tuned training run on poorly prepared data will adapt the model to the poor data. A careful evaluation stage is what reveals whether adaptation happened at all. This is why the source places the difficulty in the surrounding context and data rather than the procedure, and why a finetuning project is best judged by the quality of its data pipeline as much as by its training code.