Overview of Parameter-Efficient Finetuning Techniques

Finetuning starts from a model that has already been trained and adjusts it for a new task. The obvious approach is to let every weight in the model move during that adjustment. Early on, models were small enough that this was practical, and the approach came to be called full finetuning. As models grew, the obvious approach started to fail for ordinary people.

The question this article follows is simple: if a model is too large to adjust completely on the hardware you own, what can you change instead? Keep one idea in mind as you read. In full finetuning, the number of trainable parameters equals the total number of parameters in the model. Every later idea in this article is a way of reducing that count, or of reducing the memory cost that comes with it.

Full finetuning also typically needs a large amount of high-quality annotated data, which many people cannot afford to produce. Memory and data both push practitioners toward one question: can they update less of the model?

Memory cost is easiest to understand when you add up the pieces. Take a model with 7 billion parameters, written 7B. Suppose the weights are stored in a 16-bit format such as FP16, where each value takes 2 bytes. The first piece of the bill is the weights themselves.

The diagram shows the scale of the problem. Weights are only the first slice. The source's 56 GB figure is a total for the full finetuning run, not for the weights alone, and it still excludes activations. A GPU that cannot hold that total cannot run full finetuning on that model, however well the training code is written.

The first workaround is partial finetuning. Instead of updating every parameter, you update only some of them and freeze the rest. A frozen parameter keeps its value throughout training. Picture a model with ten layers. If you freeze the first nine and train only the last one, the number of trainable parameters falls to about 10% of what full finetuning would need.

Freezing layers lowers memory use, but it does not solve the core problem. Partial finetuning is parameter-inefficient, which means it still needs many trainable parameters before it approaches the quality of full finetuning. The source reports a study on BERT large in which roughly 25% of the parameters had to be updated to match full finetuning on the GLUE benchmark. Performance rises only as more parameters are opened up.

The natural question follows. How can you reach performance close to full finetuning while training far fewer parameters? Techniques that answer this question are called parameter-efficient. There is no official cutoff that a method must cross to earn the label.

Parameter-efficient finetuning, abbreviated PEFT, was introduced by Houlsby and colleagues in 2019. Their key move was to insert extra parameters into the model in carefully chosen places, and then train only those extra parameters. The rest of the model keeps its original values.

The extra pieces inserted by Houlsby and colleagues are called adapter modules. Their authors placed two adapter modules inside each transformer block of a BERT model. The original weights of the block are left alone, and the adapters are the only pieces that receive updates.

During finetuning, the original parameters stay unchanged. The number of trainable parameters is simply the number of parameters inside the adapters. On the GLUE benchmark, this arrangement came within 0.4% of full finetuning while using only 3% of the trainable parameters, as reported by the source.

Now trace what happens to a single input as it moves through the network. The input passes through the frozen base computation, then through the adapter, and then continues onward. The base values stay the same for every example, and only the adapter's values change during training.

Adapters keep the original weights frozen and train only the inserted modules, which saves memory and data. Lower memory is not the only benefit of PEFT. The source says PEFT enables finetuning on more affordable hardware, which makes the work accessible to many more developers. It also says that PEFT methods are generally sample-efficient: full finetuning may need tens of thousands to millions of examples before quality improves noticeably, while some PEFT methods can deliver strong results with only a few thousand examples.

The cost comes from the path itself. The adapter is an extra computation on every forward pass, so the model does more work than it did before adapters were added. The source points to this as the main drawback: the additional layers add computational steps, which increases inference latency for the finetuned model.