Parameter-Efficient Fine-Tuning with LoRA

Imagine you have built a language model, pretrained it, and fine-tuned it for a specific task. The test scores come back lower than you hoped. Where do you look next? This is the moment this article addresses. It collects practical strategies for improving an LLM's performance after the basic development steps are complete.

The chapter this material comes from closes a full cycle: implementing an architecture, pretraining the model, and fine-tuning it for specific tasks. The strategies covered here are the next things to investigate once that cycle is done. Each one is a lever you can pull, and each one needs to be judged by a measured score rather than by intuition.

The source names four strategies that can improve model performance. Each one acts on a different part of the problem, so it helps to keep them separate in your head before you decide which one to try first. Adjusting hyperparameters changes how the training run itself behaves. Changing the training data changes what the model learns from. Experimenting with prompts changes how you ask the model to respond. Choosing a larger pretrained model changes the starting point from which fine-tuning begins.

The four strategies are not interchangeable. A change to the training run can alter how the model fits the data, a change to the data can alter what there is to fit, a change to the prompt alters only the input at the moment of asking, and a change to the starting model alters the foundation everything else is built on. Knowing which lever you are pulling tells you what kind of result to expect and how to test it.

A hyperparameter change is easiest to understand as a short, repeatable sequence. You begin with a baseline configuration and a measured test score. You change one setting, retrain, and then measure again on the same test set. Only that comparison tells you whether the change helped. A score on its own means little until it is set against the baseline.

Notice that the loop always returns to the same measurement step. Every configuration is judged on the same yardstick, which is the fixed test set and the baseline score. The source lists learning rate, batch size, and number of epochs as examples of settings to adjust, and the same sequence applies to any of them.

The two strategies in this part leave the model's starting point alone. One changes what the model learns from, and the other changes how you ask it to respond. Both can be compared with the same discipline used for training settings.

The fourth strategy is to start from a larger pretrained model. The source explains the reasoning: a larger model may have greater capacity to capture complex patterns and generate more accurate responses. The source does not provide a size-versus-score table, so its evidence for this strategy is indirect. The clearest numbers it gives compare two versions of the same Llama 3 8B model.

The gap is the difference between the two averages: $82.6 - 58.51 = 24.09$. The instruct variant is about 24.09 points higher. The direction matters: the instruct model is the higher-scoring one, and the source attributes the gap to instruction fine-tuning rather than to any other difference between the two versions.

The last item is a different kind of strategy. Rather than promising a direct score gain, it addresses cost. The source introduces parameter-efficient fine-tuning with LoRA, which stands for low-rank adaptation. The chapter states that LoRA can be used for parameter-efficient fine-tuning and points to an appendix for the method itself.

This article does not describe LoRA's internal mechanics, because the source does not explain them in this section. What the chapter does set up is an experiment: take the instruction fine-tuning code from the chapter, switch it to LoRA, and then compare the training run time and the model performance against the standard approach.