Learning Rate Schedules in Finetuning

Finetuning starts from a base model that has already been trained, and then adjusts it for a specific use case. Along the way you make many choices that are not learned from the data. These settings are called hyperparameters. The exact list depends on the base model and on the finetuning method you use, so the documentation for your model or framework is always the authoritative reference. This article focuses on four hyperparameters that come up again and again and strongly influence how efficiently finetuning works: the learning rate, the batch size together with gradient accumulation, the number of epochs, and the prompt model weight.

A very common assumption is that somewhere there is a correct learning rate, a number that experts know and that works for any finetuning job. The assumption is easy to reach. Tutorials often show one specific value, and it feels natural that a best setting should exist, the way a recipe has a best oven temperature. The learning rate does not work that way. The right value depends on the model, the data, and where training currently stands, so the goal is to find a workable value for your run rather than to recall a universal one.

A useful way to hold each of these settings in your head is to ask two questions: what does this setting control during training, and what goes wrong when it is set badly? The sections that follow answer those two questions one knob at a time, beginning with the learning rate.

The learning rate sets how fast the model's parameters change with each learning step. A helpful picture is learning as a walk toward a goal, where the learning rate is the length of each step. Short steps are safe but slow: if the steps are too small, reaching the goal takes too long. Long steps cover ground quickly, but they can carry you past the goal. If the steps are too big, the model can overshoot the goal over and over and, as a result, may never converge.

In this expression, $\Delta w$ is how far a weight moves in one update, $\eta$ is the learning rate, and $g$ is the gradient, which points in the direction that reduces the loss. Read the expression as a rule about step length: the learning rate scales the gradient before it is applied. Double the learning rate and, for the same gradient, each update moves the weight twice as far.

Trace the diagram as a sequence. The weights start where they are. The gradient tells the update which direction to move. The learning rate decides how big that move is. The weights end in a new position. When the learning rate is large, the same gradient produces a big jump, which is how an update can overshoot the minimum. When it is small, the jump is so tiny that the weights hardly change from one update to the next.

Experiments are much easier to interpret when you watch the loss curve, which records the loss over the course of training. The source describes two diagnostic patterns. If the loss curve fluctuates a lot, the learning rate is probably too big, because each update overshoots and the loss bounces around. If the loss curve is stable but takes a long time to decrease, the learning rate is probably too small, because each update moves the model only a little.

Read the two columns side by side. The left column shows what a too-large learning rate looks like: the curve zigzags because every update lands on the far side of the minimum. The right column shows a too-small learning rate: the curve looks calm, which can make it tempting to accept, but it falls so slowly that training wastes time. Neither column means the run has failed outright, but each tells you which direction to adjust.

Because the right step length depends on the situation, the source recommends experimenting across the typical range, which runs roughly from $10^{-7}$ to $10^{-3}$, and seeing which value works best for your run. Each experiment is a short trial at one learning rate, and you compare their loss curves. You are looking for the value whose loss curve decreases steadily, not the one that looks best on a single number taken at one moment.

The source also offers a practical starting point rather than a blind guess. Take the learning rate that was used at the end of the pre-training phase and multiply it by a constant between 0.1 and 1. That gives you a sensible neighborhood to search. The multiplier is a choice you make, and the result is a range to explore with your own loss curves.

Once the search is running, the learning rate is not necessarily fixed for the whole training run. You can use larger learning rates at the beginning and smaller learning rates near the end. An algorithm that determines how the learning rate should change throughout training is called a learning rate schedule. A schedule is therefore just a rule that maps training progress to a learning rate, and it lets you act on the diagnostic patterns as training moves forward.

The batch size determines how many examples the model learns from in each step before it updates its weights. Batch size affects both stability and speed, and it is limited by the hardware you have. A batch size that is too small, such as fewer than eight examples, can lead to unstable training. A larger batch helps aggregate the signals from different examples, which produces more stable and reliable updates.

Bigger is not free. In general, a larger batch lets the model get through the training examples faster, but it also needs more memory to run. Because memory is finite, batch size is capped by the hardware you use. This is where the trade-off between cost and efficiency appears: more expensive compute allows faster finetuning. The source notes that, at the time of writing, compute remains a bottleneck. Models are often so large, and memory so constrained, that only small batch sizes fit.

That creates a tension. Small batches are what fit in memory, but small batches can produce unstable weight updates. The technique that addresses this tension is gradient accumulation.

Gradient accumulation changes when the weights get updated, not how many examples each batch holds. Instead of updating the model weights after every batch, you accumulate gradients across several batches and update the weights once enough reliable gradients have been collected. The weight update is then based on information gathered from several batches, which helps when each individual batch is too small to give a stable update on its own.

Read the diagram as a flow with a pause in the middle. Each batch contributes its gradients to a running total, and the weights change only at the end of that stretch. The batches still pass through the model one at a time, so the memory limit that forced you to use small batches still applies. The update you finally make, however, reflects several batches rather than one.

An epoch is one pass over the training data, so the number of epochs determines how many times each training example is trained on. Small datasets often need more epochs than large ones. For a dataset with millions of examples, one or two epochs might be sufficient. A dataset with thousands of examples might still improve after four to ten epochs.

The gap between the training loss and the validation loss gives you hints about the right number of epochs. If both losses keep decreasing steadily, the model can benefit from more epochs and more data. If the training loss still decreases but the validation loss increases, the model is overfitting to the training data, and you might try lowering the number of epochs. Watching the two curves together, rather than the training loss alone, is what reveals overfitting.

In instruction finetuning, each training example consists of a prompt and a response, and both can contribute to the model's loss during training. At inference time, however, users usually supply the prompt, and the model only needs to generate the response. Because of this, response tokens should contribute more to the loss than prompt tokens. The prompt model weight is the setting that controls how much prompts count toward the loss compared with responses.

The two extremes are instructive. At 100 percent, prompts contribute to the loss as much as responses, so the model learns equally from both. At 0 percent, the model learns only from responses. The default is typically 10 percent, which means the model learns a little from prompts but mostly from responses. The low default keeps the focus on the part of the example the model must produce at inference time.