The L2 Norm and Vector Length

Training a large language model is a long sequence of small corrections. On each step, a batch of training text produces a loss, backpropagation works out a gradient for every parameter, and the parameters move a little in the direction the gradient indicates. Most of the time these movements are reasonable in size. But now and then a batch can produce a gradient that is far larger than usual, and if that gradient is applied at full size, one step can push the parameters somewhere they should never have gone.

Follow one gradient through a standard training loop. The loss for a batch is calculated, and backpropagation then computes the gradients. Those gradients are what would normally be used to update the parameters. Clipping is applied to the gradients after backpropagation has produced them and before they are used for the update. That placement is why clipping acts directly on the updates that backpropagation generates: it is a check on the gradients, not on the data or the loss.

To decide whether a gradient is too large, we need a single number that describes its size. In gradient clipping, that number is the norm of the gradient. The norm measures the length, or magnitude, of the gradient vector within the model's parameter space. The source specifies the L2 norm, which is also called the Euclidean norm. It is computed in three steps: square each entry, sum the squares, and take the square root of the total.

Geometrically, the sum of squares is the same calculation that gives the length of an arrow drawn from the origin to a point. In two dimensions, for a point with coordinates 3 and 4, the squares add to 9 + 16 = 25 and the square root gives a length of 5. The L2 norm extends that same idea to a gradient with any number of entries.

Clipping is a short sequence of decisions. First, compute the L2 norm of the gradient. Second, compare that norm to max_norm. If the norm does not exceed max_norm, nothing changes and the gradient is used as it is. If the norm does exceed max_norm, a scaling factor is computed as max_norm divided by the norm, and every entry of the gradient is multiplied by that factor. Because the factor is applied to the whole gradient, the new norm comes out exactly equal to max_norm.

The source works through a gradient matrix G whose L2 norm is 5. With a max_norm of 1, that norm exceeds the threshold, so clipping is required. The source does not list the entries, so the worked example below uses a small matrix with the same norm of 5 so that every step can be checked.

Notice what clipping does not do. It does not throw the gradient away, and it does not change which way the gradient points. Every entry is scaled by the same factor, so the ratio between entries is preserved. The batch still contributes to learning, but its influence on the parameters is bounded by max_norm.

In PyTorch, gradient clipping is available through the clip_grad_norm_ function. Setting max_norm=1.0 in that function ensures that the norm of the gradients does not surpass 1.0. The source describes this as the practical way to enforce the threshold you just worked through by hand, so the arithmetic in the previous section is exactly what the function performs on the model's gradients. The example call above shows where it belongs: after backpropagation has filled in the gradients, and before the optimizer uses them.