Numerical Representations of Model Values

A trained model is, in large part, a collection of numerical values. Each of those values has to be stored somewhere in memory, and the amount of memory those stored values occupy is called the model's memory footprint. Large models can have billions of parameters, so the way each value is stored has a large effect on how much memory the model needs.

The central relationship in this article is simple: the fewer bits needed to represent a model's values, the lower the model's memory footprint will be. The techniques covered here all act on that relationship. Before looking at them, it helps to test your intuition about how much the bit width matters.

Storage cost is the number of values multiplied by the number of bits used for each value. Ten billion parameters at 32 bits each is 320 billion bits. Since 8 bits make one byte, that is 40 billion bytes, which is the 40 GB the source gives for the weights. The same ten billion parameters at 16 bits each is 160 billion bits, or 20 billion bytes, which is 20 GB.

A common assumption is that quantization always means converting a model's values into integers. Under that assumption, a technique that stores values in a floating-point format with fewer bits would not count as quantization at all. The belief sounds reasonable, because quantization is often associated with mapping values onto a discrete set of integer levels.

The belief is partly right. Strictly, quantization refers to converting values into an integer target format. In practice, however, the word is used more broadly, to cover all techniques that convert values to a lower-precision format. The source adopts this broader usage so that its terminology matches the wider literature, so when you read other material on this topic, expect the broad meaning unless the text specifically says integer.

To see what the strict meaning looks like in practice, consider mapping a continuous floating-point value onto a fixed set of integer levels. The source describes quantization in these terms: values are mapped onto a discrete set of integer levels. The example below uses a simple illustrative scheme, not one the source specifies. Values between -1 and 1 are scaled by 127 and rounded to the nearest integer, so each level is a whole number from -127 to 127.

Quantization is applied to specific parts of a model, so the next question is which parts to target. The source identifies the major contributors to a model's memory footprint during inference as the model's weights and its activations. The ideal target is whatever consumes the most memory, but the choice also depends on what can be quantized without hurting performance too much.

Weight quantization is more common than activation quantization. The source gives the reason: weight quantization tends to have a more stable impact on performance, with less accuracy loss. Activation quantization is not ruled out, but weights are the usual first target when the goal is to reduce memory. Intuitively, a weight is a fixed value stored with the model, so one quantization decision applies to every input. An activation is computed fresh for each input, so its range and behavior can vary from one input to the next.

The other decision is timing. The source states that quantization can happen during training or after training. Posttraining quantization, abbreviated PTQ, means quantizing a model after it has been fully trained. The model's training is already complete, and quantization is applied to its finished values.

PTQ is by far the most common approach. It is also the approach most relevant to AI application developers, who usually do not train models themselves. A developer who takes a trained model and reduces its precision for deployment is working with PTQ, so the technique fits the situation of someone who consumes models rather than builds them from scratch.