Rescaling and Shifting Data

Imagine a dataset where one feature is a person's age, with values near 20 to 40, and another feature is an income, with values in the tens of thousands. Both numbers are perfectly valid, but they live on very different scales. This article answers two questions: what can be done about that difference, and why it matters for some algorithms more than others.

The reason this adjustment is needed comes from how some algorithms behave. Neural networks and support vector machines are very sensitive to the scaling of the data. If one feature ranges over thousands while another ranges over fractions, the algorithm can be affected by those raw magnitudes rather than by the patterns the data contains. A common response is to adjust the features so that the data representation fits these algorithms better.

The two columns above differ by a factor of three in range, and the income values are in the thousands while the age values are in the twenties. Neither feature is wrong. The problem is only that the algorithm sees their raw sizes directly, so the representation needs to change before the algorithm receives it.

The adjustment described in the source is often a simple per-feature rescaling and shift. Shifting moves all of a feature's values by the same amount. Rescaling stretches or compresses them by a chosen factor. Per-feature means that each feature is handled on its own: the feature's own values determine how it is shifted and how it is scaled. A different feature in the same dataset can receive a completely different shift and scale.

The formula above is one concrete way to perform a per-feature shift and rescale. The source illustrates the general idea of shifting and scaling each feature rather than prescribing this exact recipe. Here, x is an original value in one feature, min(x) is the smallest value in that feature, and max(x) is the largest. The subtraction is the shift, which moves the minimum to zero. The division by the range is the rescaling, which makes the largest value end up at one.

Notice that the transformation changed the numbers but not their order or their relative positions. The minimum moved to zero, the maximum moved to one, and the middle value landed exactly halfway between them, because 30 sat halfway between 20 and 40.

Now apply the same procedure to the income feature, which holds 30000, 45000, and 90000. Each feature receives its own shift and its own scale, so the income feature is handled independently of age.

Notice what happened. Both features now sit on the same range from zero to one, even though their raw magnitudes were very different before. The relative spacing inside each feature is preserved: the gap between 30000 and 45000 is still a quarter of the total spread of incomes, just as the gap between 20 and 30 is still half the total spread of ages.

Because the adjustment is per feature, each column of the dataset has its own shift and its own scale. The age column was shifted by 20 and divided by 20. The income column was shifted by 30000 and divided by 60000. Mixing these up is an easy error, because the numbers look like they belong to one process when they belong to two separate features.

It helps to see scaling as one stage between the raw data and the algorithm. The raw feature values come in first. Each feature receives its shift, and its scale is applied. The adjusted values are what the algorithm finally receives. The algorithm never needs to know how the adjustment was done; it simply works with the representation it is given, which is the purpose of the adjustment in the first place.

Scaling is unsupervised because it does not use supervised information such as target labels. The shift and scale for a feature are computed from that feature's values alone. The labels never enter the calculation. This remains true even when the scaled features are fed into a supervised model, as the pipeline above shows. The scaler sits in front of the supervised model and touches only the feature values, while the labels are used only by the model.