Data Transformation

Suppose you have a dataset, a model, and a pile of candidate transformations. Which one should you apply? The honest answer is that the choice is part judgment and part experiment. The best transformation depends on the particular pairing of data and model, and nothing about one pairing guarantees the same result for another. A transformation that helps one model on one dataset may do nothing, or even hurt, when the model or the data changes.

Throughout this article, one question guides every decision: given a model, what should be changed in the data, and what should be left exactly as it is? The answer changes with the model family, so the question is not 'which transformation is correct?' but 'which transformation suits this model and this data?'

The first thing to check is the family of model you intend to use. The source makes a clear contrast: tree-based models are less sensitive to transformations, while linear models often depend on them. This means the same transformation can be essential for one model and close to irrelevant for another.

Because the right answer depends on the pairing, you cannot pick a transformation from a rule of thumb alone. You have to combine what you know about the model with what you observe in the data, then test the result. The next sections walk through the two places where this judgment is applied: the features, and the target.

Not every feature needs transformation, and different features may need different transformations. In a clean teaching example, every feature might share the same statistical properties, which makes one uniform treatment look natural. Real datasets rarely behave that way. Usually only a subset of the features needs transforming, and the rest should be left as they are.

Even among the features that do need work, one treatment rarely fits all of them. A feature whose values are spread across several orders of magnitude may need a different correction from a feature whose values already sit in a narrow band. For that reason, a sensible workflow inspects features one at a time, asks what each one looks like, and chooses a transformation for that feature specifically.

Read the flowchart as a checklist, not a fixed law. The first check asks whether the model is one that depends heavily on transformations. The second asks whether the feature's values are spread widely enough to need correction. Only features that pass both checks are candidates for a transformation, and even those must be confirmed by testing.

The discussion does not stop at the features. In a regression task, the target variable $y$ is the quantity the model is trying to predict. The source notes that it is sometimes worth transforming $y$ as well. This is a different kind of change from feature transformation. Transforming a feature alters what the model sees as input; transforming the target alters what the model is asked to predict.

The source singles out one common situation: predicting counts, such as the number of orders a customer places. Counting outcomes are a fairly routine regression task, and applying the transformation $\log(y + 1)$ to the target often helps. Here $y$ is the original count, and the transformed target is the logarithm of that count plus one.

The reason for the $+1$ is practical. A customer can place zero orders, and $\log(0)$ is undefined. Adding 1 before taking the logarithm keeps the transformation defined at zero, because $\log(0 + 1) = \log(1) = 0$. The same step also compresses large values, which is the other property the source highlights.

The index map above uses the natural logarithm for illustration. The choice of logarithm base changes the numbers on the scale but not the order of the values, so the pattern of spreading and compression is what matters. Notice that the gap from count 0 to count 1 on the transformed scale is 0.69, while the gap from count 5 to count 20 is 1.25. Large counts are squeezed together, and small counts keep more of their separation.

Count targets are often skewed: many observations sit at small values, and a few sit far out at large values. A model fitted directly to such a target can be pulled hard by those few large observations. Applying log(y + 1) reshapes the target so that the large values no longer dominate in the same way.

A good transformation choice is confirmed by results, not by habit. Compare the model's performance with and without the transformation you chose, using the same evaluation procedure each time. If a transformation does not improve the outcome, the source describes finding the right transformation as somewhat of an art, so it is reasonable to try another.

This turns the choice into an experimental loop. Pick a candidate transformation, fit the model with it, evaluate the result against the untransformed baseline, and keep the candidate only if it earns its place. Repeat with another candidate if it does not.