Data Preprocessing Techniques

A common assumption among people new to machine learning is that preprocessing is housekeeping. The data is already numbers, the model is the interesting part, and rescaling a few columns seems unlikely to change much. Under this belief, scaling is something you do to tidy up before the real work begins, and you would not expect it to decide whether a model performs well or poorly.

This article tests that belief directly. The same support vector classifier is fitted on the same cancer dataset twice: once on the raw features and once on scaled features. The model settings and the data split stay the same, so the only thing that changes is the scale of the inputs. Keep one question in mind as you read: if the model and the split are held fixed, how much can changing only the scale of the inputs matter?

The experiment has two stages, and the first is a baseline. The data is split into training and test portions. An SVC with a large regularization setting is fitted directly on the raw training features, and its accuracy is scored on the raw test features. This number is the reference point that every later result is compared against.

The second stage keeps the same split and the same SVC settings but inserts a scaling step. A MinMaxScaler is created and fitted on the training features only. It is then used to transform both the training features and the test features. The SVC is fitted on the scaled training data and scored on the scaled test data. Each state change in this sequence matters, so track which object holds which data at each point.

Notice that the classifier variable is reused. In the source, the same svm object is fitted a second time, on the scaled training data. The second fit replaces the first model rather than adding to it. When you read the trace, you should therefore compare the two scores as results of two separate fits, not as one model that improved over time.

Preprocessing is the step that transforms raw data into a format a model can use. Scaling is one kind of preprocessing, and the source treats it as a case where the effect on a supervised model can be large. The sequence of operations matters. The scaler learns its parameters from the training data, and then it applies that same learned transformation to every dataset the model will later be evaluated on.

Every preprocessing class in scikit-learn follows the same two-method pattern. The fit method learns from data, and the transform method applies what was learned. Fit is called once, on training features only. Transform is called on the training features and then on the test features, using the parameters learned during fit.

This small example shows why the fit and transform separation exists. If the scaler were refitted on the test values alone, the test value would be the minimum and maximum of its own data and would be mapped to 0 or 1 by construction. The model would then be scored on inputs that were transformed differently from the training inputs it learned from.

scikit-learn offers more than one scaler, and they differ in what they aim to produce. Both answer the same practical question, which is how to put features on a comparable scale, but they describe the result in different terms. The diagram below compares them side by side.

The reason swapping them is easy is that both follow the same fit and transform pattern. Changing the scaler therefore means changing the class name and nothing about the surrounding logic. The StandardScaler version of the experiment follows the same steps as the MinMaxScaler version.

The right scaler is an empirical question for the model and dataset at hand. The source does not establish a universal ranking, so the practical habit is to keep the fit and transform structure fixed and swap only the scaler class. Then compare the evaluation scores on held-out test data, which the scaler never saw during fitting.

Scaling changes the values of features but keeps the number of features the same. A different family of unsupervised transformations changes the representation itself. Dimensionality reduction aims to reduce the number of features in a dataset while preserving important information.

The source names three common motivations: visualization, compressing the data, and finding a representation that is more informative for further processing. These motivations pull in different directions, so it helps to see what each one asks of the reduced representation.

The source introduces three algorithms that serve these motivations. The first is one of the simplest and most widely used methods, and it applies to all of the motivations above. The other two are more specialized, and each leans toward one particular goal. Comparing how each is positioned is more useful at this stage than memorizing the descriptions.