Feature Scaling with StandardScaler

Scaling with StandardScaler gives each feature unit variance and is treated as necessary before PCA is fitted.

Programming

Why 30 Measurements Need Compressing

A scatter plot can show at most two measurements at once, one per axis. With two or three columns of data that limit is comfortable. With dozens of columns it becomes a real obstacle. The Iris dataset can be explored with a pair plot, a grid that places every pair of features against each other, but each cell of that grid shows only two measurements, so it gives only a partial picture.

The Breast Cancer dataset makes the problem sharper. It has 30 features, and the chapter's own count of pairwise plots for it comes to 420. No one can examine 420 plots closely enough to understand how the features relate to each other. The goal of this article is to see how Principal Component Analysis, usually called PCA, compresses those 30 measurements into a form that one scatter plot can display, and to understand what that compression keeps and what it discards.

Definition

PCA turns a high-dimensional dataset into a lower-dimensional one by capturing the main interactions between variables. This is what makes a dataset with 30 features viewable in a single two-dimensional scatter plot.

Before unpacking the mechanics, it helps to run the whole pipeline once and watch what it does. The program loads the Breast Cancer data, scales it, fits a PCA model that keeps two components, transforms the data, and prints the shape before and after. The steps are simple to type, but each one does a different job, and the rest of this article explains those jobs. Keep the question in mind as you read: when the program finishes, which numbers have changed, and which have stayed the same?

Tracing the Data Through the Pipeline

The pipeline has a single direction of travel. The raw data is scaled first, the scaled data is used to learn the rotation, and then the rotation is applied to produce the two-column output. Each arrow in the diagram below is one of these steps, and the label on each step names the object doing the work.

How the array's shape (samples by features) changes at each step, ending with two columns after PCA keeps two components.

Notice what does not change along this path. The row count stays the same from start to finish, because PCA changes which columns describe each sample and leaves the samples themselves alone. What changes is the number of columns. The original data describes each patient with 30 measurements. After PCA with two components, each row is described by two new values. Those two values are not copies of any original measurement. They are coordinates in a rotated space, which is why the two axes of the final scatter plot do not carry the names of individual features.

Why Scaling Must Come First

The first transformation in the pipeline is not PCA at all. It is scaling, and the source treats it as a required step before PCA is fitted. StandardScaler gives each feature unit variance, which means every column is brought to a common spread before the rotation is learned.

The reason this matters is that PCA looks for directions in which the data varies the most. If one feature has a much larger numeric range than the others, its variation is large simply because of the units it was measured in. Without scaling, features that happen to have larger numeric ranges can carry more weight than they should in the analysis. The components PCA finds would then reflect measurement units rather than the relationships between the variables.

Without scaling, the feature with the largest magnitude dominates the variance. After scaling, every feature has unit variance and contributes on a common footing.

Fit Versus Transform

Notice that the scaler is fitted with the data and then applied with transform to produce the scaled array. Fitting and transforming are separate actions. The fit call learns what the scaling needs to know, which for StandardScaler is the mean and standard deviation of each feature. The transform call applies that knowledge to produce the new array. The PCA model follows the same pattern, and seeing the two patterns side by side makes later steps easier to read.

The fit step computes per-feature statistics from the data. The transform step uses those stored statistics to produce the scaled output.

The separation matters because the statistics belong to the data used to learn them. If you fit a scaler on one set of rows and then fit a new scaler on different rows, the two scalers may hold different means and standard deviations, and their outputs will not be on the same footing. The fitted values are the reference that makes transformed data comparable. This is the reasoning behind applying a fitted scaler to new data with transform instead of fitting again on that new data.

Running the Pipeline and Reading the Shapes

The source's excerpt uses StandardScaler without showing its import line, so the imports are included below so the program runs on its own. The scaler is fitted with the data, the scaled array is produced with transform, and then a PCA object keeps two components and is fitted and transformed in the same three-step pattern.

python

from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

cancer = load_breast_cancer()

scaler = StandardScaler()
scaler.fit(cancer.data)
X_scaled = scaler.transform(cancer.data)

pca = PCA(n_components=2)
pca.fit(X_scaled)
X_pca = pca.transform(X_scaled)

print(cancer.data.shape)
print(X_pca.shape)

from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

cancer = load_breast_cancer()

scaler = StandardScaler()
scaler.fit(cancer.data)
X_scaled = scaler.transform(cancer.data)

pca = PCA(n_components=2)
pca.fit(X_scaled)
X_pca = pca.transform(X_scaled)

print(cancer.data.shape)
print(X_pca.shape)

Output (expected)

(number of rows, 30)
(number of rows, 2)

Notes on the code

load_breast_cancer returns the data as a table of samples and features
scaler.fit learns what scaling needs, and scaler.transform produces X_scaled
PCA(n_components=2) is the setting that reduces the output to two columns
pca.fit finds the rotation and pca.transform applies it to the scaled data

Read the two printed shapes as a before and after comparison. The first line describes the raw data, which has the same row count as the number of patients and 30 columns. The second line describes the projected data. The row count is unchanged, and the column count has dropped to 2. If you had left out n_components, PCA would keep all of the components by default, and the second shape would still show 30 columns, because the reduction happens only when you tell the object how many components to keep.

What do you think happens?

If you change the first shape to the second without the reduction step, what do you expect the printed shape of X_pca to be when n_components is left at its default?

(number of rows, 2)
(number of rows, 30)
(30, number of rows)

Reveal answer

Answer:

(number of rows, 30)

By default PCA keeps all of the principal components, so the number of output columns matches the number of input features. The row count is unchanged in every case.

Once the two-column result exists, it can be plotted. The chapter plots the first principal component against the second, with one point per patient. The class labels are used only to choose the color of each point. Here the two classes separate fairly cleanly, which suggests that a linear classifier, one that learns a straight dividing line, could distinguish them reasonably well. The labels were never used to find the rotation. PCA is unsupervised, so the labels appear only when points are colored in the projected plot.

Mistakes That Hide Unexpected Results

Common mistake
Expecting PCA to reduce the columns without setting n_components
What it looks like
```
pca = PCA() followed by fit and transform, then checking the shape of the output
```
Why it fails
By default PCA only rotates and shifts the data and keeps all of the principal components, so the output still has 30 columns.
Do this instead
Pass the number of components you want, such as PCA(n_components=2), when you need a reduced output.
Common mistake
Scaling after PCA instead of before it
What it looks like
Running PCA on the raw data and then scaling the two output columns
Why it fails
The source treats scaling as necessary before PCA is fitted. Features with larger numeric ranges would already have influenced the rotation before any scaling happened.
Do this instead
Scale the data first with StandardScaler, then fit and transform PCA on the scaled array.
Common mistake
Refitting the scaler on new data instead of reusing the fitted values
Why it fails
Each fit learns its own mean and standard deviation, so two fits on different rows produce outputs that are not on a common scale.
Do this instead
Fit the scaler once on the training data and apply transform to other data with the same fitted scaler.
Common mistake
Reading the two PCA axes as if they were original features

Best practice

When a PCA plot looks wrong, check the pipeline in order instead of guessing. Inspect the raw data first, then the scaled data, then the fitted PCA, then the transformed output. The first checkpoint where the values stop matching what you expect is the step that introduced the problem.

Practice Locating the Problem

MEDIUM

A student builds the pipeline with StandardScaler, PCA(n_components=2), fit, and transform, but the scatter plot shows one feature dominating the layout, with the two axes almost aligned to that feature's raw values. Using the four checkpoints (raw data, scaled data, fitted PCA, transformed output), decide which checkpoint to inspect first and what you would look for there.

Hints

Think about what the scaler is supposed to do to every column before PCA starts looking for directions of variation.
Check whether the scaled array actually has comparable spread across columns before you examine the PCA step.
If the scaled data looks fine, move forward to the fitted PCA and then the transformed output.

Your attempt (not saved)

Key Takeaways

PCA compresses a high-dimensional dataset, such as one with 30 features, into a small number of components that a two-dimensional scatter plot can display.
StandardScaler gives each feature unit variance, and the source treats this scaling as necessary before PCA is fitted, so the components reflect relationships between variables rather than measurement units.
fit learns the statistics or the principal components, transform applies them, and n_components sets how many components are kept, since the default keeps all of them.
The row count stays the same through the pipeline while the column count changes, and the new columns are coordinates in a rotated space rather than copies of original features.
When an unexpected result appears, inspect the raw data, the scaled data, the fitted PCA, and the transformed output in that order to find where it starts.

Related concepts

More concepts starting with F