Histograms for Feature Analysis

Imagine you have a dataset with many measurements per sample and two known classes. You reduce the measurements to two principal components, draw a scatter plot, and color each point by its class. The two colors form neat, separate clusters. It is natural to conclude that the projection was built to separate the classes. That reasoning feels correct because the picture is so tidy, but it is wrong, and the mistake changes how you should judge the plot. This article starts from that misconception, tests it with a prediction, and then rebuilds the workflow step by step so you can see exactly where class information enters and where it does not.

Visualizing data with many features is hard. With two features, a single scatter plot shows everything. With a handful, a pair plot gives a partial view by showing every combination of two features side by side. The Breast Cancer dataset has 30 features, and a pair plot built from all of them would contain so many panels that no one could inspect them in detail. The problem is not only the number of plots. Even a complete set of pairwise views would miss interactions that involve three or more features at once.

A cheaper first look is to compute one histogram per feature, separated by class. Each histogram counts how many samples fall into each range of values for that feature. Overlaying the two classes shows how well that single feature distinguishes them. A feature whose two histograms mostly overlap is weak at separating the classes, while one whose histograms are largely disjoint is a stronger candidate. Histograms answer a per-feature question, though. They say nothing about how features relate to one another, and that gap is what PCA is meant to fill.

Before PCA is applied, the features are scaled so that each one has unit variance. The source uses StandardScaler for this. The scaler is fitted to the raw data, and then the same fitted scaler transforms the data. The source treats this scaling step as necessary for PCA to work effectively, so it belongs in the workflow before any rotation is learned. Scaling comes first because the rotation is learned from the features as they enter PCA; the order of these steps is part of the workflow, not a detail to skip.

Using PCA resembles using any other preprocessing transformer in scikit-learn. You create the object, call fit to learn the principal components from the data, and then call transform to apply the rotation. By default PCA keeps every component, so it only rotates and shifts the data without reducing its width. To reduce dimensionality, you pass n_components when you construct the object. Setting it to 2 keeps the first two principal components, which is what makes a two-dimensional scatter plot possible.

Notice the argument list of fit. It receives only the scaled feature matrix. The class target is not passed in, and that absence is the key to the unsupervised behavior. The labels are used at exactly one point: the plotting stage. There, the first principal component is placed on one axis, the second on the other, and each point is colored according to its known class. Colors are a presentation choice layered on top of a projection that was already complete before the labels were used.

PCA is an unsupervised method. It does not use class information when it finds the rotation. Instead, it examines the correlations among the features. The original feature axes describe one measurement each, while the principal component axes are new directions in the same space, chosen according to how the features vary together. The direction of each new axis is determined by those correlations among the scaled features, not by which points belong to which class.

Once the two components are computed, the plot places the first component on the horizontal axis and the second on the vertical axis, with each point colored by its class. In the source example the two classes separate fairly well in this two-dimensional view. That observation suggests that even a linear classifier, which would learn a straight dividing line in this space, could perform reasonably well at telling the classes apart. The word suggests is important. A projection that looks separable is evidence, not proof of how a classifier will perform.

The same plot also shows spread. In the source example the malignant points are more dispersed than the benign points, which matches what the per-feature histograms suggested earlier. This link is useful: the histograms looked at features one at a time, and the PCA plot combines them into two summary directions. When the pipeline is working correctly, the two views should tell a consistent story.

When a PCA plot looks wrong, the workflow itself is the debugging map. Each stage has a shape and a meaning. Ask at which stage the shape or meaning first stopped matching what you expected. Check loading first, so that the matrix has the expected 569 rows and 30 feature columns. Check scaling next, so that the features are on a comparable scale and the scaler was fitted before it transformed. Check the fit call, which should receive only the scaled matrix. Check the transform output, which should have two columns. Finally, check the plotting step, where labels should only change colors, not positions.