Principal Components

Once you have fitted PCA and projected your data onto the new axes, the next question is usually what those axes mean. A scatter plot of the first two principal components can look clean and informative, yet the axes rarely carry a label you can read off directly. Each axis points along a direction in the original data space, and that direction is not one of the measurements you originally collected. It is a mixture of them.

This article closes that gap. You will learn where the information describing each axis is stored, how to lay it out so the numbers can be read, and why the mixing of features is the source of most of the difficulty. The goal is not just to print the numbers but to know what they say about the data and to recognize when a reading of them is wrong.

After a PCA object has been fitted, the directions it found are kept in an attribute called components_. You do not need to compute them again; you read them from the fitted object. Each direction is one principal component, and the components are ordered by importance. The first one captures the largest share of the variance in the data, the second captures the next largest, and so on.

The shape tells you the layout before you look at a single value. The first number is the count of components, here 2. The second number is the count of original features, here 30. So the array has one row per component and one column per feature. Each row is a direction, and each column is a feature that the direction draws on.

Reading the matrix correctly depends on remembering what rows and columns stand for. A row answers the question: which combination of features makes up this component? A column answers the question: how much does this one feature contribute to each component? Reading in the wrong direction produces confident but meaningless statements, so check the orientation before anything else.

A principal component is never built from a single measurement. Its direction is spread across all of the original features at once, so every feature has some weight in it. This is the reason the axes are hard to name. An axis called after one measurement would be easy to explain; an axis built from thirty interacting measurements is not, because its meaning is a pattern across all of them.

In this notation, the original features are $x_1$ through $x_{30}$, and each weight $w$ is one entry of components_. The first row of the array holds the 30 weights for the first component. Each weight says how strongly that feature pulls the component's direction. A large weight in absolute value means the feature matters more to that direction; a weight near zero means the feature barely contributes to it.

The formula above is the pattern the array encodes: one weighted sum per component, with a weight for each of the 30 features. Every term is present, which is why the component is a combination of all of them rather than of a chosen few.

The weights can also disagree with one another in sign. Some features may push the component in one direction while others push it the other way. A mixed-sign pattern tells you the component is contrasting groups of features rather than measuring one overall quantity. The same sign shared by several features means those features tend to rise and fall together along that axis, while features with opposite signs trade off against each other.

A table of 60 numbers is hard to scan. A heat map turns each coefficient into a colored cell, so you can see the sign pattern and the relative size of the weights at a glance. The source demonstrates this with the first two components of a breast cancer dataset, where the columns are the 30 features and the two rows are the components.

In the heat map, the first component has a single sign across its row, so its colored cells are uniform in tone, which marks a general correlation among the features. The second component's row is split between two tones, reflecting its mixed signs. Both rows touch every column, which is the visual version of the point that every feature participates.

To find features that move together, look for columns whose cells share a tone within the same component. Features that land in the same tone group are the ones that rise and fall together along that axis. Features in opposite tone groups trade off against each other. Before you name a group, though, check the printed numbers, since the color alone does not give you the exact weights.

When an interpretation disagrees with what you expected, the useful question is where in the chain the mismatch enters. The chain runs from the fitted object, to the components_ attribute, to the printed numbers, to the written description. Most errors happen at one of two links: the orientation of the array, or the text you wrote about the numbers.