Interactions Between Features

A linear model combines its input features with weights and adds them up. When the input has one column, that means the model draws a straight-line relationship between the input and the target. If the real relationship between input and target curves, a straight-line combination of the original column may fit poorly no matter how well the weights are chosen.

One way to give a linear model more flexibility is to add new columns that are powers of the existing ones. The new columns are called polynomial features. The model still adds weighted columns together, but now some of those columns are curved functions of the original input. This article uses a code-first path: you run the transformation, predict what the output should look like, and then use the output to explain how each column was built. The goal is to learn to read the result and to find the exact place where your expectation and the real output disagree.

Suppose the input has a single column called x. The transformer produces a new array in which the first column still holds the original values, and the remaining columns hold the same values raised to higher powers, up to the chosen degree. So a degree of 3 gives you x, then x squared, then x cubed, written as $x^{1}$, $x^{2}$, and $x^{3}$. The first output column is always the original feature, and everything after it is a higher power of that feature.

The transformer is a tool that expands a single column into a set of columns. Each new column is a fixed, predictable function of the original column, so you can work out in advance what the output must contain.

The class is imported from scikit-learn's preprocessing module. You create an object that records the settings you want, call fit on your data, and then call transform to produce the new array. The degree setting controls the highest power. The include_bias setting controls whether a constant column of ones is also added at the front. The source example uses a degree of 10 and turns the constant column off.

A shape tells you how many rows and columns the array has. For a single input column, the number of rows equals the number of samples, and the number of columns depends on the degree and on include_bias. With include_bias turned off, a single input column produces one column per power, so a degree of 10 produces 10 features, and the transformed array keeps one row per sample. Turning include_bias on adds one more leading constant column. The table below shows how the column count changes as you vary the two settings.

A shape tells you how many columns exist but not what they mean. The transformer reports the meaning of each column through its feature-name method, which gives the exponent that each feature was raised to. In the source this method is called get_feature_names. Current scikit-learn releases expose the same idea as get_feature_names_out. Its output lists the original feature first, then each power of it, in the same order as the columns of the array.

Read the names by position. Column 0 is named x0 and holds the original values, so its name tells you it is power 1. Column 1 is named x0^{^2} and holds the squared values. Column 2 is named x0^{^3} and holds the cubed values. The names step through the powers in order, and that ordering is the check you will use later.

The same pattern explains why the columns can differ so much in size. Once you see the generated columns, their magnitudes may be very different. The source points out that the second column reaches values above 20,000, which is far larger than most of the others. This follows directly from raising values to powers. A value larger than one in absolute value grows with each higher power, while a value between negative one and one shrinks toward zero. For example, an input of 300 gives $300^{2} = 90000$ in the second column, which is far larger than the original 300.

When a polynomial result looks wrong, check three things in order: the shape, the first column against the original input, and the feature names. The first place where the observed output disagrees with your expectation tells you where the problem is. If any of these checks fails, the fault is in the setup of the transformer, not in the model that comes afterward.