Interpreting Generated Feature Names

A linear model is only as useful as the table of numbers it receives. Before a model can learn anything from the Boston Housing data, the data has to be loaded, divided into parts, rescaled, and expanded into a richer set of columns. Each of those stages changes the table in a specific way, and a mistake at any one of them can quietly produce a model that looks fine but learns from the wrong numbers.

This article follows that preparation from start to finish. You will trace each stage, predict what it should produce, and then check the result against what the program reports. The central skill is reading the generated feature names. Once you can tell which original feature or pair of features each output column came from, you can check any later result against its input and locate the step where something went wrong.

The first stage imports three tools from scikit-learn, loads the dataset, and divides it into a training portion and a test portion. Read the code from top to bottom and notice that every name on the left of an equals sign is a new object created by the step on the right. The split is performed by train_test_split, which receives the input data and the target values together and returns four objects. The random_state argument fixes the random choices so that the same split is produced each time the code runs.

In the loaded dataset, the rows of boston.data are the samples and its columns are the features. boston.target holds the value the model is meant to predict. The split keeps inputs and targets aligned, so every row of X_train still belongs with the matching entry of y_train. The scaling lines come after the split, and that ordering matters, as later stages will show.

The source rescales the data with MinMaxScaler so that its values lie within a common range. Rescaling matters here because the polynomial step multiplies features together. If one feature has values far larger than the others, its products would dominate the new columns. Scaling first puts the original features on comparable footing before they are combined.

The two scaling lines follow a pattern worth memorizing. The scaler is created once. Then fit_transform is called on the training data, which both learns the scaling parameters from that training data and applies them. For the test data, the code calls transform only, which applies the parameters already learned from the training set. The test set is therefore scaled with the same rule as the training set, and the scaler never looks at the test data while it is being fitted.

The next stage asks the transformer to create new columns from the existing ones. The degree argument sets the largest number of original features that may be multiplied together in one new column. With degree=2, each output column is either an original feature, a constant, a feature multiplied by itself (its square), or a product of two different original features. The transformer is fitted on the scaled training data, and that same fitted object is then used to transform both the scaled training data and the scaled test data.

The printed shapes show the change in width. The original training matrix has 13 columns, and the expanded one has 105. The number of rows stays at 379 because each sample still produces exactly one row. Only the description of each sample becomes longer. Each of the 13 original features contributes a square term, and each pair of different features contributes one interaction term.

This expression is a way of checking the count rather than a formula printed in the source. Here n is the number of original features and N is the number of output columns. It counts one constant column, then the n original columns, then the squares and the distinct pairwise products. The term n(n+1)/2 combines the n squares with the n(n-1)/2 distinct pairs, since n + n(n-1)/2 = n(n+1)/2.

A table of 105 unlabeled numbers is hard to interpret, so the transformer can report which input produced each output. The get_feature_names method returns those labels. The labels are not arbitrary. The first is a constant, the next thirteen are the original features named x0 through x12, and the names after those show the squares and products. In the printed output, a caret indicates a square, and a space indicates a product of two features.

The full sequence follows a regular order. Names beginning with x0 come first, then x1, and so on. For each starting feature, the square comes first and is followed by the products with each later feature. The diagram below reads the list as a lookup table, showing selected positions so you can see the pattern.

Read the diagram as a lookup table. Position 0 is the constant, positions 1 through 13 are the original features, and position 14 begins the squared and product terms built from the first feature. Because the names record the origin of each column, you can check any coefficient or diagnostic later against the input it came from.

Preparation errors often do not crash the program. The shapes can look correct even when the values are wrong. A shape check confirms the column count but not the values, so the most useful habit is to ask which stage produced the surprising number and to trace each stage in the order the data passes through it.