Feature Scaling with MinMaxScaler

This article follows one realistic case from start to finish: a Ridge regression model built on 13 input features, with a polynomial expansion applied to those features. The aim is to watch the shape of the data change at each step, so that when a score or a shape looks wrong you know which line of code to inspect. The main lesson is that the cause of a failure is usually one step earlier than the line that complains.

The pipeline has three stages that depend on each other. First, every feature is rescaled to lie between 0 and 1 with MinMaxScaler. Second, PolynomialFeatures with degree=2 expands the scaled features into squares and pairwise products. Third, a Ridge model is fitted on the expanded matrix. The source states that scaling must come before the polynomial expansion, because the expansion multiplies features together. The scale of each input therefore directly affects the scale of every product built from it.

The first step loads the data, splits it into training and test sets, and rescales every feature with MinMaxScaler. The scaler is fitted on the training data, and the test data is transformed with that same scaler. After this step the training matrix still has 13 columns, but every value now lies between 0 and 1.

The second step fits a PolynomialFeatures object with degree=2 on the scaled training data. It then transforms both the training and the test sets with that same fitted object. Running the code prints two shape lines. The training matrix started with 13 columns, and the question to hold onto is what the second number in the printout means once the expansion has happened.

The third step fits a Ridge model on the expanded features. The source fits Ridge twice in the same code: once on the scaled original features and once on the polynomial features. The variable name ridge is reused, so the first model is overwritten by the second. Each model is scored on test data that was transformed the same way as its own training data. The scaled originals go with the scaled originals, and the polynomial matrix goes with the polynomial matrix.

A count of 105 tells you very little on its own. The source describes the 105 columns as a constant, the 13 original features, the squares of each original feature, and all possible products of two different original features. The phrase degree=2 means that every output column is a product of at most two original features. It does not mean that the model is restricted to quadratic curves in a single variable, and it does not mean that only two features are used. Every original feature is still present, and each new column is built from them.

The breakdown can be checked by counting. There is one constant column, and the 13 originals are kept as they are. Each original is also squared, which gives 13 more columns. Every pair of two different features contributes one product, and the number of such pairs from 13 features is $\binom{13}{2} = 78$. Adding these together gives $1 + 13 + 13 + 78 = 105$, which matches the shape printout.

The shape alone does not say which column is which. The get_feature_names method solves this. It returns a list of strings in the same order as the columns of the transformed matrix, so the position in the list and the position in the matrix are the same thing. The list begins with the constant '1', followed by the 13 original features named x0 through x12, and then the squares and products.

The notation in the names is read as follows. The caret marks a square, so x0^{^2} means the first feature multiplied by itself. A space between two names means multiplication, so x0 x1 means the first feature times the second. The name therefore records exactly which original features were multiplied to create the column. The source lists the ordering rule: column 0 is the constant, columns 1 through 13 are the originals in their input order, and column 14 is the square of the first feature. The columns after it pair the first feature with each of the other features in turn. Once you know this rule, you can locate any column without scanning the whole list.

On the source's single split, Ridge regression improved from a score of 0.621 using only the scaled original features to 0.753 using the polynomial features. The improvement comes from the interaction and squared terms, but only when the test data is transformed with the same fitted objects as the training data. Ridge is a linear regression model with an L2 penalty. It minimizes the sum of squared errors plus a penalty on the size of the coefficients, which helps it cope with the strong relationships between columns that a polynomial expansion creates.