Visualizing Cross-Validation Results with Heat Maps

A recurring question in model building is how much feature engineering to apply. If you add polynomial features, which degree should you choose? Should you add polynomial features at all? The most reliable answer is to let the outcome decide. Try several candidate settings, measure each one with cross-validation, and keep the setting that scores best. This article shows how a single grid search can make that comparison for two settings at once, one from the preprocessing stage and one from the model.

The code that drives this comparison assumes a pipeline called pipe, built earlier, and a training split made of X_train and y_train. Keep two things in view as you read: the keys in param_grid, and what the fit call receives. Each of them is explained in the sections that follow.

A pipeline contains several named steps, and each step may have parameters of its own. A grid search needs an unambiguous way to say which step a parameter belongs to. The convention used here is to write the lowercase step name, then two underscore characters, then the parameter name. The key polynomialfeatures__degree therefore means the degree parameter of the step named polynomialfeatures, and ridge__alpha means the alpha parameter of the step named ridge. The prefix is what routes each candidate value to the correct step.

Read the key from left to right. The part before the double underscore identifies the step. The part after it identifies the parameter inside that step. The value in the list is then given to that parameter. The same rule gives ridge__alpha: the step named ridge receives the candidate alpha values.

Each candidate in the grid is a full combination of values, one value for each key. The number of candidates is therefore the product of the number of values for each key. This is the rule that tells you how large a search will be before you run it.

The same arithmetic applies to the grid in this chapter. Three degrees and six alpha values give $3 \times 6 = 18$ candidates, which matches the eighteen candidate scores the visualization reads.

The search does not tune the model in isolation. Because the polynomial degree and the Ridge alpha both appear in one param_grid, each candidate is a complete choice: a degree paired with an alpha. For every pair, the pipeline builds polynomial features at that degree, fits Ridge at that alpha, and scores the result by cross-validation.

The best pair is therefore chosen with both stages taken into account at once. The preprocessing choice of degree is judged by how well the Ridge model performs on the features it produces, and the alpha is judged on the same transformed features. This is why the source treats preprocessing and model parameters as tunable together. Neither value is picked in advance; both are compared on the measured outcome.

Eighteen candidate scores are too many to read as a list, so the results are arranged as a grid. The mean cross-validation score for each candidate is taken from the search results and reshaped into a table with three rows and as many columns as needed. Since there are three degrees, the three rows correspond to degrees 1, 2, and 3. The six alpha values then form the columns. Each cell is colored by its score, which makes it possible to see at a glance where the good settings cluster.

The mean for each candidate is the average of that candidate's cross-validation scores. Cross-validation scores the model on several held-out portions of the training data, and the mean summarizes those scores as one number per candidate. The heat map therefore shows one value per degree-alpha pair, not one value per fold.

The axis labels in the plotting code name the two parameters, so the reader does not have to guess which dimension is which. Without labels, a heat map with three rows and six columns looks the same whether it is read by degree or by alpha.

Once the heat map is drawn, the cross-validation results can be read directly. Using polynomials of degree two improves the score compared with the alternatives. Degree-three polynomials perform much worse than either degree one or degree two. The best parameters found by the search are reported from the same results, and they reflect this pattern.

To read a heat map, start by locating the best cell, which is the one with the most favorable score. Its row tells you the degree and its column tells you the alpha. Then move along each axis: across a row, you see how alpha changes the score at a fixed degree; down a column, you see how degree changes the score at a fixed alpha.

The decision about the degree therefore comes from the measured outcome rather than from a guess made before the search. The heat map is the evidence for that decision, and it applies only to the values that were included in the grid.