Grid-Searching Preprocessing Steps and Model Parameters

When you tune a model with GridSearchCV, the search tries many parameter combinations and keeps one winner. In many workflows that winner is not just a bare classifier. When the object being searched is a pipeline, the winner is a complete pipeline, with its preprocessing steps and its model, all fitted together. Once the search has finished, a practical question follows: how do you get inside that fitted pipeline and read the numbers the model has learned?

The answer is a short chain of attribute lookups. Each lookup moves you one level deeper into the structure: from the search object, to the best model it found, to one step of that model, to a learned quantity inside that step. Beginners most often go wrong when they try to walk this chain by themselves, because each link is a separate lookup and a wrong key or a skipped level breaks the whole expression.

Start with the search object, the one you created with GridSearchCV. The attribute best_estimator_ is found on this search object, not on the pipeline you originally handed to it. The search keeps the record of which configuration won, and it exposes that winning configuration as best_estimator_. The source describes this model as already refit on all of the training data, so it is the version that produced the winning result.

The printed representation is your evidence. In the source example, printing best_estimator_ shows a Pipeline object whose steps are listed by name: a StandardScaler first and a LogisticRegression second. That printed text confirms that best_estimator_ is a pipeline with multiple steps, which is the fact everything later in this article depends on.

The next link is named_steps. A pipeline's named_steps attribute gives you each step by its name. You index it like a dictionary, passing the step name as a string, and you get back that step's fitted object. Read the whole expression from left to right: grid is the search object, best_estimator_ moves you to the winning pipeline, named_steps moves you to the dictionary of its steps, and the string key picks one step out of that dictionary.

The coef_ attribute belongs to the fitted LogisticRegression step, so the lookup must pass through named_steps before reading it. The coefficients are the weights the model learned for each input feature, and because this step was trained as part of the best pipeline, these weights are the ones that actually produced the search's winning result. The printed output is nested in double brackets, which signals a two-dimensional array with one row of weights and one value per input feature. Reading that structure correctly is part of interpreting the model, not just retrieving it.

A grid search can tune preprocessing parameters and model parameters together when both are steps in one pipeline. Consider a pipeline that adds polynomial features before a Ridge regression model. It has two settings that need a decision: the degree of the polynomial features, which controls how far each existing feature is raised to a power, and the alpha parameter of Ridge regression, which controls the regularization strength. Because the pipeline already contains both steps, a single search can vary the degree and the alpha value together and compare every combination on the same cross-validated scores.

The search must know which step each value belongs to. Hyperparameter values are passed to a step by prefixing the parameter name with the step name, separated by a double underscore. Each key in the param_grid is written as the step name followed by the parameter name, such as polynomialfeatures__degree and ridge__alpha. Each key therefore tells the search which step to change and which parameter to change inside that step.

Before reading any search output, it helps to know how large the search is. A grid search tries every combination of the values listed in the param_grid, so the number of candidates is the product of the number of values for each key. Cross-validation then multiplies that count, because each candidate is evaluated once per fold.

After the search runs, the results can be visualized as a heat map. The scores are read from the cv_results_ attribute, specifically the mean test score for each candidate. The 18 mean scores are reshaped into a grid with 3 rows and as many columns as needed, so each row corresponds to one polynomial degree and each column to one alpha value. Each cell is one candidate pairing, and the color reflects the averaged cross-validation score for that pairing.

The layout follows directly from the reshape call. The source places degrees on the vertical axis and alpha values on the horizontal axis. The heat map is a way of reading the search result, not a separate calculation, so its shape must always match the grid you defined. The flat array is in candidate order, and the reshape simply cuts it into rows.

When the heat map or the best score does not match what you expected, the cause is usually found in one of a few specific places. Work through them in order: first the keys of the param_grid, then the step names in the pipeline, then the reshape call, and last the axis labels. Each of these is visible in the code, so you can check them directly rather than guessing.