Precision, Recall, and the F1 Score
The scoring metric does not affect fitting and does not affect the candidates. It affects only the comparison, and the comparison decides which candidate's settings you keep. Two searches over the same grid can therefore return different winners.
Why the Same Grid Can Pick Different Winners
Earlier work taught you to compute a metric from ground truth and a model's predictions. Model selection asks a different question: which setting of a model's parameters should you keep? To answer it, a search tool has to know which number to compare across candidate settings. That number is the scoring metric, and in scikit-learn you choose it with a single argument.
What do you think happens?
If you search the same parameter grid once with the default scoring and once with AUC, will both searches keep the same gamma value?
Reveal answer
No. The scoring metric can change the winner. In the source's run, the accuracy search kept gamma=0.0001, while the AUC search kept gamma=0.01.
The grid and the data are identical in both runs. Only the number used to compare candidates changed, and that is enough to change the selected setting.
This article follows that one question. You will trace where the scoring choice enters a grid search, read two real outputs side by side, and learn the scoring strings you are most likely to need. The article does not re-derive what precision, recall, or the F1 score measure internally; the source for this section focuses on how the metric is chosen and used in scikit-learn.
Where the Scoring Argument Enters
The scoring argument, passed as a string, sets the metric that cross_val_score and GridSearchCV use to evaluate candidates.
GridSearchCV tries each candidate value in a parameter grid, scores each one, and keeps the best. Scoring is the step that turns each candidate into a comparable number. The same argument works in both tools, so you can judge a single model with cross_val_score and then search over settings with GridSearchCV using the same metric.
Follow one candidate through the loop. It is fitted, then scored, then its score is placed in the comparison with the other candidates. The metric is used only in the scoring and comparison steps. It does not change how the model is fitted, and it does not change which candidates exist in the grid. The comparison at the end is therefore the single place where the metric can change the result.
The scoring metric does not affect fitting and does not affect the candidates. It affects only the comparison, and the comparison decides which candidate's settings you keep. Two searches over the same grid can therefore return different winners.
Tracing Two Searches Over the Same Grid
The source uses a deliberately narrow grid that contains only gamma values, so the effect of the metric is easy to see. Both searches are fitted on the same training split and evaluated on the same test split. The only difference between the two runs is the scoring argument.
Tracing the accuracy search and the AUC search
The same gamma grid is searched twice. Run one uses the default accuracy. Run two uses roc_auc. Which setting does each run keep, and what score does it report?
- 1Step 1: fit the candidates on the training split
Each gamma value in the grid is fitted on the same training split in both runs. The fitted models are identical in the two runs, because scoring does not affect fitting.
- 2Step 2: score each candidate with the chosen metric
Run one scores every candidate with accuracy. Run two scores every candidate with AUC, the area under the ROC curve. Each candidate becomes one comparable number under the chosen metric.
- 3Step 3: compare and keep the best candidate
Run one keeps gamma=0.0001, with a cross-validation accuracy of 0.970. Run two keeps gamma=0.01, with a cross-validation AUC of 0.997. Each winner is best by the metric it was given.
- 4Step 4: check the result on the test set
The source states that cross-validation accuracy agrees with test set accuracy in both cases. In this run, the AUC search's chosen setting was also better in accuracy.
Under accuracy the search keeps gamma=0.0001. Under AUC it keeps gamma=0.01. The two searches disagree on the winner, and each one is correct by the metric it was given.
Read the two outputs as a trace rather than as a verdict. Notice that the AUC search found a setting that is better in AUC and, in this run, also better in accuracy. The source is explicit that this is not a claim that AUC is always superior. The lesson is that the choice of metric changed which setting was selected for this grid and this data.
Passing the Scoring Argument in Code
The same scoring argument is accepted by both cross_val_score and GridSearchCV. The example below is a generated skeleton that shows where the argument goes. The estimator is a placeholder for any classifier, and the training data names are placeholders for your own split.
- scoring is the only place the metric is named
- Changing scoring to 'accuracy' changes only the comparison step
- Leaving scoring out is equivalent to scoring='accuracy' for classification
Choosing a Scoring String for Your Task
scikit-learn offers many scoring strings. The source highlights a small set that matters most. The map below groups them by task. Learn these first, and treat the longer list as a reference to check against the documentation for your installed version.
Mistakes When a Selection Looks Surprising
Common mistake Assuming the grid itself chose the winner
What it looks likeTwo searches over the same gamma grid return gamma=0.0001 and gamma=0.01.
Why it failsThe grid is identical in both runs. Only the scoring argument changed, and it acts at the comparison step.
Do this insteadCheck the scoring argument first, then compare the candidate values using that same metric.
Common mistake Comparing values under one metric while reading the winner from another
What it looks likeThe search was run with roc_auc, but the reported numbers are read as accuracy.
Why it failsEach winner is correct only by the metric it was given, so mixing metrics makes the choice look wrong.
Do this insteadCompare the candidate scores using the same metric that the search used.
Common mistake Believing AUC is always the better metric
What it looks likeThe AUC search kept gamma=0.01 and the reader concludes AUC should always be used.
Why it failsThe source states that the finding is not that AUC is always superior. The metric choice changed the selected setting for this grid and this data.
Do this insteadPick the metric that matches your goal, and verify the effect on your own grid and data.
Common mistake Omitting scoring and assuming a different default than accuracy for classification
Checking the Available Scoring Names
The source shows how to list every available name by reading the scorer dictionary in the metrics module. That list is long and includes entries such as log_loss, adjusted_rand_score, and several precision and recall variants. The most important names to learn are the ones in the scoring map above.
Predicting a Winner Under a New Metric
Using the same gamma grid, predict which setting a search would keep if you change the scoring argument from accuracy to roc_auc, and which step of the workflow would produce the change. Then describe how you would verify your prediction.
Hints
- Decide first which step of the grid search loop reads the scoring argument.
- Remember that the candidates are fitted the same way regardless of the metric.
- Verification means running both searches and comparing the winners, using each search's own metric.
Key Takeaways
- The scoring argument, passed as a string, sets the metric that
cross_val_scoreand GridSearchCV use to evaluate candidates. - Scoring acts at the comparison step: it does not change fitting or the candidates, but it decides which setting is kept.
- In the source's run, accuracy kept gamma=0.0001 (CV accuracy 0.970) while AUC kept gamma=0.01 (CV AUC 0.997), so the metric changed the winner.
- Accuracy is the default for classification. Learn the core strings: accuracy,
roc_auc,average_precision, the f1 family, r2,mean_squared_error, andmean_absolute_error. - When a selection looks surprising, check the scoring argument first, then compare values using that same metric.