Building Text Classification Pipelines with Grid Search

Once a text classifier has been trained, its accuracy tells you how often it is right, but it does not tell you why it is right. A model can score well for reasons that have nothing to do with the meaning you care about, so a good score alone is a weak form of evidence. This article shows how to open up a trained text model and read what it learned.

For a linear model such as logistic regression, the learned weights, called coefficients, are the most direct record of what the model picked up from the training text. Each weight is attached to one feature. In a text model, a feature is a token or an n-gram: a single word, a pair of adjacent words, or a run of three words. Analyzing coefficients means reading those weights against the feature names they belong to.

The grid search is the outer object. It wraps a pipeline, and the best version of that pipeline is available as grid.best_estimator_. You do not need to re-run the search to inspect the model. You only need to walk down from the grid search object to the pipeline that won, and then to the two named steps inside it.

The pipeline has two named steps. The first is a tf-idf vectorizer, which turns text into rescaled n-gram columns. The second is a logistic regression classifier, which learns one weight per column. The names matter: you reach each component by its step name, not by guessing its position in a list of objects. Once you have the vectorizer, you can ask it for the names of its columns. Once you have the classifier, you can ask it for its coefficients. Those two outputs are the two halves you must combine.

The extraction works only because position matters. The first entry in the feature name array describes the same column as the first entry in the coefficient array. The second name matches the second weight, and so on. Nothing in the code checks this for you. The pairing is a rule about order, not a rule about content.

Because the pairing is positional, check the lengths before you trust any ranking. The two sequences should have the same number of entries, one per feature. If they do not, the pairing step is already wrong, and no later interpretation can repair it. This failure is hard to notice because the output still looks plausible. Every name still appears next to a number, but some of those numbers belong to other columns.

Once each name is paired with its weight, the weights can be read in two ways. The sign tells you which direction a feature pushes the prediction. A weight with one sign pushes the model toward one class of review, and a weight with the opposite sign pushes it toward the other. The size tells you how strongly that push happens relative to the other features. A weight near zero contributes little either way.

The visualization in the source code ranks features this way, so the strongest signals are visible without scanning the full list. The ranking uses the same pairing you built earlier. If the pairing is wrong, the ranking is wrong too, even though the chart looks clean.

A unigram model sees the word 'worth' as a single column. Its weight reflects an average over every sentence where that word appears, whatever surrounds it. Once bigrams and trigrams are added, the model gets separate columns for phrases that contain 'worth', and those phrases can point in different directions. The source highlights three of them. The phrase 'not worth' indicates a negative review. The phrases 'definitely worth' and 'well worth' indicate a positive review. The same word appears in all three, and the surrounding word changes its meaning.

When a ranked list does not match what you expected, the fastest route to the cause is to walk the pipeline in the same order the code does. The divergence usually appears at one of three points: the fitted object, the vectorizer's names, or the pairing of names with weights. The flowchart below follows that order and adds the length check from earlier.

The key to the flowchart is the decision node. If the two arrays have different lengths, the pairing is already broken, so you must trace back to whatever changed the arrays before you interpret any weight. If the lengths match, pair the arrays once, at the point where they are combined, and then rank.