tf-idf Features

Imagine a text classifier built with logistic regression on tf-idf features. Training is finished, and the model has learned one coefficient for each feature. The coefficients are just numbers. A long list of them is hard to interpret by eye, so the usual next step is to plot them and look at which features carry the most weight.

The plot is the lens for everything in this article. Before you can read it correctly, you need to understand two things: what a single coefficient means, and how the plot decides which coefficients to show. We will build up both, one step at a time, using a small example you can trace by hand.

Two inputs must line up before any chart can be read. The first is the coefficient array, a list of numbers produced by the trained model. The second is the feature name list, a separate list of the terms the model was given. The plot works only if position i in the coefficient array corresponds to position i in the feature name list. If that correspondence breaks, every bar gets labeled with the wrong term, and the chart looks confident while being wrong.

The source describes the chart as pairing each coefficient with its feature name and then displaying the largest and smallest values. Before reading the real chart, practice the core operation with a tiny stand-in. The four features and coefficients below were invented for this article, and the code uses only built-in Python.

Notice what happened. The pairs were built first, so each name stayed attached to its own number through the sort. Sorting moved the pairs around, but it never separated a name from its coefficient. This is why the pairing must happen before any sorting or selection. If you sorted the two lists separately, the names would no longer match the numbers.

The phrase one-unit change carries most of the meaning. A coefficient is a rate of change per unit of its feature, so it only means something relative to what one unit of that feature is. For tf-idf features, each feature is a numeric weight attached to a term. That is why the coefficients are the model's learned weights on those term weights. A coefficient of 1.2 on the feature great says that, holding the other features fixed, increasing that term's tf-idf weight by one unit shifts the model's log odds by 1.2 in the positive direction. A negative coefficient shifts the log odds the other way.

The source chart displays the 25 largest and 25 smallest coefficients of the trained model. The length of each bar shows the size of that coefficient. The largest coefficients are the features with the strongest positive weight in the model's ranking, and the smallest coefficients are the other extreme. Reading it means looking at both ends of the range. Looking only at the top of the list would hide half of the story, because the most negative bars are just as informative as the most positive ones.

The chart is laid out as a horizontal arrangement of feature bars in two halves: one half for the largest coefficients and one for the smallest. Each bar is labeled by its feature name. Use the labels to answer the question the chart is built for: which features does the model weight most heavily, and in which direction? The next example uses the same two-ended selection on the small invented data set.

Read the comparison carefully. The largest column holds the strongest positive weights, and the smallest column holds the most negative ones. Size alone is not the whole picture: waste has the largest magnitude in this example, even though it sits in the smallest column. The direction tells you which class the feature pushes toward, and the length tells you how strongly.

A coefficient plot can diverge from expectations in a specific way. The source states that the chart shows 25 largest and 25 smallest coefficients, which would be 50 bars in total. The call that produces it passes n_top_features=40. The source does not explain how those two numbers relate. So do not assume the chart shows exactly the count in the caption, and do not assume it shows exactly the number in the call.