Interpreting Model Coefficients

Once a text classifier has been trained, the natural question is which parts of the input it relied on. In a logistic regression model trained on tf-idf features, each feature has a coefficient, and those coefficients are what this article is about. A list of hundreds or thousands of numbers is hard to read, so the practice shown in the source is to draw them as a bar chart. The chart makes the relative sizes of the coefficients visible at a glance.

The goal here is to get you comfortable with that chart. You will see what the bars represent, learn the small piece of logic that picks out the largest and smallest values, and practice the steps that keep the feature names attached to their numbers. Selection logic is where beginners most often make mistakes, so this article slows down there. Treat every number you read from a chart as something you should be able to trace back to a value the fitted model stored.

The features here are tf-idf features. They are extracted from the text using the tf-idf technique and then passed as input to the logistic regression model. The model learns one coefficient per feature from that input. This means the chart is a picture of the trained model, not of the raw text. If you want to know what the model learned about a word, the coefficient for that word's feature is the place to look.

Two readings of a coefficient matter. Its size indicates how strongly that feature is weighted in the model, and its sign indicates which way the feature pushes the prediction. The source describes the spread of markers in its plot as showing which combinations pushed the prediction up or down, which is the same sign-based reading applied to a different model.

The source follows one concrete case. An earlier step combined day-of-week and time-of-day inputs into interaction features, which produced a linear model that performed about as well as a random forest on the same problem. Because there is one coefficient for each day and time combination, the model is easy to inspect. The first step toward reading those coefficients is building human-readable names for the original inputs. There are two groups: a list of hour labels generated in steps of three hours across the day, and a list of seven day abbreviations. The two lists are joined into one list of base feature names.

Notice the trade-off the source points out. The transformed model has many more columns than the original inputs, because each new column is a product of existing features. Interpretation shifts from reading a few effects to reading a grid of combined effects. A plot of the coefficients is what makes that grid manageable.

The program that builds the plot does three things in sequence. It asks the transformer for the names of all the interaction features it generated. It then applies a boolean test to the learned coefficients to keep only the features whose coefficient is not zero. Finally, it extracts those nonzero coefficients as a separate list. Two lists must stay in step through this process: the feature names and the coefficients. Each position in one list is supposed to describe the same position in the other.

Both lists must be filtered with the same test because the model stores coefficients in one fixed order, and the names are only useful when they sit at the same positions. The filtering idiom works like this: comparing an array to a value produces an array of True and False, and indexing with that array keeps only the True positions. The example below uses made-up numbers, not the program from the source, to show the mechanism in isolation.

Notice that 'b' and 'd' stay paired with 1.5 and negative 2.0 because both arrays were indexed with the same mask. If you had filtered only one of them, the pairing would be lost and the output would still look perfectly valid. A mismatch does not usually raise an error. It silently attaches the wrong label to a number, which is one of the most common ways a coefficient analysis goes wrong.

The chart of largest and smallest coefficients depends on a two-step selection. Picking the largest and smallest values requires sorting by the coefficient before slicing. Slicing an unsorted list selects by position, not by value, so a slice taken first returns whatever happens to sit at the ends of the original order. The safe order is to sort the pairs by value and then slice the first and last N.

The source uses an invented example to practice this selection, because the chart itself cannot be re-run here. The feature names and values are made up for illustration and are not output from a real model. The code pairs each name with its value, sorts the pairs by value, and slices off the two lowest and two highest. The middle of the list is simply left out.

The source produces the chart with a single call to a plotting helper. The call does not compute anything on its own. It receives the fitted coefficients, the feature names, and a count parameter, and then draws them. Selecting which coefficients appear is handled inside the helper, which is why the count parameter matters for what you see. The excerpt relies on two names defined earlier in the original workflow, grid and feature_names, and does not show how they were created, so treat them as given inputs.

The source describes the chart as showing the 25 largest and 25 smallest coefficients, but the call that produces it passes n_top_features=40. The excerpt does not explain how that parameter divides between the largest and smallest groups, or whether it counts the total number of bars. Do not assume a 25 and 25 split when you read the chart.

The word smallest is also ambiguous. The source does not say whether it means the lowest signed value, as in the example above, or the value closest to zero in size. The two readings can produce very different bars. Confirm which one the helper uses before comparing the two ends of the chart. Features with values in the middle of the ranking are not drawn at all, so the picture highlights the extremes rather than every feature.

A plot is only as trustworthy as its labels. When you read the plotting code closely, the axis labels in the source are attached in an unexpected way. The coefficient values are passed as the first and only data argument, so they are drawn on the vertical axis. The feature names are placed on the horizontal axis through the tick settings. Yet the horizontal axis is labeled as the feature magnitude, and the vertical axis is labeled as the feature. Read literally, the labels describe the opposite of what each axis shows.

The line responsible is the one in the plotting code that assigns the axis labels. When you find it, compare the label text against the arguments in the data call. If the label says the horizontal axis is magnitude while the coefficients are drawn vertically, the chart is misleading even though every bar is correct. Check the axis labels before drawing any conclusion from a coefficient plot.