Visualizing Model Coefficients

A trained linear regression model is more than a prediction machine. It stores one number per input column, and those numbers are the coefficients. Reading them tells you what the model has learned about each input. The practical payoff is that you can often explain a model's behavior by looking at a short list of numbers, instead of treating the model as a black box.

This article follows one concrete case. An earlier step combined day-of-week and time-of-day inputs into interaction features. The resulting linear model performed about as well as a random forest on the same problem. Because the interaction features give one coefficient for each day and time combination, the whole model can be read directly off a plot of those coefficients. The job of this article is to read those coefficients correctly and to follow the small code steps that move them from the model into a plot.

A coefficient is the expected change in the target for a one-unit change in one input column. The phrase 'one input column' matters: the change is measured for one feature at a time, while the other features are held fixed. A coefficient therefore describes the effect of a single input, not the total prediction.

In this formula, the coefficient for feature j is written as beta sub j. The expression says that if feature j goes up by exactly one unit and every other feature stays the same, the expected target moves by beta sub j. A positive coefficient means the expected target rises with that feature; a negative coefficient means it falls. The size of the coefficient tells you how large that movement is in the units of the target, so a coefficient of 2.0 means a one-unit increase is associated with a rise of about 2.0 in the expected target.

A coefficient list is only meaningful if you know which input each number belongs to. The code therefore builds a list of human-readable names for the original inputs. Here there are two groups: hour labels in steps of three hours across the day, and seven day abbreviations. The two lists are joined into one list of base feature names. The program then asks the transformer for the names of all the interaction features it generated, uses a boolean test on the learned coefficients to keep only the features whose coefficient is not zero, and extracts those nonzero coefficients as a separate list.

The diagram shows the key idea. The shaded positions, a and c, have zero coefficients and are removed. Position 1 and position 3 survive, and they survive in both lists at once. A name and its coefficient are linked only by their position index, so the mask must be applied to both lists. Because the model stores coefficients in one fixed order, the names are useful only when they sit at the same positions.

The filtering step relies on a NumPy idiom: comparing an array to a value produces an array of True and False, and indexing with that array keeps only the True positions. The example below uses made-up numbers, not output from the source model, so you can see the mask work in isolation.

Trace the output by position. The mask is False at positions 0 and 2, so 'a' and 'c' disappear from both arrays. The names 'b' and 'd' stay paired with 1.5 and negative 2.0 because both arrays were indexed with the same mask. If you had filtered only one of them, the pairing would be lost and the output would still look perfectly valid. This is why the mismatch is so dangerous: it rarely raises an error.

A plot is only as trustworthy as its labels. The plotting step draws one marker for each nonzero coefficient. It sets a wide, short figure so that many labels fit along one line, rotates the tick labels vertically so long names do not overlap, and places the feature names at the tick positions. The coefficient values are passed as the first and only data argument, so they are drawn on the vertical axis, while the feature names are placed on the horizontal axis through the tick settings.

The problem is that the axis labels in the plotting code are attached in an unexpected way. The horizontal axis is labeled as the feature magnitude, and the vertical axis is labeled as the feature. Read literally, those labels describe the opposite of what each axis shows. A reader who trusts the labels would conclude that the height of each marker is a feature name and that the horizontal position is a magnitude, which is wrong.

To locate the line responsible, compare the axis-label calls with the data arguments and tick settings in the same plotting function. The pack describes the labeling mechanism but does not show the exact source line, so you must find it yourself: look for the lines that set the horizontal and vertical axis labels, then trace each axis back to what it is actually given. If the labels and the data disagree, trust the data and correct the label before reading any conclusion from the plot.

The chart in the source is produced by a single call to a plotting helper named visualize_coefficients. The call takes three pieces of information. The excerpt relies on two names defined earlier in the original workflow, grid and feature_names, and the source does not show how they were created, so treat them as given inputs. The table below lists what each argument supplies.

Notice that the call does not compute anything on its own. It receives the coefficients and their names and then draws them. Selecting which coefficients appear is handled inside the helper, which is why the count parameter matters for what you see in the picture. Because the names are supplied separately, they must already be aligned with the coefficients, as covered in the earlier section on keeping names and coefficients in step.

Once a text classifier has been trained, the natural question is which parts of the input it relied on. In a logistic regression model trained on tf-idf features, each feature has a coefficient, and the source states that coefficients determine each feature's importance in the model. Here the features are tf-idf features, extracted from the text using the tf-idf technique and then passed as input to the logistic regression model. The model learns one coefficient per feature, so each bar in the chart corresponds to one tf-idf feature. The chart is therefore a view of the trained model, not of the raw text.

A list of hundreds or thousands of coefficients is hard to read, so the practice is to draw them as a bar chart. The chart makes the relative sizes of the coefficients visible at a glance. Larger bars mean a feature carries more weight in the model's decision. The sign of a coefficient indicates the direction in which that feature pushes the prediction. The source does not spell out which class each sign favors, so check that against the model's class ordering before reading a direction into a bar.

The source chart draws coefficient sizes for tf-idf features, produced by one call that takes the fitted coefficients, the feature names, and a count parameter. Read the chart from left to right in the pipeline sense: the text becomes tf-idf features, the model learns a coefficient for each feature, and the plotting helper turns those values into bars. Every bar you see is traceable back to a coefficient the fitted model stored.

Picking the largest and smallest values requires sorting by the coefficient before slicing, since slicing an unsorted list selects by position, not by value. The source's invented example pairs each name with its value, sorts the pairs by value, and slices off the two lowest and two highest. The example below uses new made-up values, not output from any real model, to show the same idea.

The middle of the list is simply left out. This is the same idea behind a chart of the largest and smallest coefficients: features with values in the middle are not drawn, so the picture highlights the extremes of the ranking rather than every feature.