tf-idf Features for Text

Once a text classifier has been trained, the natural question is which parts of the input it relied on. When the classifier is a logistic regression model trained on tf-idf features, the answer lives in the model's coefficients. Each feature has a coefficient, and this article is about reading those coefficients.

A trained model can have hundreds or thousands of coefficients. A plain list of that many numbers is hard to read, so the practice shown in the source is to draw them as a bar chart. The chart makes the relative sizes of the coefficients visible at a glance. By the end of this article you should be comfortable reading that chart, knowing what each bar stands for, and tracing the small selection logic that decides which bars appear.

The selection logic is where beginners most often make mistakes, so we will go slowly through it. We will use a small made-up example, because the source chart itself cannot be re-run here and the feature names and values in our example are invented for illustration only.

The text is first turned into tf-idf features, which are extracted from the text using the tf-idf technique and then passed as input to the logistic regression model. The model learns one coefficient per feature from that input. A logistic regression coefficient is a learned number for each feature, and the source states that coefficients determine each feature's importance in the model.

Because there is exactly one coefficient for each tf-idf feature, each bar in the chart corresponds to one feature. The chart is therefore a view of the trained model, not a view of the raw text. Every bar you see can be traced back to a coefficient that the fitted model stored.

The diagram above uses invented terms and values. Its purpose is to show the alignment: position 0 in the feature names pairs with position 0 in the coefficient array, position 1 with position 1, and so on. If those positions were misaligned, each bar would be labeled with the wrong word.

The source produces the chart with a single call to a plotting helper named visualize_coefficients. The call does not compute anything on its own. It receives the coefficients and their names, and then draws them. Which coefficients appear is decided inside the helper, which is why the count parameter matters for what you see in the picture.

The chart shows the largest and smallest coefficients, not the middle of the list. Picking out the extremes requires two operations in a specific order. First you sort the coefficients by value. Then you slice. Slicing an unsorted list selects by position, not by value, so the order of these two operations determines the result.

Here is a made-up example with five features. The names and values are invented and are not output from any real model. We pair each name with its value, sort the pairs by value, and then slice off the two lowest and the two highest. The middle of the list is simply left out, just as features with middle-sized values are not drawn in the chart.

A bar chart of the largest and smallest coefficients makes the relative sizes visible and lets you see the feature names alongside them. It does not tell you everything about importance. The table below separates what the chart displays from what it leaves out.