CountVectorizer and the Bag-of-Words Representation

Read any English paragraph and count the words that show up everywhere: 'the', 'and', 'is', 'of'. These words appear in almost every English document, so they tell a model very little about what any single document is about. Yet when a computer turns a document into numbers, every distinct word it sees can become a feature, and each feature takes up a column. Common words therefore keep a seat in the representation even though they contribute almost no information.

This article looks at one practical response: a pre-defined stopwords list. You will see what such a list contains, how it is handed to CountVectorizer, and what changes in the resulting features. Before reading each explanation, try to trace the code in your head and decide what you expect each stage to produce. Then compare your expectation with what the code actually does.

The scikit-learn library provides this list under the name ENGLISH_STOP_WORDS in its feature extraction text module. You can import it directly and ask how many entries it holds. The source reports that the list contains 318 stopwords. The list is a set-like collection rather than an ordered sequence, so the source converts it to a list before slicing it. The slice in the source takes every tenth entry, which is a quick way to sample the contents without printing all 318 words.

The sampled words look ordinary. The source's sample includes pronouns such as 'he' and 'me', the determiner 'the', and connecting words such as 'although' and 'into'. Notice that the sample contains no subject-specific vocabulary. The list is built around words that carry grammatical function rather than meaning about a particular topic, which is why it is a generic starting point for English text. A movie review, a news article, and a recipe all tend to contain these same function words.

CountVectorizer transforms text into a matrix of token counts. Think of the result as a grid. Each row is one document, and each column is one vocabulary word. The vocabulary is the set of distinct words the vectorizer has decided to keep, and every word in it owns exactly one column. The number stored where a row and a column meet is how often that word appears in that document. Because word order is not recorded in these counts, the representation is often called a bag of words.

Trace one row through this map. The document in this illustration contains 'movie' twice, 'fun' once, and 'long' once. Its row therefore holds 2 in the column for 'movie', 1 in the column for 'fun', and 1 in the column for 'long'. Change the vocabulary and the column for each word changes with it, but the rule stays the same: a word's column is fixed by its place in the vocabulary, and its cell records its count in that document.

CountVectorizer accepts a stop_words parameter. When you set that parameter to the string 'english', the vectorizer uses the built-in ENGLISH_STOP_WORDS list. The setting is chosen when the object is created, and the vocabulary is learned later, when fit is called on the training text. The source also notes that you can augment the built-in list and pass your own, but it mentions that option without a worked example, so treat it as a direction for further study rather than a tested procedure.

It helps to see the stopword step as one stage in a sequence rather than as a standalone operation. Text is first split into tokens. The stopword filter then removes the tokens that appear in the list. The remaining tokens form the vocabulary, and the count matrix records how often each vocabulary item appears in each document. This is a simplified conceptual model. It shows where the stopword decision sits, not every internal detail of the library.

The reasoning behind the answer is what matters. Each stopword that appears in the text is one candidate word that no longer becomes a feature. Words not on the list are untouched. Because the filter only removes candidates, the vocabulary after filtering is never larger than the vocabulary before. The source adds a more hopeful claim: although the count can only fall, the performance of a downstream model might improve. Whether it does is an empirical question, and the source frames it as something to test by trying the change and measuring the result.

When a stopwords experiment gives a surprising feature count, the cause is usually at one of a few specific points. Because the stopword setting is supplied at construction and used during fitting, the most informative first question is which stage produced the result you are looking at. The points below are the ones to check first.