Text Classification with Movie Reviews

Imagine you are asked to build a filter that decides whether a movie review is positive or negative. The learning algorithm you plan to use only accepts numbers. It has no direct way to take in a sentence such as 'Great film, I loved the ending.' Before any learning can happen, something has to turn that sentence into numbers. That step is called data representation, and the choice you make at this step quietly decides what the model is allowed to know.

A model can only learn from numbers, so the representation chosen for text determines which information the model can use. The source emphasizes this directly: as is often the case in machine learning, the representation of the data is key in NLP applications. The same review can be turned into different sets of numbers, and a model that learns from one set may succeed or fail because of the representation rather than because of the learning algorithm itself.

It helps to see the whole path from raw text to a prediction as a pipeline. Each stage hands a transformed version of the review to the next stage, and information that is dropped at an early stage can never be recovered by a later one. The diagram below traces that path for a bag-of-words representation.

A bag-of-words representation starts by splitting text into tokens, which are the units the representation counts. For this article, a token is a lowercase word with punctuation removed. The representation then records each distinct token and how many times it appears. The result is a bag: a collection of words with counts, and nothing that says where each word sat in the sentence.

Notice that the six tokens collapse into five entries. The repeated token boring is recorded once with a count of 2. This is the core move of the representation: distinct words become keys, and their frequencies become the values the model sees.

A bag-of-words representation keeps which tokens occur and how often. It does not keep the order of the tokens. This is the trade-off behind its simplicity. Two texts with the same words in different orders end up with identical counts, so the representation cannot tell them apart.

Whether that loss matters depends on the task. The source describes the bag-of-words approach as a simple and powerful solution for text classification tasks such as spam and fraud detection and sentiment analysis. For many of these problems, a strong signal such as repeated words like boring may be enough to separate one class from another, even though the arrangement of the words has been thrown away.

An n-gram is a run of n consecutive tokens. A bigram, with n equal to 2, keeps a small piece of local order that a single-token count throws away. You can picture a bigram as a window of width two that slides across the token list one position at a time, recording each pair it covers.

For the review 'Boring plot, boring acting, great ending', the window produces five bigrams. Two of them, 'boring plot' and 'boring acting', show that boring is now recorded next to other words. A single-token bag only knows that boring appeared twice; a bigram list also records what boring was next to. The source notes that inspecting the tokens and n-grams that are extracted can give powerful insights into the modeling process.

The source makes a point that goes beyond choosing a representation: you can introspect the extracted pieces directly. In text processing, it is often possible to introspect models in a meaningful way, and this holds for supervised tasks, where labels guide learning, and for unsupervised tasks, where labels are absent. The practical advice is to take full advantage of this ability when using NLP-based methods.