Topic Modeling

Imagine a newsroom archive with tens of thousands of articles and no one has ever tagged them. A reader who wants every article about the economy, or every article about a local election, has no easy way to find them. Text data like this is everywhere: customer reviews, support tickets, research abstracts, and social media posts. Most of it arrives as raw sentences with no category attached.

Topic modeling is a technique often applied to text for exactly this situation. Instead of asking a person to read everything and label it, the technique tries to discover what the documents are about. This article starts with a short pipeline that runs end to end, then explains each step and shows where the results can go wrong.

The phrase one or several topics is the key to the definition. The task does not force every document into a single bin. A document may touch more than one subject, and the method's job is to say which subjects are present. The phrase usually without supervision signals that the topics are not handed to the method in advance. They have to be found from the documents themselves.

The method used most often for this task is Latent Dirichlet Allocation, usually abbreviated LDA. LDA discovers hidden topics in a body of text. The word hidden matters: nobody labels the topics in advance. The model proposes them from patterns of word use across documents, and the person running it has to read the output and decide what each topic means.

The lesson begins with running code rather than definitions. The pipeline takes a collection of text documents and returns two outputs: a per-document summary of which topics each document leans toward, and a per-topic summary of which words define each topic. Read the code once now, then follow each stage in the sections that come after it.

Follow the diagram from left to right. Each stage depends on the one before it. Documents become a count table, the count table is fitted to produce a document-level output and a topic-word table, and the topic-word table is sorted into readable lists. A problem at any early stage shows up later as a strange topic, so the chain is worth keeping in mind whenever results look wrong.

LDA does not read raw sentences. It needs a numeric table in which each row is a document and each column is a vocabulary word, and each cell holds how often that word appears in that document. scikit-learn's CountVectorizer builds this table. In the pipeline above, the vectorizer is created with two settings, and the call to fit_transform learns the vocabulary from the training texts and converts them into the table stored in X.

The two settings shape what the model is allowed to see. A cap on vocabulary size keeps the table manageable. Dropping words that appear in a large share of documents removes terms that show up almost everywhere and therefore say little about any particular subject. The values 1000 and 0.95 used here are illustrative choices, not fixed rules. X is the only thing LDA receives, so any choice made at this stage carries straight through to the topics.

Each keyword argument in the LDA call is a hyperparameter, meaning a setting chosen by the person before fitting rather than something the model learns from the data. The table below summarizes the settings used in the lesson pipeline and what each one changes.

The call to fit_transform does two jobs at once. It fits the model to X and returns the document-level output stored in document_topics. Doing both in one step avoids the second pass that would be needed if you fitted first and transformed afterward.

Before you look at topic words, it helps to know how the fitted model stores its topics. The model keeps a table called components_, which records how important each word is for each topic. Each cell answers a single question: how important is this word to this topic? A large value means the word carries a lot of weight in that topic, and a small value means it hardly matters.

The components_ table has the same role as the components in the matrix decomposition methods covered earlier in the course, so if you remember those, you already know the layout. The next diagram shows how a row of this table maps to a topic and how its values feed the ranked word list.

The rows are the topics. The columns are the words. Reading a topic means reading one row, and reading a word's role across topics means reading one column.

A row of numbers is hard to read, so the next step converts each row into a list of its strongest words. The approach is to sort each topic's word weights and then map the sorted positions back to vocabulary names. The critical detail is that np.argsort returns positions, not values. For a row, it answers the question: which word positions, in order from smallest weight to largest, make up this row?

The number of topics is the most consequential hyperparameter in this pipeline. Ten topics are few enough that a reader can inspect every one of them. A hundred topics give a more fine-grained picture, but reading all 100 would be overwhelming, so the source inspects a hand-picked subset of them by index.

Notice that the selected topic numbers are not a sequence and carry no inherent meaning. They were picked because they look interesting and representative. A reader who picks differently will see a different slice of the model.

The source describes two cases that both fall under topic modeling. In the first, each document is assigned a single topic. The source calls this the task of clustering the documents, which connects to the clustering ideas covered earlier in the course. In the second, each document can have more than one topic, and the source connects this case to the decomposition methods covered earlier.

When a document is allowed several topics, it helps to picture the document as a bundle of topics rather than a single label. The source explains that the coefficients of the components in a document's representation indicate how strongly the document relates to each topic. A stronger coefficient means a stronger connection.

Notice what this does not claim. It does not say the article belongs to exactly one bin. Each topic receives its own degree of relatedness. This is the practical meaning of a document having multiple topics, and it is what separates this case from clustering.

The multi-topic case is linked to decomposition methods. In a decomposition method, the data is expressed in terms of a set of components. In topic modeling, each component the method learns corresponds to one topic. The representation of any single document then records how much of each component it contains, and those amounts are its coefficients.

Latent Dirichlet Allocation is the decomposition method people most often mean when they talk about topic modeling. In these terms, LDA is one specific way of learning the components, and each component is read as a topic. The source does not give the internal mathematics here, so the important takeaway is the shape of the result: topics as components, and documents as coefficients over those components.

When the topic words look wrong, the fault usually sits at one identifiable stage. Each mistake below names the stage where the divergence starts, which is the question a debugger needs answered first.