Uncovering Hidden Topics in Text Data
Topic modeling is related to the decomposition methods from Chapter 3, and each of its learned components corresponds to one topic.
Two Terms Worth Untangling
The title of this article sets two ideas side by side, and it is natural to wonder how they relate. This part of the course does not define document clustering, so this article makes no claims about it. What it does give us is a clear account of topic modeling and of the ideas you have already met in Chapter 3. The goal here is to understand what topic modeling is, what its pieces mean, and where one specific method, Latent Dirichlet Allocation, fits inside it.
Starting From Decomposition Methods
Topic modeling is related to the decomposition methods introduced in Chapter 3. A decomposition method takes a representation of data and expresses it in terms of a set of learned components. In other words, the data is rewritten as a combination of building blocks that the method has learned from the data itself.
The diagram shows the general shape of a decomposition. The data is described through components, and the coefficients say how much each component contributes. Topic modeling keeps this same shape but adds a specific interpretation to the components, which the next section takes up.
Each Component as One Topic
Topic modeling is a use of decomposition in which each of the components the method learns corresponds to one topic. The components are not arbitrary pieces. Each one is meant to stand for a subject that appears in the text collection.
This is the central move of topic modeling. A generic decomposition tells you that a representation can be split into components. Topic modeling adds the reading that each of those components is a topic. The meaning of each component comes from what the method learns from the text, not from a label supplied in advance. The source is explicit on this point: the coefficients only report how strongly each document relates to those learned topics.
Reading One Document's Coefficients
Once each component is read as a topic, the representation of a single document becomes informative. Its coefficients on the components show how strongly that document relates to each topic. The illustration below uses made-up topics and strengths to show the reading. It is not drawn from the source material.
Reading a Document's Profile
A document has coefficients that are large for Topic 1, medium for Topic 3, and small for Topic 2. What does this representation say about the document?
- 1Identify the coefficient sizes
Topic 1 has the largest coefficient, Topic 3 is in the middle, and Topic 2 is the smallest.
- 2Read each size as a strength
A larger coefficient means a stronger relationship between the document and that topic, so the document is most strongly tied to Topic 1.
- 3Note the blend
Because Topic 3 also has a meaningful coefficient, the document is not only about Topic 1. It is a mix in which Topic 1 dominates and Topic 3 contributes.
- 4Avoid over-reading
The coefficients do not tell us what Topic 1 is called. The name would come from the words the method associates with that learned topic.
The document relates most strongly to Topic 1, with a secondary relationship to Topic 3 and a weak relationship to Topic 2.
Coefficients as Degrees, Not Labels
A natural first guess is that a document is either about a topic or not. Topic modeling does not work that way. The coefficients express degree, so each document gets a graded profile across all the topics rather than a yes or no label for each one.
The shape of the profile carries the meaning. When one coefficient is much larger than the rest, the document is mostly about that one topic. When several coefficients are comparable, the document is a blend of several topics. The contrast is easiest to see by comparing the two patterns directly.
| Coefficient pattern | What it suggests about the document |
|---|---|
| One coefficient far larger than the others | The document is mostly about that single topic |
| Several coefficients of similar size | The document is a blend of several topics |
| All coefficients small | The document has weak relationships to the learned topics |
Reading the relative sizes of one document's coefficients
Where LDA Fits
Topic modeling is a general approach, and it can be carried out with particular decomposition methods. One of these is especially common. Latent Dirichlet Allocation, often shortened to LDA, is a particular decomposition method that is called by that full name. When people talk about topic modeling, they very often mean LDA.
Read the diagram from the top. Decomposition methods form the family that topic modeling is related to. Topic modeling is the idea that the learned components correspond to topics. LDA sits below it as one concrete method that carries out that idea. The idea and the method are not the same thing.
Mistakes Beginners Make
Common mistake Treating topic modeling and LDA as identical
What it looks likeSaying that topic modeling is LDA, so any topic model must use Dirichlet priors
Why it failsTopic modeling names the general idea that components correspond to topics. LDA names one concrete method that carries it out.
Do this insteadUse topic modeling for the idea and LDA for the specific method, and recognize that LDA is the one most often meant.
Common mistake Reading coefficients as yes or no labels
What it looks likeConcluding that a document is either about Topic 2 or not about it
Why it failsThe coefficients express degree. A small coefficient is a weaker relationship, not an absence.
Do this insteadDescribe the relationship in terms of strength and look at the whole profile across topics.
Common mistake Assuming the topics come with names
What it looks likeStating that Topic 1 is about sports before examining what the method learned
Why it failsThe meaning of each component comes from the learned representation. The coefficients only report how strongly each document relates to those learned topics.
Do this insteadDescribe a topic by what the method learned, and avoid attaching a name the source never supplied.
Common mistake Assuming that the title's two terms describe the same method without evidence from the course material
Checking Your Reading
A document has coefficients that are nearly equal across three topics and close to zero for all others. Describe what this representation suggests about the document, and explain why it would be wrong to say the document belongs to exactly one topic.
Hints
- Compare the sizes of the coefficients with each other, not only with zero.
- Remember that the coefficients express degree rather than a yes or no label.
Key Takeaways
- Topic modeling is related to the decomposition methods from Chapter 3, and each learned component corresponds to one topic.
- A document's coefficients on the components express how strongly that document relates to each topic, as degree rather than a yes or no label.
- Latent Dirichlet Allocation (LDA) is a particular decomposition method that is often what people mean when they say topic modeling.
- Topic modeling names the general idea, while LDA names one specific method, so the two terms should not be treated as identical.