Decomposition Methods
In topic modeling, one learned component equals one topic. The meaning of each component comes from the method's learned representation, not from a topic name supplied in advance.
Two Terms Worth Untangling
Two phrases often appear side by side when people talk about working with collections of text: document clustering and topic modeling. It is natural to wonder how they differ. The material in this part of the course does not define document clustering, so this article makes no claims about it. What it does give us is a clear account of topic modeling and of how topic modeling connects to an idea you have already met, the decomposition methods introduced in Chapter 3.
The goal here is to understand three things: what topic modeling is, what its pieces mean, and where one specific method, Latent Dirichlet Allocation, fits inside it. Keep the question in mind as you read: when a method reports that a document is related to something, what exactly is it reporting?
From Decomposition to Topics
A decomposition method takes a representation of data and expresses it in terms of a set of learned components. Topic modeling uses this same kind of idea, but with a specific interpretation: each of the components that the method learns corresponds to one topic. The components are therefore not arbitrary pieces. Each one is meant to stand for a subject that appears in the text collection.
In topic modeling, one learned component equals one topic. The meaning of each component comes from the method's learned representation, not from a topic name supplied in advance.
Reading a Document's Coefficients
Once each component is read as a topic, the representation of a document tells you something useful. A document's representation is built from the components, and each component carries a coefficient for that document. The coefficients show how strongly the document is related to each particular topic. A larger coefficient on a component means a stronger relationship between that document and the corresponding topic. A small coefficient means a weaker relationship.
The key word is degree. The coefficients are a measure of strength, not a simple yes or no label. A document does not simply belong to a topic or fail to belong; it relates to several topics, each to a different extent.
Reading one document's profile
A document's representation gives it coefficient 0.90 on Topic 1, 0.10 on Topic 2, and 0.00 on Topic 3. Which topic is this document most strongly related to, and what does the zero tell you?
- 1Compare the coefficients
Larger coefficients mean stronger relationships, so 0.90 is the largest of the three values.
- 2Identify the strongest topic
Topic 1 has the largest coefficient, so this document is most strongly related to Topic 1.
- 3Interpret the weaker values
Topic 2 has a small coefficient, so the document relates to it only weakly. Topic 3 has a coefficient of zero, which means no relationship is recorded in this representation.
The document relates most strongly to Topic 1, weakly to Topic 2, and not at all to Topic 3. The reading comes from degrees of relation, not from a yes or no label.
Topic Modeling and LDA
Topic modeling is a general approach, and it can be carried out with particular decomposition methods. Among them, one is especially common. Latent Dirichlet Allocation, often shortened to LDA, is a particular decomposition method called in full by that name. When people talk about topic modeling, they very often mean LDA.
This does not make the two terms identical. Topic modeling names the idea that components correspond to topics. LDA names one concrete method that carries out that idea. Reading the hierarchy from the top, decomposition methods are the broader family that topic modeling is related to, and LDA sits inside that family as one specific method used for topic modeling.
Mistakes When Reading Topic Models
Common mistake Treating topic modeling and LDA as the same thing
Why it failsThe source states that the two terms should not be treated as identical. Topic modeling is the general idea that components correspond to topics, while LDA is one concrete method that carries it out.
Do this insteadSay topic modeling when you mean the approach, and say LDA when you mean the specific method.
Common mistake Reading a coefficient as a yes or no answer
What it looks likeSeeing a coefficient of 0.10 on a topic and concluding the document is simply not about that topic.
Why it failsThe coefficients express degree. A small coefficient means a weaker relationship, not an absent one written as a binary label.
Do this insteadCompare coefficients against each other to judge relative strength, and read each one as a degree of relation.
Common mistake Assuming each topic already has a name
Why it failsThe source says the meaning of each component comes from the method's learned representation. Topics are not supplied in advance.
Do this insteadTreat components as learned entities, and interpret them from the representation rather than from a label you assumed beforehand.
Applying the Reading Rule
What do you think happens?
A document has coefficient 0.75 on Topic A and coefficient 0.25 on Topic B. Before reading on, which topic does the document relate to more strongly?
Reveal answer
Topic A
Larger coefficients mean stronger relationships, and coefficients express degree rather than a yes or no label. Since 0.75 is larger than 0.25, the document relates more strongly to Topic A.
A colleague says that because their model uses LDA, they are doing decomposition, and therefore they are doing something different from topic modeling. Using the distinction in this article, explain what is correct and what is confused in that statement.
Hints
- Recall that decomposition methods are the broader family, and topic modeling is related to them.
- Recall that LDA is one specific method often used for topic modeling, not a separate field that excludes it.
Key Takeaways
- Topic modeling is related to the decomposition methods from Chapter 3, and each learned component it produces corresponds to one topic.
- A document's coefficients on the components express how strongly that document relates to each topic, as a degree rather than a yes or no label.
- Latent Dirichlet Allocation (LDA) is one particular decomposition method, and it is often what people mean when they say topic modeling.
- Topic modeling names the general idea and LDA names one specific method, so the two terms should not be treated as identical.