Text Data and Document Representation
Topic modeling is an umbrella term for assigning each document to one or multiple topics, usually without supervision.
A Large Pile of Unlabeled Documents
A large share of the text data people work with is made of documents: news articles, reports, messages, and similar units of written language. Imagine a collection so large that nobody can read every item one by one. A natural question arises: what is each document about? This is the problem that topic modeling addresses.
Topic modeling does not ask a person to read every document and label it by hand. Instead, it looks at the collection as a whole and works out which topics the documents concern. Keep this in mind as we follow a single article through the process, because the question of how many topics that article can end up with is the heart of the lesson.
Tracing One Article Through the Process
Consider what happens to a single document under topic modeling. The document enters the process as a piece of text. The process then assigns it to one or several topics. Because the definition allows more than one topic, the document does not have to leave the process with a single label.
The important point is the fan-out at the topic assignment step. In topic modeling, the same document may leave along more than one arrow. The article itself is unchanged by the process; what changes is how many topic labels it is allowed to carry. A football club's financial troubles, for instance, could legitimately belong to both 'sports' and 'finance'.
What do you think happens?
Suppose the same football-finance article is handled by document clustering instead. How many topic labels can it receive at the assignment step?
Reveal answer
Exactly one
Document clustering is the special case in which each document is assigned a single topic. The article is limited to exactly one of 'sports' or 'finance', so the fan-out seen in topic modeling does not occur.
What Usually Means Without Supervision
The definition contains two phrases that carry most of its meaning. The first is 'one or multiple topics'. A document is not forced into a single bin; it may be connected to several topics at once.
The second phrase is 'usually without supervision'. Supervision in this sense means that the correct answer is supplied in advance, so the method learns from examples that someone has already labeled. Topic modeling is usually applied in the setting where such labels are not given. The topics have to be uncovered from the documents themselves.
Organizing a News Collection by Topic
The source uses news data as its illustration. A news collection might be categorized into topics such as 'politics', 'sports', 'finance', and so on. Each of these labels names a subject that a news story can be about. The example is useful because almost every reader already understands these categories, which makes it easier to see what it means for a document to be assigned to a topic.
Sorting a small news collection
A collection contains three articles: one about a parliamentary vote, one about a club's match result, and one about a bank's profit warning that also mentions the club's transfer budget. Which topics might each article carry under topic modeling?
- 1Read the collection as a whole
The process looks at all three articles together rather than labeling each one in isolation by hand.
- 2Identify candidate subjects
The vote article concerns politics, the match result concerns sports, and the bank article concerns finance.
- 3Allow more than one topic
The bank article also mentions the club's transfer budget, so it may connect to both finance and sports rather than being forced into one bin.
- 4Record the result
The vote article is primarily politics, the match article is sports, and the bank article can carry both finance and sports.
The collection is organized by topic, and the mixed article keeps more than one topic label.
Clustering Versus Topic Modeling
The source separates two situations according to how many topics a document may receive. If each document is assigned a single topic, the task is document clustering. If each document can have more than one topic, the task is topic modeling. Document clustering is therefore not a separate world from topic modeling. It is the special case in which every document is limited to exactly one topic.
| Feature | Document clustering | Topic modeling |
|---|---|---|
| Topics per document | Exactly one | One or multiple |
| Typical supervision | Usually without labels | Usually without supervision |
| Football-finance article | Forced into one topic | Can carry both sports and finance |
| Relation between tasks | Special case of topic modeling | Broader task that allows the special case |
The two tasks differ mainly in the limit on topics per document.
Mistakes Beginners Make
Common mistake Treating topic modeling as if every document must have one topic
What it looks likeLabeling a football-club finance story only as 'sports' because it mentions a match.
Why it failsThe definition explicitly allows a document to be connected to one or several topics at once.
Do this insteadAsk how many topics the document may carry, and allow more than one when the content supports it.
Common mistake Assuming that clustering and topic modeling are unrelated methods
Why it failsThe source says document clustering is the special case in which every document is limited to exactly one topic.
Do this insteadThink of clustering as topic modeling with the one-topic limit applied.
Common mistake Believing that 'without supervision' means the topics are chosen by a person in advance
What it looks likeExpecting a labeled training set with the answers already given.
Why it failsSupervision means the correct answer is supplied in advance. Topic modeling is usually applied where such labels are not given.
Do this insteadExpect the topics to be uncovered from the documents themselves.
Practice: Counting Topic Labels
A news article about a city council's decision to fund a new stadium also discusses how the stadium will change ticket revenue for the local club. Under topic modeling, which topics could this article reasonably carry? Under document clustering, how many topics would it receive, and which one would you choose?
Hints
- Ask whether the article concerns more than one subject.
- Under document clustering, each document is limited to exactly one topic.
- A good answer names the candidate subjects first, then applies the one-topic or multiple-topic limit.
Key Takeaways
- Topic modeling is an umbrella term for assigning each document to one or multiple topics, usually without supervision.
- Supervision here means the correct answers are supplied in advance; topic modeling usually works where such labels are not given.
- A news collection can be organized by topics such as politics, sports, and finance, and a single article may carry more than one of them.
- Document clustering is the special case of topic modeling in which each document is assigned exactly one topic.
- The key difference between the two tasks is whether a document may carry more than one topic.