Identifying Categorical Data
A column of text is not automatically categorical; categorical data comes from a fixed list of possible values.
Why Words Alone Do Not Make a Category
A common first instinct when a column is full of words is to label it categorical. The reasoning seems sound: the values are text, text looks like labels, and labels sound like categories. That instinct is incomplete. Whether a column counts as categorical depends on what its values are allowed to be, not on whether they are written as words.
What do you think happens?
A survey column holds the full name of each respondent, one name per row, with no two names the same. Is this column categorical?
Reveal answer
No, because the values are not drawn from a fixed list
A column of names, addresses, or free-text comments is made of strings too, yet it does not behave like a categorical variable. Being text is not enough; the values must come from a fixed set of possible values.
This article corrects that shortcut. You will see what makes a variable categorical, how to test a column rather than guess at it, and what to do when the test reveals labels that are really the same thing written in different ways.
Fixed Lists Versus Free Text
Categorical data comes from a fixed list of possible values. The test is not whether a value is written as a word but whether the column draws its values from a limited, predefined set. A column that records one of a few allowed answers for each row is categorical. A column where nearly every row contains a value nobody else has, such as a name or a comment, is not.
The left column has three values that repeat across many rows. The right column has a different value almost everywhere. Both are strings, but only the left one behaves like a categorical variable, because only the left one has a fixed list of possible values that rows are drawn from.
Walking Through the Three Checks
Once you suspect a column is categorical, test that suspicion. The test moves from a rough visual impression to a precise count and then to a judgment about meaning. Each check answers a different question, and no single check settles the matter alone.
- 1Visual scan: look at the column and ask whether it shows a very large variety of different strings. If it does, the variable is unlikely to be categorical. This check is fast but imprecise, so treat it only as an early warning.
- 2Unique values: compute the distinct values across the whole column. This turns the impression into a list. A short list where each entry is a sensible label makes the column a good candidate. A long list suggests free text or identifiers.
- 3Frequency counts: build a histogram of how often each value appears. A reasonable pattern supports categoricity. A few entries appearing only once each can reveal typos, stray entries, or a category that was only used briefly.
- 4Meaning review: confirm that each remaining value is a real, distinct label that belongs to the same kind of thing as the others.
Tracing a Column Through the Checks
Testing a ten-row survey column
A survey asks Do you use the service? and records the answer in one column of ten rows. Is this column categorical?
- 1Visual scan
The column shows only a few words repeated over and over, such as Yes and No. It does not show a wide variety of different strings, so the early warning does not fire.
- 2Unique values
The distinct values are Yes, No, and Maybe. That is a short list, and each entry is a sensible answer to the survey question.
- 3Frequency counts
Yes appears six times, No appears three times, and Maybe appears once. The pattern is reasonable for a survey answer, and no value looks like a stray entry.
- 4Meaning review
Yes, No, and Maybe are three distinct labels for the same kind of answer. The column draws from a fixed list of possible values, so it is categorical.
The column is categorical, with three categories: Yes, No, and Maybe.
The count of distinct values is three, and it stays three no matter how many rows the survey has. That fixed, small list is the signature of categorical data. If the same count instead grew with every new row, the column would be behaving like free text or identifiers.
Merging Spelling Variants Into One Label
Sometimes the check reveals a column that is categorical in principle but messy in practice. The same category appears under several spellings or formats. A typical source of this problem is a misspelling introduced by a mid-survey correction, which creates duplicate categories. Consolidation fixes this by merging labels that have the same meaning, so that each real category has one label.
Distinct labels: NY, New York, new york (trailing space) -> 3 categories
Three labels for one real place, so the category count is inflated.
Distinct labels: New York -> 1 category
One label per real category. The number of rows is unchanged.
Common mistake Calling a column categorical because its values are text
What it looks likeA column of customer names is treated as categorical because every entry is a string.
Why it failsBeing a string does not make a column categorical. Names come from no fixed list, so almost every row holds a different value.
Do this insteadCheck whether the column draws from a fixed set of possible values, then run the unique value and frequency checks.
Common mistake Skipping the frequency check after finding a short list
What it looks likeA column shows four distinct labels, but one of them appears only once in a survey of hundreds.
Why it failsA short list can still hide a typo or a stray entry that the unique value list alone does not reveal.
Do this insteadBuild a histogram of how often each value appears, and investigate entries that appear only once.
Common mistake Leaving near-duplicate labels unmerged
What it looks likeThe labels NY and New York are kept as two separate categories.
Why it failsThey refer to the same real category, so the category count is inflated and the groupings are split.
Do this insteadMerge labels that have the same meaning into one canonical label.
Judging a Column for Yourself
A column records the city of birth for each participant in a study. Across 200 rows, the unique value list has more than 150 different entries. Using the three checks, decide whether this column is categorical, and explain which check gave you the answer.
Hints
- Ask whether the values are drawn from a fixed, limited list.
- Compare the number of distinct values to the number of rows.
- Think about whether a few labels might be spelled differently for the same city.
Putting the Checks Together
- 1A column of text is not automatically categorical. Categorical data comes from a fixed list of possible values.
- 2The three checks move from a visual scan, to the unique values, to frequency counts, and finish with a meaning review. No single check settles the question alone.
- 3A short unique value list with a reasonable frequency pattern suggests a categorical column, while a list that keeps growing with each row suggests free text or identifiers.
- 4Consolidation merges labels that share a meaning, such as NY and New York, restoring one label per real category without changing the number of rows.
Key Takeaways
- A column of text is not automatically categorical; categorical data comes from a fixed list of possible values.
- Test a suspected categorical column with a visual scan, a unique value list, a frequency histogram, and a meaning review.
- A short list of unique values that keeps the same size as the rows grow indicates free text or identifiers, not categories.
- Consolidation merges spelling variants and inconsistent labels into one label per real category, changing labels but not rows.