Working With String Categories in Tabular Data

Many datasets are full of numbers that were never meant to be measured. A column might hold 0, 1, 2, and so on, while the thing it describes is a set of named options: a job type, a country, a survey answer. Once the labels have been replaced by digits, the table looks like it contains quantities, and it is easy to forget what the digits actually stand for. This article looks at how that happens, why it causes trouble when a model is trained, and how to decide what kind of feature an integer column really is.

The idea to hold onto is simple. Categorical data is often encoded as integers, either for ease of storage or because of how the data was collected. The meaning of such a column has to be recovered from context. The rest of this article shows how to do that step by step.

A common assumption goes like this: if a column contains integers, then its values are counts or measurements, so the column is continuous and the numbers can be compared, averaged, and fed to a model as ordinary magnitudes. This belief is the one to watch for. It treats the data type of the stored value as if it revealed the meaning of the feature.

It helps to see how the same information can appear in two different forms. In the adult dataset, categorical variables such as workclass were stored as strings like Private. Storing text has one advantage: it clearly marks a variable as categorical. It also has a drawback, because spelling errors become possible. Categorical data is frequently stored as integers instead, either because integers are easier to store or because of the way the data was collected.

What resolves the ambiguity is knowing what the numbers mean. Once you know that the codes indicate employment status, it becomes clear that these are very distinct states, and that they should not be modeled by a single continuous variable. The numbers themselves carry no such information. Their meaning lives in the documentation or in your understanding of the data collection, and you have to go and find it.

The central question for any integer feature is whether the encoded values have an ordering. Ordering means that moving from one code to another corresponds to a real, meaningful step along some scale. When no such ordering exists between the semantics that are encoded, the feature must be treated as discrete, and in practice it is typically one-hot encoded so that each state gets its own indicator. The workclass example falls into this group, since its codes name employment statuses that do not lie along any scale.

The top row shows an integer feature with a natural order. A rating of four stars is higher than a rating of two, so each step along the row corresponds to a real move on a scale. The bottom row shows codes that only label options. Code 0 is not less than code 1 in any meaningful sense, and the distance between code 0 and code 2 says nothing about how the options relate. A model that receives these numbers as magnitudes will read a meaning into the gaps that was never there.

Read the flowchart from the top. The first question is not about the data type at all. It asks what the numbers mean. If the values measure amounts, the feature is a quantity. If they name states, the second question asks whether moving between codes corresponds to a real step along a scale. A no answer leads to a discrete treatment, usually one-hot encoding. A yes answer leads to an ordered feature, which still needs a deliberate encoding choice, as discussed below.

Not every integer feature is clearly categorical. A five-star rating has a natural order, so the feature is not simply a set of unrelated labels. Even so, the source is careful on this point: for a case like five-star ratings, the better encoding depends on the particular task and data, and on which machine learning algorithm is used. There is no single correct answer that applies everywhere.