Continuous and Categorical Features

Most of the features you have worked with so far are either numbers that measure a quantity or items chosen from a fixed list. Many real applications, however, contain a third kind of information: text. Consider a system that must decide whether an email is legitimate or spam. The content of the email will almost certainly carry information that matters for that decision. The same holds when someone wants to learn a politician's opinion on immigration from speeches or tweets, or when a customer service system must decide whether a message is a complaint or an inquiry from its subject line and content.

Before looking at text in detail, recall the two feature types already covered. A continuous feature describes a quantity. A categorical feature is an item from a fixed list. Text data is a third kind. It is not a number that can be compared or averaged, and it is not one of a fixed set of labels. Each kind of feature asks a different question about the data, and the difference determines how the data must be handled.

The comparison shows the key distinction. Continuous and categorical features each have a fixed kind of value per example. Text features are strings, and those strings can differ widely in length. That variable shape is the reason text cannot simply be dropped into an algorithm the way a number can.

A text feature is represented as a string made of characters, and strings have variable length. To see what that means, follow three values through the same check. The table below uses invented example strings and counts their characters, including spaces. Each row is one example from a hypothetical email dataset.

Every row holds one value for one example, just as a continuous or categorical feature would. But the value itself is a sequence of characters whose size changes from row to row. A continuous feature gives each example a single quantity, and a categorical feature gives each example one item from a list. The text feature gives each example a string whose shape is different in each case. This is the practical meaning of saying text has variable length.

Machine learning algorithms cannot directly handle raw text. Preprocessing is the work of converting raw text into a structured format suitable for analysis. The source describes this as a necessary step: the text has to be processed before any algorithm is applied to it. The stages below show the order in which that conversion moves a value from raw text toward something an algorithm can use.

The pipeline makes one point clear. Raw text is not the input the algorithm receives. Preprocessing sits between them, and it is a required stage rather than an optional refinement. The source does not describe the individual steps inside preprocessing, so this article does not list them.

The source gives examples where text carries the information a decision needs: the content of an email for spam detection, speeches or tweets for learning a politician's opinion, and the subject line and content of a message for a customer service system. In each case the information lives in the words themselves. A rough check that follows from the definitions is to ask three questions of the values. Does each example hold a quantity that can be compared or averaged? Does each example hold one item from a fixed list of labels? If neither is true, and each value is a string of characters, the feature is text.

The section introduces four kinds of string data that you may encounter in practice. The supplied material announces these four kinds but does not list or describe them. Because of that, this article does not name them or state how they differ. What can be said with confidence is that string data is the general form of the text features discussed above, and that the four kinds are a further classification that the surrounding material develops.