Features and Representations of Data

Most introductory machine learning examples start from a question that has a known answer, such as whether an email is spam or how much a house will sell for. Each example in the data comes with a correct answer attached in advance, so the algorithm can be shown what to predict. Unsupervised learning is different. The data arrives without labels. Nobody has attached a correct answer to any example, so the algorithm cannot be told what to predict. Instead, it works directly with the data it is given.

Keep one question in mind as you read: if the data has no labels to guide the work, what can an algorithm usefully do with it? The answer to that question starts with understanding how data is represented in the first place.

A dataset is always stored in some representation. Each example is described by a set of features, which are the properties that were recorded for it. The original representation is simply the set of features that were recorded. Nothing forces that description to stay as it is.

The diagram shows the key point. Each example on the left has a one-to-one counterpart on the right, so the transformation does not add new examples or remove the ones that were there. It changes how each example is described. This is the source of most confusion about the topic, so it is worth checking each time: count the examples before and after, and the count should match.

An unsupervised transformation is an algorithm that creates a new representation of a dataset. The new representation may be easier for humans or other machine learning algorithms to understand. Because there are no labels, the goal of the transformation is a new description of the data, not a predicted answer.

The definition carries two phrases that hold most of its meaning. The first is new representation. The output is a different description of the same data. It is not a prediction and not a label. The second is easier to understand. The reason for creating the new description is usefulness. A transformation is judged by whether the new form helps people or downstream algorithms, not by whether it matches an answer that was given in advance, because no such answer exists for unlabeled data.

A common application of unsupervised transformations is dimensionality reduction. Its starting point is a high-dimensional representation of the data, meaning a representation with many features for each example. Dimensionality reduction finds a new way to represent that same data so that the essential characteristics are still captured, but with fewer features per example.

The goal is best described as summarizing. The new representation is not meant to keep every detail of the original. It is meant to keep the characteristics that matter most, in fewer numbers per example. Some information is therefore given up on purpose, in exchange for a shorter description that is still useful.

A common choice of target for dimensionality reduction is two dimensions, chiefly for visualization. Two features can be drawn on a flat page or screen, with one feature on each axis. A dataset reduced to two dimensions can therefore be plotted and looked at directly. A dataset with four features, or forty, cannot be drawn that way on a flat surface, because there is no direct way to place it on a plane with one axis per feature.

In the reduced representation, each example has exactly two values, so each example becomes one point. The first value is read as the horizontal position and the second as the vertical position. Once every example is a point, the whole dataset can be drawn as a scatter of points and inspected by eye. Read this way, the two-feature representation is a practical tool for looking at data that would otherwise be impossible to picture.