Classification Versus Clustering

Classification and clustering both attach a label to every item, and that resemblance is why the two are easy to confuse.

Programming

Why Cluster Numbers Look Like Classes

Many learners meet clustering for the first time through a familiar pattern. They have a table of examples, each row receives a number, and the numbers look exactly like the class labels from a classification problem. It is natural to conclude that cluster 0 means something specific, such as 'low values', and that cluster 2 means something else. This article tests that belief directly, because it is the most common way people misread the output of k-Means.

What do you think happens?

If k-Means labels one group as 0 and another as 2, does 0 mean something specific about those points?

Yes, cluster 0 always means low values
No, the numbers are only names the algorithm assigned to the groups

Reveal answer

Answer:

No, the numbers are only names the algorithm assigned to the groups

Clustering has no ground truth. Nothing in the algorithm tells you what group 0 represents. The only information the labels carry is that points sharing a number are similar to one another.

What Each Method Receives and Returns

Classification and clustering both attach a label to every item, and that resemblance is why the two are easy to confuse. The difference lies in what the learner is given. A supervised classifier is trained with labels supplied in advance, so the model learns which label goes with which pattern. k-Means receives only the feature matrix X. It is never given the true group for each row, so it has no ground truth to check its groups against. Its job is to discover groups and report which points belong together.

What is different about the inputs and outputs of a supervised classifier, where labels are given in advance, compared with k-Means, where no labels are given and groups are discovered?

The practical rule is this: a classifier's label has a meaning fixed by the training labels, while a cluster number has only the meaning you attach to it after inspecting the points. Keep that distinction in mind for everything that follows.

Fitting KMeans with a Chosen Cluster Count

Applying k-Means with scikit-learn follows a short sequence. You create an instance of the KMeans class and set the number of clusters you want the algorithm to find. The parameter is called n_clusters, and you must choose it before the algorithm runs. Then you call the fit method with your data. The number of clusters is therefore an input you decide on, not something the algorithm discovers for you.

Code walkthrough

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

X, y = make_blobs(n_samples=60, centers=3, random_state=0)

kmeans = KMeans(n_clusters=3, random_state=0)
kmeans.fit(X)

print(kmeans.labels_[:10])
print(kmeans.cluster_centers_)

1
The first line imports the KMeans class from scikit-learn.
2
make_blobs creates the feature matrix X and a true-group array y. Only X is used for clustering.
3
KMeans(n_clusters=3) creates an unfitted model that is set to look for three clusters.
4
fit(X) runs the algorithm on the feature matrix and stores the results on the kmeans object. It does not return a new table.
5
labels_ holds one cluster number per training row. Because three clusters were requested, the values run from 0 to 2.
6
cluster_centers_ holds the mean position of each cluster, with one row per cluster and one column per feature.

make_blobs generates a small synthetic two-dimensional dataset
y holds the true group memberships, but it is not passed to KMeans
n_clusters must be chosen before fit runs
fit groups the points and stores its results on the model object

Reading labels_ as Names, Not Meanings

During fitting, each training point in X is assigned a cluster label, and you retrieve all of them from the labels_ attribute. Those labels are the answer to the question 'which points belong together?' They are not answers to the question 'what kind of point is this?' The only information they give you is that points sharing a number are similar to each other. Nothing says that 0 is small, that 2 is large, or that 1 is anything in particular.

Renumbering is also possible between runs. Because the initialization is random, running the algorithm again can produce a different numbering of the same groups. The grouping can stay identical while the numbers attached to the groups change. The table below shows two runs over four points, A to D. Each run produces the same three groups, but the numbers are permuted.

How can a learner see that cluster numbers are names rather than meaningful values, when two runs group the same points but number the groups differently?

Worked example

Checking whether two label lists agree

Run 1 gives the labels [0, 0, 1, 2] for points A, B, C, D. Run 2 gives the labels [2, 2, 0, 1]. Do the two runs describe the same groups?

1
Find the points that share a label in run 1
Points A and B both have label 0, so they form one group. C and D have labels 1 and 2, so each is alone.
2
Find the points that share a label in run 2
Points A and B both have label 2, so they form one group. C and D have labels 0 and 1, so each is alone.
3
Compare the groups, ignoring the numbers
Both runs put A and B together and keep C and D separate. The partitions match.
4
Conclude
The numbers differ, but the groups are identical. The numbers are names, so the difference is only a renaming.

Result

Both runs describe the same three groups. Comparing the raw label values directly would wrongly suggest the runs disagree.

Reading cluster_centers_ as Summaries

A cluster is more informative when you can see its center. The cluster centers are stored in the cluster_centers_ attribute, and each center is the mean position of all the points assigned to that cluster. Because the data here is two-dimensional, each center is a pair of coordinates. The cluster_centers_ attribute therefore has one row per cluster and one column per feature. You can read column 0 and column 1 of a row directly from that attribute to place the center on a plot.

How does each training row relate to one entry in labels_, and how does each label relate to a row in cluster_centers_?

The number of clusters is a choice you make, and it changes the result. The same data clustered with two centers and with five centers produces different partitions. With two clusters, the points are split into two groups. With five, the algorithm carves the same data into more, smaller groups. Each run uses its own fitted model, so you can compare the assignments directly. Notice also that reusing the name kmeans for each new model replaces the previous object. If you want to keep several models, store them under different names.

Assigning New Points with predict

A fitted model can also label points it has never seen. The predict method takes new data and assigns each new point to the closest existing cluster center. The important detail is that the model does not change. Predicting does not refit the centers or move them toward the new points. It applies the centers that fit already learned. On the training set, predict agrees with labels_.

What do you think happens?

After you call predict on new points, do the entries in cluster_centers_ change?

Yes, the centers move toward the new points
No, the centers stay exactly as fit left them

Reveal answer

Answer:

No, the centers stay exactly as fit left them

Predicting does not refit the centers or move them. It only assigns each new point to its closest existing center.

python

new_points = [[0.0, 0.0], [5.0, 5.0]]
new_labels = kmeans.predict(new_points)
print(new_labels)
print(kmeans.cluster_centers_)

new_points = [[0.0, 0.0], [5.0, 5.0]]
new_labels = kmeans.predict(new_points)
print(new_labels)
print(kmeans.cluster_centers_)

Notes on the code

new_points holds two points the model was never fitted on
predict returns the closest center's label for each new point
cluster_centers_ is printed afterward to confirm it is unchanged

MEDIUM

Without running any code, describe what changes and what stays the same if you call predict on a new point and then call fit again on a different dataset. Which attributes are read, and which are replaced?

Hints

Predict reads the existing centers and does not modify them.
Calling fit again replaces the fitted results stored on the model object.

Your attempt (not saved)

Mistakes Beginners Make with Cluster Output

The errors below come up repeatedly when people first read k-Means output. Each one comes from treating clustering like classification.

Common mistake
Treating cluster 0 as meaning low values
What it looks like
Reporting that cluster 0 holds the small measurements because it has the lowest number.
Why it fails
The algorithm assigns numbers arbitrarily. Nothing in the fit ties 0 to any property of the points.
Do this instead
Describe a cluster by its members and by its center in cluster_centers_, then name it based on what you observe.
Common mistake
Comparing label numbers across two runs
What it looks like
Concluding that the points labeled 1 in run one and run two are the same group because the numbers match.
Why it fails
A rerun can renumber the same groups, so matching numbers do not guarantee matching groups.
Do this instead
Compare which points share a label within each run, not the raw numbers between runs.
Common mistake
Expecting centers to be data points
What it looks like
Searching X for an exact row equal to a row of cluster_centers_.
Why it fails
Each center is a mean of assigned points and may lie where no training example exists.
Do this instead
Treat cluster_centers_ as summaries and place them on a plot with the points rather than looking them up in X.
Common mistake
Expecting predict to update the model
What it looks like
Calling predict on new data and assuming the centers now reflect those new points.
Why it fails
Predict only assigns labels using the fitted centers and leaves the model unchanged.
Do this instead
Call fit again on the combined data if you want the centers to incorporate the new points.
Common mistake
Overwriting a model variable and then trying to compare with the old model

Debugging a Surprising Clustering Result

When a clustering result seems wrong, the fastest path is to find the step where your expectation and the code's behavior first diverged. Work through the sequence in order: check the data passed to fit, check n_clusters, then inspect labels_ and cluster_centers_. Each earlier step feeds the later ones.

1
Confirm the feature matrix X is the data you intended, and that no label column was passed in by mistake.
2
Confirm n_clusters is the number you meant, because it was chosen before fit ran.
3
Read labels_ to see which points share a label, ignoring the specific numbers.
4
Read cluster_centers_ to interpret what each group's average position represents.
5
If you need predictions for new data, use predict, and remember it does not refit the model.

Key Takeaways

k-Means is created with n_clusters, which you choose before fit, and fit(X) groups the points without ever seeing a ground truth.
labels_ holds one cluster number per training point, but those numbers are arbitrary names that can be renumbered between runs.
cluster_centers_ holds the mean position of each cluster and is the best place to interpret what a group represents.
predict assigns new points to their closest existing center without changing the fitted model, and on the training set it agrees with labels_.

Related concepts

More concepts starting with C