Dendrograms and Reading Cluster Trees

Most introductions to clustering ask you to pick one grouping of your data. Hierarchical clustering asks a different question: what happens if we watch the grouping form, one step at a time, and keep every stage of that process? Instead of producing a single answer, it produces a family of answers. Each one is a valid grouping of the same points, but each has a different number of clusters.

Agglomerative clustering is the method that produces a hierarchical clustering. Its defining feature is the direction of its journey. It does not start from a few large groups and split them apart. It starts from the other end: every data point begins as its own one-point cluster.

The source describes the whole procedure as iterative. Each point makes a journey from being a single-point cluster to belonging to some final cluster. Notice what this says about the start and the end. At the start, the number of clusters equals the number of points. At the end, every point belongs to one of the final clusters. Everything in between is the process of points being gathered together.

Each intermediate step of the process is itself a clustering of the data. The steps differ only in how many clusters they contain. Early steps have many small clusters, and later steps have fewer and larger ones. Because every stage is a complete clustering, you do not have to commit to one stage in advance. You can look at the whole sequence and decide which level of grouping is useful for your purpose.

A concrete trace makes the iterative idea easier to see. Take five points, numbered 1 to 5. The source's teaching example follows the same pattern: points 1 and 2 join first, point 3 later joins them, and points 4 and 5 form their own pair. Follow how the number of clusters drops with each stage, and notice that every point is present in every stage.

The stages are linked. The clusters at one level are built from the clusters at the level before, and a cluster that appears in a later stage is made from clusters that existed earlier. This linking is what makes the result a hierarchy rather than a list of unrelated groupings.

A tree diagram shows that structure. The bottom row holds the individual points, and each line of merging moves upward until everything sits in one final cluster. Read the diagram from the bottom up. Points 1 and 2 join first, point 3 later joins them, and points 4 and 5 form their own pair. The top node is the stage where everything has merged. This diagram is a small teaching illustration. It follows the same pattern the source describes, in which every stage in between is a valid clustering.

Because each stage is a valid clustering, it is sometimes useful to look at all of them on one picture. The source does this with an overlay: it draws every clustering from the run together. The lines in the picture show the cluster assignments, and the data points are numbered so you can follow any one point through the stages. The overlay shows how each cluster breaks up into smaller clusters, which is exactly what a hierarchy is meant to reveal.

The source produces this picture with a helper plotting function from the mglearn package, which the course text uses for its illustrations. That call is a plotting call, not a clustering call on your own data. Treat it as a way to view the structure, not as the algorithm itself.

When you read an overlay, ask which lines stay together as you move from one stage to the next. A group of points that stays linked across several stages is a stable part of the structure. A group that splits apart quickly was gathered only at a fine level of detail.