Transformers and Estimators in Machine Learning

A pipeline is easiest to understand by watching data move through it. This article starts with a small, runnable pipeline with two steps. The first step is a transformer that adds one to every number. The last step labels each number as low or high by comparing it with a cutoff. Before you read the trace, make a prediction: when you call predict on the raw numbers 3, 5, and 8, which values reach the labeling step?

The surprise in that answer is useful. If your guess was 3, 5, and 8, your mental model of the pipeline is missing the hand-off between steps. The rest of this article builds that missing piece, one step at a time, so that you can predict the path of data through any pipeline and find the exact point where a result goes wrong.

A pipeline stores its steps as a list of (name, estimator) tuples. The estimator for step i sits at index 1 of that tuple, so pipeline.steps[0][1] is the first estimator, pipeline.steps[1][1] is the second estimator, and so on. The name at index 0 is a label you choose. When you trace a pipeline, the index is what tells you which step you are looking at.

Read the diagram as a lookup. To reach the estimator for the first step, go to position 0 of the steps list, then take index 1 of the tuple you find there. The name tells you which step it is; the estimator is what actually does the work of fitting, transforming, or predicting.

The code below is a toy pipeline written for this article. The AddOne class is a transformer: it has a transform method that adds one to every number. The LowHighLabeler class is the final step: it has a fit method and a predict method that compares each value with a cutoff. Read the code first and notice how the steps are stored and which method calls are made. Note that this is a sketch that uses plain Python lists so that the data flow is easy to follow, not a production pattern.

The printed output is low, high, high. The label for 3 is low because its transformed value 4 is below the cutoff of 6. The labels for 5 and 8 are high because their transformed values, 6 and 9, are at or above the cutoff. The labeling step never sees the raw input 3, 5, 8; it sees 4, 6, 9. That is the hand-off the trace makes visible.

Fitting and prediction use the same steps but call different methods on them. The two paths are described separately below, because the difference between them is the key to reading any pipeline trace.

Every step except the last one is called with fit_transform during fitting and with transform during prediction. That means every intermediate step must be able to transform data. The last step is different: during fitting it is only fitted on the data it receives, and during prediction the pipeline calls its predict method. The source notes that the last step only needs a fit method. It does not have to provide predict.

This is why a pipeline does not have to end in a classifier. The source describes a pipeline built from a scaler and PCA. PCA has no classifier at all, but it does have a transform method. Because the earlier step only needs transform, and the last step only needs fit, you can call transform on the whole pipeline and get the output of PCA applied to the data that the earlier step has already processed. Pipelines are therefore not limited to classifiers, and they can be built from transformers alone.

When the final output of a pipeline is wrong, do not guess at the cause. Trace the data backward through the intermediate outputs and find the first step where the values stopped matching your expectation. The flowchart below shows that search.

In the toy pipeline, suppose you expected the labeling step to receive 3, 5, and 8. You would check its input and find 4, 6, and 9 instead. The first step's transform is the place where the values changed, and that is expected behaviour, so the mismatch points to your mental model rather than a bug. The point of the trace is to find the exact step where the data first differs from what you assumed, so you know which step to reason about.