Estimators and the fit and transform methods

Many machine learning workflows pass through several stages in a fixed order. Raw data is cleaned and reshaped, then turned into features, and then fed to a model. Keeping track of that order by hand is error-prone. The difficulty grows when the same sequence has to be applied to new data later. A pipeline is the tool that packages such a sequence into a single object, so the stages are defined once and then run together.

This article answers two questions. The first is what a pipeline is allowed to contain. The second is what happens internally when you ask a pipeline to learn from data. The answer to the first question is more permissive than many beginners expect.

An estimator, in this context, is a component that learns from data. A pipeline chains several estimators in a row, so that the output of one becomes the input of the next. The chain is not fixed to any particular set of tasks. The number of steps is a design choice, and the chain can be any length that makes sense for the problem.

The source gives a concrete arrangement: a pipeline containing feature extraction, feature selection, scaling, and classification, for a total of four steps. Nothing about that particular arrangement is special. It is one valid chain among many.

The same chain could end in a regression estimator that predicts a continuous value, or in a clustering estimator that groups the samples. Changing only the last step does not change how the earlier steps are connected. The first three stages still hand their results forward in the same way.

Notice the asymmetry. Each non-final step is touched twice: first to learn from its input, and then to produce the output that moves on. The last step is touched once, to learn from the data it receives. This asymmetry is the reason only the non-final steps need a transform method.

The requirement above only makes sense once you see the order of calls. When you call Pipeline.fit, the pipeline does not simply fit every step on the original data. It works through the steps in order, and each step receives the output of the step before it.

Calling rules and data values are separate concerns. The calling rules tell you which methods run and in what order. The data determines what each step learns. Each class in a stand-in example has a fit method that learns something and, where needed, a transform method that returns new data. A helper function follows the rule above: for every step except the last, call fit, then transform, and pass the result onward; for the last step, call fit only.

Trace the data as well as the calls. The first two steps change the values, and the last step learns a summary of the values it is given. If you swapped the order of the first two steps, the intermediate values would change, even though the calling rules would stay exactly the same. This is why the order of steps matters even when the rules are simple.