Pipelines

When you build a machine learning model, the raw data you start with is rarely the data the model should see. Values may sit on very different scales, and a preprocessing step has to transform them before a classifier can learn from them. Keeping track of those steps by hand, in the right order, every time you train or test a model is tedious and easy to get wrong. The Pipeline class in scikit-learn lets you package several processing steps into one object, so the whole sequence can be handled as a single unit.

Keep one question in mind as you read: when you call a single method on a Pipeline, what happens to the data before the final model sees it? The rest of this article works through that question step by step.

The key word is single. Without a pipeline, a scaler lives in one variable, a classifier lives in another, and you must remember to apply them in sequence every time. With a pipeline, you hold one object, and that object already knows its ordered list of steps. The source introduces the Pipeline class in the context of tools such as cross_val_score and GridSearchCV, where it provides a way to combine processing steps with the model being evaluated.

Think of a Pipeline as an ordered chain. Data enters at the first step. That step's output becomes the input to the next step, and so on, until the final element, which is the supervised model. The most common arrangement described in the source is a preprocessing step such as scaling, followed by a classifier. In that arrangement the classifier never receives the raw features directly. It receives whatever the preprocessing step hands it. The preprocessing is part of the pipeline's definition, not a separate action you must remember to perform first.

Because the result is a single estimator, it can be passed to the same tools you would use with any other scikit-learn model. The Pipeline class itself has fit, predict, and score methods and behaves like any other model in scikit-learn. Once you know how to train and evaluate one estimator, you already know how to use a Pipeline, because the interface matches. The chaining happens behind that familiar interface.

The single line pipe.fit(X_train, y_train) stands in for several operations. The arguments are the training inputs and their labels, and both are given to the pipeline, which decides how each component uses them. The program never calls fit on the scaler or on the SVM directly. The pipeline object takes care of that.

The sequence is easier to follow as a set of calls over time. The pipeline is the caller, and it hands work to each component in turn.

The diagram shows the calls in order, but the important question is what the data looks like at each stage. The scaler's output is a different set of numbers from the data you started with, and the classifier is trained on that second version. Keeping the two versions distinct in your head is the key to tracing a fit correctly.

The following example is an original illustration for this article. It assumes that X_train, y_train, X_test, and y_test already exist as the feature and label splits of a dataset. It chains a scaling step with a logistic regression classifier. Each step is given a name, so you can refer to it later.

Once the pipeline is fitted, you need a way to judge how well it works. The source describes pipe.score for this purpose: to evaluate the model on test data, you call pipe.score. Like fit, the call goes to the whole pipeline. The test data passes through the same scaler and then into the same classifier that were set up during training.

The important detail is that score does not retrain the steps. The scaling uses what the scaler learned from the training data during fit, and the classifier uses what it learned during fit. The test data is only transformed and predicted on. This is why test data should stay separate from the training call. If you fit the pipeline on the test data, the evaluation no longer tells you how the model handles data it has never seen.

When the pipeline's output is unexpected, you do not need to guess which step caused it. The source advises tracing the input to each stage in order to find where the result diverges from expectations. Start at the front of the chain and check each stage's output against what you anticipated. The first stage whose output looks wrong is the place to look.

The flowchart is a simple habit to adopt. Inspect the data before the scaler, then the data after it, then the classifier's predictions. Each check narrows the search, and you stop at the first stage where the data stops matching your expectation.