Pipeline Class

Each step in a pipeline has a name, and the name is how the pipeline refers to that step.

Programming

A Chain That Lives Only in Your Head

A common first instinct is to treat a machine learning workflow as a set of separate actions. You scale the numbers, then you pick a feature, then you train a model, and then you check how well it works. Each action feels simple on its own, so it seems reasonable to run them one after another by hand and trust that you did the bookkeeping correctly.

Real machine learning work is rarely a single isolated use of a model. It is a sequence of processing steps, which may include feature extraction, preprocessing, and the model itself. Each step must receive its input in the form the previous steps prepared it, and the same chain must be repeated faithfully every time the model is evaluated on new data. The Pipeline class exists to make that chain a single, explicit object instead of an arrangement of lines in your code.

EASY

Before reading the next section, write down what you think could go wrong if you prepare a training set with one set of transformations, then a few lines later evaluate on a test set but apply only some of those transformations. What would the model receive as input at evaluation time, and how would you find out?

Hints

Think about which transformations were applied to the training data and which ones were actually applied to the test data.
Consider whether the model was trained on data that had been prepared in a different way from the data it is tested on.

Your attempt (not saved)

Why Hand-Run Preprocessing Goes Wrong

When every step is handled separately, the chain lives only in your head and in the order of your code. Nothing in the program ties the steps together. That makes it easy to apply a transformer to the training data and forget it on the test data, or to apply two transformers in the wrong order. The source notes that these errors can happen when processing chains are built without the Pipeline class, and that the Pipeline class reduces their likelihood.

The diagram below traces where each transformer is fitted and where it is only applied. A transformer that learns something from data, such as the average and spread used by a scaler, must learn that from the training split only. The test split then receives the already-fitted transformer, so the evaluation reflects what the model would see on unseen data.

Which data is each transformer fitted on, and which data only has the already-fitted transformer applied to it, when the train and test sets are split?

What a Pipeline Object Holds

A Pipeline encapsulates multiple steps into a single Python object. The steps are the transformers and the final estimator, and they are kept in the order you declared them. The whole chain is one object that follows the scikit-learn interface of fit, predict, and transform. The source describes this as the central idea: because the whole chain is one object, you work with the chain rather than with each step separately.

Each step in a pipeline has a name, and the name is how the pipeline refers to that step. The single Pipeline object is the thing your code works with, while the named steps inside it do the individual transformations. The diagram shows the order in which raw data passes through each transformer and then the final model, and what each step hands to the next.

In what order does raw data pass through each transformer and the final model, and what does each step hand to the next?

How fit, predict, and transform Flow Through

Because the chain is one object, you call fit on the pipeline rather than fitting each step yourself, and you call predict on the pipeline rather than passing data through each step manually. The same object is reused every time the chain is applied. This is what allows the pipeline to be handed to evaluation tools as if it were a single model.

fit: you give the pipeline training data, and the steps are fitted in order so that each one learns from the data as the previous steps prepared it.
transform: transformer steps prepare data in the same way they were prepared during training.
predict: you give the pipeline new data, and it passes that data through the same prepared steps before the final model produces its output.

A Small Pipeline in Code

The example below wraps a scaler and a classifier into one pipeline. The code is an illustrative example written for this article, using the scikit-learn library that the source's interface comes from. The names scale and model are labels we chose for the two steps. X holds the input features and y holds the labels, and both are assumed to be already loaded.

Code walkthrough

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

pipe = Pipeline([
    ('scale', StandardScaler()),
    ('model', LogisticRegression()),
])

scores = cross_val_score(pipe, X, y, cv=5)
print(scores.mean())

1
The first three imports bring in the Pipeline class and the two steps; the fourth brings in the cross-validation helper.
2
The Pipeline is built from a list of named steps. The scaler is listed first, so it runs before the classifier.
3
cross_val_score receives pipe as if it were one model. Every split repeats the full chain of scaling then classifying.
4
The mean of the fold scores summarizes how the whole chain performed, not just the classifier.

Import the Pipeline class and the two steps used in the chain.
Declare the steps as a list of (name, object) pairs; the order in the list is the order the data passes through.
Wrap the chain in one Pipeline object that can be handled like a single model.
Pass the whole pipeline to cross-validation; the evaluation code never needs to know what is inside it.

python

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Manual version: the scaler is fitted on ALL data before any split
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression()
model.fit(X_scaled[:800], y[:800])
# The test rows were already used to fit the scaler,
# and nothing in the code ties the two steps together.

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Manual version: the scaler is fitted on ALL data before any split
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression()
model.fit(X_scaled[:800], y[:800])
# The test rows were already used to fit the scaler,
# and nothing in the code ties the two steps together.

Notes on the code

The scaler learns its statistics from every row, including the rows later used for testing.
Nothing enforces that the test rows are prepared the same way as the training rows.

Cross-Validation Refits the Whole Chain

The source singles out model evaluation with cross-validation as a situation where capturing every processing step in a pipeline is essential. Data is evaluated across repeated splits, so each split needs its own correctly chained processing. The cross-validation code never needs to know which steps exist inside the pipeline. It treats the whole chain as one model, so every fold repeats the entire sequence of processing steps.

What happens to the training and validation split, and to the fitted steps, on each fold of cross-validation when the Pipeline is passed in?

Worked example

Tracing One Fold Through the Pipeline

Five-fold cross-validation is run on a pipeline with a scaler and a classifier. What happens to the steps during one fold?

1
Split the data
The data is divided into a training part and a validation part for this fold.
2
Fit the scaler on the training part
The scaler learns its preparation from the training part only, because the pipeline is fitted on that part.
3
Fit the classifier on the transformed training part
The classifier learns from the data as the scaler prepared it, in the order the steps were declared.
4
Transform and predict the validation part
The validation part passes through the already-fitted scaler and then the classifier, so it is prepared the same way as in training.
5
Record the score and move on
The fold score is kept, and the next fold repeats all of these steps from a new split.

Result

Each fold produces a score for the whole chain, and no step ever sees the validation part during its fitting.

Grid Search and Named Parameters

The source identifies parameter selection with grid search as the second situation where capturing all steps in a pipeline matters. Many candidate settings are compared, and every candidate must be evaluated through the same complete chain for the comparison to be fair. Steps can be given parameters that are searched over, and the pipeline lets those candidate settings travel with the steps they belong to. The source does not go into the search mechanics, so the key lesson is narrower: the pipeline is what keeps the processing consistent while the search runs.

python

from sklearn.model_selection import GridSearchCV

param_grid = {'model__C': [0.1, 1.0, 10.0]}
search = GridSearchCV(pipe, param_grid, cv=5)
search.fit(X, y)

from sklearn.model_selection import GridSearchCV

param_grid = {'model__C': [0.1, 1.0, 10.0]}
search = GridSearchCV(pipe, param_grid, cv=5)
search.fit(X, y)

Notes on the code

The key model__C names the parameter C belonging to the step labeled model.
Each candidate value is evaluated through the full pipeline, including the scaler.

The prefix before the double underscore tells the search which step a setting belongs to, and the part after it names the parameter inside that step. Changing the candidate value therefore changes only the classifier, while the scaler stays part of the same chain for every candidate.

Mistakes When Building Pipelines

Common mistake
Applying a transformer to the training data and forgetting it on the test data
What it looks like
Scaling X_train but passing the raw X_test to the model.
Why it fails
The model receives test input prepared differently from the input it was trained on, so the evaluation is not a faithful test.
Do this instead
Put the transformer and the model in one Pipeline and call predict on the pipeline with the test data.
Common mistake
Applying two transformers in the wrong order
Why it fails
Each step must see its input in the form the previous steps prepared it, so a reversed order gives the model different data than intended.
Do this instead
Declare the steps in the order the data should flow, since a Pipeline keeps the steps in the order you declared them.
Common mistake
Fitting a transformer on the full dataset before splitting
What it looks like
Calling fit on all rows, then splitting for cross-validation.
Why it fails
The test data influences how the transformer prepares the training data, which inflates the evaluation.
Do this instead
Pass the untransformed data to a pipeline and let cross-validation refit the whole chain on each split.
Common mistake
Writing a separate preprocessing function that is called in some places but not others

Best practice

Treat the pipeline as the unit you evaluate. If you want to compare a different scaler or a different model, swap one named step and rerun the same evaluation, rather than rewriting the processing chain each time. The source notes that this keeps trial-and-error experimentation far less tedious and leads to more succinct code.

Trace the Chain

MEDIUM

A colleague builds a pipeline with a scaler followed by a classifier, then calls cross-validation on it. Another colleague prepares a training set by scaling it by hand, fits a classifier, and then scales the test set by fitting a new scaler on the test set. For each approach, identify where the scaler is fitted, which data the classifier sees during evaluation, and which approach is more likely to produce an unfair evaluation.