Fitting a Pipeline Like Any Other Estimator

A pipeline is defined by a list, and each item in that list is a two-item tuple.

Programming

One Call, Two Trained Components

A machine learning workflow often has more than one stage. Before a support vector machine (SVM) is trained, the data may first need to be scaled. You could write those stages as separate manual calls, but that spreads the workflow across several lines and makes it easy to apply the stages in the wrong order or to forget one. The Pipeline class in scikit-learn offers a different approach: you describe the whole workflow once, as a single object, and then treat that object like any other scikit-learn estimator.

Look at the central line of the example program you will meet in this article: pipe.fit(X_train, y_train). It is a single call. Inside it, a scaler and an SVM are both trained. The program never calls fit on the scaler or on the SVM directly; the pipeline object takes care of that. The question this article answers is what happens inside that call, in what order, and with which data.

What do you think happens?

The pipeline has a scaler and a classifier, and you have only one fit call. Which component gets trained first?

The scaler
The SVM

Reveal answer

Answer:

The scaler is fitted first. The SVM is fitted afterwards, on the scaled version of the training data.

A pipeline runs through its steps in a fixed order. The scaler comes first in the sequence, so it learns from the training data before the final model does.

Tracing a List of Named Steps

Before reading any scikit-learn mechanics, it helps to see the same shape built from plain Python values. A pipeline is defined by a list, and each item in that list is a two-item tuple. A loop can unpack each tuple into two variables, and this unpacking is the same mechanism that lets you read a step's name and its estimator separately.

Code walkthrough

steps = [('scaler', 'MinMaxScaler instance'), ('svm', 'SVC instance')]
for name, estimator in steps:
    print(name, '->', estimator)

1
Line 1 builds a list containing two tuples. Their order is the order of the workflow.
2
Line 2 starts a loop that reads one item at a time from the list, from first to last.
3
Line 2 also unpacks each tuple into two variables, name and estimator.
4
Line 3 prints the two parts. The loop visits scaler first and svm second, which is the order the pipeline will use.

steps is a list; each of its two items is a tuple with exactly two entries
The for loop visits the items in order, first ('scaler', ...) and then ('svm', ...)
Unpacking assigns the first entry of each tuple to name and the second to estimator

Output

scaler -> MinMaxScaler instance
svm -> SVC instance

As the loop moves through the list, which tuple is read at each position, and where does a malformed item break the sequence?

Trace the loop by hand: at position 0 the loop reads the scaler tuple, at position 1 it reads the SVM tuple, and each read gives a name and an estimator. The diagram marks the position where the sequence would break. The next section shows what a broken item looks like and how to find it.

Anatomy of a Single Step

A pipeline step is a tuple with two parts, and the parts play different roles. The name is a label you choose so that you can refer to the step later. The source says it can be any string of your choosing. The estimator is the object that does the actual work, and it must be an instance, meaning it was created by calling the class.

What are the two parts of one step, and how do the name and the estimator object fit together inside the tuple?

In the source example, the name scaler belongs to a MinMaxScaler instance, and the name svm belongs to an SVC instance. The name is what you read; the instance is what does the work. Keeping those two roles separate makes the rest of the pipeline easier to read.

Building the Pipeline Object

A Pipeline object is created by passing it a list of steps, and that list expresses the workflow for training a model. The example below assumes that MinMaxScaler and SVC have already been imported, so the focus stays on how the pipeline is assembled. The list holds two steps: a scaler named scaler, and an SVM named svm. Notice that each step is a tuple, written with parentheses inside the square brackets.

python

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC

pipe = Pipeline([
    ('scaler', MinMaxScaler()),
    ('svm', SVC()),
])

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC

pipe = Pipeline([
    ('scaler', MinMaxScaler()),
    ('svm', SVC()),
])

Notes on the code

The list order sets the order of the workflow: scaler first, svm second
Each item is a (name, estimator instance) tuple
The pipeline variable holds one object that stands in for both steps

Once the pipeline object exists, it can be fitted in the same way as any other scikit-learn estimator. The variable pipe now holds one object representing the whole sequence, not two separate models.

Finding a Broken Step Definition

When a pipeline definition fails, the error often points to the pipeline call rather than to the step that caused it. A practical way to find the faulty step is to inspect the list of steps before passing it in. Each item should be a tuple with exactly two entries. The check below uses plain Python values and reports the type and length of each item.

Code walkthrough

steps = [
    ('scaler', 'MinMaxScaler instance'),
    ['svm', 'SVC instance'],
]
for i, item in enumerate(steps):
    print(i, type(item).__name__, len(item))

1
Line 1 starts the list of steps.
2
Line 3 puts a list in the second position, using square brackets instead of parentheses.
3
Line 5 loops over each item with its position.
4
Line 6 prints the position, the type name, and the number of entries. Position 1 reports list, so that is the item to change, not the whole list.

Position 0 is a tuple, as the source requires
Position 1 is a list with the right number of entries but the wrong type

Output

0 tuple 2
1 list 2

The output shows where the problem is. The second item has the right number of entries, so a length check alone would miss it. Checking the type as well locates the faulty step precisely, and the same kind of check works on the real steps you intend to pass to Pipeline.

Common mistake
Writing a step as a list instead of a tuple
What it looks like
```
['svm', SVC()] in place of ('svm', SVC())
```
Why it fails
The source requires each step to be a tuple containing a name and an estimator instance.
Do this instead
Use parentheses for each step: ('svm', SVC()).
Common mistake
Passing an estimator class instead of an instance
What it looks like
```
('svm', SVC) instead of ('svm', SVC())
```
Why it fails
The estimator is the object that does the work, and the source defines it as an instance created by calling the class.
Do this instead
Call the class with parentheses so that the step holds an instance.

What pipe.fit Does, Step by Step

A pipeline is a sequence of data processing steps followed by a model. Calling fit on the pipeline runs through that sequence in a fixed order. Each processing step learns from the training data and then passes transformed data forward, and the final model learns from whatever arrives at the end. For the scaler and SVM pipeline, pipe.fit calls fit on the first step, transforms the training data with it, and then fits the final model on the transformed data.

Definition

pipe.fit trains a pipeline in order: it calls fit on the first step, transforms the training data, and then fits the final model on the transformed data.

Following the Training Data from Scaler to SVM

python

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

pipe = Pipeline([('scaler', MinMaxScaler()), ('svm', SVC())])
pipe.fit(X_train, y_train)
print(pipe.score(X_test, y_test))

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

pipe = Pipeline([('scaler', MinMaxScaler()), ('svm', SVC())])
pipe.fit(X_train, y_train)
print(pipe.score(X_test, y_test))

Notes on the code

The data is generated synthetically, so the example is self-contained
One pipe.fit call trains both components; there is no direct fit call on the scaler or the SVM
pipe.score is called after fitting, on data kept separate from the training call

Scaling is a preprocessing step that transforms data to have similar magnitudes. That is the reason the scaler sits at the front of the pipeline: the training data it hands to the SVM is a different set of numbers from the data you started with. Keep two versions of the data distinct when you trace a fit. The raw training values enter the pipeline; the scaled values reach the SVM. The final model never sees the raw training values, only the scaled ones that the scaler produced.

When pipe.fit(X_train, y_train) runs, how does the training data move from the scaler to the SVM, and what does each step pass on to the next?

The diagram traces the training data from its starting point through the scaler and into the SVM. The scaler is fitted first, and the SVM is fitted on scaled data rather than on the original training values. This is the order in which pipe.fit visits the steps, and it is fixed by the order of the list you wrote.

Evaluating a Fitted Pipeline with pipe.score

Once the pipeline is fitted, you need a way to judge how well it works. The source describes pipe.score for this purpose: to evaluate the model on test data, you call pipe.score. Test data should stay separate from the training call. Like fit, the call goes to the whole pipeline, so the test data passes through the same scaler and then into the same SVM that were set up during training. The scaler is not refitted on the test data; it transforms that data using what it already learned.

MEDIUM

Trace a pipeline by hand. Given the steps [('scale', MinMaxScaler()), ('model', SVC())], which step is fitted first, which step receives scaled data, and which call evaluates the pipeline on test data? Then, if the list were written as [('scale', MinMaxScaler()), ['model', SVC()]], at which position would the definition break, and why?

Hints

Items in the list are read in order, from position 0 upward.
A step must be a tuple with exactly two entries, a name and an estimator instance.
Check the type of each item, not only its length.

Your attempt (not saved)

Key Takeaways

A Pipeline object is created from a list of steps, and each step is a tuple holding a name you choose and an estimator instance.
Calling pipe.fit runs through the steps in order: it fits the first step, transforms the training data, and fits the final model on the transformed data.
The scaler is fitted first and the SVM is fitted on scaled data, not on the original training values.
A malformed step, such as a list instead of a tuple, can be located by checking the type and length of each item in the list before passing it to Pipeline.
pipe.score evaluates the fitted pipeline on separate test data, passing it through the already fitted steps.

Related concepts

More concepts starting with F