Concepts / Pipeline class in scikit-learn

Pipeline class in scikit-learn

The most common use case of a Pipeline is linking a preprocessing step, such as scaling, with a classifier. Whatever preprocessing is done is tied directly to the model that depends on it.

  • Programming

Why Separate Stages Need Bookkeeping

Machine learning work often happens in stages. Raw data frequently needs to be transformed before a learning algorithm can use it. A common example is scaling, where the numeric values of the data are adjusted so that the features are on a comparable footing. Only after that transformation does the data go to a supervised model such as a classifier.

You can keep these stages as separate objects and call them one after another. That works, but it leaves you responsible for two things: remembering the correct order of the stages, and passing each stage's output into the next one. As the workflow grows, that bookkeeping is easy to get wrong.

The Pipeline class in scikit-learn answers this problem. It links the preprocessing and the supervised model into one unit, so that a single object handles the whole path from raw input to prediction.

Tracing Data Through the Stages

Think of a Pipeline as a sequence of named stages. Data enters at the first stage, where a preprocessing step such as scaling transforms it. The transformed data then passes to the next stage, which is a supervised model like a classifier. The classifier is the final stage, and it produces the predictions.

From the outside, you supply the raw data and receive results. The movement between stages happens inside the Pipeline. The diagram below traces that movement for a two-step pipeline, showing what each stage receives and what it hands on.

Read the diagram from left to right. The scaling step never hands raw values to the classifier; it hands on its transformed output. The classifier is the only stage that produces predictions. This is the reason the stages are described as a sequence: each one depends on the output of the one before it.

The Pipeline as a Single Model

Definition

The Pipeline class in scikit-learn chains preprocessing steps together with supervised models, and it presents the whole chain through the same interface as any other model.

The defining feature of a Pipeline is that it presents itself as a model. The Pipeline class itself has fit, predict, and score methods, and it behaves just like any other model in scikit-learn. Once you have built a Pipeline, you do not need to remember its internal stages in order to use it. You call the same methods you would call on a standalone model, and the Pipeline takes responsibility for the steps inside it.

Key point

The most common use case of a Pipeline is linking a preprocessing step, such as scaling, with a classifier. Whatever preprocessing is done is tied directly to the model that depends on it.

What Happens During fit, predict, and score

Because the Pipeline has the same methods as a standalone model, a call such as fit on the Pipeline is the single entry point for training the whole workflow. Inside the Pipeline, the preprocessing stage handles the data first, and the supervised model is the stage that learns from the transformed data. The sequence diagram below follows one call to fit from start to finish.

The diagram shows that you make one call, and the Pipeline routes the data. The scaling step receives the raw training features and returns transformed values. The classifier then trains on those transformed values together with the labels. The chain of stages is hidden behind the one call you wrote.

The predict and score methods follow the same pattern: the input you supply is passed through the preprocessing stage first, and the classifier then produces predictions from that transformed input. Because each call goes through the whole chain, the data the classifier sees during prediction is prepared in the same way as the data it was trained on.

Building a Pipeline from Named Steps

The example below is an original illustration of the pattern described above. It chains a scaling step with a logistic regression classifier. Each stage is given a name, and the Pipeline is built from a list of these named pairs in the order the data should flow. The variables X_train, X_test, y_train, and y_test are assumed to already exist from an earlier train/test split.

Code walkthrough
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression()),
])

pipe.fit(X_train, y_train)
predictions = pipe.predict(X_test)
accuracy = pipe.score(X_test, y_test)
  1. 1
    Line 1 to line 3: the three classes are imported from scikit-learn.
  2. 2
    Lines 5 to 8: the Pipeline is built from an ordered list of named stages, with the scaler listed first so that data reaches it first.
  3. 3
    Line 10 a single fit call trains every stage in order.
  4. 4
    Line 11 predict returns predictions for X_test, passing it through the scaler and then the classifier.
  5. 5
    Line 12 score returns a score for the Pipeline on the test data, using the same interface a standalone model would have.
  • The import lines bring in the Pipeline class, a scaling transformer, and a logistic regression classifier.
  • Each element of the list is a (name, step) pair; the first pair is the preprocessing stage and the second is the classifier.
  • fit trains the whole chain: the scaler transforms the training features, then the classifier learns from them.
  • predict and score take raw test data and pass it through the same chain.
Before
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
clf = LogisticRegression()
clf.fit(X_train_scaled, y_train)
predictions = clf.predict(X_test_scaled)

Each stage is a separate object, and you pass each stage's output into the next one by hand.

After
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression()),
])
pipe.fit(X_train, y_train)
predictions = pipe.predict(X_test)

The stages live inside one Pipeline object, and you call fit and predict on that single object.

Mistakes When Chaining Steps

The most common mistakes come from the bookkeeping the Pipeline is meant to remove. When stages are handled separately, it is easy to forget a stage or to hand the wrong data to the next one.

  • Common mistake

    Passing raw data straight to the classifier and skipping the preprocessing stage

    What it looks like

    clf.fit(X_train, y_train) when the scaling step was meant to run first

    Why it fails

    The classifier is supposed to learn from the output of the preprocessing stage. When you bypass that stage, the classifier works with values that were never transformed.

    Do this instead

    Put both stages in one Pipeline and call fit on the Pipeline, so the order is handled for you.

  • Common mistake

    Forgetting that the output of one stage must become the input of the next

    What it looks like

    Calling predict on the classifier with unprocessed test features after fitting it on scaled training features

    Why it fails

    Each stage depends on the output of the stage before it. Breaking that link means the stages no longer describe one workflow.

    Do this instead

    Wrap the stages in a Pipeline so that predict on the Pipeline passes the input through the whole chain each time.

  • Common mistake

    Treating the Pipeline as a collection of separate objects instead of one model with fit, predict, and score

Tracing a Pipeline Yourself

EASY

Suppose a Pipeline is built as a list with a scaling step named 'scaler' first and a logistic regression classifier named 'clf' second. Before you run anything, trace what the scaler receives when you call fit on the Pipeline, what the classifier receives, and what the Pipeline returns when you call predict on new test data.

Hints
  • Start from the first stage in the list: that is where raw data enters.
  • Ask which stage's output becomes the input of the next stage.
  • Remember that predict and score pass the input through the same chain before the classifier produces results.

Once you have traced the answer, check it against the diagrams above. If you can explain why the classifier never receives the raw training features directly, you understand how the Pipeline chains the stages.

Key Takeaways

  • The Pipeline class in scikit-learn chains preprocessing steps, such as scaling, together with supervised models like classifiers.
  • A Pipeline has fit, predict, and score methods and behaves like any other model, so you call those methods directly instead of managing each stage.
  • A Pipeline is built from an ordered list of named stages; data enters at the first stage and each stage passes its output to the next.
  • Handling stages separately leaves you responsible for their order and for passing outputs along, which is the bookkeeping a Pipeline removes.

Related Topics

Related concepts

More concepts starting with P