Building pipelines with make_pipeline

In scikit-learn, a pipeline is the tool that lets you chain those processing steps together so they behave like one object.

Programming

Predicting Who Sees the Raw Text

A text classifier needs numbers, not strings, so something has to convert each document into numeric features before learning can happen. In scikit-learn, a pipeline is the tool that lets you chain those processing steps together so they behave like one object. This article starts with a small text classification setup and asks you to watch what happens to the data as it moves through it.

What do you think happens?

When grid.fit(text_train, y_train) is called on a pipeline whose first step is TfidfVectorizer and whose second step is LogisticRegression, which object receives the raw text, and does LogisticRegression ever receive the raw strings?

Reveal answer

Answer:

The TfidfVectorizer receives the raw text first. It converts the text into a TF-IDF matrix of numeric features, and the LogisticRegression step receives that matrix, not the strings.

The source describes the vectorizer as transforming text into a matrix of TF-IDF features and says that matrix, not the raw strings, is what the classifier receives. Keep your guess in mind as you read the next sections, because the rest of the article checks it against the mechanism.

Tracing One Fit Call Through the Chain

The pipeline has two steps, and they run in the order they were listed. The text enters the vectorizer first. The vectorizer produces a TF-IDF matrix, which is a table of numeric features with one row per document. That matrix is then handed to the classifier, which learns from those features. The text never reaches the classifier directly.

When I pass raw text to pipe.fit(), where does the data go at each step, and what form does it take when it reaches the classifier?

Notice what this diagram answers. The raw strings stop at the first box. Everything after the vectorizer works with numbers. If your mental model had LogisticRegression reading text, this is the point where that assumption breaks, and it is the most important thing to check against your prediction.

Building the Pipeline with make_pipeline

Definition

make_pipeline chains TfidfVectorizer(min_df=5, norm=None) and LogisticRegression so that text flows through the vectorizer into the classifier. A pipeline is a way to chain multiple data processing steps together.

The source snippet begins by importing the tools it needs. make_pipeline comes from the pipeline module, TfidfVectorizer comes from the text feature extraction module, LogisticRegression comes from the linear model module, and GridSearchCV comes from the model selection module. Each of these is a standard scikit-learn module.

python

from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV

pipe = make_pipeline(TfidfVectorizer(min_df=5, norm=None), LogisticRegression())
param_grid = {'logisticregression__C': [0.1, 1, 10]}
grid = GridSearchCV(pipe, param_grid, cv=5)
grid.fit(text_train, y_train)
print(f'{grid.best_score_:.2f}')

from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV

pipe = make_pipeline(TfidfVectorizer(min_df=5, norm=None), LogisticRegression())
param_grid = {'logisticregression__C': [0.1, 1, 10]}
grid = GridSearchCV(pipe, param_grid, cv=5)
grid.fit(text_train, y_train)
print(f'{grid.best_score_:.2f}')

Notes on the code

The imports are required; without them the names used below are undefined.
make_pipeline builds one object from the vectorizer and the classifier, listed in the order they run.
min_df=5 and norm=None are the vectorizer settings from the source snippet.
The candidate C values in param_grid are illustrative. The source does not list them.
text_train and y_train are assumed to exist; this article does not define a dataset.

The pipe object is a single estimator. Calling pipe.fit(text_train, y_train) runs the vectorizer on the text and then fits the classifier on the resulting matrix. You do not call the vectorizer and the classifier separately, and you do not pass the matrix between them yourself. You give the pipeline the text, and each step is applied in the order it was listed.

Question	Without a pipeline	With make_pipeline
How many fit calls do you write?	One per step, in order	One call on the pipeline
Who passes the matrix to the classifier?	You do, by storing the output	The pipeline does
What object do you pass to GridSearchCV?	Each step separately, which is awkward	The single pipe object

What make_pipeline removes from the workflow, based on the source's description of combining the two steps.

Reading a Step Name Inside a Parameter Key

GridSearchCV is a tool for hyperparameter tuning using cross-validation. It needs to know which parameter to change, and the pipeline contains several objects, so the key in param_grid has to say which step owns the parameter. The source uses the key 'logisticregression__C' rather than the plain 'C'. The part before the double underscore names the step, and the part after it names the parameter of that step.

How does a key like 'logisticregression__C' find the right object inside the pipeline, and which step does each part of the key point to?

Worked example

Decoding a parameter key step by step

Given the key 'logisticregression__C' in param_grid, which pipeline step does it target, and which parameter of that step does it set?

1
Find the double underscore
The key contains one double underscore, which separates the step name from the parameter name.
2
Read the prefix
The text before the separator is logisticregression. This is the step name, and the source says it comes from the lowercase name of the class in the pipeline.
3
Read the suffix
The text after the separator is C. This is the parameter of that step, which the source describes as the regularization strength of LogisticRegression.
4
Match to the pipeline
GridSearchCV looks up the step named logisticregression inside pipe and sets its C parameter to each candidate value during the search.

Result

The key targets the C parameter of the LogisticRegression step, not the vectorizer and not a top-level parameter.

The same naming rule applies to any step. The source's own key is logisticregression__C, and a key such as tfidfvectorizer__min_df would follow the same pattern for the vectorizer's min_df setting. Because the pipeline is one estimator object, GridSearchCV can tune a parameter of any step inside it while still evaluating the whole chain with cross-validation.

Reading the Search Result

After grid.fit runs, the snippet reads grid.best_score_ and prints it formatted to two decimal places. This value is the best cross-validation score found across the candidate settings in param_grid. The source does not show the printed number, so no expected value is given here. When you run the code, the output is the single line the print call produces.

python

print(f'{grid.best_score_:.2f}')

print(f'{grid.best_score_:.2f}')

Notes on the code

grid.best_score_ is the best cross-validation score, not a score on separate test data.
The format specifier .2f rounds the value to two decimal places for display.

Mistakes That Make a Pipeline Run Diverge

A pipeline run can go wrong in three places: the imports, the step names, and the parameter keys. The flowchart below orders the checks so that the cheapest and most common problems are ruled out first.

If the run fails or ignores a parameter, which check should I perform first, and what does each check rule in or out?

Common mistake
Using the plain key 'C' in param_grid
What it looks like
```
param_grid = {'C': [0.1, 1, 10]}
```
Why it fails
The key does not name a step, so the search cannot tell which object inside the pipeline should receive the parameter.
Do this instead
Use the step-prefixed key, such as 'logisticregression__C'.
Common mistake
Assuming the classifier receives raw text
What it looks like
Expecting LogisticRegression to read the strings from text_train
Why it fails
The vectorizer runs first and produces a numeric TF-IDF matrix, which is what the classifier receives.
Do this instead
Pass the raw text to the pipeline and let the vectorizer step handle the conversion.
Common mistake
Forgetting an import
What it looks like
Calling make_pipeline without importing it from sklearn.pipeline
Why it fails
The name is undefined, so the snippet cannot run.
Do this instead
Include all four imports the snippet uses before building the pipeline.
Common mistake
Guessing the step name from the variable name
What it looks like
Writing 'vectorizer__min_df' because the variable is called vectorizer
Why it fails
The source derives the step name from the lowercase class name, not from a variable name.
Do this instead
Use tfidfvectorizer__min_df for the TfidfVectorizer step.

Practice Tracing a Pipeline

MEDIUM

A colleague writes param_grid = {'tfidfvectorizer__min_df': [2, 5]} and then runs GridSearchCV on the same pipe built from TfidfVectorizer and LogisticRegression. Using the checks in this article, identify whether the key is correct, which step it targets, and what the search does with each candidate value.

Hints

Split the key at the double underscore and read each part separately.
Ask which object inside the pipeline owns a parameter named min_df.
Remember that GridSearchCV tries each candidate value and scores each one with cross-validation.

Your attempt (not saved)

Summary of the Pipeline Mechanism

1
make_pipeline chains TfidfVectorizer and LogisticRegression so that one fit call runs the whole chain in listed order.
2
The vectorizer converts raw text into a TF-IDF matrix, and the classifier receives that matrix rather than the strings.
3
GridSearchCV tunes a step's parameter through a key such as 'logisticregression__C', where the prefix names the step and the suffix names the parameter.
4
When a run diverges, check the imports first, then the step names, then the parameter keys.

At a glance

make_pipeline chains TfidfVectorizer and LogisticRegression so one fit call processes the whole chain in the order the steps were listed.
The TfidfVectorizer turns raw text into a TF-IDF matrix, and the classifier receives that matrix rather than the strings.
GridSearchCV tunes a step's parameter through a prefixed key such as 'logisticregression__C', where the prefix names the step and the suffix names the parameter.
When a pipeline run diverges, check the imports, then the step names, then the parameter keys.

Related concepts

More concepts starting with B