Accessing Estimators Inside a Pipeline
make_pipeline is a convenience function that creates a pipeline and automatically names each step based on its class, so you do not have to write step names yourself.
Reaching Into a Finished Pipeline
A pipeline chains several processing steps together so that data passes through them in a fixed order. Once a pipeline exists, you will often want to look inside it. Perhaps you want the coefficients of a linear model, or the components extracted by PCA. These are properties of individual steps, not of the pipeline as a whole, so you need a way to get from the pipeline to one step and then to that step's attributes.
This article covers two tasks in sequence. The first is creating the pipeline, which can be done either by writing a name for every step or by using the convenience function make_pipeline. The second is accessing the steps once the pipeline exists, using the named_steps attribute. Keep both tasks in mind as you read.
Creating a Pipeline by Naming Every Step
When you build a pipeline by hand, you pass a list of (name, estimator) pairs. Each pair gives one step a label you chose yourself, and the estimator is the object that does the work for that step. The names are not decoration: they are the handles you will later use to find each step.
- Pipeline receives a list of (name, estimator) pairs
- 'scaler' is the name we chose for the scaling step
- 'clf' is the name we chose for the linear classifier step
This version is precise. You decide exactly what each step is called, so you can reach it later by a name you already know. The cost is repetition: every step needs its own string, and when you do not care what the steps are called, those strings are extra work that adds nothing to the result.
What make_pipeline Does for You
make_pipeline is a convenience function that creates a pipeline from the step objects alone. It assigns each step a name automatically, based on the step's class, so you do not have to write the names yourself.
- No pairs of strings are written
- The function generates a name for each step from its class
- The steps still run in the order they were given
The trade-off is control. Because you did not choose the names, you cannot assume what they are when you want to reach a particular step later. You must find out what names were generated. The next section shows how.
Finding the Generated Step Names
For the pipeline built with make_pipeline, the step names are the keys of the named_steps dictionary. Printing those keys is the most direct way to see them. Once you know a key, you can use it to get the matching step.
named_stepsis a dictionary from step names to estimatorskeys()lists the names thatmake_pipelinegenerated
The named_steps Lookup
The named_steps attribute is a dictionary from step names to estimators. It is the easiest way to access the individual steps of a pipeline. You give the step's name as the key and receive the estimator stored under that key.
Because named_steps is a dictionary, the lookup works exactly like any other dictionary lookup. This is why names matter. Whether you wrote them yourself or accepted the automatic ones, the key is the handle that connects you to the step. Think of the pipeline as not handing you its steps directly; it exposes them through this dictionary, and each key leads to exactly one estimator.
Reading a Fitted Step's Attributes
The source's main reason for reaching into a pipeline is to read a step's own attributes. For a linear model, the attribute coef_ holds the coefficients. The example below uses the hand-named chain, looks up the classifier by its name, and reads coef_. It assumes the pipeline has already been fitted on training data, which is a generated illustration.
- fit runs the pipeline's steps on the training data
named_steps['clf'] returns the classifier estimator stored under 'clf'- coef_ is an attribute of the classifier, not of the pipeline
Reaching a coefficient in two lookups
Given a fitted pipeline whose classifier step is named clf, how do you get its coefficients?
- 1Start at the pipeline
The variable explicit holds the whole pipeline.
- 2Look up the step
explicit.named_steps['clf'] uses the key clf to return the classifier estimator.
- 3Read the attribute
The classifier's coef_ attribute holds the coefficients of the fitted linear model.
clf.coef_ gives the coefficients, reached through the pipeline and its named_steps dictionary.
Mistakes Beginners Make When Accessing Steps
Common mistake Guessing the names of steps created by make_pipeline
What it looks likeauto.named_steps['scaler'] when the pipeline was built with make_pipeline(StandardScaler(), LogisticRegression())
Why it failsThe scaler and classifier were given no names, so 'scaler' and 'clf' were never created as keys. The generated names come from the classes.
Do this insteadPrint list(auto.named_steps.keys()) and use the keys that actually appear.
Common mistake Expecting the pipeline itself to have the classifier's attributes
What it looks likeauto.coef_Why it failscoef_ belongs to the linear model step, not to the pipeline as a whole.
Do this insteadLook up the step first, then read its attribute, such as auto.named_steps[...].coef_ with the correct key.
Common mistake Assuming make_pipeline changes the order in which steps run
Common mistake Treating make_pipeline as a different kind of pipeline
Why it failsThe source states that the result is still a pipeline that runs the steps in the order you gave them.
Do this insteadThink of make_pipeline as a shortcut for naming, with the same data flow as the explicit version.
Checking What Each Step Is
Before you rely on a step, confirm what it is. The keys of named_steps tell you the names, and each value is the estimator object itself, so you can check its type or read its attributes. This is the habit that prevents the most common errors in this topic: looking up a name that does not exist, or reading an attribute from the wrong step.
A pipeline was built as make_pipeline(StandardScaler(), LogisticRegression()) and then fitted. Without guessing, write the steps needed to read the coefficients of the classifier. Start by finding the generated names, then look up the classifier step and read its coef_ attribute.
Hints
- Start with the keys of
named_steps. - The classifier is the step whose class is LogisticRegression.
- The coefficients are an attribute of that step, not of the pipeline.
Summary of Accessing Pipeline Steps
- 1
make_pipelinecreates a pipeline from step objects alone and names each step automatically from its class. - 2Writing (name, estimator) pairs yourself gives you control over the names, which helps when you plan to reach steps by name later.
- 3The
named_stepsattribute is a dictionary from step names to estimators and is the easiest way to reach an individual step. - 4After looking up a step, you can read that step's own attributes, such as coef_ for a linear model or the components extracted by PCA.
- 5Read the generated names from
named_stepsrather than guessing them, becausemake_pipelinedoes not let you choose them.
At a glance
make_pipelinecreates a pipeline from step objects alone and names each step automatically from its class.- Explicit (name, estimator) pairs give you control over step names;
make_pipelinetrades that control for less writing. - The
named_stepsattribute is a dictionary from step names to estimators and is the easiest way to reach one step. - Once you reach a step, you can read its own attributes, such as coef_ for a linear model.
- Always read generated names from
named_stepsinstead of guessing them.