Tuning a Pipeline with Grid Search
A Pipeline object is created by passing it a list of steps, which expresses a workflow for training a model.
A Workflow Written Once
Many machine learning workflows have more than one stage. Before a support vector machine (SVM) is trained, the data may first need to be scaled. You could write each stage as a separate manual call, but that spreads the workflow across several lines and makes it easy to apply the stages in the wrong order or to forget one. The Pipeline class in scikit-learn offers a different approach: you describe the whole workflow once, as a single object.
This article focuses only on the definition step: how you build the Pipeline object. The source example uses a scaler followed by an SVM. The grid search that would normally tune the SVM is left out for now, and training and evaluating the pipeline come later in the chapter. Read the definition below and try to say in your own words what the variable pipe will hold before you read on.
Why Bundle the Stages Together
Consider what goes wrong with separate manual calls. Each stage becomes its own line, and you must remember the order every time you run the workflow. A pipeline removes that burden. The stages are declared once, in a fixed sequence, and the pipeline carries the data through that sequence in order.
Following the Data Through Two Steps
The steps in a pipeline are not independent. The source describes the workflow as training an SVM after scaling the data, so the scaler comes first and the SVM comes second. When the pipeline is fitted, the data enters the first step, the scaler transforms it, and the result is passed on to the second step, the SVM, which is trained on it. The diagram below traces that path.
What a Single Step Is Made Of
Each step in a pipeline is a tuple with two parts: a name and an estimator instance. The name is a label you choose, and it can be any string you like, so that you can refer to the step later. The estimator is the object that does the actual work. It is an instance, meaning it has been created by calling the class.
In the source example, the name scaler belongs to a MinMaxScaler instance, and the name svm belongs to an SVC instance. The name is what you use to refer to the step, while the estimator is what performs the scaling or the training.
Building the Pipeline Object
The example below assumes that MinMaxScaler and SVC have already been imported, so the focus stays on how the pipeline is assembled. The list passed to Pipeline holds two tuples, one per step.
- The outer square brackets form a list of steps.
- Each parenthesised pair is a tuple holding a name and an estimator instance.
- The name 'scaler' is a label;
MinMaxScaler()is the object that does the work. - Calling the class with parentheses creates the instance.
What do you think happens?
Before reading on, what does the variable pipe hold after this line runs?
Reveal answer
A Pipeline object that holds the two named steps
The source states that a Pipeline object is created by passing it a list of steps. The steps are not fitted yet; the pipeline exists as a single object that can be fitted later like any other scikit-learn estimator.
Tracing the Tuple List
The structure of the pipeline is easier to see when it is built from plain Python values, without scikit-learn. A list of two-item tuples can be looped over, and each tuple can be unpacked into two variables. This unpacking is the same mechanism that lets you read a step's name and its estimator separately.
scaler -> MinMaxScaler()
svm -> SVC()- The loop visits the tuples in list order, first position 0, then position 1.
- Each tuple is unpacked into name and estimator on every pass.
Mistakes When Writing the Steps
A pipeline definition is a plain Python list of tuples with estimator instances in them, so most errors come from getting one of those three pieces wrong. The two errors the source names are shown below.
Common mistake Writing a step as a list instead of a tuple
What it looks likePipeline([['scaler', MinMaxScaler()], ('svm', SVC())])Why it failsThe source requires each step to be a tuple. The first item has the right shape, but the list in the second position breaks the required structure.
Do this insteadUse parentheses for each step: Pipeline([('scaler', MinMaxScaler()), ('svm', SVC())])
Common mistake Passing an estimator class instead of an instance
What it looks likePipeline([('scaler', MinMaxScaler), ('svm', SVC())])Why it failsThe source says the estimator must be an instance, created by calling the class. MinMaxScaler without parentheses is the class itself, not an object that can do the work.
Do this insteadCall the class: ('scaler', MinMaxScaler()) and ('svm', SVC())
Finding the Faulty Step
A practical way to locate the problem is to inspect the list of steps before passing it to Pipeline. Each item should be a tuple with exactly two entries. The check below reports the position, type, and length of each item, using plain Python values so the logic is easy to follow.
0 tuple 2
1 list 2- Position 1 is a list, not a tuple, even though it has two entries.
- The check points at the second item, so only that item needs to change.
The output shows where the problem is. The first step is a tuple, as the source requires. The second step is a list, even though it has the right number of entries. Because the failure is located at the second item, you change that item rather than rewriting the whole list. The same kind of check works on the real steps you intend to pass to Pipeline.
Practice Finding the Bad Step
Consider this definition: steps = [('scaler', MinMaxScaler), ('svm', SVC())]. Which step is wrong, what kind of error does it contain, and how would you correct it?
Hints
- Check whether each item is a tuple with two entries.
- Check whether the second part of each tuple is an instance or a class.
- Calling a class with parentheses creates an instance.
Write a list of steps for a pipeline that scales data with MinMaxScaler and then trains an SVC. Name the scaler 'scaler' and the SVM 'svm'. Then run the position and type check from the previous section on your list and confirm that every item is a tuple.
Hints
- Use parentheses around each name and estimator pair.
- Remember to create the instances with parentheses after each class name.
Pulling the Pieces Together
- A Pipeline object is created by passing it a list of steps, which expresses the training workflow in one place.
- Each step is a tuple: a name you choose, plus an estimator instance created by calling the class.
- The data passes through the steps in list order, so the scaler runs before the SVM is trained.
- Common errors are writing steps as lists instead of tuples, and passing classes instead of instances.
- When the error points at the Pipeline call, inspect each item in the list to find the faulty step.
Key Takeaways
- A Pipeline object is created from a list of named steps and expresses a model training workflow as a single object.
- Each step is a tuple of a name, which can be any string, and an estimator instance created by calling the class.
- Steps run in list order: the scaler transforms the data before the SVM is trained on it.
- Writing steps as lists, or passing classes instead of instances, are the errors to check first.
- When a Pipeline call fails, inspect each step's type and length to find the faulty item.