Estimator Parameters and Model Construction

Parameters such as regularization, complexity control, and the number of clusters are set when the model object is constructed.

Programming

A Model That Has Not Seen Any Data

Imagine you have just created a scikit-learn model object and printed it. The object exists, and its settings are in place, but nothing in it has learned anything from data yet. It is closer to a configured tool sitting on a workbench than to a finished model. Its settings have been chosen, but it has not yet seen a single example.

Three methods govern what happens next. The fit method builds the model from data. The predict method turns a fitted model into a new output, such as a class label. The transform method turns a fitted model into a new representation of the input. This article follows the order in which these methods are used and explains how data moves through them, starting with the state of the object at each step.

Following an Estimator Through Its Lifecycle

An estimator changes state as you use it. It begins unfitted, holding only the settings you gave it. A call to fit changes it into a fitted model that has learned from the data. Only then can you apply it. Calling predict produces a new output, and calling transform produces a new representation of the input. Neither of those two calls replaces fit, which must come before either of them.

How does an estimator's state change from the moment it is constructed to the moment it is fitted and used?

Read the diagram from top to bottom. The model is constructed, fit learns from the data, and the fitted model is then used. The two branches at the bottom are alternatives. You pick one based on what you need from the model.

Settings Chosen at Construction

The source guidance is that you should set any parameters of a model when you construct the model object, not later. These parameters control things such as regularization, complexity control, and the number of clusters the algorithm should find. In other words, parameters are decisions made before any data is supplied.

The fit step then uses those decisions while it builds the model. The decisions you made at construction shape how the learning happens, but the values the model ends up with come from the data. The distinction matters because the two kinds of values come from different places.

Which values are supplied when the model object is built, and which are computed from the data during fit?

Anatomy of a Call to fit

Every estimator has a fit method, and its job is to build the model. The first argument is always the data X. In scikit-learn, X is expected to be a NumPy array or a SciPy sparse matrix in which each row represents a single data point. The entries are assumed to be continuous, meaning floating-point numbers. If your data is integer-valued, convert it to floating-point before fitting.

What shape and role does X have, and when is y passed or omitted in the call to fit?

Supervised algorithms need one more input. The y argument is a one-dimensional NumPy array holding the target values: the known output labels for classification or the known responses for regression. Unsupervised methods such as clustering take X without a y argument. So y is required only when the algorithm is supervised, and it is left out of the call otherwise.

python

from sklearn.cluster import KMeans
import numpy as np

X = np.array([[1.0, 2.0], [1.5, 1.8], [8.0, 8.0], [8.5, 7.9]])

km = KMeans(n_clusters=2)   # construction: the cluster count is set here
km.fit(X)                   # fit: X only, no y for this unsupervised method

from sklearn.cluster import KMeans
import numpy as np

X = np.array([[1.0, 2.0], [1.5, 1.8], [8.0, 8.0], [8.5, 7.9]])

km = KMeans(n_clusters=2)   # construction: the cluster count is set here
km.fit(X)                   # fit: X only, no y for this unsupervised method

Notes on the code

n_clusters is a construction-time setting
X is a float NumPy array with one row per data point
No y is passed because clustering is unsupervised

Choosing Between predict and transform

The source frames the two ways of applying a learned model around the kind of result you want. If you want a prediction in the form of a new output, such as a label or a value, you use predict. If you want a new representation of the input data X, you use transform. The decision rests on one question: do you need a new output or a new representation of the data?

When the goal is a new output label or value, how does the data path differ from the path that produces a new representation of the input?

The two columns differ in what comes out, not in how the model is fit. Both start from a fitted model. The predict path produces an output that answers a question about each row, such as which class the row belongs to. The transform path produces a reworked version of the input rows, which is useful when a later step needs the data in a different form. The source lists the uses of transform as preprocessing, dimensionality reduction, feature extraction, and feature selection.

Preprocessing with a Standard Scaler

A common use of transform is preprocessing. You fit a preprocessing estimator on training data, and then you transform that data. The transformed result has the same rows as the input, with each row describing the same data point in a new form. The scaler is fit once, and its transform method is then applied to the data it was fit on.

Worked example

Scaling training data with fit and transform

Scale a small training matrix so that its features are on comparable ranges, then confirm that the number of rows is unchanged.

1
Construct the scaler
StandardScaler() is created with no data. Its settings are fixed at this point, before any data is supplied.
2
Fit on X
fit receives the training matrix X. It is unsupervised preprocessing, so no y is passed.
3
Transform the same X
transform returns a new matrix. Each row still stands for the same data point, now in a new representation.
4
Check the shape
The output has the same number of rows as the input, so the transform output keeps the data points aligned with the original rows.

Result

The scaled matrix has the same shape as the input matrix, with each row describing the same data point in a new form.

python

from sklearn.preprocessing import StandardScaler
import numpy as np

X_train = np.array([[1.0, 200.0], [2.0, 340.0], [3.0, 180.0]])

scaler = StandardScaler()        # construction: settings chosen
scaler.fit(X_train)              # fit: learns from X_train
X_scaled = scaler.transform(X_train)  # transform: new representation of X_train

print(X_scaled.shape)

from sklearn.preprocessing import StandardScaler
import numpy as np

X_train = np.array([[1.0, 200.0], [2.0, 340.0], [3.0, 180.0]])

scaler = StandardScaler()        # construction: settings chosen
scaler.fit(X_train)              # fit: learns from X_train
X_scaled = scaler.transform(X_train)  # transform: new representation of X_train

print(X_scaled.shape)

Output (expected)

(3, 2)

Notes on the code

The scaler is constructed before any data is supplied
fit receives X_train and learns from it
transform returns a reworked version of the same three rows

Mistakes Learners Make with fit, predict, and transform

Common mistake
Setting model parameters after the object is already built, or treating them as values that will be filled in later by fit
Why it fails
The source says parameters such as regularization and the number of clusters are set when the model object is constructed. They are decisions made before any data is supplied.
Do this instead
Pass every construction-time setting when you create the object, for example KMeans(n_clusters=3), and then call fit.
Common mistake
Passing integer-valued data directly to fit without converting it
Why it fails
In scikit-learn, X is expected to hold continuous floating-point values, so integer entries do not match what the framework expects.
Do this instead
Convert the data to floating-point first, for example with astype(float), before calling fit.
Common mistake
Passing y to an unsupervised algorithm, or leaving y out of a supervised one
Why it fails
y is the one-dimensional array of known targets. It is needed for supervised algorithms and not for unsupervised methods such as clustering.
Do this instead
Pass y only when the algorithm is supervised, and check whether the method is supervised before you build the call.
Common mistake
Calling predict or transform on an estimator that has not been fitted
Why it fails
Both methods apply a learned model. fit must come first, and neither predict nor transform replaces it.
Do this instead
Follow the order: construct, fit, then use predict or transform.
Common mistake
Choosing predict when the goal is a reworked version of the input rows, or choosing transform when the goal is a label
Why it fails
predict gives a new output such as a class or value, while transform gives a new representation of X.
Do this instead
Ask whether you need a new output or a new representation of the data, and pick the method that matches.

Predict Before You Check the Shape

What do you think happens?

After you fit a StandardScaler on a matrix with 3 rows and 2 columns and then call transform on that same matrix, how many rows does the result have?

2 rows
3 rows
It depends on the number of clusters

Reveal answer

Answer:

3 rows

transform produces a new representation of the input, and each row still describes the same data point. The row count stays the same, which is why the scaled output keeps the same shape as the input.

MEDIUM

A colleague wants to group customer records into segments and says they will call fit with both X and y, then call predict to get segment labels. Which parts of their plan are correct, and which need fixing?

Hints

Ask whether the algorithm is supervised or unsupervised before deciding whether y belongs in the fit call.
Consider whether the goal is a new output label or a new representation of the data.
Think about where the number of segments should be set.

Your attempt (not saved)

Summary of the Three Methods

1
Parameters such as regularization, complexity control, and the number of clusters are set when the model object is constructed, before any data is supplied.
2
fit builds the model from data. Its first argument is X, a NumPy array or SciPy sparse matrix of floating-point values with one row per data point. The optional y argument holds targets and is used only for supervised algorithms.
3
fit must come before predict or transform. predict produces a new output such as a classification, regression, or clustering result, and transform produces a new representation of the input.
4
Choose the applied method by asking whether you need a new output or a new representation of the data.

At a glance

Parameters such as regularization, complexity control, and the number of clusters are set when the model object is constructed.
fit builds the model from X, a floating-point NumPy array or SciPy sparse matrix with one row per data point, and takes y only for supervised algorithms.
predict gives a new output such as a classification, regression, or clustering result; transform gives a new representation of the input.
Choose the applied method by asking whether you need a new output or a new representation of the data.

Related concepts

Supervised and Unsupervised Learning Inputs

More concepts starting with E