Supervised Learning with Classifiers and Regressors
Whether an algorithm does preprocessing, supervised learning, or unsupervised learning, scikit-learn packages it the same way.
Why Every Algorithm Looks the Same
Imagine you have learned how to use one scikit-learn tool. You then open a different tool for a different job and find the steps are almost identical. That consistency is deliberate. Whether an algorithm does preprocessing, supervised learning, or unsupervised learning, scikit-learn packages it the same way. This article calls that shared packaging the estimator interface.
Once you recognize the pattern, you can work with any algorithm in the library by asking the same three questions. How do I create the object? What settings does it need? How do I give it data to learn from? This article answers those three questions in order, and at each step it traces what the estimator is holding as it moves from a blank object to a trained model.
Creating an Estimator Object
To apply an algorithm, you first instantiate an object of its class. Instantiating simply means creating one concrete object from a class. Logistic regression is a supervised classification algorithm, and it makes a clear first example. Creating an estimator takes two lines: one to import the class, and one to instantiate it.
- LogisticRegression is the class, the blueprint that defines what the algorithm does.
- logreg is a variable that points to one object built from that blueprint.
Notice that the class name and the variable name are different things. The class is the blueprint. The variable logreg points to one object built from that blueprint. You could create several objects from the same class, and each would have its own state. They would be independent of one another. Changing one does not change another.
What an Estimator Holds Before and After Training
An estimator object has two jobs. The first is that it contains the algorithm. The second is that it stores the model learned from data using that algorithm. Before any data has been provided, the object holds the algorithm but no learned model. After training, the same object holds both. Keeping the algorithm and the learned model in one object lets you treat a trained model as a single thing you can carry around in your code.
The arrow from fit to the fitted state is the important part. The fit call does not hand you back a separate model that you must manage yourself. It changes the state of the object you already created. The learned model ends up stored inside the estimator.
Setting Parameters at Construction Time
Some values control how an algorithm behaves. The source is clear about when they must be chosen: you set any parameters of the model when you construct the model object. The parameters it names include regularization, complexity control, and the number of clusters to find. Regularization and complexity control are settings that limit how flexibly a model can fit the data. The number of clusters is how many groups an unsupervised method should look for.
- KMeans is the clustering class.
n_clusters=3 is a construction-time setting that controls the number of groups to find.- The setting is passed when the object is created, before any data is given to
fit().
Passing X and y to fit
Every estimator has a fit method, and its job is to build the model. This is the point where the algorithm actually reads data and produces the learned model that the estimator stores. Until you call it, the estimator is configured but untrained. The fit method always takes the data X as its first argument. For supervised algorithms, it also takes a second argument, y.
The y argument is a one-dimensional NumPy array containing the known target values. For classification these are output labels, and for regression they are numeric responses. Unsupervised algorithms do not need y, because they are not given known answers to learn from. The k-means style clustering in the previous section is one such case, so its fit call takes only X.
- X has four rows, one per sample, and two columns, one per feature.
- y has one label per row of X, in the same order.
fit(X, y)is the supervised call, with X first and y second.
- The unsupervised call takes only X, because no known answers are provided.
Shape and Value Rules for X
The data X has strict expectations. It is represented as a NumPy array or a SciPy sparse matrix, and in both forms each row represents a single data point. The columns hold the measurements, or features, of that point. The number of rows counts your examples, and the number of columns counts the values describing each example. Passing a matrix with the rows and columns swapped describes a different dataset entirely, and the estimator would read the numbers incorrectly.
The source also states that X is always assumed to contain continuous, floating-point entries. In practice, the values should be decimal numbers such as 1.0 or 2.5 rather than text labels or category names. Integers stored as floats are fine, which is why 1.0 and 2.0 work in the example above. Categories and text must be converted into numbers before they can serve as X.
The target y follows a parallel rule. Each entry of y lines up with one specific row of X, so the nth label belongs to the nth sample. If X has four rows, y must have four entries, in the same order.
Checking a Dataset Before fit
Deciding whether a dataset is ready for X
A dataset has 5 patients. Each patient has 3 measurements, and the sex of each patient is recorded as the text values 'male' or 'female'. Can this table be passed directly as X, and if not, what must change?
- 1Check the orientation
Each patient is one data point, so the 5 patients must be 5 rows. The 3 measurements are features, so they must be 3 columns. The matrix should be 5 rows by 3 columns, with each row holding one patient.
- 2Check the value types
The 3 measurement columns are numeric and can be stored as floats. The sex column contains text, which the source says must be converted into numbers before it can serve as X.
- 3Convert the category
Replace 'male' and 'female' with numeric codes, such as 0 and 1, and store those codes as floats in the matrix. The row count stays at 5.
- 4Prepare y separately
If the task is supervised, the known outcome for each patient goes into a one-dimensional array y with 5 entries, in the same patient order as the rows of X.
The table cannot be passed directly, because of the text column. After the category is converted to numeric values, the 5 by 3 matrix is valid as X, and a 5-entry array is the matching y.
Mistakes Beginners Make
Common mistake Treating the algorithm as a bare function instead of creating an object
Why it failsEvery scikit-learn algorithm is implemented as a class, and you work with an object created from that class.
Do this insteadInstantiate the class first, for example logreg = LogisticRegression(), and then call methods on logreg.
Common mistake Confusing the class name with the variable name
What it looks likeLogisticRegression is the class, and logreg is the variable that points to one object.
Why it failsThe class is a blueprint. The variable names one concrete object built from it.
Do this insteadUse the class name to create objects, and use a separate variable name to refer to each object.
Common mistake Expecting a trained model before calling fit
Why it failsBefore data is provided, the estimator holds the algorithm but no learned model.
Do this insteadCall fit with the data so the algorithm builds the model and stores it in the object.
Common mistake Passing category names as text inside X
What it looks likeA column containing the words 'red', 'green', and 'blue'.
Why it failsX is assumed to contain continuous, floating-point entries, not text.
Do this insteadConvert categories into numbers before building X.
Common mistake Passing a supervised estimator without y
Why it failsSupervised algorithms take y as a second argument, containing the known target values.
Do this insteadCall fit(X, y) for supervised algorithms, and reserve fit(X) for unsupervised ones.
Common mistake Putting features in rows and samples in columns
Why it failsEach row must represent a single data point, so transposing the matrix describes a different dataset.
Do this insteadArrange the matrix so the number of rows counts your examples and the number of columns counts the features.
Predict Before You Reveal
What do you think happens?
Right after you run logreg = LogisticRegression() and before you call fit, does the object already contain a learned model?
Reveal answer
No, it holds the algorithm but no learned model yet.
Before any data has been provided, the estimator holds the algorithm and no learned model. The model appears only after fit has been called with data, and it is then stored inside the same object.
Checking Your Own Data Layout
A colleague has 8 houses. For each house, they recorded 4 numeric features, and they stored the data as a matrix with 4 rows and 8 columns. They want to call fit(X, y) for a regression estimator, where y holds the sale price of each house. Identify what is wrong with the layout and describe the correct shape of X and y.
Hints
- Each row of X must represent one complete data point, which here is one house.
- The number of rows should count the examples, and the number of columns should count the features per example.
- The y array must have one entry per row of X, in the same order.
Key Takeaways
- Every scikit-learn algorithm, including preprocessing, classification, regression, and clustering, is implemented as a class called an estimator, and you work with an object created from that class.
- An estimator holds the algorithm, and after fit it also holds the learned model. The model lives inside the same object, so fit changes the object's state rather than returning a separate model.
- Parameters such as regularization, complexity control, and the number of clusters are set when the estimator is constructed, before any data is provided.
- fit takes the data matrix X first, with one row per data point and numeric float entries, and takes a one-dimensional target y second for supervised algorithms. Each entry of y lines up with the matching row of X.