Features and Labels in Supervised Learning

The Wisconsin Breast Cancer dataset is a real collection of tumor measurements, labeled benign or malignant, that scikit-learn loads with load_breast_cancer.

Programming

A Real Dataset to Start With

Many small datasets used early in a machine learning course are invented for teaching. The Wisconsin Breast Cancer dataset is different. It is a real collection of clinical measurements taken from breast tumors, and scikit-learn ships it so that you can practice on realistic data without collecting anything yourself. Each tumor in the collection carries a label of benign (harmless) or malignant (cancerous). The learning task is to predict whether a tumor is malignant from measurements of the tissue.

This article starts by running code rather than explaining the structure first. You will load the dataset, look at what the loaded object contains, and then explain what you saw. Each section below asks you to check your reading against the actual output.

What do you think happens?

When you load the dataset with load_breast_cancer, what kind of object do you get back: a plain list, a NumPy array, or something else?

A plain Python list of tumors
A NumPy array of numbers only
A Bunch object that behaves like a dictionary

Reveal answer

Answer:

A Bunch object that behaves like a dictionary

The object returned by load_breast_cancer is a Bunch. A Bunch stores the numbers together with information about the dataset, so one object carries both the data and the context needed to interpret it.

Loading the Dataset and Listing Its Keys

Code walkthrough

from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
print(type(cancer))
print(cancer.keys())

1
Line 1 imports the loader function that ships the Wisconsin Breast Cancer data with scikit-learn.
2
Line 2 calls the loader and keeps the result in a variable named cancer.
3
Line 3 prints the type, which the source identifies as a Bunch.
4
Line 4 prints the keys. The output includes an entry named DESCR, which points to a text description of the data.

load_breast_cancer is imported from scikit-learn's datasets module
Calling it returns the dataset as a single object stored in cancer
type(cancer) shows the kind of object that was returned
keys() lists the names of the values stored inside the object

Read the printed keys as a menu of what the object holds. You do not need to memorize the full list yet. The important entries for this article are data, target, feature_names, and target_names, which the next sections explain. The DESCR entry is worth reading too, because it is the source of context about what each measurement means and where the data came from.

What a Bunch Object Is

A Bunch is the container scikit-learn uses for most of its bundled datasets. It is dictionary-like: it stores named values, and you read each value by its name. What makes it useful is that it holds the actual data and also the information needed to interpret that data. A bare table of numbers would not tell you which column is which or what the answers mean. A Bunch keeps those descriptions alongside the numbers.

Definition

A Bunch is a dictionary-like object that stores a dataset's values together with the descriptive information needed to read them, such as names of columns and categories.

Where Features, Labels, and Names Live

A learning task always separates inputs from answers. In this dataset the feature values are the measurements a model would look at, and the labels are the answers it would try to predict. The source stores these in four different places inside the Bunch: the feature values in data, the integer labels in target, the names of the measurements in feature_names, and the names of the label categories in target_names.

Which keys in the loaded object hold the feature table, the labels, the feature names, and the label names?

The labels stored in target are integers, not words. The word labels live in target_names, and each integer is a position into that list. That is why you cannot read the answer for a tumor directly from target without also consulting target_names. The feature names work the same way in reverse: they describe the columns of data and are not part of the numbers themselves.

Reading the Shape of the Feature Table

The stored data attribute holds the feature values, and its shape tells you the size of that table. The source describes the dataset as 569 data points, each with 30 features, so the shape is (569, 30). Printing the shape is the quickest way to confirm the dataset has the size you expect before you do anything else with it.

python

print(cancer.data.shape)

print(cancer.data.shape)

Output (expected)

(569, 30)

Notes on the code

The first number counts rows, one per tumor
The second number counts columns, one per measured feature

In the shape (569, 30), which dimension counts tumors and which counts measurements?

Worked example

Reading the feature table's shape

The printed shape is (569, 30). What does each number tell you about the dataset?

1
Identify the first axis
The first number, 569, is the number of rows. Each row is one tumor.
2
Identify the second axis
The second number, 30, is the number of columns. Each column is one measured feature.
3
Connect to the source
The source describes 569 data points, each with 30 features, so the printed shape matches the description.

Result

The table has 569 tumors as rows and 30 measured features as columns.

Aligning Rows With Labels

The feature table and the label array are two parallel structures. Row i of data describes one tumor, and entry i of target is the label for that same tumor. Alignment is by position, so the row index and the label index must refer to the same sample. The diagram below shows the first three positions of each structure and how they pair up.

If I pick row 0 of the feature table, which entry in the label array describes it, and how do the two arrays line up by position?

To find the label for a tumor, take its row number and use the same number to index target. Then use that integer as a position into target_names to get the word. Each step is a lookup by position, so an error at any step gives a wrong answer without raising an error.

Checking the Per-Class Counts

The class balance is the second thing to read. The source reports 212 malignant tumors and 357 benign tumors. These two counts add up to 569, which equals the number of rows. That sum is a useful check you can apply yourself: if your per-class counts do not total the number of rows, something is wrong in how you are counting.

python

import numpy as np
print(cancer.target_names)
counts = np.bincount(cancer.target)
print(list(zip(cancer.target_names, counts)))
print(counts.sum())

import numpy as np
print(cancer.target_names)
counts = np.bincount(cancer.target)
print(list(zip(cancer.target_names, counts)))
print(counts.sum())

Notes on the code

target_names prints the category words
bincount counts how many times each integer appears in target
zip pairs each name with its count, so the pairing is visible
sum checks that the counts total the number of rows

Mistakes Beginners Make With Bunch Objects

Common mistake
Treating target as the answer words
What it looks like
```
print(cancer.target[0]) prints an integer, and the student reads it as the word for the tumor.
```
Why it fails
The labels in target are integers. The words live in target_names, and each integer is a position into that list.
Do this instead
Look the integer up in target_names, for example cancer.target_names[cancer.target[0]].
Common mistake
Assuming the name-to-count pairing without checking
What it looks like
Reporting that a particular word has a particular count based on memory of the source output.
Why it fails
The pairing depends on the order in target_names. A swapped mapping still prints plausible numbers, so nothing crashes.
Do this instead
Pair the names and counts in code, as with zip, and test the assumption against the data.
Common mistake
Confusing feature names with the feature values
What it looks like
Searching data for the text of a measurement name.
Why it fails
feature_names describes the columns of data. data holds only the numbers.
Do this instead
Use feature_names to label columns when you display or report results.
Common mistake
Skipping the shape check
What it looks like
Starting analysis without printing cancer.data.shape.
Why it fails
A wrong size is easy to miss and will propagate into every later step.
Do this instead
Print the shape first and confirm it is (569, 30).
Common mistake
Ignoring DESCR, which explains what each measurement means and where the data came from

Practice Reading the Loaded Object

EASY

Load the dataset and print the shape of cancer.data. Then print cancer.target_names and the paired counts from np.bincount. Without looking back at the article, state what the first number of the shape counts and what the two per-class counts should add up to.

Hints

The first number of the shape counts rows, and each row is one tumor.
The per-class counts should add up to the number of rows in cancer.data.
Use zip to pair each name in target_names with its count from np.bincount.

Your attempt (not saved)

Summary of the Loaded Structure

The Wisconsin Breast Cancer dataset is loaded as a Bunch, a dictionary-like object that keeps the numbers together with their descriptions. Feature values live in data with shape (569, 30), labels live in target as integers, and the word labels and feature names live in target_names and feature_names.

At a glance

The Wisconsin Breast Cancer dataset is a real set of tumor measurements labeled benign or malignant, loaded with load_breast_cancer.
The loaded object is a Bunch, and its values can be read with dot notation or bracket notation because it behaves like a dictionary.
Features live in data with shape (569, 30), integer labels live in target, label names live in target_names, and feature names live in feature_names.
The counts of 212 malignant and 357 benign tumors add up to 569, but the name-to-count pairing should be checked in code rather than assumed.
Always print the shape and paired counts before modeling, and read DESCR for the meaning of each measurement.

Related concepts

More concepts starting with F