Feature Engineering and Derived Features

Before you train any model, you need to know what data you are working with. The Boston Housing dataset is a real-world regression dataset, and it is useful for learning feature engineering because its column count changes when we add new inputs built from old ones. Keep one question in mind as you read: if a dataset starts with 13 features, and we add every product of two features to the list, how many features do we end up with?

The shape of a dataset is reported as a pair of numbers in a fixed order: data points first, then features. For the Boston Housing dataset, the shape is 506 data points and 13 features. Reading this tuple in the correct order is the first check you should make on any dataset, because swapping the two numbers would tell you something false about the table.

Each row of the table is one neighborhood in Boston, and each of the 506 rows is a separate data point. Each column is one feature, meaning a single measured property of that neighborhood. The features include information such as the crime rate, proximity to the Charles River, and highway accessibility. So a single data point is not a house. It is one neighborhood described by 13 numbers, and the whole table is a stack of 506 such descriptions.

The measurements are collected for a reason, and that reason is the prediction task attached to the dataset. The goal is to predict the median value of homes in several Boston neighborhoods during the 1970s, using the other measurements as inputs.

Because the target is a continuous quantity, a number that can take many values, rather than a category such as yes or no, the dataset is a regression problem. Its purpose is to estimate a number, not to assign a label.

The 13 original measurements are not the only inputs the source uses. Feature engineering creates new inputs from existing ones, and one common way to do this is to form interaction features. An interaction feature is the product of a pair of original features. Instead of using the crime rate and the highway accessibility as two separate inputs, the expanded version also includes a third column, whose value for each neighborhood is the crime rate multiplied by the highway accessibility.

The multiplication happens row by row. Each neighborhood has its own crime rate and its own highway accessibility value, so each neighborhood gets its own product. The new column is built by taking the two input columns, multiplying them position by position, and storing the results in a fresh column.

The expanded version of the dataset is loaded with a dedicated function, and its shape shows how many columns the expansion produces. The source states the count directly: the 104 features are the 13 original features together with the 91 possible combinations of two features drawn from those 13. Adding the two parts gives the total.

The rows do not change. The expansion adds columns, so the table is still 506 rows tall, and only the width grows from 13 to 104.

The number 91 is worth checking by hand. Choosing two different features from 13 gives $\binom{13}{2} = 78$ pairs. The remaining 13 combinations must be squares, where a single feature is multiplied by itself. Since $78 + 13 = 91$, the count reconciles with the source. This check is our own arithmetic rather than a detail the source spells out, but it shows exactly which combinations the number includes.

Feature engineering is a step between the raw data and the model. The raw features are read from the table first. Derived features, such as interaction products, are then computed from them. The model is trained on the full set of inputs, both original and derived, and it produces the single output the task asks for: an estimate of the median home value.