Binning Continuous Features

Suppose you want a linear model to predict a target from a continuous x value, but the relationship between x and the target is not one straight line. It bends, rising steeply in some stretches and flattening or falling in others. One common response is to cut the x-axis into segments, called bins, and let the model treat each segment differently. This article follows that idea step by step. You will see how binning gives each segment its own offset, why that still leaves the model with a single slope, and how interaction features give every bin its own slope as well.

Binning replaces a continuous x value with the identity of the segment it falls into. The bin edges partition the x-axis into non-overlapping ranges, and every value lands in exactly one range. For the small worked dataset used later in this article, there are two bins. Bin 0 holds points with x below 2.0, and bin 1 holds points with x at or above 2.0. Because each point belongs to exactly one bin, a point is always in one bin and never in two.

Read the diagram as a lookup. Each x value points to exactly one bin, and the bin edge at 2.0 is the boundary between the two segments. Once every point has a bin label, you can represent that label as a set of indicator columns, one column per bin, where the column for the point's bin holds 1 and every other column holds 0.

A model that receives only binned input can shift its prediction up or down from one bin to the next, because each bin receives its own offset. The model still has only one x-axis feature, though, and that feature has only one slope. Every bin's line therefore rises or falls at the same rate. The lines for different bins can start at different heights, but they are parallel. The downward slope is applied everywhere, regardless of which bin a point sits in.

An interaction feature combines two other features so the model can respond to their combination. Here the two features are the bin indicator, which records which bin a data point is in, and the original x value. Their product is the feature that gives each bin its own slope. The product is the original x value when the point is inside a bin and zero everywhere else. Because it is zero outside its bin, the linear model can use it only to adjust predictions for points in that bin.

The construction in the source is a single line that multiplies the x column by the bin indicator columns and stacks the result beside the indicators. The product multiplies the one x column against each indicator column, producing one product column per bin. The stacking step then joins the indicator columns and the product columns side by side.

The source dataset has 10 bins and 100 data points. That gives 10 indicator features and 10 product features, a total of 20 features, so the shape is (100, 20). The number of columns is always twice the number of bins in this construction.

The vectorized version hides every value, so it helps to trace the same idea with plain Python lists on four points and two bins. Bin 0 holds x below 2.0 and bin 1 holds x at or above 2.0. Each row has two indicator values followed by two product values.

Follow the row for x = 2.5. Its indicator columns are 0 and 1, which mark it as belonging to bin 1. Its first product column is 0.0, because the point is outside bin 0. Its second product column is 2.5, the original x value, because the point is inside bin 1. This is the zero-outside-the-bin behavior in concrete form.

When the linear model is fitted on the product representation, every bin gets its own offset and its own slope. The weight on a bin's indicator column acts as that bin's offset, and the weight on that bin's product column acts as its slope. Because a product column is zero for every point outside its bin, its weight only affects predictions inside that bin. The indicator and product weights of each bin together define an independent line for that bin.

Notice what the worked example shows. Bin 0 falls as x rises, while bin 1 rises. A single shared slope could not produce that pattern, because it would force both bins to move in the same direction at the same rate.