Visualizing Histograms of Feature Values

Before a model learns anything, the features it receives have a shape, and that shape can make learning easy or hard. Many modeling methods work best when each input feature, and in regression also the target, is loosely Gaussian distributed. In practice this means that if you draw a histogram of one feature, it should look something like the familiar bell curve. Real measurements rarely arrive in that form, so a common job of a data practitioner is to reshape features before modeling them.

This article focuses on one important family of features: count data. The aim is to understand what makes count data distinctive, why its raw histogram rarely looks bell-shaped, and how to spot the point in a workflow where the data stops behaving the way you expected. The ideas are easiest to see in running code, so the article moves from observation to explanation, starting with a question you can answer before reading the reveal.

The definition rests on two properties, and each one has a concrete consequence. Integer values mean the feature can take only a discrete set of possible numbers. A count of 3 or 4 is possible, but a count of 3.5 is not, because there are no fractional counts between two whole values. The fact that counts are never negative means the lowest possible value is a hard floor, usually zero.

Together these two properties often produce a distribution that is lopsided. There is a crowd of small values, a steady fall-off as values grow, and an occasional very large value far out on the right. Nothing in the definition forces this lopsidedness, but the combination of integers and a floor at zero makes it the usual outcome for counts.

Counting code tends to fail quietly, so it helps to know where to look when a tally disagrees with expectation. The routine below assumes every value is a non-negative integer. It creates a list of slots sized from the largest value, and each observed value is used directly as an index into that list. Because the list is indexed by value, the only place the assumption can break is the indexing step, so that is where the trace should focus. The routine below is a small generated example that follows that same design.

Trace the running tally one iteration at a time. After the value 2, the slots are [0, 0, 1, 0]. After the value 0, they become [1, 0, 1, 0]. After the value 3, they become [1, 0, 1, 1]. The next value is -1. The code runs counts[-1] += 1, and index -1 refers to the last slot, which is the slot for 3. That slot jumps from 1 to 2, giving [1, 0, 1, 2]. The expected tally for the valid values 2, 0, 3, 2, 1 is [1, 1, 2, 1], so the running tally first differs from expectation on the line that increments the slot for -1. Nothing crashes, and the final list still looks like a plausible tally, which is exactly why this kind of bug is dangerous. The tally has silently mixed a negative value into a non-negative slot.

Tallying values reveals patterns that a plain list hides. Reading a long list of integers one entry at a time shows almost nothing, because the pattern lives in how often each value occurs, not in any single entry. A histogram turns the list into a picture of frequency. The horizontal axis should show the count values themselves, and the vertical axis should show how often each value occurred.

Count histograms usually sit sharply peaked near the low end. Most observations take small values, the frequency for higher values drops quickly, and the long right-hand tail still contains a few large values. Because the values are bounded below by zero but have no fixed upper bound, the shape is pulled toward the right. This is a clear departure from the symmetric bell curve that a model would prefer, and that departure is the reason a transformation is worth considering.

Consider a generated tally of monthly logins for a sample of users. The counts might show 40 users with zero logins, 25 with one, 14 with two, 8 with three, 5 with four, 3 with five, 2 with six, and 2 with eleven. Nearly all of the mass sits in the first few bins, while the bins on the far right hold only a handful of users each. The lone high value at eleven is the kind of long-tail point that stretches the right side of the plot.

The shape of a feature is not fixed by its underlying meaning alone. The source's synthetic dataset was built to mimic counts found in real settings, and its construction shows where the lopsided shape comes from. The process starts with values drawn from a standard normal distribution, which are bell-shaped. Each of those values is then passed through an exponential function and scaled by 10. The scaled result is used as the rate for a Poisson distribution, and each final count feature entry is a random draw from a Poisson distribution with that rate.

A Poisson distribution is a probability distribution used to model count data, so the final features are counts even though an intermediate step was bell-shaped. The bell-shaped latent value is pushed through an exponential, which stretches the upper end, and then the discrete Poisson sampling converts it into integers. The result has the integer-valued, non-negative, skewed character described earlier. In other words, the skew is introduced by the exponential stretch and the integer sampling, not by any single entry.

NumPy's bincount function is the tool the source uses for this job. It counts how many times each non-negative integer appears in an array and returns a sequence in which position k holds the number of appearances of the value k. Because positions start at zero, the first entry always refers to the value zero, even when zero never occurs in the data. This is the source of many off-by-one confusions, so keep it in mind when reading any bincount output.

When the bins are read in order, the lower values are the most frequent, the counts for higher values fall off quickly, and the right-hand tail holds a few large values. The source also notes one especially high value that appears only twice. The overall picture is a sharply peaked, right-skewed histogram. Reading the bins in this order is what reveals the pattern that a plain list hides.

The source describes using transformations like log and exp to move a feature toward a Gaussian shape. It calls this approach hacky but simple and efficient. The transformation is especially useful for integer count data, because counts pile up at small values and stretch out at large ones, and a logarithm compresses that stretch. The exponential function is the mirror image of the logarithm, which is why both appear in the source's workflow: exp appears when the data is generated, and log appears when the data is reshaped toward a bell curve.

The contrast in the comparison is the central motivation for the transformation. The gap between the skewed count shape and the bell-shaped target is what a log transform is meant to close. Because the transform changes the shape of the feature, it changes what the model sees, so the transformed values should be checked with a histogram before modeling.