Feature Augmentation Strategies

Machine learning is often presented as a way to let data speak for itself, so that a person no longer has to write rules about how the world works. That goal is real. It does not mean that people who understand an application have nothing left to contribute. The stage where raw information is turned into the inputs a model actually sees is called feature engineering, and it is one of the most natural places for that understanding to matter.

The reason this stage matters so much is simple: a learning algorithm can only find patterns in the information that reaches it. The features a model receives largely determine what it can learn. If a factor that drives the outcome is never represented in the inputs, the model has no direct way to account for it.

Consider a travel agency that wants to predict what a flight will cost. Its records include the price, the date, the airline, the departure city, and the destination city. Those fields are a reasonable starting point, and a model trained on them may already perform decently. The useful question is what else a knowledgeable person would expect to influence the price, and whether that influence can be seen in the fields the agency already has.

Some factors are visible in the raw columns. If the price of a flight tends to rise in a certain season, and the season can be read directly from the date, then the model has a fair chance of capturing that pattern. The date is doing real work in that case.

Other holidays are harder. Some are anchored to astronomical events such as the phases of the moon, and others are set by public or school authorities and can change from year to year. A holiday that is not anchored to a fixed day leaves no trace in a column that only stores the calendar date. The record says when the flight left, but not that a holiday was close by. Some holiday effects are therefore invisible if flights are recorded only by Gregorian date.

The remedy described in the source is simple. A feature can be added that records whether a flight was on a public or school holiday, was in the period just before one, or was in the period just after one. The information needed to build that feature comes from outside the original records, namely a calendar of holidays. Once that calendar is joined onto each flight, the model receives a field it could not have derived from the date alone. In this way, prior knowledge about the task is written into the features the algorithm learns from.

An added feature of this kind is an indicator for being on, before, or after a holiday. It encodes an expert's expectation that holidays matter for prices, and it does so without forcing the algorithm to use it. The indicator is a claim the data can then confirm or fail to support.

A common worry is that adding many features will make a model worse. The source takes a more relaxed view. Including a feature does not force the learning algorithm to rely on it. If the holiday indicator carries no useful signal for flight prices, the algorithm can give it little weight and proceed with the other inputs. The source also notes that augmenting the data with information that turns out to be noninformative does no harm.

This is what makes the approach practical. The cost of adding a candidate feature is small, while the cost of missing a factor that does matter can be large. A domain expert's best guess about what influences the outcome is therefore worth encoding, and the data can then indicate whether the guess was correct.