Feature Engineering Fundamentals
Feature engineering is a natural place to use domain expertise, because the features a model receives largely determine what it can learn.
A Flight Price Problem
Imagine a travel agency that wants to predict what a flight will cost. Its records already hold useful fields: the price, the date, the airline, the departure city, and the destination city. A model trained on those fields may already perform decently. The question this article answers is a different one: what else would a knowledgeable person expect to influence the price, and can that influence be seen in the fields the agency already has?
This question sits at the heart of feature engineering. The stage where raw information is turned into the inputs a model actually sees is where a person's understanding of the application can matter most. Holding that idea in mind will help you follow every step that comes next.
What the Model Actually Sees
A learning algorithm can only find patterns in the information that reaches it. Raw records are not the model's inputs by default. Between the stored records and the model sits a step where raw fields are transformed, combined, and augmented into the features the algorithm learns from. Feature engineering is that step.
Notice where expert knowledge enters the path. It does not replace the model. It shapes what the model receives. The holiday calendar in the diagram is not part of the original flight records, so without an explicit step to bring it in, the model never sees it.
Factors Visible and Hidden in Raw Fields
A model can learn a factor only when that factor is represented in its inputs. Some factors show up directly in the raw columns. A season, for example, can be read straight from a date, so if flight prices tend to rise in a certain season, the model has a fair chance of capturing that pattern. Other factors exist only outside the dataset, and no amount of training on the raw columns will reveal them.
Fixed and Moving Holidays
Holidays illustrate the distinction well. Some holidays fall on the same calendar day every year, so their effect on prices is tied to the date and can be learned from it. Other holidays move. Some are anchored to astronomical events such as the phases of the moon, and others are set by public or school authorities and can change from year to year.
When a holiday is not anchored to a fixed day, a record that says only what calendar date a flight took off does not reveal that the holiday was nearby. The date is present, but the fact the model needs is absent. This is the gap that feature engineering is meant to close.
Encoding a Holiday Indicator
The remedy described in the source is simple. A feature can be added that records whether a flight was on a public or school holiday, was in the period just before one, or was in the period just after one. The information needed to build that feature comes from outside the original records, namely a calendar of holidays. Once that calendar is joined onto each flight, the model receives a field it could not have derived from the date alone. In this way, prior knowledge about the task is written into the features the algorithm learns from.
Flagging a flight near a movable holiday
A flight records only its departure date. A movable holiday falls the day after that departure. Show how the holiday indicator changes what the model can see.
- 1Read the raw date
The raw record holds only the calendar date of departure. On its own, the date does not say whether a holiday is nearby.
- 2Consult the outside calendar
The holiday calendar lists the movable holiday on a date that is one day after the departure.
- 3Join the calendar to the flight
The flight is matched to the calendar, and the matching produces a new field.
- 4Set the indicator
Because the flight falls in the period just before a holiday, the indicator for before-a-holiday is set. The model now has a field it could not have derived from the date alone.
The flight carries a before-holiday flag. The model can now use the fact that a holiday was nearby, which the date alone never revealed.
Why an Unhelpful Feature Does No Harm
A common worry is that adding many features will make a model worse. The source takes a more relaxed view. Including a feature does not force the learning algorithm to rely on it. If the holiday indicator carries no useful signal for flight prices, the algorithm can give it little weight and proceed with the other inputs. The source also notes that augmenting the data with information that turns out to be noninformative does no harm.
| Situation | What the algorithm can do with the feature | What it means for the model |
|---|---|---|
| The feature carries real signal | Give it meaningful weight in learning patterns | The added knowledge helps the model capture a factor it could not otherwise see |
| The feature carries no useful signal | Give it little weight and proceed with the other inputs | The added feature does no harm, because the algorithm is not forced to rely on it |
A helpful feature and an unhelpful feature, compared
Mistakes Beginners Make With Feature Engineering
Common mistake Assuming the model will discover factors that are not in its inputs
What it looks likeExpecting a flight model trained only on date, airline, and cities to know that a movable holiday falls nearby.
Why it failsA model cannot learn a factor whose influence is not represented in its inputs.
Do this insteadIdentify the factor, then build a feature that represents it, such as a holiday indicator.
Common mistake Treating a holiday calendar as a field that already exists in the records
What it looks likeTrying to derive a holiday flag from the departure date alone for a holiday that moves each year.
Why it failsThe information needed to build the feature comes from outside the original records, so it cannot be computed from the date alone.
Do this insteadJoin the external calendar onto each flight, then derive the flag from the joined data.
Common mistake Believing every added feature must improve the model
What it looks likeDropping a holiday indicator after one test because it did not visibly help.
Why it failsA feature that carries no signal is not harmful, and the source treats noninformative augmentation as causing no harm. Its absence of benefit is itself a result about the data.
Do this insteadKeep plausible expert features as candidates and let the data show which ones carry signal.
Common mistake Forgetting that the feature is only useful if it is computed consistently for every record
Checking Your Understanding
A travel agency's flight records include the price, departure date, airline, and both cities. A movable holiday is set to fall three days after some flights depart. Which of the following is visible in the raw fields, and which requires a feature built from outside the records? Explain your reasoning, and describe the holiday indicator you would add.
Hints
- Ask whether the factor can be read directly from a stored field or whether it exists only in an external source.
- Recall that a movable holiday is not anchored to a fixed calendar day.
- A before-holiday or after-holiday flag can be set for a window around the holiday.
Key Takeaways
- Feature engineering is a natural place to apply domain expertise, because the features a model receives largely determine what it can learn.
- A model cannot learn a factor whose influence is not represented in its inputs, so some effects, such as a movable holiday, are invisible in a raw date column.
- A holiday indicator encodes prior knowledge by joining an outside calendar onto each record, producing a field the model could not derive from the date alone.
- Including a feature does not force the algorithm to use it, so an unhelpful feature carries no real cost, which makes encoding plausible expert guesses a low-risk practice.