Introduction to Feature Engineering

Imagine building a program that sorts incoming email into spam and legitimate mail. Writing a rule for every possible spam message is not realistic, so the sensible choice is to let the program learn from examples. Once you make that choice, a deeper question appears: how much of the learning process has to be designed by people, and how much can the program work out for itself?

A learning model cannot read an email the way a person does. It works on measurable quantities, and these are called features. A feature is a property of the data that the model can use to tell one kind of example from another. An email is therefore not something the model understands as a message; it is something the model must reduce to numbers that describe it.

Keep this idea in mind as the central thread of the article: every model works from a set of measurements, and the open question is where those measurements come from.

For spam classification, the source describes several features an expert might choose. The first is how often certain trigger words appear, such as prize, win, and free. The second is how many exclamation marks a message contains. The third is whether words are written in all uppercase. The fourth is whether suspicious links are present.

Once those features have been computed for every email, the results form a table of values. That table becomes the dataset used to train the model. Each row describes one email in terms of the expert's chosen measurements, and each column is one feature.

The clearest way to see the difference is to compare the two pipelines side by side. In both cases the raw email eventually reaches a classifier. The traditional pipeline, however, places a human-designed feature step between the raw email and the classifier. The deep learning pipeline removes that step from the human's hands.

Read the diagram from left to right in each column. The traditional column contains a feature step that a person has designed in advance, which is why it is the only human-defined stage in that path. The deep learning column has an automatic feature step in the same position. The source states that the features are extracted automatically rather than being handed to the model by a person. It does not describe how those internal features are formed, so this article does not go further into the mechanism.

The practical consequence is that the expert's job changes. In the traditional approach, the designer must decide in advance that exclamation marks and uppercase words are worth counting. In the deep learning approach, that decision is no longer part of the designer's list of tasks. The designer works instead on the surrounding system: gathering examples, setting up training, and judging whether the model's predictions are good.

Now follow the traditional workflow from start to finish. Each stage depends on the stage before it, so the order matters. The path runs from human labeling of training data to a model that can make predictions on new mail. Labeling means assigning each example a category, such as spam or not spam.

The key point is that labels remain required in both approaches. The source states that deep learning does not remove the need for collected labels, which are gathered by experts or users. Deep learning changes who works out the relevant characteristics of the input, but it does not change the need for examples that are marked with the correct answer.

Artificial intelligence is a broad field, and machine learning and deep learning currently dominate it. They are not the only approaches. The field also includes rule-based systems, genetic algorithms, expert systems, fuzzy logic, and symbolic reasoning. Each of these takes a different route toward intelligent behavior.

When this article compares traditional machine learning with deep learning, it is comparing two members of this wider family. It is not describing the whole of AI. Keeping that boundary in mind prevents the common error of treating the terms as interchangeable.

The source introduces large language models, or LLMs, as models that can solve a variety of problems. For this article, the important point is their position. LLMs are one of the modern model families discussed in the context of machine learning. Their architecture and other details are developed in later sections, so this article does not explain them.