Types of Machine Learning Problems
When the problem is too intricate for a person to specify in full, the rules become the bottleneck, and every case the designer did not anticipate is handled by no rule at all.
A Spam Folder Problem
Imagine you are responsible for an email program. You want incoming messages that look like junk to land in a spam folder instead of the inbox. The question this article opens with is a broad one: how do we get a program to make decisions like this on its own, and what is different about the approach that machine learning takes?
Keep this spam filter in mind as you read. It is a concrete problem that makes the contrast between two ways of building intelligent behavior easy to see. Follow one message through the first approach, then compare it with the second.
What do you think happens?
A spam filter is built from a fixed list of blocked words. A junk message uses a spelling that is not on the list. What do you think the filter does with it?
Reveal answer
It delivers the message to the inbox
The filter only acts on words it was told about. If no word in the message matches an entry on the list, no rule fires, so the message is not marked as spam. This is the first hint that a rule system is only as complete as the rules its designer wrote down.
Tracing a Handcoded Rule System
In the early days of intelligent applications, many systems relied on handcoded rules: explicit if and else decisions that process data or adjust to user input. A person with expertise in the problem writes each rule down, and the program simply applies them.
A spam filter built this way might use a blacklist of words. If an incoming message contains a word on the list, the message is marked as spam and moved to the spam folder. The designer decides which words belong on the list, and that decision is the entire intelligence of the system.
Tracing two messages through the blacklist
The blacklist contains the words free and winner. Trace two incoming messages: message A says Congratulations, you are a winner, and message B says Meeting notes attached.
- 1Read message A
The rule system scans the text for any word on the blacklist. It finds winner.
- 2Apply the rule to message A
A blacklisted word is present, so the if condition is true and the message is marked as spam and moved to the spam folder.
- 3Read message B
The rule system scans the text and finds no word that appears on the blacklist.
- 4Apply the rule to message B
The if condition is false, so no rule fires and the message is delivered to the inbox.
Message A goes to the spam folder and message B goes to the inbox. Every decision came from a word that a person chose in advance.
Where Rule Systems Fall Short
Hand-written rules are not a bad idea by default. Crafting decision rules manually is feasible for some applications, particularly those where humans understand the process being modeled well. When the rules can be stated clearly by someone who knows the domain, a rule system can work well.
The source states that using handcoded rules to make decisions has two major disadvantages. The point is to recognize that this is a real limitation, not just a matter of style. A rule system depends entirely on what the designer thought to write down. When the problem is too intricate for a person to specify in full, the rules become the bottleneck, and every case the designer did not anticipate is handled by no rule at all.
Where the Decision Logic Comes From
Machine learning takes a different route. Rather than having a person spell out the decision rules, machine learning is a method of creating intelligent applications by presenting a program with a large collection of data. The program is not handed the rule. It works out what matters from the examples it sees.
The source describes this as a method that allows algorithms to determine the characteristics needed to identify a specific entity, such as a face, from a large collection of data. The key shift is in where the knowledge comes from. In a handcoded system, the knowledge lives in the rules a person wrote. In a machine learning system, the knowledge is extracted from the data.
Comparing the Two Approaches
Placing the two approaches side by side makes the difference easier to hold onto. Both aim to produce an intelligent application, but they disagree about where the decision logic comes from. In the first column, a person supplies the logic. In the second, the logic is found in the data.
| Question | Handcoded rules | Machine learning |
|---|---|---|
| Where does the decision logic come from? | A person who knows the domain | Characteristics extracted from data |
| What does the designer decide? | Which words, conditions, or checks to write down | The data presented to the program |
| What happens when the problem changes? | The rules must be revised by hand | The program learns from the examples it is given |
| Where does it fit best? | Processes that humans understand well enough to model | Decisions that are hard to capture in explicit rules, such as faces in images |
Two ways of building an intelligent application
Common mistake Assuming machine learning always replaces hand-written rules
Why it failsThe source says manually crafting decision rules is feasible for some applications, especially where humans understand the process well. Rules are not a bad idea by default.
Do this insteadChoose the approach based on whether a person can state the decision rules clearly for the problem.
Common mistake Thinking a rule system can handle any case the designer forgot
What it looks likeA blacklist filter does not flag a junk message whose wording matches none of its entries.
Why it failsA rule system only knows what the designer wrote down, so unanticipated cases fall through.
Do this insteadRecognize that the rules are the limit of the system when the problem is too intricate to specify in full.
Common mistake Believing the program is handed the rule in a machine learning system
Trying a New Decision Yourself
A company wants to decide which incoming support tickets are urgent. Someone proposes writing an if and else rule for each urgent case, while another person suggests presenting the system with a large collection of past tickets and letting it find what makes a ticket urgent. Which approach matches the handcoded rule system described in this article, and what is the main risk of that approach?
Hints
- Ask where the decision logic lives in each proposal: with the person writing rules, or in the data.
- Think about which urgent cases a person would fail to write down in advance.
Name one decision from everyday life that you would find hard to capture in explicit if and else rules, and explain in one or two sentences why a rule written by hand would struggle with it.
Hints
- Choose a task where you recognize the answer instantly but cannot easily say how you know.
Key Takeaways
- Machine learning is a method of creating intelligent applications by presenting a program with a large collection of data.
- A handcoded rule system makes each decision through explicit if and else checks written by a person, such as a blacklist of words in a spam filter.
- Handcoded rules are feasible where humans understand the process well, but the source states they have two major disadvantages, and they can only cover cases the designer thought to write down.
- In machine learning, an algorithm determines the characteristics needed to identify something, such as a face, so the knowledge is extracted from data rather than written as rules.
- Spam filtering and face identification are examples of decisions that are hard to capture in explicit rules, which is why they motivate the machine learning approach.