Deep Learning Basics
Classical programming starts with rules written by humans and uses them to process data.
The Programming Question
Most programming begins with a human deciding which rules the computer should follow. Machine learning begins with a different question: can a computer examine data, discover useful rules, and use those rules on information it has not seen before? The key change is where the rules come from. In classical programming, humans write them. In machine learning, the computer produces them from data and expected answers.
Machine learning does not begin by giving the computer a complete, explicitly ordered set of rules. It begins with examples containing data and expected answers.
From Examples to Rules
The machine learning process described here has three important stages. First, humans provide data together with the answers expected for that data. Second, the computer uses those paired examples to produce rules. Third, the learned rules are applied to new data, producing answers for cases that were not part of the original material.
Classifying Messages
A human provides many messages as data and supplies an expected category for each message. How can those examples support an answer for a new message?
Provide paired examples: The computer receives messages together with the expected answer for each message, such as one category or another.
Produce rules: The computer uses the paired data and answers to learn rules.
Receive a new case: A new message arrives that was not part of the original material.
Apply the rules: The learned rules are applied to the new message to produce an answer for that case.
The original examples lead to learned rules, and those rules generate an answer for a new message.
Classical and Machine Learning
The clearest comparison is to track what humans provide and what the computer produces. In classical programming, humans provide a program containing rules and data for those rules to process. The computer follows the supplied rules and produces answers. In machine learning, humans provide data together with the expected answers. The computer uses that material to produce rules, which can later be applied to new data.
| Approach | Humans provide | Computer produces |
|---|---|---|
| Classical programming | A program containing rules and data | Answers after following the supplied rules |
| Machine learning | Data together with expected answers | Rules that can be applied to new data |
The central difference is the direction of the relationship between rules, data, and answers.
Using Rules on New Data
Learning does not end when the computer produces rules. The purpose of those rules is to handle new data. A new case moves into the learned rules, and the rules produce an answer for that case. The new case was not part of the original examples, so this stage shows how machine learning changes the usual programming path: the computer first produces rules from examples, then uses those rules on additional information.
What do you think happens?
A computer has used paired messages and expected categories to produce rules. What should happen when a message arrives that was not in the original examples?
Reveal answer
Answer: The computer applies the learned rules to produce an answer.
The learned rules are intended to be used on new data, including cases that were not part of the original material.
Lady Lovelace's Objection
The historical question behind machine learning concerns whether a computer can do more than follow instructions explicitly ordered by people. In 1843, Ada Lovelace argued that Charles Babbage's Analytical Engine could perform whatever people knew how to order it to perform, but could not originate anything. Alan Turing later discussed this idea as Lady Lovelace's objection while considering whether general-purpose computers could learn and show originality.
Machine learning addresses the learning part of that historical question by changing where the rules come from. Rather than asking only what instructions a human should write, it asks whether a computer can examine data, discover useful rules, and then use those rules on information it has not seen before. This does not erase the role of humans: people provide the data and expected answers. It does change the computer's role from merely following human-written rules to producing rules from the supplied material.
The historical connection is about the origin of rules. Lady Lovelace's objection asks whether a computer can originate anything; machine learning explores whether a computer can learn rules from data instead of receiving every rule directly from a human.
Mistakes About Machine Learning
Saying that machine learning starts with rules written by humans.
That describes the classical programming arrangement. In the machine learning arrangement, humans provide data and expected answers, and the computer produces rules.
Fix:
State that machine learning starts with data and expected answers from which the computer produces rules.Stopping after the computer produces rules.
The learned rules are meant to be applied to new data to generate answers for new cases.
Fix:
Describe both stages: producing rules from examples and applying those rules to new data.Claiming that humans provide nothing in machine learning.
The described machine learning process depends on humans providing data together with expected answers.
Fix:
Identify the human contribution as the data and expected answers, while identifying learned rules as the computer's production.Treating machine learning as unrelated to the historical question about computers and originality.
The source connects the objection to the question of whether computers can learn rather than merely follow explicitly ordered instructions.
Fix:
Explain that machine learning addresses the learning part of that question by changing where the rules come from.
Check Your Understanding
Compare the two programming approaches in your own words. For each approach, identify what humans provide, what the computer does with that material, and what the computer produces. Then describe how a new message would be handled after machine learning has produced rules.
Hints
- Use the sequence humans provide, computer processes or learns, and computer produces.
- For classical programming, focus on a human-written program containing rules.
- For machine learning, focus on data together with expected answers, followed by learned rules and their application to new data.
A Complete Trace
Trace the machine learning process for a new message.
Human input: Humans provide many messages and the expected category for each message.
Rule production: The computer uses the paired messages and categories to produce learned rules.
New data: A message arrives that was not included in the original examples.
Answer production: The learned rules are applied to the new message and produce an answer for that case.
Data and expected answers lead to rules, and the rules are then used to answer a new case.
What to Remember
- Classical programming uses human-written rules to process data and produce answers.
- Machine learning uses data and expected answers to produce rules.
- Learned rules can be applied to new data to generate answers for cases not included in the original examples.
- The historical connection to Lady Lovelace's objection concerns whether computers can learn rather than merely follow explicitly ordered instructions.
- Machine learning changes where the rules come from, while humans still provide the data and expected answers.
Key Takeaways
- Classical programming begins with human-written rules and data; the computer follows those rules to produce answers.
- Machine learning begins with data and expected answers; the computer uses them to produce rules.
- Those learned rules can be applied to new data to produce answers for new cases.
- Lady Lovelace's objection provides the historical question of whether a computer can learn rather than merely follow explicitly ordered instructions.