Symbolic AI and Rule-Based Systems
Classical programming starts with rules written by humans and uses them to process data.
Two Ways to Program
Programming does not always begin with a human writing every rule that a computer must follow. In classical programming, humans write the rules and provide data for those rules to process. In machine learning, humans provide data together with expected answers, and the computer produces rules from that material. Those learned rules can then be used on new data.
What do you think happens?
In which approach does the computer produce rules from examples: classical programming or machine learning?
Reveal answer
Answer: Machine learning
Classical programming starts with rules written by humans. Machine learning starts with data and expected answers, from which the computer produces rules.
The Classical Programming Path
The classical programming model has a direct arrangement. A human supplies a program containing rules and also supplies data. The computer follows those supplied rules while processing the data, then produces answers. The path from input to output is designed by the programmer before the computer runs the program.
| Question | Classical programming | Machine learning |
|---|---|---|
| What do humans provide? | A program containing rules and data | Data together with expected answers |
| What does the computer produce? | Answers after following the supplied rules | Rules learned from the provided material |
| What happens to new data? | The human-written rules process it | The learned rules are applied to it to produce answers |
From Examples to Rules
Machine learning changes the usual starting point for programming. Instead of asking only what instructions a human should write, it asks whether a computer can examine data, discover useful rules, and use those rules on information it has not seen before. The human supplies examples in the form of data and expected answers. The computer uses those paired examples to produce rules. The rules are then available for processing new data.
The expected answers are important because they accompany the original data and give the computer the intended answer for each example. The result is not merely a collection of examples: it is a set of rules that can later be applied to new data.
A Message-Classification Trace
Generated teaching scenario: imagine that a human gives a computer many messages as data. For each message, the human also supplies an expected answer indicating which of two categories the message belongs to. The paired messages and answers provide the material from which the computer learns rules.
From Message Examples to a New Answer
Trace what happens when a computer receives message data together with expected category answers, followed later by an unfamiliar message.
Provide examples: The human supplies many messages as data and gives the expected category answer for each message.
Produce rules: The computer uses the paired messages and answers to produce rules.
Receive new data: A new message arrives that was not part of the original material.
Apply the rules: The learned rules are applied to the new message.
Produce an answer: The rules produce an answer for the new case.
The computer moves from human-provided examples and expected answers to learned rules, then uses those rules to answer a case that was not in the original examples.
Lovelace and the Learning Question
The historical question behind machine learning concerns whether computers can do more than follow explicitly ordered instructions. In 1843, Ada Lovelace argued that Charles Babbage's Analytical Engine could perform whatever people knew how to order it to perform, but could not originate anything. Alan Turing later discussed this position as Lady Lovelace's objection while considering whether general-purpose computers could learn and show originality.
Machine learning addresses the learning part of that question by changing where the rules come from. In classical programming, the rules are explicitly supplied by a human. In machine learning, the computer examines data and expected answers and produces rules that can be used on new information. This does not erase the human contribution: humans still provide the data and expected answers. It changes the role of the computer from only following supplied rules to producing rules from the provided material.
Mistakes About Rule Sources
Saying that classical programming learns rules from data.
Classical programming starts with rules written by humans, which the computer then uses to process data.
Fix:
Describe classical programming as human-supplied rules plus data producing answers.Saying that machine learning begins with rules written explicitly for every case.
Machine learning starts with data and expected answers, from which the computer produces rules.
Fix:
Identify the data and expected answers as the starting material for learning.Stopping after the computer produces learned rules.
The learned rules can be applied to new data to generate answers.
Fix:
Trace the complete path: examples and expected answers, learned rules, new data, and an answer.Claiming that humans provide nothing in machine learning.
Humans provide the data together with the answers expected for that data.
Fix:
Separate the human contribution from the computer's contribution: humans provide examples and expected answers; the computer produces rules.
Check Your Understanding
A human provides a collection of data and an expected answer for each item. The computer produces rules and then receives new data. Explain the sequence from the original material to the answer for the new data. Then state whether this describes classical programming or machine learning, and explain why.
Hints
- Identify what the human provides before the computer works.
- Identify what the computer produces from the paired data and answers.
- Include the step in which the produced rules are applied to new data.
Compare the following two descriptions: one human writes rules and supplies data; another human supplies data with expected answers and the computer produces rules. For each description, name the programming approach and state what the computer produces.
Hints
- Classical programming begins with human-written rules.
- Machine learning begins with data and expected answers.
- The output of the machine-learning stage is rules that can later be applied to new data.
Key Takeaways
- Classical programming starts with rules written by humans and uses those rules to process data.
- Machine learning starts with data and expected answers, from which the computer produces rules.
- Learned rules can be applied to new data to generate answers for cases outside the original material.
- The distinction between the two approaches is mainly about what humans provide and where the rules come from.
- Lady Lovelace's objection provides a historical backdrop for asking whether computers can learn rather than merely follow explicitly ordered instructions.
Key Takeaways
- Classical programming uses human-written rules to process data.
- Machine learning uses data and expected answers to produce rules.
- The learned rules can be applied to new data to produce answers.
- The key difference is the source of the rules and the role assigned to the computer.
- Lady Lovelace's objection connects machine learning to the historical question of whether computers can learn rather than only follow ordered instructions.