Classification tasks
Machine learning discovers data-processing rules from examples of inputs and expected outputs.
From Examples to Classes
A classification task asks a machine-learning system to produce a class as the output for an input. The system is not given a complete list of hand-written rules for every possible input. Instead, machine learning discovers data-processing rules from examples of inputs and expected outputs.
The central idea is not simply producing an output. The system must discover rules that turn available input into a form useful for the task, such as classification.
The Learning Cycle
Learning requires three basic ingredients: input data, a way to measure performance, and feedback used to adjust the algorithm. Examples provide inputs together with expected outputs. The system can then compare its result with the expected output through a performance measurement. Feedback from that measurement is used to adjust the algorithm, helping it discover a more useful data-processing rule.
Tracing One Learning Cycle
A system receives examples of inputs and expected classes. Trace the role of each part of the learning process.
1. Receive examples: The system receives input data together with expected outputs.
2. Produce a result: The system uses its current data-processing rule to produce an output for an input.
3. Measure performance: The result is assessed using a performance measurement.
4. Use feedback: Feedback from the performance measurement is used to adjust the algorithm.
The rule changes through the learning cycle rather than being supplied as a complete hand-written list of rules.
What Representation Means
A representation is a way to encode or view data. It describes how the available input is presented so that a learning system can process it.
Representation does not mean that the underlying data has become a different real-world thing. It means that the data is encoded or viewed in a particular way. The choice of representation affects how directly a task can use the information.
Why Useful Representations Help
A representation is useful when it makes the information needed for a task more directly available to the learning process. For classification, a useful representation can make the expected classes easier to distinguish. The learning system is therefore concerned not only with which class it outputs, but also with how it changes the way it views the input.
Comparing Representations
Suppose the same input can be viewed in two different ways before a classification task. What should we ask about the two representations?
1. Identify the available information: Both representations describe the same available input data.
2. Examine task usefulness: Ask which representation presents information more directly for distinguishing the expected classes.
3. Connect usefulness to learning: A more useful representation can make discovering a rule for the classification task easier.
The better representation is not defined by being a different real-world object. It is useful because it makes the task easier for the learning process.
Representation Learning
Learning representations means finding transformations of input data that move the model closer to the expected output. The model is not merely memorizing an answer for each example. Its task is to discover rules that turn the available input into a form useful for the task, such as classification.
The search for appropriate representations is central to machine learning and deep learning because useful transformations help connect available inputs to expected outputs.
Mistakes About Classification
Thinking that machine learning begins with a complete list of hand-written rules for every input.
Machine learning discovers data-processing rules from examples of inputs and expected outputs.
Fix:
Focus on the learning process: examples are used to discover a rule rather than supplying a complete list of rules.Treating representation as the final class output.
A representation is a way to encode or view data. It is an input form that can support the task, not necessarily the expected output.
Fix:
Separate the represented input from the class that the system is expected to produce.Ignoring performance measurement and feedback.
Learning requires performance measurement and feedback used to adjust the algorithm.
Fix:
Include the full cycle: examples, output, performance measurement, feedback, and algorithm adjustment.Assuming that every representation makes a task equally easy.
Different representations of the same data can support different tasks more effectively.
Fix:
Ask whether a representation makes the information needed by the classification task more directly available.Equating representation learning with memorizing an answer for each example.
Representation learning searches for transformations that move the model closer to the expected output.
Fix:
Describe the goal as discovering useful transformations and rules for processing new available inputs.
Check Your Understanding
Explain, in your own words, why a classification system needs more than input data alone. Include the roles of expected outputs, performance measurement, feedback, and representation.
Hints
- Start with what examples provide.
- Explain how performance measurement produces useful feedback.
- End by describing how a representation can make classification easier.
A task seems difficult using one representation of its input. What question should you ask before concluding that the task itself cannot be learned?
Hints
- Compare the current representation with another possible way to encode or view the data.
- Ask which representation makes the task's information more directly usable.
Key Takeaways
- Machine learning discovers data-processing rules from examples of inputs and expected outputs.
- Learning requires input data, performance measurement, and feedback used to adjust the algorithm.
- A representation is a way to encode or view data.
- Different representations of the same data can make different tasks more or less effective.
- Representation learning searches for transformations that make the input more useful for tasks such as classification and is central to machine learning and deep learning.
Key Takeaways
- Classification uses learned data-processing rules to connect inputs with expected classes.
- Examples, performance measurement, and feedback form the basic learning cycle.
- A representation is an encoding or view of data, and a useful representation can make a task easier.
- Representation learning seeks transformations that move input data closer to the expected output.
- Finding useful representations is central to machine learning and deep learning.