Deeply Layered Artificial Neural Networks
An ANN is a network of interconnected units, not a single isolated unit.
From Unit to Network
An artificial neural network, or ANN, is not one isolated computational unit. It is a network of interconnected units. Individual units have some properties similar to neurons in nervous systems, but the central idea is the complete connected system: many units working together to represent a relationship.
When you study an ANN, focus on the relationships among its units, not only on what one unit does by itself.
Connected Units as One System
The word network changes the way the system should be understood. A single unit can be considered in isolation, but an ANN is defined by interconnected units. The network provides the structure used to represent a relationship. Therefore, the relevant question is not simply whether one unit can express a rule. The more useful question is how the connected collection of units can represent a relationship as a system.
Reading the Network View
Suppose a description refers to three interconnected units as one artificial neural network. What should you identify as the main system?
Identify the units: The three units are the individual components of the system.
Identify the connections: The relationships among the units are essential because the system is described as interconnected.
Name the system: The complete collection of connected units is the artificial neural network, rather than any one unit considered alone.
The ANN is the connected network of units. Treating one unit as if it were the whole ANN misses the central definition.
Representing Nonlinear Relationships
Many machine learning tasks involve relationships that are not simply a straight, easily described rule. Artificial neural networks are widely used for nonlinear function approximation: they use a network of connected units as the tool for representing such a relationship. The important contrast is between an isolated simple rule and a connected system intended to represent a relationship that does not have a simple straight description.
Consider a machine learning task whose relationship cannot be described by a simple straight rule. In the source description, this is the kind of task for which ANNs are widely used. The example does not require selecting one isolated unit; it requires viewing the connected network as the representation tool.
Information Through Deep Layers
A deeply layered ANN extends the network idea across many layers. The basic idea remains a network of interconnected units. What changes is the depth of the network: there are many layers of connected units between the beginning and the end of the system. This layered structure is a useful way to think about how a machine learning system can represent a relationship as a connected system rather than as one isolated rule.
When explaining a deeply layered ANN, describe both parts of the idea: the units are interconnected, and the network contains many layers. Mentioning depth without mentioning connected units leaves out the basic ANN concept.
Training and Expanded Capabilities
Artificial neural networks have a long history. More recent progress in training deeply layered ANNs has contributed to some of the most impressive abilities of machine learning systems. This places training advances in context: the underlying concept is still a network of interconnected units, while improved training of networks with many layers has helped those systems achieve stronger capabilities.
Mistakes in Interpretation
Treating an ANN as one isolated unit
The defining description is a network of interconnected units, not a single isolated unit.
Fix:
Describe the units together with their connections as the ANN.Explaining ANN use without mentioning nonlinear relationships
The source identifies nonlinear function approximation as a major reason ANNs are widely used.
Fix:
Connect ANN use to representing relationships that are not simply straight, easily described rules.Assuming that depth alone defines the whole concept
Deep layering is part of the concept, but the foundational ANN idea is still a network of interconnected units.
Fix:
Explain depth as many layers within the connected network.Separating training advances from machine learning abilities
The source places progress in training deeply layered ANNs in the story of impressive machine learning capabilities.
Fix:
State that advances in training have contributed to stronger machine learning abilities.
Check Your Understanding
Explain, in your own words, why an artificial neural network should be described as a connected system rather than a single isolated rule. Then explain why nonlinear function approximation is an important use of ANNs.
Hints
- Begin with the definition of an ANN as a network of interconnected units.
- Contrast a simple straight, easily described rule with a relationship that is not simply straight.
- End by connecting progress in training deeply layered ANNs to machine learning abilities.
Key Takeaways
- An artificial neural network is a network of interconnected units, not one isolated unit.
- ANN units have some properties similar to neurons in nervous systems.
- ANNs are widely used for nonlinear function approximation because many machine learning relationships are not simply straight, easily described rules.
- A deeply layered ANN preserves the network idea while organizing interconnected units across many layers.
- Progress in training deeply layered ANNs has contributed to impressive abilities in machine learning systems.
Key Takeaways
- An ANN is defined by interconnected units working as a network.
- The network perspective is more important than treating one unit as the whole system.
- ANNs are widely used to approximate nonlinear relationships that are not simply straight, easily described rules.
- Deeply layered ANNs extend the connected-network idea across many layers.
- Advances in training deeply layered networks have contributed to expanded machine learning capabilities.