Weights and Connections in Neural Networks
Feedforward describes the absence of loops in an ANN's connections.
The Loop Test
A neural network may have an input on one side and an output on the other, but that description alone does not tell you whether it is feedforward. The decisive question is structural: do the connections contain a loop? A feedforward artificial neural network has no loop in its connections. If even one loop exists, the network is classified as recurrent.
Forward Information Flow
In a feedforward artificial neural network, information moves through the network's connection pattern without returning through a loop to an earlier unit. The network is organized into an input layer, one or more hidden layers, and an output layer. The input layer is where the input enters, hidden layers are internal layers, and the output layer produces the output.
Feedforward describes the connection pattern, not merely the fact that a network has an input and eventually produces an output.
Layers and Internal Units
Each layer has a distinct position. The input layer receives the input. The output layer produces the output. Any layer that is neither the input layer nor the output layer is a hidden layer. A feedforward ANN may have one or more hidden layers.
Weights on Connections
The units in a feedforward ANN are connected by links. Every link has an associated real-valued weight. Therefore, a connection is not identified only by the units it joins; its associated weight is also part of the network's specification. In terminology used for biological neural networks, a weight roughly corresponds to the efficacy of a synaptic connection.
Classifying a layered network
Consider a network with an input layer, one hidden layer, and an output layer. Every link between its units has a real-valued weight, and no link returns to an earlier unit. Is it feedforward or recurrent?
Locate the layers: The network has the input, hidden, and output layer roles expected in a feedforward ANN.
Inspect the connections: The connections do not form a loop, so no unit's output can influence its own input through a returning path.
Use the decisive test: The absence of loops is the feature that determines the classification. The real-valued weights describe the links but do not change this structural test.
The network is feedforward.
When a Loop Changes the Label
A network can still have recognizable input, hidden, and output layers while being recurrent. The classification changes when at least one loop exists. In particular, a feedforward ANN has no path through which a unit's output can influence its own input. One loop is enough to make the network recurrent.
| Feature | Feedforward ANN | Recurrent ANN |
|---|---|---|
| Connection pattern | No loops | At least one loop |
| Input, hidden, and output layers | Can be identified | Can still be identified |
| Unit output influencing its own input through a path | No | Possible because a loop exists |
Calling every network with an input and an output feedforward.
Having an input and an output is not sufficient. The decisive feature is whether the connections contain a loop.
Fix:
Inspect the complete connection pattern and classify the network as recurrent if even one loop exists.Assuming that a hidden layer is disconnected or inactive.
Hidden means that the layer is internal rather than the input or output layer.
Fix:
Identify hidden layers by position: any layer that is neither input nor output is hidden.Ignoring weights when describing the network's connections.
Weights are an additional part of the network's specification.
Fix:
Describe each connection together with its associated real-valued weight.
Classify the Pattern
A diagram contains an input layer, two hidden layers, and an output layer. All connections move through the layers without returning to an earlier unit. Each connection has a real-valued weight. Classify the network and justify your answer using the connection pattern.
Hints
- Count the layer roles, then inspect whether any connection creates a loop.
- The number of hidden layers does not determine whether the network is feedforward.
- Use the absence or presence of loops as the decisive evidence.
What do you think happens?
Suppose the same network gains one connection from an output-side unit back to an earlier hidden unit. What is its classification?
Reveal answer
Answer: It is recurrent.
The presence of even one loop changes the classification from feedforward to recurrent, even though the input, hidden, and output layers can still be identified.
Key Takeaways
- A feedforward ANN is defined by the absence of loops in its connections.
- A feedforward ANN is organized into an input layer, one or more hidden layers, and an output layer.
- The input layer receives input, the output layer produces output, and hidden layers are internal layers.
- Every connection between units has an associated real-valued weight.
- The presence of even one loop makes the network recurrent.
Key Takeaways
- Feedforward refers to a network's connection pattern, specifically the absence of loops.
- Input, hidden, and output layers describe the positions and roles of layers in the network.
- Every link between units has an associated real-valued weight.
- A single loop is enough to classify the network as recurrent rather than feedforward.