Introduction to Artificial Neural Networks
Feedforward describes the absence of loops in an ANN's connections.
The Decisive Structural Question
When people first see a neural network, they may focus on the fact that information enters through one side and eventually produces an output. That description is not enough to classify the network as feedforward. The decisive question is whether the network's connections contain a loop. A feedforward artificial neural network has no loop in its connections. If at least one loop exists, the network is recurrent instead.
Feedforward describes the connection pattern, not merely the existence of an input and an output. The absence of loops is the defining feature.
Tracing a Feedforward Network
To trace a feedforward network, begin at the input layer and follow its connections toward the output layer. The network can contain one or more hidden layers between them. The important restriction is that the connections do not create a path through which a unit's output can influence its own input. Information therefore moves through the network without returning to an earlier point in a loop.
Classifying a Three-Layer Network
A network has an input layer, one hidden layer, and an output layer. Its connections lead from the input layer to the hidden layer and from the hidden layer to the output layer. No connection creates a loop. How should the network be classified?
Locate the entrance: The input layer is the network's entrance because it receives the input.
Locate the internal layer: The layer between the input and output layers is a hidden layer. It is internal to the network, not disconnected or inactive.
Locate the exit: The output layer is the network's exit because it produces the output.
Check for a loop: The connections do not form a loop, so the network satisfies the defining structural condition for a feedforward ANN.
The network is a feedforward artificial neural network.
Layer Positions and Roles
A feedforward ANN is organized into input, hidden, and output layers. The input layer receives the input. The output layer produces the output. Any layer that is neither the input layer nor the output layer is a hidden layer. A feedforward ANN may have one or more hidden layers.
Weights on the Connections
The units in an ANN are connected by links. Every link has an associated real-valued weight. In terminology used for biological neural networks, a weight roughly corresponds to the efficacy of a synaptic connection. These weights are part of the network's specification, while the presence or absence of loops determines whether the network is feedforward or recurrent.
For classification, inspect the connection pattern first: a loop makes the network recurrent. The real-valued weights describe the links but do not remove or create the structural loop criterion by themselves.
When One Loop Changes the Classification
A network can still have recognizable input, hidden, and output layers while being recurrent. The deciding issue is not whether most connections move toward the output. The presence of even one loop changes the classification to recurrent.
A Mostly Forward Network with a Loop
A network has an input layer, a hidden layer, and an output layer. Most connections move from input to hidden to output, but one connection leads from the output back to a hidden unit. Is the network feedforward?
Identify the layers: The input, hidden, and output layers can still be identified from their positions and roles.
Inspect the extra connection: The connection from the output back to a hidden unit returns to an earlier part of the network.
Apply the loop rule: That return connection creates a loop. The rule depends on whether any loop exists, not on whether most connections point forward.
The correct classification is recurrent, not feedforward.
Calling every network with an input and an output feedforward.
Having an input and an output does not establish that the connections are loop-free.
Fix:
Trace the connections and check whether any path forms a loop.Treating hidden layers as disconnected.
Hidden means that the layer is internal rather than the entrance or exit.
Fix:
Identify hidden layers as layers that are neither the input layer nor the output layer.Using weights alone to classify the network.
Weights are part of the network specification, but the feedforward or recurrent classification is determined by whether the connections contain a loop.
Fix:
Check the connection pattern for loops, then note that each link also has an associated real-valued weight.
Check Your Classification
A diagram contains an input layer, two hidden layers, and an output layer. Every connection moves toward a later layer, and no connection returns to an earlier layer. Classify the network and explain which observation supports your answer.
Hints
- Start by checking whether any connection creates a loop.
- The number of hidden layers does not decide the classification.
- Use the absence or presence of a loop as your main evidence.
What do you think happens?
A network has input, hidden, and output layers, but one link creates a loop. What is its classification?
Reveal answer
Answer: Recurrent
The presence of even one loop changes the classification to recurrent. The network may still have identifiable input, hidden, and output layers.
Essential Takeaways
- A feedforward ANN has no loops in its connections.
- The input layer receives input, the output layer produces output, and internal layers are hidden layers.
- A feedforward ANN may contain one or more hidden layers.
- Every connection between units has an associated real-valued weight.
- The presence of even one loop makes the network recurrent rather than feedforward.
Key Takeaways
- Feedforward refers to the absence of loops in an ANN's connections.
- A feedforward ANN is organized into input, hidden, and output layers.
- Each connection between units has an associated real-valued weight.
- A single loop is enough to classify the network as recurrent.