Convnet Filter Visualization
DeepDream uses the input to a convnet as the object being optimized.
The Input Becomes the Object
Many convnet explanations focus on what happens inside the network. DeepDream reverses the usual direction of attention: instead of changing the convnet to fit an image, it changes the input supplied to the convnet. The input is the object being optimized, and the goal is to make a chosen filter more strongly activated.
The central operation is gradient ascent on the convnet's input.
One Optimization Step
A filter-visualization process starts with an input and a selected filter. The convnet evaluates that input, and the selected filter has an activation. Gradient ascent then provides the direction for adjusting the input. The adjusted input is supplied to the convnet again, with the aim of increasing that filter's activation.
Maximizing a Chosen Filter
Suppose a learner selects one filter and wants to visualize the input that activates it most strongly.
Select the objective: Choose the activation of one specific filter as the objective.
Evaluate the input: Supply the current input to the convnet and observe the selected filter's activation.
Use gradient ascent: Use the gradient-ascent direction to adjust the convnet input toward a stronger activation.
Repeat the process: Supply the adjusted input to the convnet again and continue optimizing the input toward the selected filter's objective.
The optimized input is being used to visualize what the selected filter responds to, because the input has been adjusted to maximize that filter's activation.
Gradient Ascent Over Time
The important state change is not a change to the convnet itself. The input changes from one optimization step to the next. At each step, the current input is passed through the convnet, the selected filter's activation supplies the objective being maximized, and gradient ascent determines how to adjust the input for the next step.
DeepDream and Filter Visualization
DeepDream and convnet filter visualization should not be treated as unrelated procedures. The source describes DeepDream as almost identical to the filter-visualization technique. Both optimize the convnet input with gradient ascent so that a selected filter's activation is maximized.
| Aspect | DeepDream | Convnet filter visualization |
|---|---|---|
| What is changed | The input supplied to the convnet | The input supplied to the convnet |
| Central operation | Gradient ascent on the input | Gradient ascent on the input |
| Objective | Maximize the activation of a selected filter | Maximize the activation of a selected filter |
| Relationship | Almost identical to filter visualization | The technique whose central idea DeepDream uses |
What the Process Reveals
The purpose of the process is to visualize a convnet filter. Repeatedly modifying the input to amplify a chosen filter reveals an input that the convnet has been optimized to associate with a stronger activation of that filter. DeepDream can therefore be understood as an image-directed use of a convnet: the image changes so that the selected filter responds more strongly.
Mistakes Beginners Make
Assuming the convnet is modified during DeepDream.
The source states that DeepDream changes the input supplied to the convnet and uses that input as the object being optimized.
Fix:
Track the input as the changing object; the selected filter's activation is the objective.Describing gradient ascent as an adjustment to the filter.
Gradient ascent provides the direction for adjusting the convnet input.
Fix:
Say that the input is adjusted in a direction intended to maximize the selected filter's activation.Treating DeepDream and filter visualization as completely different methods.
The source describes DeepDream as almost identical to convnet filter visualization.
Fix:
Start with their shared central idea: optimize the convnet input with gradient ascent to maximize a selected filter's activation.Adding an unsupported distinction between the two techniques.
The source mentions a few simple differences but does not specify them.
Fix:
Report only the shared mechanism and acknowledge that the unspecified differences should not be filled in by guesswork.
Check the Mechanism
A learner says: The goal of DeepDream is to train the convnet until it recognizes the starting image. Correct the statement using the roles of the input, gradient ascent, and selected filter activation.
Hints
- Identify which object is optimized.
- Identify what gradient ascent changes.
- Identify the objective that is maximized.
A Complete Trace
Trace the roles in a filter-visualization step from the current input to the next input.
Current input: The current input is supplied to the convnet.
Selected activation: The activation of the chosen filter is treated as the objective.
Gradient-ascent direction: Gradient ascent provides the direction for adjusting the input.
Next input: The adjusted input is supplied again so the selected filter's activation can be optimized further.
The input changes step by step while the convnet is used to guide those changes toward a larger activation of the selected filter.
Key Takeaways
- DeepDream uses the convnet input as the object being optimized.
- Its objective is to maximize the activation of a specific filter.
- Gradient ascent supplies the direction for adjusting the input.
- Convnet filter visualization and DeepDream share the same central mechanism and are described as almost identical.
- The process visualizes a filter by repeatedly modifying the input used by the convnet.
Key Takeaways
- DeepDream optimizes the input rather than changing the convnet.
- Gradient ascent adjusts the input toward stronger activation of a chosen filter.
- Filter visualization and DeepDream have the same central idea in the available material.
- Tracing the process means following the input, filter activation, gradient direction, and updated input.
- The optimized input provides a visualization of what strongly activates the selected filter.