Pretrained ImageNet Models

From Image to Dream

DeepDream does not begin by creating an image from nothing. It starts with an existing image and changes the image so that a convolutional network responds more strongly inside selected layers. As the pixels are repeatedly adjusted, patterns learned by the network become visually prominent. The result is an altered version of the starting image rather than an unrelated collection of new pixels.

One Ascent Step

The network evaluates the current image and produces activations in its layers. DeepDream uses those activations to define an objective. Gradient ascent then determines how the input pixels should change so that the objective becomes larger. Repeating the adjustment makes the image a progressively stronger expression of features learned by the network.

The important direction is from the objective back to the input pixels. DeepDream is not merely recording what the network sees. It uses the response to decide how to alter what the network will see next. Because the process is repeated, small pixel changes accumulate into visible distortions connected to patterns already available in the original image.

Whole Layers and Existing Images

DeepDream and filter visualization use the same broad strategy: change the input in the direction that increases an internal network response. Their distinctive choices are different. Filter visualization can maximize one specific filter, while DeepDream maximizes an entire layer. Since a whole layer combines many learned feature responses, the resulting image can contain a mixture of visual patterns instead of one isolated feature.

The Loss Behind the Change

Gradient ascent needs one numerical objective to increase. DeepDream defines this objective as a weighted sum of L2-based activation contributions from selected layers. Each layer has a coefficient, so its contribution can have more or less influence on the total objective. The cited formulation normalizes contributions by the number of activation values and leaves out border pixels when calculating the activation contribution, helping avoid border artifacts.

You can interpret the loss as a collection of layer measurements being combined into one score. First, activation values from a selected layer contribute an L2-based magnitude. That contribution is normalized over the activation values, with border positions excluded in the cited formulation. Then the layer's weight determines how strongly that contribution affects the total. Gradient ascent changes the image to make this combined score larger.

• Selected layers provide the activation contributions.
• Each contribution is based on the L2 magnitude of the layer's activations.
• The contributions are normalized by the number of activation values.
• Border activation positions are left out in the cited formulation.
• Layer coefficients weight the contributions before they are combined.
• The resulting weighted total is the objective increased by gradient ascent.

Why DeepDream Uses Octaves

DeepDream does not process the image at only one size. It defines several image scales called octaves. Processing begins with a smaller version and moves through increasingly larger versions. Each successive scale is 1.4 times the previous scale, which is 40 percent larger. Working across these scales allows feature maximization to act while the image is examined at different sizes, improving the quality of the visualization.

Choosing the Network

DeepDream depends on the features learned by the convolutional network that evaluates the image. The source example uses the Inception V3 model available with Keras, while also noting that other pretrained convolutional networks can be used. Because different architectures learn different features, changing the pretrained network can change the appearance of the resulting visualization.

Common Reasoning Errors

• Thinking DeepDream creates an image from blank input.

  DeepDream starts with an existing image. Blank, slightly noisy input is associated with the contrasting filter-visualization procedure described in the source.

  Fix: Remember that DeepDream repeatedly nudges an existing image toward stronger network responses.

• Treating DeepDream as maximization of one filter.

  DeepDream maximizes an entire layer, which combines many learned feature responses.

  Fix: Use whole-layer maximization to explain why the output can contain a mixture of visual patterns.

• Describing gradient ascent as changing the network weights.

  The source describes gradient ascent as repeatedly adjusting the input pixels in the direction that increases the objective.

  Fix: Track the changing object correctly: the image pixels are adjusted so the network's selected activations become larger.

• Ignoring the scale progression.

  DeepDream defines several image scales called octaves and moves from smaller to increasingly larger versions.

  Fix: Include the octave sequence and the 1.4 scale factor between successive scales.

• Treating the loss as one unweighted activation value.

  The objective is a weighted sum of L2-based activation contributions, with normalization over activation values and border positions omitted in the cited formulation.

  Fix: Explain how selected-layer contributions are measured and then combined according to their coefficients.

Check Your Model

Key Takeaways

1. DeepDream modifies an existing image by maximizing activations inside a convolutional network.
2. Unlike single-filter visualization, it can maximize whole layers, allowing many learned feature responses to influence the result.
3. Gradient ascent repeatedly changes input pixels in the direction that increases the objective.
4. Octaves are successive image scales, beginning with a smaller version and increasing by a factor of 1.4 between scales.
5. The objective combines weighted, normalized L2-based activation contributions from selected layers, with border positions omitted in the cited formulation.