Latent Representations

From Images to Points

A creative AI system does not have to work only with complete images. It can learn a compact space of representations for images. In this space, each point is a low-dimensional vector, and the point can be mapped to a realistic-looking image. This is called a latent space of images.

The word latent means that the representation is not the visible image itself. The learner works with a point in a compact representation space, while a separate mapping can turn that point back into image space. The important change in viewpoint is this: instead of choosing every image pixel directly, a system can choose a point in a learned representation space and then use that point to produce an image.

Following a Latent Point

The mapping can be followed in two steps. First, a point is selected in the low-dimensional latent space. Second, that point is mapped into image space, where it becomes an image represented as a grid of pixels. The point is therefore useful because it gives the system a compact way to specify an image representation.

Sampling New Images

A point in a learned latent space does not have to be selected from a previously stored list of images. A point can be selected deliberately or sampled at random. After the point is passed through the mapping back to image space, the result can be an image that the system has never seen before.

The word unseen refers to the generated result, not to a claim that the system has created an image without using anything it learned. The system first learns the latent space and then uses a selected or sampled point in that space. The resulting mapped image can nevertheless be an image that was never present in the system's earlier collection of seen images.

The VAE Decoder

A variational autoencoder, or VAE, learns a latent space of image representations and provides a decoder. The decoder maps a latent point to an image. In particular, the decoder maps a selected or randomly sampled latent point into an image grid of pixels.

The decoder is the part that performs the return journey from representation space to image space. It accepts a latent point and produces the image grid of pixels associated with that point. Because the input point may be selected or randomly sampled, the decoder can be used both to map a chosen representation and to generate a new image from a sampled representation.

Smooth Organization

VAEs are useful for learning well-structured latent spaces. In such a space, directions can represent meaningful axes of variation. This gives the latent representation an organization that is useful for deliberately choosing points and for understanding how representations vary.

The source material describes meaningful axes of variation as a benefit of a well-structured latent space. The practical idea is that changing position in the representation space can be used to explore variation in the images produced by the decoder. The latent space is therefore not merely a storage area for unrelated points; its organization is part of what makes it useful for image generation.

VAE and GAN Scope

The supplied material gives a clear account of VAE latent spaces: a VAE learns a latent space of image representations, provides a decoder, and is useful for learning a well-structured space in which directions can represent meaningful axes of variation. The supplied material names generative adversarial networks, or GANs, as a comparison topic but does not describe how a GAN organizes its latent space.

Mistakes to Avoid

• Treating a latent point as if it were already a finished image.

  The source describes the point as a low-dimensional representation. It must be mapped into image space to produce an image grid of pixels.

  Fix: Describe the point as an image representation and the decoder or mapping as the step that produces the image.

• Assuming generation requires selecting an image that already existed.

  The source states that sampling from a learned latent space can produce images that have never been seen before.

  Fix: Explain that a selected or randomly sampled point can be passed through the decoder to produce an unseen image.

• Describing the decoder as the latent space itself.

  The latent space is the low-dimensional vector space. The decoder is the mapping that takes a latent point to an image.

  Fix: Keep the representation space and the mapping from that space to image pixels as separate parts of the explanation.

• Claiming a detailed VAE-GAN contrast that is not supported by the supplied material.

  The supplied material explains the VAE side but does not specify how GAN latent spaces are organized.

  Fix: State the supported VAE properties and identify the GAN details as outside the scope of this source.

Check Your Understanding

Key Takeaways

1. A latent space of images is a low-dimensional vector space whose points can be mapped to realistic-looking images.
2. A point may be selected deliberately or sampled at random, and mapping it into image space can produce an image that has never been seen before.
3. In a VAE, the decoder maps a selected or sampled latent point into an image grid of pixels.
4. VAEs are useful for learning well-structured latent spaces in which directions can represent meaningful axes of variation.
5. The supplied material supports a detailed description of VAE latent spaces but does not provide enough information for a detailed technical comparison with GAN latent spaces.