State Representation

Tile coding converts continuous states into coarse, tile-based features.

Programming

From Location to Feature

A learning system may receive a state described by continuous values, but it may still need a feature representation that is easier to process. Tile coding provides one way to convert that continuous state into coarse, tile-based features. Instead of preserving every detail of the state's exact location, it records which region of the input space contains the state.

How does a multi-dimensional continuous state get converted into a set of tile-based features?

The central transition is from an exact location to region membership. A feature records which receptive field contains the state. It does not preserve every detail about where the state lies inside that region. That loss of detail is what makes the representation coarse.

Tilings and Tiles

A tiling is the complete partition of the input space. A tile is one receptive-field region inside that complete partition.

The distinction is one of whole structure versus individual region. A tiling describes how the complete space is divided. A tile is one part of that division. For a two-dimensional state space, a uniform grid is a simple example of a tiling: the complete grid is the tiling, while each individual cell is a tile.

What is the relationship between a complete tiling that partitions the space and one tile within that tiling?

Identifying the Tile

Imagine a two-dimensional state space divided by a uniform grid. A particular state lies inside the grid cell named Tile B. What does the tiling tell us, and what does the tile tell us?

Identify the whole partition: The uniform grid is the tiling because it describes the complete division of the two-dimensional input space.

Identify the containing region: Tile B is one tile because it is one receptive-field region inside that complete grid.

Create the feature: The state is represented by the tile-based feature associated with Tile B. The representation records membership in that region rather than the state's every exact coordinate.

The tiling supplies the partition, while the tile supplies the particular region-based feature selected by the state.

A State Inside a Tile

Suppose a continuous state has several values and falls inside one tile of a multi-dimensional partition. Tile coding does not need to retain every detail of the state's position within that tile for this feature representation. It identifies the receptive field containing the state and uses that region to identify a feature.

Representing a Point by Region Membership

Consider a two-dimensional state space partitioned into square tiles. A state lies somewhere inside Tile 4. How is that state represented by one tiling?

Start with the continuous state: The state has an exact location in the two-dimensional space, but its values are continuous.

Find the containing tile: The state is checked against the partition, and Tile 4 is the receptive-field region containing it.

Record tile membership: The feature representation identifies Tile 4. It does not preserve every detail of the state's location inside Tile 4.

With this tiling, the state is represented by membership in Tile 4.

Which tile or feature position becomes active when a particular state falls within a tile?

Why One Tiling Is Coarse

Using only one tiling gives identical tile-based treatment to states that lie inside the same tile. Two states can have different exact continuous values but still receive the same feature treatment if they remain in the same receptive-field region. When a state crosses a tile boundary, its identified tile can change. This is state aggregation: many exact states are grouped according to the one region they share.

Why does one tiling assign a whole region of states the same representation, while a broader coarse-coded representation can use information from more than one partition?

The single-tiling case is therefore narrower than full coarse coding. One tiling supplies one region-membership view of the state. A fuller coarse-coded representation can use information from multiple tilings, so the state is described through more than one region-membership view rather than through only one partition.

Computational Value

Tile coding is useful for multi-dimensional continuous spaces because it converts continuous states into coarse, tile-based features. The system can work with region membership instead of processing every detail of the exact continuous location in the same form. This makes tile coding flexible and computationally efficient as a feature representation.

How does tile coding replace detailed continuous-state processing with region-based feature information?

Common Misunderstandings

Treating a tiling and a tile as the same thing.
A tiling is the complete partition, while a tile is one receptive-field region inside that partition.
Fix: Use tiling for the whole division of the space and tile for one region.
Assuming tile coding preserves the exact continuous location.
The feature records which receptive field contains the state, not every detail of its position within that region.
Fix: Think in terms of region membership.
Assuming one tiling gives a detailed representation of every state.
States inside the same tile receive identical tile-based treatment with one tiling.
Fix: Recognize the single-tiling representation as state aggregation.
Thinking coarse means useless.
The coarseness is the mechanism that turns continuous states into computationally efficient tile-based features.
Fix: Evaluate the trade-off between retained region information and omitted within-region detail.

Check Your Understanding

MEDIUM

A two-dimensional continuous state space is divided into a uniform grid. Two states have different exact values, but both fall inside the same tile. A third state crosses into a neighboring tile. Explain which states receive the same tile-based treatment under one tiling, what changes for the third state, and why this representation is called coarse.

Hints

Start by separating exact location from region membership.
Ask whether the first two states share a receptive-field region.
A boundary crossing changes the containing tile.

Your attempt (not saved)

Practice Answer

Explain the representation of the two states inside one tile and the third state inside a neighboring tile.

Compare the first two states: Although their exact continuous values differ, they share one receptive-field region, so one tiling gives them the same tile-based treatment.

Compare the third state: The third state lies in a different tile, so its region-membership feature differs from the first two.

Explain coarseness: The representation groups states by the tile they occupy and does not retain every detail inside each tile.

One tiling aggregates states within the same tile and distinguishes them from states in another tile.

Key Takeaways

Tile coding converts multi-dimensional continuous states into coarse, tile-based features.
A tiling is the complete partition of the input space; a tile is one receptive-field region within that partition.
A state is represented through the tile or region that contains it, not through every detail of its exact position inside the region.
With one tiling, states in the same tile receive identical tile-based treatment, which produces state aggregation.
Tile coding is useful because region-based features provide a flexible and computationally efficient representation for continuous spaces.

Key Takeaways

Tile coding represents continuous states through membership in regions of a partitioned input space.
The complete partition is a tiling; each receptive-field region in it is a tile.
One tiling groups all states inside the same tile together, so it provides state aggregation rather than preserving exact within-tile detail.
Tile coding is computationally useful because it turns continuous states into coarse, tile-based features.