Feature Construction
RBF networks are function approximators whose features are radial basis functions.
From State to Features
Feature construction asks how an input state is turned into information that a function approximator can use. An RBF network makes a specific choice: its features are radial basis functions. Each feature describes the input in relation to a center, varying according to the input's distance from that center. The RBF network then uses those features to approximate a target function.
A State Through an RBF Representation
A Nearby State and Three Centers
Consider an input state and three radial basis functions, each associated with a different center.
Compare distances: The input state is compared with each center. One center may be relatively near the state, while the others may be farther away.
Construct feature responses: Each radial basis function produces a feature value based on the input's distance from its center. The feature responses therefore describe the same state from several center-based perspectives.
Approximate the target: The RBF network uses the constructed feature representation in a function approximator.
The original state has been converted into a collection of radial basis function features. Changing the state changes its distances from the centers and therefore changes the feature representation.
The important idea is not merely that an RBF network has several features. It is that the features are organized around centers and respond to distance from those centers. A state near one center can receive a different pattern of feature responses from a state that is farther away or closer to another center.
What do you think happens?
Suppose the input state moves closer to one center while the other centers remain where they are. What should happen to the RBF feature representation?
Reveal answer
Answer: The feature values can change because the distances to the centers have changed.
Fixed centers do not mean fixed feature responses. They mean that the centers remain unchanged while different input states can produce different responses according to their distances from those centers.
Why Fixed Features Are Linear
An RBF network with fixed features is a linear RBF network. The radial basis functions define the representation of the input, but that representation does not change during learning. Learning is therefore applied to the linear function approximator built on top of those fixed features. The network can produce different approximations by changing the parameters of that linear approximator without moving the feature centers or changing their widths.
The word linear refers to what learning changes in the approximator. It does not mean that the radial basis features are absent or that the input-to-feature relationship is a simple straight line. The defining condition is that the feature representation stays fixed while the linear approximator is learned.
When RBF Learning Becomes Nonlinear
| Feature behavior | Linear RBF network | Nonlinear RBF network |
|---|---|---|
| RBF feature centers | Fixed during learning | Can be changed during learning |
| RBF feature widths | Fixed during learning | Can be changed during learning |
| What learning changes | The linear approximator built on the features | The approximator and the feature representation |
| Potential fitting ability | Uses the given representation | May fit target functions more precisely |
| Computational and tuning burden | Lower than when centers and widths are also adapted | Greater computational complexity and often more manual tuning |
Adapting centers and widths can help the network fit target functions more precisely because the representation itself can be adjusted. That potential precision has a cost: nonlinear RBF methods require more computational complexity and can require more manual tuning before learning becomes robust and efficient.
RBF Methods and Tile Coding
| Consideration | RBF network methods | Tile coding |
|---|---|---|
| Representation | Uses radial basis functions as features | Uses tile-code regions |
| Feature evaluation | Requires evaluating radial basis function responses | Uses the tile-code representation |
| Computational demand | Has additional computational complexity compared with tile coding | Provides the comparison baseline described by the source |
| Adaptation | Nonlinear versions can change centers and widths | The source does not describe center and width adaptation for tile coding |
| Tuning | Nonlinear RBF networks can require more manual tuning | The source does not specify an equivalent tuning burden |
RBF networks and tile coding both construct a representation before function approximation, but they do so differently. RBF methods describe an input through radial basis function responses. Tile coding describes it through tile-code regions. According to the source, RBF network methods carry additional computational complexity compared with tile coding, and nonlinear RBF networks add further tuning effort because their centers and widths are learned.
The High-Dimensionality Problem
Higher-dimensional state spaces make both RBF feature construction and tile coding more difficult to make effective and control. As the number of state dimensions increases, there are more dimensions in which centers or tile-code regions must provide useful coverage. The representation can therefore become harder to place and control across the state space.
When a state representation has many dimensions, examine not only the approximator but also the feature construction. Ask whether the centers or regions cover the state space in a useful and controllable way, and remember that an adaptable RBF representation may require additional tuning and computation.
Check Your Classification
A method uses radial basis functions as its features. During learning, the feature centers and widths remain fixed, while the linear approximator is learned. Classify the method and explain why.
Hints
- First identify what kind of features the method uses.
- Then ask whether learning changes the feature representation.
- Use the distinction between fixed features and learned centers or widths.
Classifying Two RBF Methods
Classify two methods: Method A keeps centers and widths fixed and learns only the linear approximator. Method B learns the linear approximator while also changing centers and widths.
Classify Method A: Its RBF features remain fixed, so it is a linear RBF network.
Classify Method B: Its feature representation changes during learning because centers and widths are adapted, so it is a nonlinear RBF network.
Compare the trade-off: Method B may fit target functions more precisely, but it brings greater computational complexity and often more manual tuning.
Method A is linear because only the approximator is learned. Method B is nonlinear because learning also changes the RBF feature representation.
Calling every RBF network nonlinear.
The source distinguishes linear and nonlinear RBF networks by what learning changes, not simply by the presence of radial basis functions.
Fix:
Classify the network as linear when the features are fixed and the learned part is the linear approximator.Assuming fixed centers mean fixed feature values for every input.
Different input states can have different distances from the fixed centers.
Fix:
Separate fixed feature parameters from feature responses: fixed centers and widths can still produce different responses for different states.Describing the extra precision of nonlinear RBF networks without mentioning its cost.
The potential fitting improvement is accompanied by greater computational complexity and often more manual tuning.
Fix:
State both sides of the trade-off: potentially more precise fitting, but more computation and tuning effort.Ignoring state-space dimension when discussing feature construction.
Higher-dimensional state spaces make centers and tile-code regions harder to place, cover, and control.
Fix:
Include the dimensionality of the state space when assessing whether a representation will be effective and manageable.
Feature Construction Summary
- An RBF network is a function approximator whose features are radial basis functions.
- Fixed RBF features produce a linear RBF network because learning changes the linear approximator while the representation remains fixed.
- Learning the RBF centers and widths changes the representation itself, making the network a nonlinear function approximator.
- Nonlinear RBF networks may fit target functions more precisely, but they require more computational complexity and often more manual tuning than simpler alternatives such as tile coding.
- Higher-dimensional state spaces make the placement, coverage, and control of RBF centers and tile-code regions more difficult.
Key Takeaways
- RBF networks construct features with radial basis functions that respond to an input's distance from feature centers.
- The fixed-versus-learned status of the features determines whether the RBF network is linear or nonlinear.
- Adapting centers and widths can improve fitting precision but increases computation and tuning effort.
- RBF methods involve additional computational complexity compared with tile coding.
- Increasing state-space dimensionality makes feature placement, coverage, and control more difficult.