Higher-Order Polynomial Bases
Higher-order bases offer greater approximation capability for complicated functions.
Why More Expressive Bases Matter
When the function being approximated is complicated, a simpler approximation may not be expressive enough to represent it accurately. Higher-order polynomial bases provide additional expressive capacity, so they can improve the approximation of complicated functions.
What do you think happens?
Suppose a current approximation cannot represent a complicated target function accurately. What is one reason to consider a higher-order polynomial basis?
Reveal answer
Answer: It can provide greater approximation capability
The value of a higher-order basis is its extra expressive capacity. However, using a higher order does not guarantee a smaller basis set or remove the need to manage the number of functions.
The important distinction is between capability and practicality. Increasing polynomial order can make the approximation more expressive. It does not mean that the resulting full basis is automatically manageable.
How the Full Basis Expands
For a polynomial basis of positive order N, the number of basis functions grows exponentially as the dimension of the state space increases. This makes basis size a central concern. The difficulty is therefore not merely that higher order adds a few extra functions; the full set can become very large when the state space has more dimensions.
Choosing Between Expressiveness and Manageability
A target function is complicated, so a higher-order polynomial basis may improve its approximation. The state space also has enough dimensions that using the complete higher-order basis may create a very large set of functions. What should the practitioner reason about?
Identify the approximation need: Because the target function is complicated, a more expressive approximation may be needed.
Recognize the size problem: For positive order N, the number of functions in the full basis grows exponentially as state-space dimension increases.
Avoid treating order as the only decision: Higher order may improve approximation capability, but the complete basis may be impractical.
Retain a subset: Use guidance about the target function or an adapted automated method to select a more manageable subset of basis functions.
The practical choice is not simply the highest possible order. It is an expressive basis whose number of retained functions is controlled through informed or automated selection.
Guidance for Basis Selection
When the full higher-order basis is too large to use comfortably, selecting a subset makes higher-order approximation more manageable. The source identifies two kinds of guidance for that selection: prior beliefs about the target function and adapted automated methods.
Prior beliefs provide human or domain-informed guidance about the target function. Adapted automated methods provide another route for deciding which functions to retain. Both approaches serve the same practical purpose described here: reducing the full higher-order set to a subset that is more manageable.
The Central Trade-Off
Higher order is valuable because it can improve the approximation of complicated functions. However, the number of functions in the full basis can become very large as state-space dimension increases. These are different concerns: one concerns how expressive the approximation can be, while the other concerns how many functions are retained.
| Design question | What it addresses | Source-grounded response |
|---|---|---|
| How can the approximation represent a complicated function? | Expressive capability | Consider a higher-order polynomial basis. |
| How can the basis remain manageable as dimensions increase? | Number of retained functions | Select a subset of basis functions. |
| How should the subset be selected? | Selection guidance | Use prior beliefs or an adapted automated method. |
Treat polynomial basis selection as a trade-off rather than a contest in which the highest order automatically wins. First recognize the need for expressive capacity, then consider how to control the number of retained functions.
Mistakes in Basis Decisions
Assuming that the highest order is automatically the best choice.
The full basis can become very large as state-space dimension increases.
Fix:
Balance the need for expressive capacity with the practical need to control the number of functions.Confusing higher approximation capability with a larger usable basis.
The source distinguishes the value of higher order from the size of the full basis.
Fix:
Consider selecting a subset of the higher-order basis.Ignoring state-space dimension when thinking about basis size.
For positive order N, basis count grows exponentially with state-space dimension.
Fix:
Include state-space dimension in the practicality assessment.Treating subset selection as arbitrary removal.
The source identifies prior beliefs and adapted automated methods as ways to guide selection.
Fix:
Use knowledge about the target function or an adapted automated selection method.
Check Your Reasoning
A complicated target function is being approximated in a state space whose dimension is increasing. Explain why a higher-order polynomial basis might help, why using its full set of functions might become impractical, and name two kinds of guidance that could support selecting a subset.
Hints
- Begin with the relationship between target-function complexity and approximation capability.
- Then connect positive polynomial order and increasing state-space dimension to basis count.
- Finish with the two selection-guidance sources identified in the article.
Practice Answer
Explain the design decision in the scenario above.
Approximation need: The target is complicated, so greater expressive capacity may be needed. A higher-order polynomial basis can provide that capacity.
Practical limitation: For positive order N, the number of functions in the full basis grows exponentially as state-space dimension increases.
Selection response: Rather than retaining the entire set automatically, select a subset using prior beliefs about the target function or an adapted automated method.
Higher order addresses approximation capability, while subset selection addresses the number of functions used.
Key Takeaways
- Higher-order polynomial bases can approximate complicated functions more accurately because they provide greater expressive capacity. For positive order N, the full basis grows exponentially as state-space dimension increases, making basis size a central concern. Selecting a subset makes higher-order approximation more manageable. Prior beliefs about the target function and adapted automated methods can guide that selection. The main design decision is a trade-off between approximation capability and the number of functions retained.
Key Takeaways
- Higher-order polynomial bases provide greater approximation capability for complicated functions.
- For positive order N, the full basis count grows exponentially as state-space dimension increases.
- A selected subset can make higher-order approximation more manageable.
- Prior beliefs and adapted automated methods can guide basis selection.
- Increasing expressive power and controlling basis size are separate but connected design goals.