Reinforcement Learning and Artificial Intelligence
Reinforcement learning is connected to both mathematical disciplines and the sciences of biological learning.
A Connected Field
Reinforcement learning is not an isolated branch of artificial intelligence. It has been shaped by neighboring disciplines, and it has also contributed ideas back to them. To understand these connections, ask two questions: which disciplines help reinforcement learning handle difficult learning problems, and which disciplines help explain the kinds of learning that reinforcement learning resembles?
The four disciplines most directly connected with reinforcement learning in this topic are statistics, optimization, psychology, and neuroscience. Statistics and optimization belong mainly to its mathematical integration. Psychology and neuroscience explain its biological and behavioral relationships.
The Mathematical Route
Statistics and optimization represent reinforcement learning's mathematical integration. This connection is part of a broader movement in artificial intelligence and machine learning toward stronger integration with mathematics. It is more than a change in terminology: reinforcement learning methods can use parameterized approximators.
Why Approximation Matters
Suppose a learning problem has a small number of possible states and actions, and then imagine that the number of possibilities grows dramatically.
Small problem: A direct table can represent the possibilities in a small problem.
Growing problem: As the number of states and actions grows, a direct table becomes impractical. This is the kind of difficulty associated with the curse of dimensionality.
Parameterized approach: A reinforcement learning method can use a parameterized approximator rather than depending on an impractically large table. This is why parameterized approximators matter in the relationship between reinforcement learning and the curse of dimensionality.
Parameterized approximators provide a way for reinforcement learning methods to address the curse of dimensionality identified in operations research and control theory.
The Biological Exchange
Psychology and neuroscience connect reinforcement learning with the sciences of biological learning. Their role is not limited to supplying metaphors. They help explain the kinds of learning reinforcement learning resembles, while reinforcement learning also contributes models back to these fields.
The relationship is two-way. Biology inspires reinforcement learning. In the opposite direction, reinforcement learning has contributed a psychological model of animal learning that better matches some empirical data, and it has contributed an influential model of parts of the brain's reward system.
A useful way to interpret this exchange is to separate inspiration from contribution. Psychology and neuroscience provide biological and behavioral relationships that shape reinforcement learning. Reinforcement learning then supplies models of animal learning and aspects of reward that can contribute to psychology and neuroscience.
Two Kinds of Connection
| Connection | Disciplines | Main relationship described in the source |
|---|---|---|
| Mathematical | Statistics and optimization | Reinforcement learning methods can use parameterized approximators to address the curse of dimensionality. |
| Biological and behavioral | Psychology and neuroscience | Biology inspires reinforcement learning, while reinforcement learning contributes models of animal learning and parts of the brain's reward system. |
Treating reinforcement learning as an isolated artificial intelligence topic.
The source describes reinforcement learning as shaped by several neighboring areas and as contributing ideas back to those areas.
Fix:
Study both the mathematical disciplines that help address difficult learning problems and the biological sciences that explain related forms of learning.Describing psychology and neuroscience as one-way sources of inspiration.
The relationship is explicitly two-way.
Fix:
Remember that reinforcement learning contributes a psychological model of animal learning and an influential model of parts of the brain's reward system.Mentioning parameterized approximators without connecting them to dimensionality.
Their importance in this topic comes from addressing the curse of dimensionality known from operations research and control theory.
Fix:
Connect parameterized approximators with the difficulty of representing a growing number of states and actions.
Check Your Understanding
Classify each statement as primarily mathematical or primarily biological and behavioral: statistics; optimization; models of animal learning; models of parts of the brain's reward system. Then explain why parameterized approximators matter to the mathematical connection.
Hints
- Statistics and optimization belong to the mathematical integration.
- Psychology and neuroscience explain the biological relationships.
- Relate parameterized approximators to the curse of dimensionality.
A Complete Classification
Organize the four disciplines and explain the direction of influence for psychology and neuroscience.
Group the mathematical disciplines: Place statistics and optimization in the mathematical integration of reinforcement learning.
Explain the approximation connection: Parameterized approximators matter because they address the curse of dimensionality associated with operations research and control theory.
Group the biological disciplines: Place psychology and neuroscience in the biological relationships of reinforcement learning.
Trace both directions: Biology inspires reinforcement learning, while reinforcement learning contributes models of animal learning and parts of the brain's reward system.
Reinforcement learning has a mathematical connection concerned with statistics, optimization, and dimensionality, as well as a biological and behavioral connection concerned with psychology, neuroscience, learning, and reward.
Key Takeaways
- Statistics and optimization form the main mathematical connections emphasized in this topic.
- Parameterized approximators matter because they address the curse of dimensionality in operations research and control theory.
- Psychology and neuroscience explain reinforcement learning's biological and behavioral relationships.
- The relationship with psychology and neuroscience is two-way: biology inspires reinforcement learning, and reinforcement learning contributes models of learning and reward.
- Reinforcement learning is best understood as part of a network of mathematical and biological disciplines, not as an isolated branch of artificial intelligence.
Key Takeaways
- The four disciplines most directly connected with reinforcement learning here are statistics, optimization, psychology, and neuroscience.
- Statistics and optimization represent mathematical integration, including the use of parameterized approximators to address the curse of dimensionality.
- Psychology and neuroscience provide biological and behavioral connections involving learning and reward.
- The psychology and neuroscience relationship is reciprocal: they influence reinforcement learning, and reinforcement learning contributes models back to them.