Stimulus Representation
The TD model places time and changes in prediction at the center of its account of classical conditioning.
Following Prediction Through Time
Classical conditioning is often described by listing which events occur together. The TD model asks a more time-sensitive question: how does predictive information change from one moment to the next during a conditioning episode? This shift places time and changes in prediction at the center of the model.
A temporal difference is a change in predictive information between neighboring moments of an episode, not merely a label for the final outcome.
To understand stimulus representation, follow one episode from beginning to end. At the beginning, the learner has one set of expectations. As the stimulus unfolds and later events become available, the learner has new information. The TD perspective compares these successive points rather than treating the entire episode as one undifferentiated event.
A Conditioning Episode Unfolds
The diagram shows the central movement of the TD account. The learner does not treat the episode as a single indivisible event. Instead, the episode supplies information in sequence. Predictions at one moment can be compared with predictions at a later moment, and the change between them organizes learning.
Reading One Episode as Successive Moments
Suppose a conditioning episode has a beginning, an unfolding stimulus, and a later outcome. How would a TD-style explanation describe the episode?
Beginning: The learner starts with an initial set of expectations before all information in the episode is available.
Stimulus unfolds: As the episode continues, later moments provide new predictive information. The learner's current prediction can therefore differ from the earlier prediction.
Outcome arrives: The later event supplies information that can be compared with the prediction carried from the preceding moment.
Learning is organized: The relevant learning signal is the time-sensitive change in predictive information across these moments, rather than only the fact that the stimulus and outcome occurred in the same episode.
The TD model explains the episode through successive changes in prediction across time.
This account distinguishes the TD model from a description that records only whether a stimulus and an outcome appeared together. Timing matters because information becomes available at different points in the episode.
What the Representation Encodes
A temporal learning rule still needs a way to represent the stimulus across the episode. The representation determines which distinctions across time are available to the model. If the representation preserves useful differences between moments, the model has a richer basis for comparing successive predictive states.
Microstimulus representation provides a way to preserve the temporal structure of an extended stimulus by representing it as a sequence of time-specific microstimuli. In this conceptual view, different parts of the episode can provide distinct information for prediction. The representation is a modeling choice; the source does not claim that every microstimulus has one fixed physical form.
Representation Changes the Prediction
A coarse representation may treat the episode as one undifferentiated event. A time-sensitive representation preserves distinctions among moments. Because the TD model compares predictive information across time, these two representations can give the model different opportunities to organize learning and can lead to different predictions about conditioning.
A model can have a temporal learning rule and still fail to capture useful timing if its stimulus representation does not preserve the relevant distinctions across the episode. This is why evaluating the TD model requires attention to representation as well as to the learning rule.
The Historical Learning-Rule Family
The TD model emerged through several stages rather than appearing as a finished theory in one publication. Its history includes a close relationship between the Rescorla-Wagner model and the Least-Mean-Square learning rule, also called the LMS or Widrow-Hoff rule.
| Element | Role in the history |
|---|---|
| LMS rule | Introduced by Widrow and Hoff in 1960; it provides part of the established error-based learning-rule tradition. |
| Rescorla-Wagner model | Sutton and Barto's 1981 work recognized a near identity between this model and the LMS rule. |
| TD model | Developed gradually through an early version in 1981, later TD algorithms in 1984 and 1988, and presentations in 1987 and 1990. |
Historical relationships described in the source
This history places the TD model alongside an established error-based learning-rule tradition rather than presenting it as an unrelated account of conditioning. The distinctive emphasis of the TD perspective is its focus on changes in prediction across successive moments.
Testing the Model Beyond the Rule
The model's learning rule is only part of the evaluation. Later work examined different stimulus representations and their possible neural implementations in relation to response timing and response topography. The microstimulus representation therefore became important not because it replaced the TD idea of temporal prediction, but because it offered a way to examine how stimulus information might be represented across an episode.
When researchers evaluate a temporal model, they must ask both how the model updates learning and how the stimulus is represented while those updates occur.
Prediction-Error Information Flow
The flow should be read as a conceptual sequence. Stimulus information is represented at a particular point in the episode. That information supports a prediction. A later point supplies new information, allowing the model to organize learning around the change between predictions. The resulting update affects how later predictive information is understood.
What do you think happens?
If an extended stimulus is represented as one undifferentiated event instead of as time-specific parts, what is most likely to change?
Reveal answer
Answer: Which distinctions across time are available for prediction and learning
The source emphasizes that stimulus representation affects how the TD model performs. A representation that preserves useful distinctions across time gives the model a richer basis for comparing successive moments.
Mistakes About Stimulus Representation
Treating classical conditioning as only a question of co-occurrence
The TD model focuses on when information becomes available and how predictions change across time.
Fix:
Analyze the episode as a sequence of moments with changing predictive information.Assuming the TD learning rule determines all model behavior by itself
The source states that the way a stimulus is represented can affect how the TD model performs.
Fix:
Evaluate the learning rule together with the representation used to encode stimulus information.Treating microstimuli as fixed physical objects
The source presents microstimulus representation as a modeling representation and does not claim that every microstimulus has one fixed physical form.
Fix:
Describe microstimuli as time-specific representational units without assigning them an unsupported fixed physical identity.Describing the TD model as a theory that appeared fully formed in one publication
The model developed through several stages, including work in 1981, 1984, 1987, 1988, and 1990.
Fix:
Describe the TD model as a gradual development within a broader history of learning rules.
Apply the Representation Test
A researcher compares two TD-model implementations. Implementation A represents an entire conditioning episode as one undifferentiated event. Implementation B represents the episode through time-specific microstimuli. Explain why the two implementations may produce different conditioning predictions, even if they use the same general TD learning idea.
Hints
- Start by identifying what information each representation preserves across time.
- Then connect that difference to the TD model's comparison of predictive information at successive moments.
- Finally, explain why this makes representation part of the evaluation rather than a minor implementation detail.
Answering the Representation Test
Why may two implementations using the same TD learning idea produce different predictions?
Compare what is encoded: The first implementation preserves little distinction among moments because it treats the episode as one event. The second preserves distinctions by using time-specific microstimuli.
Connect encoding to prediction: The TD model organizes learning around changes in predictive information. The second representation gives the model more time-specific information to compare.
State the consequence: Because the available temporal distinctions differ, the two implementations can organize learning differently and produce different conditioning predictions.
The representation matters because it determines which temporal distinctions the TD model can use when comparing successive predictions.
Key Takeaways
- The TD model explains classical conditioning through changes in prediction across successive moments of an episode.
- Its historical development belongs to an error-based learning-rule tradition connected to the Rescorla-Wagner model and the LMS or Widrow-Hoff rule.
- Stimulus representation matters because it determines which distinctions across time are available to the model.
- Microstimulus representation offers a way to represent an extended stimulus as time-specific parts while evaluating the TD model.
- A complete evaluation must consider both the temporal learning rule and the representation used to encode stimulus information.
Key Takeaways
- The TD model treats classical conditioning as a process of changing predictive information over time.
- The model developed gradually within a history that links the Rescorla-Wagner model and the LMS learning rule.
- Stimulus representation affects what temporal distinctions the TD model can use for learning and prediction.
- Microstimulus representation became important in later evaluations because it provides a time-sensitive way to encode an extended stimulus.
- Understanding TD performance requires examining both its learning rule and its stimulus representation.