Models of Classical Conditioning

Why Conditioning Needs a Model

Classical conditioning is not only a matter of recording which events occur together. A useful model must also explain how learning changes from one trial to the next. Prior conditioning can alter what an animal learns later, and the time at which information becomes available can affect how predictions change during a conditioning episode.

Reinforcement in Historical Context

The word reinforcement has not always had one fixed meaning in animal learning. According to the source, the term first appeared, to the best of its knowledge, in the 1927 English translation of Pavlov's monograph. In that use, reinforcement referred to a non-action-contingent case: the event was not described as depending on an action.

Later traditions used the term differently. Mackintosh proposed using reinforcement for either strengthening or weakening a pattern of behavior. Skinner used reinforcement only for strengthening behavior and treated weakening as something produced by punishment. These differences are historically important because the same word can point to different ideas depending on the theoretical tradition.

Kamin Blocking Across Two Stages

Kamin blocking shows that prior conditioning can affect subsequent learning. The situation has two stages. First, one stimulus is repeatedly involved in conditioning. Later, that previously conditioned stimulus appears together with a new stimulus. The question is whether the new stimulus will also become a predictor of the outcome. In blocking, prior conditioning prevents or greatly reduces the expected new learning about the added stimulus.

Surprise in Rescorla-Wagner

This idea gives blocking a learning-based explanation. Before the compound stage, Cue A has already acquired predictive significance. When Cue A and Cue B appear together, the outcome is not as unexpected as it would have been for a learner with no prior conditioning. Since the outcome is already prepared for, the new Cue B may contribute little or no additional learning.

Alternative Conditioning Models

The Rescorla-Wagner model is one model among several used to study classical conditioning. The source lists models associated with Klopf, Grossberg, Mackintosh, Moore and Stickney, Pearce and Hall, and Courville, Daw, and Touretzky. This broader list matters because it places Rescorla and Wagner within a history of animal learning theory rather than treating their model as the only available explanation.

Temporal Differences in a Conditioning Episode

Imagine following one conditioning episode from its beginning to its end. At the beginning, the learner has one set of expectations. As the episode unfolds, later events provide new information. The TD perspective compares these successive points rather than treating the whole episode as one undivided event. Learning is therefore connected to changes in predictive information over time.

Three Learning Rules in One History

The TD model developed within a larger history of learning rules. Sutton and Barto's 1981 work recognized a near identity between the Rescorla-Wagner model and the Least-Mean-Square learning rule, also called the LMS or Widrow-Hoff rule. Widrow and Hoff introduced the LMS rule in 1960. This connection places the TD model alongside an established error-based learning-rule tradition rather than presenting it as an unrelated account of conditioning.

The development was gradual. The early version of the TD model made a specific prediction: temporal primacy overrides blocking. The source reports that Kehoe, Scheurs, and Graham later showed this effect in the rabbit nictitating membrane preparation. This is an example of a model being evaluated against an experimental result, not merely a restatement that timing exists.

Why Representation Changes the Evaluation

The TD model does not depend only on its learning rule. The way a stimulus is represented can also affect how the model performs. Later work examined different stimulus representations and their possible neural implementations in relation to response timing and response topography.

A temporal model needs a way to encode stimulus information across the episode. If the representation preserves useful distinctions between early and later moments, the model has a richer basis for comparing successive predictions. If those distinctions are not represented, the model has less information with which to organize time-sensitive changes.

Ludvig, Sutton, and Kehoe introduced the microstimulus representation in 2008. In 2012, they evaluated the TD model on previously unexplored classical-conditioning tasks and examined the influence of several stimulus representations, including the microstimulus representation. The important lesson is that evaluating a temporal learning rule also requires asking how the stimulus is represented.

Common Reasoning Errors

• Treating conditioning trials as independent events

  Kamin blocking specifically depends on prior conditioning changing the conditions for later learning.

  Fix: Trace the learning history before interpreting what happens when the new stimulus is introduced.

• Defining learning as simple stimulus-outcome co-occurrence

  The Rescorla-Wagner account emphasizes whether the outcome is surprising given what is already predicted.

  Fix: Ask what the learner already expects before deciding how much new learning remains.

• Treating the TD model as a model of only the final outcome

  The TD model focuses on changes in predictive information across different moments.

  Fix: Compare neighboring moments and ask how the prediction changes as new information becomes available.

• Assuming the TD learning rule is sufficient without specifying a representation

  The source states that stimulus representation, including microstimulus representation, affects how the TD model performs.

  Fix: Treat representation as part of the evaluation of the model.

• Assuming reinforcement has one universal historical meaning

  The source describes meaningful differences among these traditions.

  Fix: Identify the historical tradition and its definition before comparing claims.

• Inventing mechanisms for every alternative model named in the source

  The source lists these alternatives but does not provide their detailed mechanisms here.

  Fix: Use the supplied material only to identify them as part of the broader history of conditioning models.

Practice the Model Trace

Key Takeaways

1. The meaning of reinforcement changed across historical traditions in animal learning, so the term must be interpreted in context.
2. Kamin blocking demonstrates that prior conditioning can affect later learning about a newly added stimulus.
3. The Rescorla-Wagner model links learning with surprise: when an outcome is already expected, additional learning may be small.
4. The TD model explains conditioning through changes in predictive information across time and developed within a history connected to the Rescorla-Wagner model and the LMS learning rule.
5. Stimulus representation matters for TD evaluations, and microstimulus representation provides a way to examine stimulus information through time-specific components.