Neuronal Plasticity

When Learning Signals Arrive Late

A neuron may participate in an action before the consequence of that action becomes available as a learning signal. An eligibility trace solves this timing problem by preserving synapse-specific information about earlier activity. When a later reinforcing neuromodulatory signal arrives, it can use that record to modify the efficacy of a synapse that is still eligible.

A Synapse-Specific Activity Record

An eligibility trace records that a particular synapse participated in earlier activity involving the neuron. The trace belongs to the synapse, not merely to the neuron as a whole. That distinction matters because different synapses may have participated in different activity. A later reinforcing signal can therefore modify the efficacy of an eligible synapse rather than indiscriminately changing every synapse on the neuron.

From Eligibility to Synaptic Change

The sequence has two separated moments. First, activity involving a synapse is recorded as eligibility. Later, a reinforcing neuromodulatory input arrives. If the synapse is eligible at that later moment, the signal can modify the synapse's efficacy. The trace therefore acts as a bridge between the earlier activity and the later learning signal.

Trace Dynamics Across the Waiting Period

Think of the trace as carrying information across a delay. Activity involving a synapse produces an eligibility record. The system may then wait for the consequence associated with the neuron's activity. If the reinforcing neuromodulatory signal arrives while the synapse remains eligible, the record can be used to modify synaptic efficacy. The source describes the role of the trace in preserving earlier activity and connecting it to a later signal; it does not specify a particular numerical decay rule or time constant.

Actor and Critic Traces

Actor and critic learning rules both use eligibility traces, but their traces carry different information. A contingent actor trace depends on the actor unit's input and its output. A non-contingent critic trace does not include the critic unit's output. The difference concerns what information is included in the trace, not whether the synapse has an activity record.

Klopf and Corticostriatal Plasticity

Klopf's hedonistic-neuron hypothesis provides an intuition for why an eligibility trace is useful. In this view, a neuron participates in a closed loop: its activity can influence later input through feedback within the nervous system and body or through the external environment. The neuron adjusts synaptic efficacies according to rewarding or punishing consequences associated with its action potentials. A synapse becomes eligible when it participated in the neuron's firing, and a reinforcing signal selects that eligible synapse for modification.

The neural interpretation becomes more specific in corticostriatal synapses. These synapses carry signals from the cortex to principal neurons in dorsal and ventral subdivisions of the striatum, and they also receive signals from dopamine neurons. Eligibility traces provide the bridge between earlier activity at a corticostriatal synapse and later neuromodulatory input relevant to changing its efficacy.

Common Reasoning Errors

• Treating eligibility as an immediate synaptic change.

  Eligibility is the recorded condition that allows a later reinforcing signal to modify the synapse.

  Fix: Separate the earlier eligibility record from the later modification of synaptic efficacy.

• Treating the eligibility trace as a property of the whole neuron.

  The trace preserves synapse-specific information about earlier activity.

  Fix: Ask which particular synapse participated and became eligible.

• Calling every actor and critic trace contingent.

  Actor traces include actor output, while critic traces do not include critic output.

  Fix: Check whether the unit's output is included: actor traces are contingent, critic traces are non-contingent.

• Assuming the reinforcing signal must arrive at exactly the same moment as synaptic activity.

  The purpose of the eligibility trace is to connect earlier synaptic activity with a later reinforcing neuromodulatory signal.

  Fix: Look for the eligibility record that preserves the earlier activity during the delay.

Check Your Understanding

Key Takeaways

1. An eligibility trace preserves synapse-specific information about earlier activity.
2. A later reinforcing neuromodulatory signal can modify the efficacy of a synapse that is eligible.
3. Actor traces are contingent because they depend on actor input and actor output.
4. Critic traces are non-contingent because they do not include critic output.
5. The mechanism connects actor-critic learning rules with Klopf's hedonistic-neuron hypothesis and corticostriatal synaptic plasticity.