Mean Squared Value Error

An Immediate Correction

Imagine a value-prediction system receiving examples one after another. A stochastic gradient descent method does not wait until every example has been processed before changing its parameters. Instead, it makes a small adjustment to its weight vector immediately after each example. The immediate adjustment responds to the error associated with the example just considered.

Reading the Update

θt+1 = θt − α∇f(θt)

In this update, θt is the current weight vector and θt+1 is the vector after the update. The symbol α is a positive step-size parameter. The term ∇f(θt) is the gradient of the chosen scalar expression with respect to the components of the weight vector.

The gradient is a derivative vector rather than a single number. Each component describes how the error changes when the corresponding weight changes. Together, the components identify the direction of greatest increase in the error. The subtraction in the update is therefore essential: moving in the opposite direction points toward the most rapid decrease indicated by the gradient.

Tracing a Single Step

This example shows two separate roles in the update. The gradient determines the direction of the change, while α determines how far the weight vector moves in that direction. A larger positive step size would scale the same gradient more strongly; a smaller positive step size would produce a smaller adjustment.

From Local Steps to MSVE

A single stochastic update focuses on one example's error. It does not directly calculate an adjustment using every example at once. The connection to Mean Squared Value Error comes from repetition: the method processes many examples, making a small correction after each one. These local corrections can collectively reduce an average objective such as MSVE.

The local and broad interpretations should not be confused. Locally, each update tries to reduce the squared error for the example currently being considered. Broadly, repeated updates across examples can reduce the average error measured by MSVE. When the examples follow the same distribution as the states used for the MSVE objective, corrections based on observed examples can support that average-performance goal.

Mistakes About Stochastic Updates

• Thinking stochastic means that the weight vector changes only occasionally or unpredictably.

  In this context, stochastic refers to using one example for each update, and the adjustment happens immediately after that example.

  Fix: Treat each processed example as an opportunity for an immediate small weight-vector adjustment.

• Moving in the direction of the gradient when trying to reduce error.

  The gradient points toward the direction of greatest error increase.

  Fix: Use the negative gradient for a descent step, as in θt+1 = θt − α∇f(θt).

• Treating the gradient as a single number.

  The gradient is a derivative vector with one partial derivative for each component of the weight vector.

  Fix: Read each gradient component as information about the corresponding weight component.

• Assuming that one example directly minimizes MSVE.

  A single update focuses on one local example error, while MSVE is an average objective supported by repeated corrections across examples.

  Fix: Separate the local purpose of one update from the broader effect of many updates.

Check Your Reasoning

Key Takeaways

1. Stochastic gradient descent updates the weight vector immediately after each example rather than waiting for all examples.
2. The gradient is a vector describing how the error changes with respect to the weight components.
3. The gradient points toward the steepest error increase, so descent uses its negative.
4. The positive step-size parameter controls how large each update is.
5. Each update addresses one example locally, while many updates across suitable examples can reduce an average objective such as MSVE.