Concepts / Variance of a Random Variable

Variance of a Random Variable

Chebyshev's Inequality provides an upper bound on the probability of deviation from a random variable's mean.

  • Programming

Why Spread Matters

A random variable does not always equal its mean. To describe how widely its values can spread, we use variance. Chebyshev's Inequality then connects that spread to a probability statement: it gives an upper bound on the chance that the random variable is at least a chosen distance away from its mean.

Variance measures expected squared distance from the mean, while Chebyshev's Inequality uses that variance to bound the probability of a large deviation.

From Values to Variance

The construction of variance follows three steps. First, compare the random variable Z with its mean E[Z]. This produces the deviation Z − E[Z]. Next, square the deviation. Squaring makes the deviation nonnegative and gives more weight to larger distances. Finally, take the expected value of the squared deviation. The result is Var[Z], the variance of Z.

Var[Z] = E[(Z − E[Z])²]

subtract E[Z]squaretake expectationZpossible valueZ − E[Z]deviation from mean(Z − E[Z])²squared deviationVar[Z]expected squared deviation
How does each value's distance from the mean become the expected squared deviation?

Chebyshev's Bound

Chebyshev's Inequality states that the probability that Z differs from its mean by at least a chosen distance a is at most the variance divided by the squared distance: P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².

Read the inequality from left to right. The left side describes the deviation event: Z is at least a distance a from its mean. The right side is the bound that can be calculated from two inputs: the variance in the numerator and the squared deviation threshold in the denominator.

part of |Z − E[Z]| ≥ apart of |Z − E[Z]| ≥ aZ ≤ E[Z] − aE[Z] − a < Z < E[Z]+ aVar[Z] / a²upper boundZ ≥ E[Z] + a
How do the mean-centered interval and its tail regions correspond to the probability bound?

Applying the Calculation

A numerical Chebyshev bound

Suppose a random variable has variance 4. Find an upper bound for the probability that it differs from its mean by at least 2.

Identify the inputs: The variance is 4 and the deviation threshold is a = 2.

Substitute into Chebyshev's Inequality: Use P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a², giving P(|Z − E[Z]| ≥ 2) ≤ 4 / 2².

Evaluate the denominator: The squared threshold is 2² = 4.

Compute the bound: The right-hand side is 4 / 4 = 1.

The probability that Z is at least 2 units from its mean is bounded above by 1.

squarenumeratordenominatorupper-boundsVar[Z]variancea²squared thresholdVar[Z] / a²calculated boundP(|Z − E[Z]| ≥ a)bounded probabilityadeviation distance
Given a variance and a deviation distance, what calculation produces the upper bound?

Changing the Bound

Quantity that changesPosition in the boundEffect described by the formula
Variance Var[Z]NumeratorA larger variance produces a larger value of Var[Z] / a².
Deviation threshold aDenominator after squaringA larger threshold produces a larger a², making Var[Z] / a² smaller when the variance is fixed.

The formula makes the roles of spread and distance visible. Variance appears in the numerator, so increasing variance increases the calculated bound when the threshold stays fixed. The threshold is squared in the denominator, so increasing the allowed distance from the mean increases the denominator and decreases the bound when variance stays fixed.

EASY

A random variable has variance 9. Use Chebyshev's Inequality to find an upper bound for the probability that it differs from its mean by at least 3.

Hints
  • Use P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².
  • Substitute Var[Z] = 9 and a = 3.

Averages of i.i.d. Variables

The source also considers the average of m independent and identically distributed random variables. In this setting, the variance of the average decreases as m increases. When the common variance is written as σ², the variance of the average is σ² divided by m. Thus, increasing the number of variables makes the variance term used in a Chebyshev bound smaller.

Var[average of m i.i.d. random variables] = σ² / m
increase mincrease mcontinuem = 1variance σ²m = 2variance σ² / 2m = 3variance σ² / 3m variablesvariance σ² / m
As the number of i.i.d. random variables increases, how does the variance of their average change?

Comparing averages

Compare the variance of an average formed from 2 i.i.d. variables with the variance of an average formed from 8 i.i.d. variables, assuming the common variance is σ².

Use the average-variance formula: For m i.i.d. variables, the variance of the average is σ² / m.

Substitute m = 2: The first average has variance σ² / 2.

Substitute m = 8: The second average has variance σ² / 8.

Compare the results: The denominator is larger for 8 variables, so σ² / 8 is smaller than σ² / 2.

The average based on 8 i.i.d. variables has the smaller variance.

Common Calculation Mistakes

  • Using the deviation threshold without squaring it.

    Chebyshev's bound uses the squared deviation threshold in the denominator.

    Fix: Square a before dividing the variance by it.

  • Treating variance as an ordinary distance from the mean.

    Variance is the expected value of the squared deviation, not the unsquared deviation itself.

    Fix: Use Var[Z] = E[(Z − E[Z])²].

  • Reading the Chebyshev result as the exact deviation probability.

    The inequality provides an upper bound.

    Fix: Use the language at most or no greater than for the right-hand side.

  • Forgetting that the average's variance depends on the number of variables.

    For an average of m i.i.d. variables, the variance is σ² / m.

    Fix: Include m in the denominator.

Practice Check

What do you think happens?

A random variable has variance 16. Which expression is the Chebyshev upper bound for the probability that it differs from its mean by at least 4?

  • 16 / 4
  • 16 / 4²
  • 4 / 16²
  • 16 × 4²
Reveal answer

Answer: 16 / 4²

Chebyshev's Inequality places the variance in the numerator and the squared deviation threshold in the denominator.

MEDIUM

Explain in one or two sentences why increasing m makes the variance of the average σ² / m smaller.

Hints
  • Look at the position of m in the expression σ² / m.
  • Connect the smaller variance to the variance term in Chebyshev's bound.

Key Takeaways

  1. Variance is the expected squared distance from a random variable's mean: Var[Z] = E[(Z − E[Z])²].
  2. Chebyshev's Inequality states that P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².
  3. A larger variance produces a larger Chebyshev bound when the deviation threshold is fixed.
  4. A larger deviation threshold produces a smaller bound because the threshold is squared in the denominator.
  5. For an average of m i.i.d. random variables with common variance σ², the variance is σ² / m, so the variance decreases as m increases.

Key Takeaways

  • Variance measures expected squared distance from the mean.
  • Chebyshev's Inequality converts variance and a deviation threshold into an upper bound on deviation probability.
  • The bound is Var[Z] divided by the squared threshold.
  • The variance of an average of m i.i.d. random variables is σ² / m and decreases as m increases.

Related Topics

  • Markov's Inequality
  • Expected Value
  • Independent and Identically Distributed Random Variables