Measure Concentration

One Vector, Several Measurements

Many later arguments take a vector as input and then measure or compare it. The vector itself does not change when we choose a measurement rule. What changes is the feature we ask the rule to emphasize: geometric length, the total magnitude across coordinates, or the single largest coordinate magnitude.

Column Coordinates in R^d

The setting is finite-dimensional Euclidean space. A vector with d coordinates belongs to R^d. In this chapter, vectors are written as column vectors, so their coordinates are arranged vertically rather than across a row. The symbol d records how many coordinates the vector has; it does not change the fact that the vector is represented as one vertical object.

Combining Two Vectors

For two vectors u and v in R^d, their inner product is written as ⟨u, v⟩. It combines corresponding coordinates: multiply the first coordinates, multiply the second coordinates, and continue through coordinate d. Adding those coordinatewise products produces one scalar. The inner product therefore connects two vectors rather than measuring only one vector by itself.

⟨u, v⟩ = u₁v₁ + u₂v₂ + ⋯ + u_dv_d

Euclidean Length from Self-Combination

When the two inputs to the inner product are the same vector, the result contains the squared coordinate magnitudes. Taking the square root converts that quantity into the Euclidean norm, also called the ℓ2 norm. Thus, the inner product supplies the quantity inside the square root.

∥u∥₂ = √⟨u, u⟩

Three Ways to Measure u

The ℓ2, ℓ1, and ℓ∞ norms use different rules on the same coordinates. The ℓ2 norm reflects overall geometric length by combining squared coordinate magnitudes and taking a square root. The ℓ1 norm totals the magnitudes of all coordinates. The ℓ∞ norm keeps only the largest coordinate magnitude.

∥u∥₁ = |u₁| + |u₂| + ⋯ + |u_d|

∥u∥∞ = max{|u₁|, |u₂|, …, |u_d|}

A Reliable Calculation Order

1. First identify the requested measurement: inner product, ℓ2 norm, ℓ1 norm, or ℓ∞ norm.
2. For an inner product, pair corresponding coordinates, multiply each pair, and add the products.
3. For the ℓ2 norm, use the vector with itself in the inner product and then take the square root.
4. For the ℓ1 norm, take the absolute value of every coordinate and add all those magnitudes.
5. For the ℓ∞ norm, take the absolute value of every coordinate and select the largest magnitude.
6. Before finalizing the result, check whether the rule uses all coordinates, squared magnitudes, or only one largest magnitude.

• Using the largest coordinate itself for the ℓ∞ norm.

  The ℓ∞ norm uses the largest coordinate magnitude, not the coordinate with a particular sign.

  Fix: Compare 2, 1, and 3 after taking absolute values. The result is 3.

• Adding signed coordinates for the ℓ1 norm.

  The ℓ1 norm totals coordinate magnitudes.

  Fix: Compute |2| + |−1| + |3| = 6.

• Stopping at the self-inner-product when the question asks for the Euclidean norm.

  The self-inner-product supplies the quantity inside the square root.

  Fix: Take the square root after calculating ⟨u, u⟩, giving ∥u∥₂ = √14.

• Treating the three norms as measurements of three different vectors.

  The vector remains the same; only the measurement rule changes.

  Fix: Keep the coordinates fixed and apply the requested norm definition.

Practice Check

Key Takeaways

1. Vectors in this chapter are column vectors in finite-dimensional Euclidean spaces, and a vector with d coordinates belongs to R^d.
2. The inner product ⟨u, v⟩ combines corresponding coordinate products from two vectors into one scalar.
3. The Euclidean norm is obtained by taking the square root of a vector's inner product with itself: ∥u∥₂ = √⟨u, u⟩.
4. The ℓ1 norm adds all coordinate magnitudes, while the ℓ∞ norm keeps only the largest coordinate magnitude.
5. The same vector can receive different measurements because each norm emphasizes a different feature.