Linear Algebra Review

Vectors in this setting are column vectors in finite dimensional Euclidean spaces.

Programming

One Vector, Three Measurements

Later arguments in this chapter often take a vector as input and then measure or compare it. The vector itself does not change when you choose a different norm. What changes is the measurement rule. For the vector u = (2, −1, 3), the Euclidean norm combines squared coordinate magnitudes and takes a square root, the ℓ1 norm adds coordinate magnitudes, and the ℓ∞ norm keeps only the largest coordinate magnitude.

How do the Euclidean, ℓ1, and ℓ∞ norms differ in what aspect of a vector they measure?

Column Vectors in R^d

The setting for this chapter is finite-dimensional Euclidean space. A vector with d coordinates belongs to R^d. The symbol d records how many coordinates the vector has. Under the convention used here, those coordinates are arranged vertically, so the vector is a column vector rather than a row written across the page.

A vector in R^d is represented as one vertical object containing d coordinates.

u = [u₁; u₂; ...; u_d]

For example, the source vector u = (2, −1, 3) has three coordinates, so it belongs to R^3. Written with the chapter's column-vector convention, its coordinates are understood as being arranged vertically. The number of coordinates changes from one vector space to another, but the vertical convention remains the same.

From Inner Product to Euclidean Norm

The inner product connects two vectors. For vectors u and v in R^d, write their inner product as ⟨u, v⟩. When the same vector appears twice, ⟨u, u⟩ is the inner product of that vector with itself. The Euclidean norm, also called the ℓ2 norm, is obtained by taking the square root of this self-inner product.

||u||₂ = √⟨u, u⟩

How does the inner product of a vector with itself become its Euclidean norm?

Working with u = (2, −1, 3)

Three norm calculations

Calculate the Euclidean norm, ℓ1 norm, and ℓ∞ norm of u = (2, −1, 3).

Euclidean norm: The Euclidean norm combines the squared coordinate magnitudes and then takes a square root: ||u||₂ = √(2² + (−1)² + 3²) = √14.

ℓ1 norm: The ℓ1 norm totals the coordinate magnitudes: ||u||₁ = |2| + |−1| + |3| = 6.

ℓ∞ norm: The ℓ∞ norm keeps only the largest coordinate magnitude: ||u||∞ = max(|2|, |−1|, |3|) = 3.

For u = (2, −1, 3), the Euclidean norm is √14, the ℓ1 norm is 6, and the ℓ∞ norm is 3.

Norm	Operation	Feature measured	Value for u = (2, −1, 3)
Euclidean or ℓ2	Square coordinate magnitudes, add them, then take the square root	Combined magnitude of all coordinates	√14
ℓ1	Add coordinate magnitudes	Total coordinate magnitude	6
ℓ∞	Keep the largest coordinate magnitude	Largest individual coordinate magnitude	3

The same vector receives different measurements because each norm uses a different rule.

Common Calculation Mistakes

Treating the ℓ1 norm as a signed sum
The ℓ1 norm totals coordinate magnitudes, so negative coordinates are converted to their absolute values.
Fix: Compute |2| + |−1| + |3| = 6.
Using the largest signed coordinate for the ℓ∞ norm
The ℓ∞ norm keeps the largest coordinate magnitude, not the largest signed value.
Fix: Compare |2|, |−1|, and |3|, giving ||u||∞ = 3.
Stopping at the self-inner product when calculating the Euclidean norm
The inner product with itself supplies the quantity inside the square root.
Fix: After finding the self-inner product, take its square root.
Assuming the three norms describe three different vectors
The measurements can all be applied to the same vector. Only the measurement rule changes.
Fix: Keep the vector fixed and identify which rule each norm applies.

A Second Check

EASY

Let v = (−4, 0, 2). Calculate ||v||₁ and ||v||∞. Then describe, in words, what each result measures.

Hints

For the ℓ1 norm, convert every coordinate to its absolute value and add.
For the ℓ∞ norm, compare the absolute values and keep the largest.

Your attempt (not saved)

Practice check

For v = (−4, 0, 2), calculate ||v||₁ and ||v||∞.

ℓ1 norm: Add the coordinate magnitudes: |−4| + |0| + |2| = 6.

ℓ∞ norm: Keep the largest coordinate magnitude: max(|−4|, |0|, |2|) = 4.

The ℓ1 norm is 6 and the ℓ∞ norm is 4. The first measures total coordinate magnitude; the second measures the largest coordinate magnitude.

Key Takeaways

Vectors in this chapter are column vectors in finite-dimensional Euclidean spaces.
A vector with d coordinates belongs to R^d.
The Euclidean norm, or ℓ2 norm, is ||u||₂ = √⟨u, u⟩.
The ℓ1 norm totals coordinate magnitudes, while the ℓ∞ norm keeps the largest coordinate magnitude.
Different norms measure different features of the same vector; changing the norm does not change the vector.