Abstract Linear Algebra 20 | Gram-Schmidt Orthonormalization

TL;DR

Gram–Schmidt orthonormalization converts a basis {u1,…,uk} of a finite-dimensional inner-product subspace U into an orthonormal basis {b1,…,bk}.

Briefing Cornell Notes

Briefing

Gram–Schmidt orthonormalization turns any basis of a finite-dimensional inner-product subspace into an orthonormal basis that fits the geometry of the inner product—making tasks like orthogonal projections far easier. Starting with a k-dimensional subspace U and an arbitrary basis {u1, u2, …, uk}, the process produces an orthonormal basis {b1, b2, …, bk} where each bi has length 1 and different bj are orthogonal (their inner products are 0). This “geometry-respecting” basis is especially valuable because orthogonal projection formulas become simple once an orthonormal basis is available.

The method works in two repeated phases for each new vector: first enforce orthogonality to all previously constructed basis vectors, then normalize. For the first vector, b1 is just u1 scaled to unit length. The scaling factor uses the norm induced by the inner product: ||u1|| = sqrt(⟨u1, u1⟩), so b1 = u1 / ||u1||. The key point in an abstract inner-product space is that the norm must come from the inner product—one cannot assume the standard Euclidean norm from R^n.

To build b2, the process removes from u2 its component along b1. It computes the orthogonal projection of u2 onto span{b1}, which is straightforward because span{b1} is one-dimensional and b1 is already orthonormal: proj_{b1}(u2) = ⟨u2, b1⟩ b1. The vector orthogonal to b1 is then the “normal component” b2~ = u2 − proj_{b1}(u2). Finally, b2 is obtained by normalizing b2~: b2 = b2~ / ||b2~||.

The same pattern continues. When constructing b3, the previously built vectors b1 and b2 span a two-dimensional subspace, so the orthogonal projection of u3 onto that subspace is the sum of its projections onto each orthonormal direction: proj_{span{b1,b2}}(u3) = ⟨u3, b1⟩ b1 + ⟨u3, b2⟩ b2. Subtracting this from u3 yields the orthogonal component b3~ = u3 − proj_{span{b1,b2}}(u3), and normalizing gives b3. Inductively, for the final step, UK is projected onto span{b1, …, b_{K−1}} by summing ⟨UK, bj⟩ bj for j = 1 to K−1, then subtracting to get the orthogonal component and normalizing to produce BK.

By the end of k steps, the procedure outputs a full orthonormal basis for U. The repeated structure—projection onto the span of earlier orthonormal vectors, subtraction to get the orthogonal component, then normalization—makes the algorithm systematic and well-suited for computation, with the added benefit that orthogonal projection calculations become especially clean once the orthonormal basis is in hand.

Cornell Notes

Gram–Schmidt orthonormalization converts an arbitrary basis {u1,…,uk} of a k-dimensional inner-product subspace U into an orthonormal basis {b1,…,bk}. It builds vectors sequentially: each new bi is obtained by subtracting from ui its orthogonal projection onto the span of the previously constructed orthonormal vectors, then normalizing the result to unit length. The norm used in normalization must be the one induced by the inner product, namely ||v|| = sqrt(⟨v,v⟩). Once an orthonormal set {b1,…,b_{i−1}} is available, projecting ui onto their span becomes a simple sum of one-dimensional projections ⟨ui,bj⟩ bj. The payoff is a basis tailored to the inner product, which simplifies orthogonal projections and related calculations.

How does Gram–Schmidt produce b1 from u1, and what norm is used in an abstract inner-product space?

b1 is obtained by scaling u1 to unit length: b1 = u1 / ||u1||. The norm comes from the inner product, so ||u1|| = sqrt(⟨u1,u1⟩). In an abstract inner-product space, this induced norm is the only correct choice; it’s not safe to use a “standard” Euclidean norm unless the inner product is the usual one.

What is the formula for the orthogonal component used to build b2, and why does it guarantee orthogonality to b1?

First compute the orthogonal projection of u2 onto span{b1}: proj_{b1}(u2) = ⟨u2,b1⟩ b1 (since b1 is already orthonormal). Then form the orthogonal component b2~ = u2 − proj_{b1}(u2). By construction, b2~ has zero inner product with b1, so it is orthogonal to b1. Normalizing gives b2 = b2~ / ||b2~||.

How is the orthogonal projection of u3 onto span{b1,b2} computed once b1 and b2 are orthonormal?

With an orthonormal basis {b1,b2}, projection onto their span is the sum of one-dimensional projections: proj_{span{b1,b2}}(u3) = ⟨u3,b1⟩ b1 + ⟨u3,b2⟩ b2. The orthogonal component is then b3~ = u3 − proj_{span{b1,b2}}(u3), and normalization yields b3.

What is the general projection-and-subtraction step for constructing bK from UK?

Assume {b1,…,b_{K−1}} is already orthonormal. Project UK onto span{b1,…,b_{K−1}} by summing one-dimensional projections: proj_{span} (UK) = Σ_{j=1}^{K−1} ⟨UK,bj⟩ bj. Subtract to get the orthogonal component: bK~ = UK − proj_{span}(UK). Finally normalize: bK = bK~ / ||bK~||.

Why does having an orthonormal basis make orthogonal projection easier?

Once the basis vectors are orthonormal, projecting onto their span reduces to a clean sum of scalar multiples: proj_{span{b1,…,bm}}(v) = Σ_{j=1}^{m} ⟨v,bj⟩ bj. Without orthonormality, projection formulas require more complicated computations (e.g., involving Gram matrices).

Review Questions

Given an inner product ⟨·,·⟩, what is the correct expression for the norm used to normalize u1 in Gram–Schmidt?
In Gram–Schmidt, how do you compute the orthogonal projection of UK onto span{b1,…,b_{K−1}} when {b1,…,b_{K−1}} is orthonormal?
Why does subtracting the orthogonal projection of ui onto the span of earlier bj produce a vector orthogonal to every earlier bj?

Key Points

1
Gram–Schmidt orthonormalization converts a basis {u1,…,uk} of a finite-dimensional inner-product subspace U into an orthonormal basis {b1,…,bk}.
2
Normalization must use the norm induced by the inner product: ||v|| = sqrt(⟨v,v⟩).
3
Each new basis vector bi is built by subtracting from ui its orthogonal projection onto span{b1,…,b_{i−1}}.
4
When {b1,…,b_{i−1}} is orthonormal, the projection onto their span is the sum Σ_{j=1}^{i−1} ⟨ui,bj⟩ bj.
5
After forming the orthogonal component bi~ = ui − proj, the algorithm normalizes it to get bi = bi~/||bi~||.
6
The resulting orthonormal basis makes orthogonal projection computations straightforward and consistent with the geometry of the inner product.

Highlights

The induced norm in abstract inner-product spaces is ||v|| = sqrt(⟨v,v⟩), and it must be used for normalization.

Orthogonal projection onto an orthonormal span becomes a simple sum: proj_{span{b1,…,bm}}(v) = Σ ⟨v,bj⟩ bj.

Each step follows the same template: project, subtract the projected component, then normalize.

The algorithm guarantees orthogonality at every stage by construction, not by guesswork.

Topics

Gram–Schmidt
Orthonormal Bases
Orthogonal Projection
Inner Product Spaces
Linear Algebra Algorithms