Hilbert Spaces 12 | Bessel's Inequality

TL;DR

For any inner product space and orthonormal system {eα} indexed by I, every vector x satisfies ∑_{α∈I} |⟨eα,x⟩|² ≤ ||x||².

Briefing Cornell Notes

Briefing

Bessel’s inequality links the “energy” of any vector to its coordinates along an orthonormal system, guaranteeing that the total squared inner-product contributions never exceed the vector’s own norm. Concretely, for any inner product space and any orthonormal system {eα} indexed by a set I, the inequality

∑_{α∈I} |⟨eα, x⟩|² ≤ ||x||²

holds for every vector x. This matters because it provides a universal bound—even when the orthonormal system is infinite or even uncountable—without requiring orthogonal projections to exist.

The argument starts with a familiar geometric fact from Hilbert spaces: if U is a closed subspace spanned by finitely many orthonormal vectors e1,…,eN, then the orthogonal projection of x onto U is obtained by summing the one-dimensional projections onto each basis direction. The normal component n = x − proj_U(x) is orthogonal to U. Writing n explicitly as

n = x − ∑_{k=1}^N ⟨e_k, x⟩ e_k,

and testing orthogonality by taking ⟨n, u⟩ = 0 for an arbitrary u in U (which itself is a linear combination of the e_k), the orthonormality condition ⟨e_k, e_l⟩ = δ_{kl} collapses the resulting double sums. The calculation shows ⟨n, u⟩ = 0, confirming that n really is the perpendicular component.

From there, the inequality emerges by comparing lengths: since n is a vector, ||n||² ≥ 0. Expanding ||n||² = ⟨n, n⟩ produces ||x||² minus the sum of squared coefficients |⟨e_k, x⟩|². This yields the finite version of Bessel’s inequality:

∑_{k=1}^N |⟨e_k, x⟩|² ≤ ||x||².

The key upgrade is that the same bound survives in general inner product spaces and for arbitrary index sets I, not just finite ones. When I is infinite (even uncountable), the left-hand side is defined using a supremum over all finite subsets J ⊂ I: one considers finite partial sums ∑_{β∈J} |⟨eβ, x⟩|² and takes the least upper bound. The inequality guarantees this supremum stays finite and never exceeds ||x||², unlike a generic nonnegative series where partial sums could grow without bound.

Finally, the proof for the infinite/uncountable case repeats the same inner-product expansion on an arbitrary finite subset J, then passes to the supremum. The result is a robust “energy bound” for orthonormal systems: squared inner products behave like coordinates whose total cannot outrun the vector’s norm. Equality is not guaranteed for every orthonormal system, but the discussion points toward special orthonormal systems that can achieve it in later work.

Cornell Notes

Bessel’s inequality provides a universal bound on how much a vector x can “align” with an orthonormal system {eα}. For any inner product space and any orthonormal system indexed by I, the squared coefficients satisfy

∑_{α∈I} |⟨eα, x⟩|² ≤ ||x||².

When I is infinite or uncountable, the sum is defined via a supremum over all finite subsets J ⊂ I of the partial sums ∑_{β∈J} |⟨eβ, x⟩|². The inequality is proved by first establishing the finite case using orthogonal projections (x minus its projection onto span{e1,…,eN} is orthogonal to that span), then expanding ||x − projection||² = ⟨n,n⟩ and using orthonormality to collapse double sums. The nonnegativity of ||n||² forces the bound.

How does orthogonal projection onto a span of orthonormal vectors lead to a bound on squared inner products?

For a finite orthonormal system e1,…,eN spanning U, the orthogonal projection is proj_U(x)=∑_{k=1}^N ⟨e_k,x⟩ e_k. The normal component n=x−proj_U(x) is orthogonal to U. Since ||n||²=⟨n,n⟩≥0, expanding ⟨x−∑⟨e_k,x⟩e_k, x−∑⟨e_l,x⟩e_l⟩ yields ||x||² minus ∑_{k=1}^N |⟨e_k,x⟩|². Nonnegativity forces ∑_{k=1}^N |⟨e_k,x⟩|² ≤ ||x||².

Why do double sums collapse in the proof?

When expanding ||n||², terms involve ⟨e_k,e_l⟩. Orthonormality gives ⟨e_k,e_l⟩=δ_{kl}, where δ_{kl} is 1 if k=l and 0 otherwise. This turns a double sum over k and l into a single sum because only the diagonal terms survive.

What changes when the orthonormal system is infinite or uncountable?

The expression ∑_{α∈I} |⟨eα,x⟩|² may not be a standard countable series. Instead, it is defined using nonnegative partial sums over finite subsets J ⊂ I: take S(J)=∑_{β∈J} |⟨eβ,x⟩|², then define the “sum” as sup_{J finite} S(J). Bessel’s inequality guarantees this supremum is bounded by ||x||², so it cannot blow up to infinity.

Why is Bessel’s inequality valid in any inner product space, even without orthogonal projections?

The proof can be run on any finite subset J ⊂ I: for that finite orthonormal system, the finite inequality holds. Since the left side for the infinite/uncountable case is defined as a supremum over these finite partial sums, the same bound carries over. This avoids needing existence of orthogonal projections, which is guaranteed in Hilbert spaces but not required for the inequality itself.

What does the inequality mean geometrically?

It says the squared “coordinate energy” of x along orthonormal directions cannot exceed the total energy ||x||². In the finite projection picture, the missing part is exactly the squared norm of the perpendicular component n=x−proj_U(x), so the difference ||x||²−∑|⟨e_k,x⟩|² equals ||n||²≥0.

Review Questions

State Bessel’s inequality for an orthonormal system indexed by I and explain how the left-hand side is defined when I is uncountable.
In the finite case, write the normal component n and show how orthonormality leads to the collapse of double sums.
Why does taking the supremum over finite subsets preserve the inequality sign in Bessel’s inequality?

Key Points

1
For any inner product space and orthonormal system {eα} indexed by I, every vector x satisfies ∑_{α∈I} |⟨eα,x⟩|² ≤ ||x||².
2
In the finite case, the inequality follows from ||x−proj_U(x)||²≥0, where proj_U(x)=∑_{k=1}^N ⟨e_k,x⟩e_k.
3
Orthonormality ⟨e_k,e_l⟩=δ_{kl} collapses double sums that appear when expanding ⟨n,n⟩.
4
For infinite or uncountable index sets, the “sum” is defined as sup over all finite subsets J ⊂ I of the partial sums ∑_{β∈J} |⟨eβ,x⟩|².
5
Bessel’s inequality does not require orthogonal projections to exist; it can be proved by applying the finite result on each finite subset and then taking a supremum.
6
The inequality guarantees the supremum of partial sums is always finite and never exceeds ||x||², preventing the left side from diverging.
7
Equality in Bessel’s inequality is not automatic for every orthonormal system, motivating the search for special systems that can achieve it.

Highlights

Bessel’s inequality bounds the total squared inner-product contributions of x along an orthonormal system by ||x||².

The proof hinges on orthonormality collapsing double sums via ⟨e_k,e_l⟩=δ_{kl}.

For uncountable index sets, the “sum” is defined through a supremum over finite partial sums, and the inequality ensures it stays finite.

The difference ||x||²−∑|⟨eα,x⟩|² equals the squared norm of the perpendicular component in the finite projection setting.