Linear Algebra 26 | Steinitz Exchange Lemma

Steinitz’s exchange lemma is the key tool for making “dimension” well-defined: it guarantees that any two bases of the same subspace contain the same number of vectors. That matters because the intuitive idea—lines need one basis vector, planes need two—only works if the count of basis elements cannot change when switching between different bases.

The setup starts with a subspace U of R^N and a fixed basis B = {v1, v2, …, vK}. Then a new collection of vectors A = {a1, a2, …, aL} is chosen from U such that the vectors in A are linearly independent. Steinitz’s exchange lemma says it’s possible to build a new basis of U by “exchanging” some vectors from B with the vectors from A. More precisely, after inserting the L independent vectors from A, the basis must still have K vectors total, so exactly K − L vectors from the original basis B remain, while the rest are replaced by vectors from A.

To justify the claim, the proof is first illustrated in the simplest nontrivial case L = 1. Here there is just one extra vector a1, and the combined family B ∪ {a1} has K + 1 vectors. Because B already spans U (it is a basis), the new vector a1 can be written as a linear combination of the basis vectors v1, …, vK. That immediately implies the enlarged family is linearly dependent. Since a1 is also not the zero vector (because A is linearly independent), at least one coefficient in the expression of a1 must be nonzero; call it the coefficient corresponding to vj.

Using that nonzero coefficient, the proof rearranges the linear combination to solve for vj in terms of a1 and the remaining basis vectors. This produces a new family C obtained by removing vj from B and adding a1. The argument then checks two properties.

First, C is linearly independent: any linear combination of C that equals the zero vector forces the coefficient of a1 to be zero (otherwise the equation could be rearranged to produce an alternative representation of a1 using fewer basis vectors, contradicting uniqueness of coordinates from the original basis). With that coefficient gone, the remaining coefficients multiply the original basis vectors, which are already linearly independent, so all coefficients must be zero.

Second, C spans U: take any vector u in U. Since B is a basis, u can be written using the v’s. Wherever vj appears, substitute the expression for vj in terms of a1 and the other v’s. This rewrites u as a linear combination of vectors in C. Therefore C is a basis.

For L > 1, the same exchange idea is reused, exchanging multiple vectors rather than just one. The payoff is foundational: the number of vectors in a basis of U is fixed, which makes the dimension of a subspace a meaningful invariant rather than a basis-dependent guess.