LU Decomposition - An Example Calculation [dark version]

TL;DR

LU decomposition factors a square matrix A into L·U, with L lower triangular and U upper triangular.

Briefing Cornell Notes

Briefing

LU decomposition rewrites a square matrix A as the product of a lower triangular matrix L and an upper triangular matrix U, and the method is essentially Gaussian elimination—except every elimination step is recorded into L. The key practical constraint emphasized is that the algorithm must run without row exchanges; if Gaussian elimination would require swapping rows, those swaps should be handled beforehand, otherwise the LU factors produced by the standard procedure won’t match.

The walkthrough uses a 4×4 example matrix A. The process begins by introducing an identity matrix I of the same size and placing A on the right side of an augmented setup, effectively starting from I·A = A. The algorithm then transforms the identity matrix into the lower triangular factor L while simultaneously performing Gaussian elimination on A to transform it into the upper triangular factor U. Concretely, each elimination step targets a pivot element and uses row operations to create zeros below it in the working matrix on the right; the multiples used to eliminate entries are not discarded. Instead, those multipliers are written into L at the corresponding positions.

In the first column, the pivot is the (1,1) entry. To eliminate the element below it in row 2, the method forms a new row 2 by subtracting 2 times row 1 (written as row2 − 2·row1). The algorithm records the multiplier 2 into L at the position (2,1), and the arithmetic confirms the elimination produces a zero in the target spot. The same pattern repeats for rows 3 and 4: row3 − 3·row1 and row4 − 2·row1 eliminate the entries beneath the first pivot, with the corresponding multipliers (3 and 2) stored in L. After finishing this column, the working matrix on the right has zeros below the first pivot, while the left side has accumulated the lower-triangular structure.

The procedure then moves to the second column. The pivot is now the (2,2) entry of the partially reduced matrix. Again, elimination is performed to create zeros below this pivot: the algorithm updates row 3 by adding 4 times row 2 (equivalently subtracting −4·row2), stores −4 in L at (3,2), and computes the resulting row entries. It then updates row 4 by subtracting 1 times row 2, stores −1 in L at (4,2), and completes the second-column elimination so only the upper-triangular portion remains nonzero.

Finally, the third column requires one last elimination step. Row 4 is updated by subtracting 3 times row 3, and the multiplier 3 is recorded into L at (4,3). After this, the left side is a lower triangular matrix L with ones on its diagonal, and the right side is an upper triangular matrix U. Multiplying L and U reconstructs the original matrix A, provided the elimination proceeded without row exchanges. The example ends by noting that handling cases needing row swaps requires a separate generalization, referenced for later discussion, but the core mechanics—Gaussian elimination plus bookkeeping of multipliers—remain the backbone of LU decomposition.

Cornell Notes

LU decomposition factors a square matrix A into L·U, where L is lower triangular and U is upper triangular. The method is Gaussian elimination in disguise: elimination operations create zeros to form U, while the multipliers used during elimination are stored into L. The walkthrough stresses a crucial condition—no row exchanges during the algorithm—because the recorded multipliers assume the elimination path is fixed. Using a 4×4 example, the process eliminates entries column by column: first under the (1,1) pivot, then under the (2,2) pivot, and finally under the (3,3) pivot. The result is L with ones on the diagonal and U containing the remaining upper-triangular entries, and their product returns A.

Why does LU decomposition require avoiding row exchanges during the elimination steps?

The standard LU procedure records the exact multipliers used to eliminate entries (e.g., when forming row2 − 2·row1, the multiplier 2 is written into L). If row swaps occur, the elimination path changes and those stored multipliers no longer correspond to the same sequence of operations that produced U. The transcript advises handling any necessary row exchanges before applying the LU algorithm so the recorded multipliers remain consistent.

How does the algorithm decide what numbers to put into L?

Every time Gaussian elimination uses a row operation of the form row_i ← row_i − m·row_k to create a zero, the multiplier m is stored in L at position (i,k). For example, eliminating the entry below the first pivot used row2 − 2·row1, so 2 is placed into L at (2,1). Similarly, row3 − 3·row1 stores 3 at (3,1), and later steps store multipliers like −4 and −1 into L at (3,2) and (4,2).

What role does the identity matrix play at the start?

The identity matrix is used as a “workspace” on the left side. Starting from I·A = A, the algorithm transforms I into L while simultaneously transforming A into U. As elimination proceeds, the left matrix stops being the identity and becomes lower triangular, with ones on the diagonal, because the elimination multipliers are written into the appropriate lower-triangular positions.

How does the method proceed from one column to the next?

After finishing elimination in the first column (making all entries below the first pivot zero), the pivot shifts to the next diagonal position in the reduced matrix—here, the (2,2) entry. The same elimination logic is applied: create zeros below the new pivot using row operations, store the multipliers into L, and update the right-hand working matrix so it gradually becomes upper triangular.

What confirms that the computed L and U are correct?

The transcript’s check is that multiplying L and U reconstructs the original matrix A. This works because the elimination steps on the right produce U, while the recorded multipliers on the left produce L that represent the same sequence of operations—assuming no row exchanges were used during the LU algorithm.

Review Questions

In LU decomposition, what exactly is stored in L during elimination, and where is it stored?
Why does the transcript insist that row exchanges must be avoided during the LU algorithm itself?
Describe the sequence of pivot columns used in the 4×4 example and how each pivot leads to zeros below it.

Key Points

1
LU decomposition factors a square matrix A into L·U, with L lower triangular and U upper triangular.
2
The procedure is equivalent to Gaussian elimination, but it records the elimination multipliers into L.
3
Each time a row operation row_i ← row_i − m·row_k is used to create a zero, the multiplier m is written into L at (i,k).
4
The algorithm starts by pairing the identity matrix I with A so that I is transformed into L while A is transformed into U.
5
The standard LU algorithm assumes no row exchanges during elimination; any needed swaps should be handled beforehand.
6
In the worked 4×4 example, L ends with ones on its diagonal and U contains the remaining upper-triangular entries.
7
When no row exchanges are used, multiplying the resulting L and U reproduces the original matrix A.

Highlights

LU decomposition is Gaussian elimination plus bookkeeping: the multipliers used to eliminate entries become the entries of L.

Avoid row exchanges during the LU algorithm; otherwise the stored multipliers no longer match the elimination that produced U.

In the example, L is built column by column as zeros are created under each pivot in U.

The final check is straightforward: L·U reconstructs the original matrix A.

Topics

LU Decomposition
Gaussian Elimination
Triangular Matrices
Matrix Factorization