Get AI summaries of any video or article — Sign up free
Hamming codes part 2: The one-line implementation thumbnail

Hamming codes part 2: The one-line implementation

3Blue1Brown·
6 min read

Based on 3Blue1Brown's video on YouTube. If you like this content, support the original creators by watching, liking and subscribing to their content.

TL;DR

Hamming-code parity-check results can be read as the binary index of a single flipped bit, not just as a yes/no error flag.

Briefing

Hamming codes can locate a single flipped bit with an error position that drops out directly from XOR—so the receiver’s core job can shrink to one line of computation. Instead of “binary searching” through candidate error locations using multiple parity checks, the parity results can be read as the error index itself in binary; that same logic can be packaged into a single XOR reduction over the indices of all 1 bits. The payoff is practical: the redundancy grows only logarithmically with block size, making single-error correction efficient even for large blocks.

The mechanism starts with four parity checks for a 16-bit block. Each check corresponds to a specific bit pattern in the binary representation of the position labels 0000 through 1111. For any error at position 7 (binary 0111), the first three parity groups flip while the last does not—so the four parity outcomes, read bottom-to-top, spell out 0111, i.e., the error location. This isn’t a special case: the parity groups are constructed so that every position has a unique binary signature across the checks, meaning the syndrome (the parity-check results) uniquely identifies which single bit flipped.

Why parity bits land in positions that are powers of two (1, 2, 4, 8) becomes clearer once positions are viewed as binary numbers with one “1” bit. Those power-of-two indices have exactly one set bit in their binary label, so each parity bit participates in exactly one parity group. That structure generalizes cleanly: for a block with more positions, each additional binary digit required to label indices contributes another parity check, letting the scheme scale to larger powers of two.

The transcript then reframes the same idea using XOR. XOR of many bit strings is equivalent to adding them mod 2 without carries, so it naturally computes parities across multiple groups at once. Concretely, label each position in binary, select the indices where the received bits are 1, and XOR all those indices together. If the block is “well-prepared” (parity bits chosen so the total XOR is 0), then any single-bit flip changes the set of included indices: a 0→1 adds that index to the XOR sum, while a 1→0 removes it. Because XOR is its own inverse (adding the same index twice cancels), the final XOR result becomes exactly the binary position of the error—or 0 if no error occurred.

A Python sketch illustrates the receiver-side logic: enumerate indices, filter to those where bits are 1, then reduce the list with XOR. The same approach scales beyond 16 bits; the core computation remains an XOR reduction over indices of 1s. Efficiency improves as blocks grow: for 256 bits, only about 3% of space is redundancy, and for extremely large blocks the number of parity bits grows slowly (logarithmically). The tradeoff is error model limits: Hamming codes correct only single-bit errors, and larger blocks face higher odds of multiple flips. Burst errors also motivate interlacing blocks, while more advanced codes like Reed-Solomon handle bursts better.

Finally, the transcript situates the elegance historically. Hamming’s discovery is portrayed as non-obvious—he experimented widely before landing on the parity-check structure that encodes error locations. The broader context links Hamming’s work to Shannon’s information theory, emphasizing that efficient error correction and the “bits-first” worldview were still coalescing in the mid-20th century. The result feels simple in hindsight, but the underlying design space and conceptual leap were anything but.

Cornell Notes

Hamming codes can identify the location of a single flipped bit by turning parity-check outcomes into the error index written in binary. For a 16-bit block with four parity checks, each check corresponds to one binary digit of the position labels 0000–1111, so the syndrome uniquely spells out the error location (e.g., 0111 for position 7). The same logic can be implemented more directly in software: XOR together the indices of all positions whose received bits are 1. If the block was correctly prepared, the total XOR is 0; a single-bit flip changes the XOR to the exact error position, or leaves it 0 when no error occurred. This yields logarithmic redundancy growth with block size, but performance depends on the assumption of at most one bit error per block.

How do four parity checks for a 16-bit block end up encoding an error position in binary?

Each position in the 16-bit block is assigned a conceptual index label from 0000 to 1111. The four parity checks correspond to whether the last bit, second-to-last bit, third-to-last bit, and highest-order bit of that index label are 1. When a single bit flips, it changes which parity groups are “odd,” producing a syndrome that matches the binary digits of the flipped position. Reading the four check results bottom-to-top yields the binary index of the error (the example error at position 7 produces 0111 because 7’s label has 1s in the last three binary digits but 0 in the highest-order digit).

Why do parity bits sit at positions like 1, 2, 4, and 8?

Those positions are powers of two, and in binary they have exactly one set bit (e.g., 1 is 0001, 2 is 0010, 4 is 0100, 8 is 1000). That means each parity bit participates in exactly one of the parity groups defined by the binary digits of the indices. In larger blocks, the same pattern holds: each parity bit aligns with a unique binary digit position, so the parity-check structure stays consistent as the block grows.

How does XOR replace multiple parity checks with a single operation?

XOR is addition mod 2 without carries. If you XOR together many bit strings component-by-component, each output bit effectively computes the parity (even/odd count) of the corresponding group. In the Hamming-code setting, you can label every index in binary, then select the indices where the received bits are 1 and XOR all those indices together. The resulting four low bits match the four parity-check outcomes because each binary digit column counts (mod 2) how many selected indices fall into the corresponding parity group.

Why does the XOR of indices of 1 bits equal the error location after a single-bit flip?

If the block is correctly prepared, the XOR of all selected indices is 0. A 0→1 flip adds that index to the XOR sum, so the total becomes the index of the flipped bit. A 1→0 flip removes that index; XOR has the property that adding the same value twice cancels (since 1+1=0 mod 2), so the net effect is the same: the final XOR equals the error’s position. If no bit flips, the XOR remains 0.

What scaling behavior makes Hamming codes efficient, and what breaks that efficiency?

The number of parity bits grows with the number of bits needed to label positions, so redundancy scales like log2(block size). As block size doubles, one more parity bit is needed, so the fraction of redundancy shrinks (the transcript cites about 3% redundancy for 256 bits). The limitation is the error model: Hamming codes only correct single-bit errors, and larger blocks increase the chance of multiple flips. Burst errors can also overwhelm a single block, so interlacing blocks is used to spread bursts across many blocks.

How does the transcript connect this to broader coding and information theory?

It contrasts Hamming codes with Reed-Solomon coding, noting Reed-Solomon can be tuned to handle more errors per block and is particularly good with burst errors. It also frames Hamming’s work historically: Hamming’s path to the parity-check structure was described as meandering, and the transcript links the timing to Claude Shannon’s information theory, which helped establish that efficient error correction is possible in principle.

Review Questions

  1. In a 16-bit Hamming-code setup with four parity checks, what binary relationship between an index label and the parity groups guarantees that the syndrome uniquely identifies a single-bit error?
  2. Explain why XOR of the indices of 1 bits yields 0 for a correctly prepared block and becomes the error position after a single-bit flip.
  3. What assumptions about error frequency and error patterns (single-bit vs burst/multiple-bit) determine when Hamming codes are practical?

Key Points

  1. 1

    Hamming-code parity-check results can be read as the binary index of a single flipped bit, not just as a yes/no error flag.

  2. 2

    Parity groups are constructed by mapping each position to a binary label and assigning each parity check to one binary digit of that label.

  3. 3

    Parity bits naturally belong at power-of-two positions because those indices have exactly one set bit in their binary representation, making each parity bit touch only one parity group.

  4. 4

    XOR can compute the same syndrome in one step: XOR together the indices of all positions whose received bits are 1.

  5. 5

    A correctly prepared block yields XOR=0; a single-bit flip changes the XOR to the exact error position, while no flip leaves it at 0.

  6. 6

    Redundancy grows logarithmically with block size, improving efficiency for larger blocks, but the scheme fails to correct multiple-bit errors.

  7. 7

    Burst errors often require interlacing or switching to stronger codes like Reed-Solomon for better burst resilience.

Highlights

The four parity checks for a 16-bit block don’t just detect an error—they spell out the error’s position in binary when read in the right order.
Each parity bit sits at a power-of-two index because those indices have a single 1 in their binary label, so parity-bit participation stays clean and non-overlapping.
On the receiver side, the entire single-error-location logic can collapse to XOR-reducing the indices of all 1 bits.
XOR’s self-canceling property (adding the same index twice returns to 0 mod 2) is what makes the error location pop out after a flip.
Hamming codes are efficient in redundancy but limited to single-bit errors; burst and multi-bit error likelihood drive practical design choices.

Topics