What is Intercoder reliability in research (and why you don't need it)

Intercoder reliability—having multiple coders align their coding and then using statistical tests to quantify agreement—is often pushed as a credibility booster for qualitative research, but it clashes with the core assumptions that make qualitative inquiry work. The practice typically treats coding as something that should converge on a single “common interpretation,” measured through coder-to-coder consistency. That framing matters because qualitative research frequently rests on constructivism, where meanings are shaped by people and contexts rather than discovered as one objective truth.

The transcript lays out a chain of problems that follow from that mismatch. First comes philosophical and epistemological misalignment: constructivist qualitative work assumes multiple realities and high subjectivity, while intercoder reliability implicitly aims for a universal interpretation shared across researchers. That leads to an “illusion of objectivity,” where agreement is treated like evidence of truth, even though qualitative methods usually emphasize interpretive depth, context, and the situated nature of meaning.

From there, the approach risks oversimplification. By forcing interpretations into a single agreed-upon codebook, researchers may reduce emergent, nuanced findings into a homogenized view. The process can also narrow the iterative practice of moving back and forth through the data—where new insights are discovered and interpretations evolve—because the goal becomes alignment rather than exploration. The transcript even flags an ethical risk: prioritizing coder agreement can cause some participant-relevant meanings to be overlooked, effectively muting voices that qualitative research is meant to foreground.

Methodologically, the transcript argues that reliability is the wrong target. In qualitative research, many scholars prefer validity over reliability, since reliability is tied to replicability—something qualitative studies often cannot (and should not) guarantee in the same way. Instead of focusing on whether different coders would produce the same coding, qualitative researchers should focus on whether interpretations are credible and well-supported, which is framed as a validity concern.

Another methodological concern is the neglect of reflexivity. Reflexivity requires researchers to examine how their own assumptions, biases, and presence influence the research process. Intercoder reliability, by emphasizing objectivity and alignment, can send the opposite message—suggesting reflexivity is something to avoid rather than a tool for transparency.

Finally, the transcript points to a practical translation problem: intercoder reliability is rooted in quantitative research assumptions, and those assumptions are difficult to carry into qualitative work without distorting what qualitative research is trying to achieve. The takeaway is not that coding discussion is inherently bad, but that adopting intercoder reliability as a formal requirement should be questioned, carefully planned, and justified as the right fit for the study’s goals rather than treated as a default marker of rigor.