Selective Reporting & Misrepresentation of Data | eSupport for Research | 2022

TL;DR

Selective reporting suppresses negative or undesirable findings, often leading to biased analysis or writing that undermines reproducibility.

Briefing Cornell Notes

Briefing

Selective reporting—often tied to outcome reporting bias and reporting bias—happens when researchers deliberately or carelessly present only part of the results from a study, leaving out negative or “undesirable” findings. The motivation is frequently to suppress outcomes that don’t fit expectations, but the downstream effect is serious: the published findings can become skewed by bias during analysis or writing, making the results hard or impossible to reproduce. When selective reporting enters the scientific record, reproducibility suffers because readers and other researchers are not seeing the full evidence base that the original study produced.

The discussion also links selective reporting to a broader ecosystem of biases that can distort research from start to finish. Design bias can begin at the planning stage if the research team uses a limited or non-representative population or dataset—such as relying on a narrow healthcare dataset rather than a sufficiently broad demographic base. Procedural bias can arise after the design is approved, when researchers steer the experiment through a predetermined path rather than following the intended procedure neutrally. Personal bias is described as especially difficult to avoid because it stems from the researcher’s character and may go unrecognized even by the people involved.

Once results are ready for dissemination, reporting bias can expand into selective evidence dissemination. The framing of what gets reported can lead to fragmented reporting, where only the “preferred” subset of analyses is emphasized. Selective publication is another pathway: work deemed “not suitable” for a journal may be pushed into conference proceedings, abstracts, or other outlets, creating publication bias when the weight of evidence across venues becomes uneven. The transcript also highlights inclusion bias during literature reviews, where authors may select literature or databases they are comfortable with, exclude recent findings that don’t align with existing work, or rely on non-existent, outdated, or poorly cited databases. These choices can prevent proper comparison, cross-validation, and a fair synthesis of prior evidence.

All of these biases—reporting, publication, and inclusion—can compound into dissemination bias, distorting what the scientific community ultimately learns. The final section shifts to misrepresentation of data, defined as communicating honestly collected data in a deceptive way. Misrepresentation can involve misleading interpretation, unfounded extrapolation beyond what the data support (for example, extending findings to populations not actually studied), and ignoring limitations that should constrain conclusions.

A practical way to spot misrepresentation is through comparison: when a study is framed by a proper protocol and statistical plan, the accurate article should include pre-specified methods, focus interpretation on primary analyses, highlight limitations, and avoid overreach. A distorted article, by contrast, may misreport methods, misreport results, and misinterpret findings—sometimes through negligence, and sometimes intentionally to achieve a desirable outcome or “beautify” the narrative. The transcript closes by emphasizing that readers can often detect non-reproducibility and that adherence to publication ethics guidance (including COPE-style expectations mentioned) helps reduce these problems, including redundant or duplicate publication practices.

Cornell Notes

Selective reporting and misrepresentation distort the scientific record by presenting only favorable or incomplete evidence. Selective reporting—closely linked to outcome reporting bias—can suppress negative findings, skew analysis or writing, and undermine reproducibility. Bias can enter at multiple stages: design bias from limited populations, procedural bias from predetermined experimental paths, and personal bias that may go unnoticed. During dissemination, selective publication and inclusion bias in literature reviews can further skew what gets compared and synthesized. Misrepresentation of data includes misleading interpretation, unfounded extrapolation, and ignoring limitations, often detectable by comparing pre-specified methods and primary analyses in an accurate article versus a distorted one.

What makes selective reporting unethical, and why does it threaten reproducibility?

Selective reporting involves deliberately not fully or accurately reporting research results, often to suppress negative or undesirable findings. Because the analysis or writing stage becomes biased toward a preferred narrative, other researchers may not be able to reproduce the reported conclusions. The transcript frames this as a key consequence: findings can become skewed, so the published evidence no longer reflects the full study results.

How do design bias, procedural bias, and personal bias differ in where they enter a study?

Design bias starts when the research team creates the process for the experiment—such as using a limited or non-representative dataset or population (e.g., a narrow healthcare dataset). Procedural bias occurs after the design is approved, when researchers steer the experiment through a predetermined path rather than following the intended procedure neutrally. Personal bias is tied to researcher character and is described as harder to avoid because it may go unrecognized even by those involved.

What is selective publication, and how can it create publication bias?

Selective publication happens when work considered “not suitable” for a journal is instead routed to conference proceedings, abstracts, or other formats. This can produce publication bias because the overall distribution of evidence across venues becomes uneven, and the transcript notes that publication bias is hard to quantify if the weight of findings across outlets is not consistent.

How does inclusion bias affect literature reviews and evidence synthesis?

Inclusion bias arises when authors select literature or databases that they are comfortable with or that align with their desired comparisons. The transcript also notes problems such as using non-existent, outdated, or improperly cited databases, and it emphasizes that proper database selection enables comparison and cross-validation. Excluding recent or discordant findings can skew the review’s conclusions.

What counts as misrepresentation of data in this framework?

Misrepresentation is described as communicating honestly reported data in a deceptive manner—through misleading interpretation, unfounded extrapolation to populations not supported by the data, and ignoring limitations. The transcript contrasts accurate articles (with pre-specified methods, primary analysis focus, and limitations highlighted) against distorted articles that misreport methods/results and misinterpret findings.

How can readers detect distorted reporting using the protocol-method-results-interpretation structure?

The transcript suggests comparing what should be present under a proper protocol: pre-specified methods and derivations, results tied to primary analyses with appropriate inference, and explicit limitations. Distorted articles may misreport methods, misreport results, and misinterpret conclusions, sometimes due to negligence or intentionally to achieve a desirable finding. Non-reproducibility is a key red flag.

Review Questions

Which stage of research is most associated with design bias, and what example of dataset limitation is given?
How do selective publication and inclusion bias each distort what other researchers can learn from the literature?
What specific elements of an accurate article (methods, primary analysis, limitations) are used as a checklist to spot misrepresentation?

Key Points

1
Selective reporting suppresses negative or undesirable findings, often leading to biased analysis or writing that undermines reproducibility.
2
Outcome reporting bias is treated as a common research ethics problem because incomplete reporting prevents others from verifying results.
3
Bias can originate early (design bias from limited or non-representative populations), during execution (procedural bias from predetermined experimental paths), or from researcher behavior (personal bias).
4
Selective publication can shift evidence into conferences or abstracts instead of journals, creating publication bias through uneven evidence distribution.
5
Inclusion bias in literature reviews can come from choosing comfortable or aligned databases, excluding recent findings, or using outdated/non-existent sources.
6
Misrepresentation of data includes misleading interpretation, unfounded extrapolation beyond supported populations, and ignoring limitations that should constrain conclusions.
7
Comparing pre-specified methods, primary analyses, and limitations between an accurate protocol-based article and a distorted one helps identify misreporting and non-reproducibility risks.

Highlights

Selective reporting can make published findings non-reproducible because bias enters during analysis or writing, not just during interpretation.

Design bias, procedural bias, and personal bias represent three different entry points where distortion can begin—planning, execution, and researcher behavior.

Selective publication (moving “unsuitable” work into proceedings or abstracts) can create publication bias by changing where evidence appears.

Inclusion bias can stem from using outdated, non-existent, or poorly cited databases, blocking proper comparison and cross-validation.

Misrepresentation is often detectable by checking whether pre-specified methods and primary analyses are faithfully reported and whether limitations are acknowledged.

Topics

Selective Reporting
Outcome Reporting Bias
Publication Bias
Inclusion Bias
Data Misrepresentation

Mentioned

Akash Bhoi
COPE

Selective Reporting & Misrepresentation of Data | eSupport for Research | 2022 | Dr. Akash Bhoi