Datacolada Has Given Up on p-Curve

When p-curve debuted in 2014, it was billed as a powerful tool for detecting publication bias and estimating evidential value from the distribution of statistically significant results. Its creators — Leif Nelson, Joe Simmons, and Uri Simonsohn of Data Colada — vigorously defended the method in a series of posts and journal replies through 2018.

Those early defenses addressed challenges like:

  • Whether excluding p > .05 distorts results
  • How p-curve handles effect size heterogeneity
  • The method’s robustness to extreme (“ambitious”) p-hacking

By late 2018, the authors had answered their critics in technical detail and released p-curve 4.0.

Then… silence.

From 2019 onward, the Data Colada blog shifted almost entirely to fraud detection, replication case studies, and other bias-related topics. In the same period, criticism of p-curve intensified:

  • Brunner & Schimmack published simulation studies questioning its statistical properties.
  • Montoya (2024) showed that p-curve often yields unreliable or irreproducible conclusions in applied case studies.
  • Morey & Davis-Stober (2025) dissected the statistical foundations of p-curve’s evidential value tests and average power estimator, arguing that the method is flawed in principle.

Despite this, Nelson, Simmons, and Simonsohn have issued no substantive public defense of p-curve since 2018. The only post-2018 “activity” was a 2024 bug fix to the p-curve app after Richard Morey reported an error in the half-p-curve test.

Given their earlier pattern of responding quickly to criticism, the long silence is telling. While the authors have never publicly conceded that p-curve has serious problems, the lack of engagement with major post-2018 critiques suggests they have effectively abandoned the method as an active research program.

When eminent psychologists die, their passing is announced in obituaries in the American Psychologist. Not so for psychological theories or methods. In contrast, psychological theories and methods never die; they simply fade away. Maybe that should change. I say, “p-curve is dead. Long live z-curve.”


2 thoughts on “Datacolada Has Given Up on p-Curve

  1. I would prefer if psychological theories evolved rather than fade away to be replaced by the next flashy toy.

    I wish for the best possible evolution therefore for the Z-curve!

    1. Agreed. Although z-curve was developed independently, you can say that it is an evolved version of p-curve. We are not just giving up on bias detection and correction, we are making it better.

Leave a Reply