I wrote the first draft. ChatGPT edited it, and I made final edits. All mistakes are my own and I am willing to correct them if DT Wegener or anybody else points them out in the comment section or by email.
Calling someone a liar implies that they know the truth but deliberately say otherwise. That is difficult to prove, since often people are simply misinformed or fail to understand. In fact, we are usually the last to realize when we are wrong, because the moment we recognize our mistake, we have already corrected our false belief.
Duane T. Wegener, however, has had ample opportunity to correct his misconceptions about statistical power—its concept, its history, and its uses—yet he has shown no signs of doing so. His ability to publish extensively on the topic and cite relevant sources makes it implausible to dismiss him as incompetent. The more reasonable explanation is willful ignorance.
Willful ignorance by itself is common and not always problematic. But when people in positions of influence use their authority to spread falsehoods, the consequences are serious. Wegener uses his standing among experimental social psychologists to reinforce misconceptions about power, and in doing so, he misinforms new generations of students. This is particularly troubling because experimental social psychology is already the poster child of bad science. The replication crisis has led to some positive reforms, but Wegener is one of the few outspoken voices that is fighting reforms with misinformation.
Wegener has co-authored several misleading articles on power, but his first-author paper, Evaluating Research in Personality and Social Psychology: Considerations of Statistical Power and Concerns About False Findings, most clearly reveals his willful ignorance. Although the article focuses on statistical power, it appeared in a journal for personality and social psychologists—meaning its reviewers and editors were not statistical experts. Peer review cannot function when the peers are equally ignorant, willful or not.
So what is the misrepresentation at the core of his paper? It is captured in this sentence:
“Traditionally, statistical power was viewed as relevant to research planning but not evaluation of completed research.”
Wegener focuses on null-hypothesis significance testing (NHST) and ignores that power is an older concept that plays a much more important role in Neyman-Pearson’s approach to statistic than in NHST that is mostly based on Fisher.
In Neyman-Pearson’s approach power is the opposite of a the probabilty of a type-II error, which implies falsely accepting the null-hypothesis or rejecting the hypothesis that is favored by the researcher. In this framework, power is important because studies with low power lead to false conclusions (e.g., burning fossil fuel does not contribute to global warming). Not so in NHST. In NHST low power only results in non-significant results that are deemed inconclusive and are not published. This explains how psychologists often ignore power to get a significant result, but journals publish 90% significant results.
Wegener et al. (2022) support their claim that power was never used to evaluate published results by quoting Cohen, the biggest authority on power analysis in the behavioral “sciences.”
“Jacob Cohen, the primary advocate of consideration of statistical power, was clear, however, in describing power as relevant to pre-data-collection study planning but not to post-data-collection study evaluation.” (Wegener et al., 2022, p. 1106)
This claim is inconsistent with Cohen’s (1988) book. On page 4 he wrote
“Consider a completed experiment which led to nonrejection of the null hypothesis. An analysis which finds that the power was low should lead one to regard the negative results as ambiguous…”
This is a straightforward endorsement of power as an evaluative tool. Cohen and others also conducted multiple meta-analyses of power to show that studies are underpowered. These studies are not cited by Wegener et al. (2022), presumably because they are inconsistent with the message that power was never used to evaluate research.
Wegener leans heavily on a passage from a brief comment by Cohen (1973) on an article that used power to criticize research in education.
“Cohen (1973, p. 227) noted that, “power analysis is a powerful, in fact the only rational, guide to planning the relevant details of the research. But once data are gathered and analyzed, it recedes into the background.” (Wegener et al., 1106).
The full context makes it clear that Cohen was not rejecting the general claims about low power in education research. He was just pointing out that the effect size estimate of a study is more important than the assumed effect size before a study.
I was most pleased by the recent publication by Brewer, “On the Power Statistical Tests in the American Educational Research Journal” (1972), understandably delighted with his heavy reliance, in accomplishing his on my power handbook (Cohen, 1969). I strongly agree with his stress importance of power analysis. Further, his survey’s confirmation of finding of a decade ago (Cohen, 1962; 1965) that the neglect of power analysis results in generally low power is very useful, although not surprising. (p. 225).
Stripped of context, Wegener’s quotation makes it appear that Cohen flatly opposed using power for evaluation. That is a distortion. Cohen consistently criticized psychologists for conducting meaningless NHST rituals and advocated for power and effect size estimation as part of a solution. Wegener’s framing makes Cohen appear to support the very practices he opposed.
That is why Wegener’s statement is not merely wrong but a self-serving untruth—one that shields a failing methodology from criticism. Whether one calls this lying or willful ignorance, the effect is the same: misleading a field already struggling with credibility. True reform requires ending the null-hypothesis ritual and success rates of 90% that are void of real significance.