UPDATE 5/13/2019 Our manuscript on the z-curve method for estimation of mean power after selection for significance has been accepted for publication in Meta-Psychology. As estimation of actual power is an important tool for meta-psychologists, we are happy that z-curve found its home in Meta-Psychology. We also enjoyed the open and constructive review process at Meta-Psychology. Definitely will try Meta-Psychology again for future work (look out for z-curve.2.0 with many new features).
Since 2015, Jerry Brunner and I have been working on a statistical tool that can estimate mean (statitical) power for a set of studies with heterogeneous sample sizes and effect sizes (heterogeneity in non-centrality parameters and true power). This method corrects for the inflation in mean observed power that is introduced by the selection for statistical significance. Knowledge about mean power makes it possible to predict the success rate of exact replication studies. For example, if a set of studies with mean power of 60% were replicated exactly (including sample sizes), we would expect that 60% of the replication studies produce a significant result again.
Our latest manuscript is a revision of an earlier manuscript that received a revise and resubmit decision from the free, open-peer-review journal Meta-Psychology. We consider it the most authoritative introduction to z-curve that should be used to learn about z-curve, critic z-curve, or as a citation for studies that use z-curve.
Cite as “submitted for publication”.
Feel free to ask questions, provide comments, and critic our manuscript in the comments section. We are proud to be an open science lab, and consider criticism an opportunity to improve z-curve and our understanding of power estimation.
Latest R-Code to run Z.Curve (Z.Curve.Public.18.10.28).
[updated 18/11/17] [35 lines of code]
call function mean.power = zcurve(pvalues,Plot=FALSE,alpha=.05,bw=.05)
Z-Curve related Talks
Presentation on Z-curve and application to BS Experimental Social Psychology and (Mostly) WS-Cognitive Psychology at U Waterloo (November 2, 2018)