Since the early 2010s, psychology has been grappling with a/nother crisis, dubbed the replication crisis or the credibility crisis. The crisis has motivated many psychologists to examine the foundations of empirical psychology and to suggests improvements.
In 2016, a group of psychologists founded the Society for the Improvement of Psychological Science (SIPS). Key founding members were Simine Vazire and Brian Nosek. Brian Nosek is also the co-founder and director of the Center for Open Science that hosts the Open Science Framework to allow researchers to share pre-prints and data.
To start with the positive, SIPS and COS have changed psychological science in ways that seemed impossible before 2010. The availability of open data makes it possible to reanalyze data and pre-prints have made it possible to share information faster. I love psychological science in 2025 more than in 2010.
However, all social movements have similar dynamics that we would better understand if social psychologists actually studied social phenomena rather than wasting their time on laboratory studies with a single participant responding to stimuli on a computer screen. Fortunately, we do not need science to understand familiar group dynamics that have been replicated for thousands of years. Every group forms a hierarchy with leaders and followers. Every group creates an identity to distinguish in-group and out-group members and is biased towards in-group members. At least the last part has been studied by social psychologists.
In the beginning, many critics of the status quo were also out-group members of organizations that represent the establishment. A representative of mainstream social psychology, Susan Fisk, branded critics of the status quo as terrorists. That did not go over well, and she had to retract her words, but she did not change her attitudes. These times of heated debates on social media and in more polite tone in journal articles are over.
Now it seems as if the crisis is over and psychology has been reformed for the better. Former “terrorists” even wrote an article in a series edited by accuser Susan Fisk with the optimistic title “Psychology’s Renaissance” (Nelson, Simmons, & Simonsohn, 2018). We no longer talk about a credibility crisis; now we have a credibility revolution (Vazir, 2018).
Another sign of progress is that many journals now reward researchers for good practices with open science badges. A leader of this initiative was the APS journal Psychological Science. Psychologists now get a badge if they preregistered their hypotheses and data analysis plan before they conducted a study. In medicine, you do not get a badge. You simply cannot publish a clinical trial without preregistration. This form of nudging psychologists towards better practices is based on psychological insights that reward is better than punishment and that nudges are better than strict rules. Call me conservative in my approach to science (I am progressive in my social values), but I always disagreed with this approach. John et al. (2012) compared the use of statistical tricks to get significant results, now called p-hacking, scientific doping. It gives unethical scientists an advantage in the publish-or-perish game of science. In sports, doping is banned and punished if athletes are found to be cheating. Why not hold scientists who actually are tasked to find the truth to a lower standard than athletes who perform only for our entertainment? Isn’t scientific cheating worse? The American Physician Association thinks so, but the American Psychological Association does not.
The latest big news was that the co-founder of SIPS, Simin Vazir, became the editor of the journal Psychological Science. Psychological Science is the main journal of the American Psychological Society that split from the American Psychological Association to represent research workers rather than clinical psychologists. Publishing in Psychological Science is considered a sign of excellence because the journal has a high rejection rate because it can only publish a few articles a year to be profitable for APS and the publisher Sage because it still prints paper copies that are sold to libraries at tax-payers expense.
And here the problem of the happy marriage between Open Science and the establishment starts. Just like Ben & Jerry’s ceased to be an independent, fun, and progressive ice-cream company, when it was sold to Unilever, Open Science faces the normal pressures of profit margins at traditional journals. Most papers have to be rejected and the selection of the papers that get in and get the stamp of excellence is difficult. After all, the journal gets submissions from all areas of psychology. How do you compare a manuscript that investigates language development in 1-year olds to a paper that examines cultural variation in the influence of religion on pro-social behavior. Which is better? Inevitably, the decision is subjective. Moreover, the high number of rejections means that many articles are not even sent out for review by experts and get a desk rejection. An editor skims the paper and then decides that it is not interesting enough for the wide readership of the journal. Editors are also to busy to explain their reasoning, which would be difficult given that it is really just a subjective judgment that lacks any objective criteria.
This is not a problem per se. After all, we all understand the motives of publishers to make money and maintain their profit margins of 30% or more that they make of a scientific publication created by researchers, published mainly for researchers, and all paid for by you, the taxpayer. The question is only whether this model of scientific publishing is compatible with open science? I think it is not. Essentially, the subjective judgment of a single person is a black box that cannot be questioned and decides on the careers of researchers who may get a grant or not because they published in a prestigious journal. The process is anything but transparent or open. It does not matter whether the editor is considered benevolent and working in the interest of the people or abuses their power for their own agenda. The editor has full discretion to determine what gets published in their journal. To be fair, this is about the only reason to take on this job that has little other incentives, aside from some financial compensation (I do not know how much an editor of Psych Science gets paid). In short, it is impossible to be an open scientist and the editor of Psychological Science.
What we need to do to make psychological science more open and transparent is to create new journals that are run by scientists for scientists and do not aim to maximize profits for publishers. More and more journals of this kind are appearing, and they are the future of open science. I am proud to have contributed a little to the creation of such a journal, Meta-Psychology, edited by Rickard Carlsson. Unlike Psychological Science, this journal does not have a black box decision process. The review process and editorial decisions are open and transparent to anybody with a web browser. This is what I call Open Science. The problem for these journals is that they lack prestige and many young researchers are forced to get publications in journals that are assumed to publish better work. But is the research published in Psychological Science really better?
Psychological Science has used the Open Science badges to brand itself as a leader in Open Science. The badges do not only reward scientists. They also reward journals because they can now signal that the research published in these journals is more credible, but what does this really mean? Sharing materials makes it easier to reproduce a study design, but does it tell us that the study was actually good? No. Sharing data is useful, but if the data were p-hacked, sharing the p-hacked data will reproduce the p-hacked p-values below .05. So, are these articles more trustworthy? So, what about pre-registration. Are pre-registered articles really better than those that are not preregistered? I have some data to answer this question, and I will publish the results, but you will not see these results in Psychological Science because Simin Vazire decided that the answer to this question is not of interest to readers of Psychological Science. When I asked her why not, she declined to elaborate on the cognitive processes inside her Black Box that delivered the verdict. So, there you have it. Badges solved the replication crisis and if you do not believe so, you might still be considered a “terrorist, ” or at least a party pooper who does not believe that psychology has solved its deep fundamental problems with a few band aids with cute symbols on them.