After a desk-rejection for JPSP, my co-author and I submitted our ms. to PSPB (see blog https://replicationindex.com/2021/07/28/the-race-implicit-association-test-is-biased/). After several months, we received the expected rejection. But it was not all in vane. We received a detailed review that shows how little social psychologists really care about African Americans even when they claim to study racism and discrimination.
As peer-reviews are considered copyrighted material belonging to the reviewer, I cannot share the review in full. Rather I will highlight important sections that show how little authors with the authority of an expert reviewer pay attention to inconvenient scientific criticism of their work.
Here is the key issue. Our paper provides new evidence that the race IAT is an invalid measure of African Americans’ attitudes towards their own group and the White out-group. This new evidence is based on a reanalysis of the data that were used by Bar-Anan and Nosek (2014) to claim that the race IAT is the most valid measure to study African Americans’ implicit attitudes. Here is wat the reviewer had to say about this.
(6) It has been a while since I read the Bar-Anan and Nosek (2014) article, but my memory for it is incompatible with the claim that those authors were foolish enough to simply assume that the most valid implicit measures was the one that produced the biggest difference between Whites and Blacks in terms of in-group bias, as the present authors claim (pp. 7-8).
So, the reviewer relies on his foggy memory to question our claim instead of retrieving a pdf file and checking for himself. New York University should be proud of this display of scholarship. I hope Jost made sure to get his Publons credit. Here is the relevant section from Bar-Anan and Nosek (2014 p. 675; https://link.springer.com/article/10.3758/s13428-013-0410-6).
A lazy recollection is used to dismiss the results of a new statistical analysis. This is how closed, confidential, back-room, peer-review works, which means it does not work. It does not serve the purpose to present all scientific arguments in the open and let data decide between opposing views. Pre-publication peer-review is not a reliable and credible mechanism to advance science. For this reason, I will publish as much as possible in open-peer review journals (e..g, Meta-Psychology). Open science without open exchange of ideas and conflicts is not open, trustworthy, or credible.
Easy to agree on the questions regarding IAT. But what on earth does that mean: “It is a general observation that White researchers have speculated about African American’s self-esteem and mental states often without consulting African Americans.” So? Should we question the scientific validity of conclusions of Black researchers studying behavior of Whites without consulting them? Of rich professors, like you, who study poor participants, neurotypical scientists, like you, who study DSM diagnosed individuals, male researchers studying females, heterosexuals studying homosexuals without consulting those groups first? Scientifically skilled researchers studying science deniers, conspiracy theorists? What metascientific reasoning might be behind such claims?
Psychology is not like chemistry. Whether we like it or not, the interpretation of data is often influenced by information that goes beyond the data and scientific theory. This means interpretations can be biased by personal beliefs. Adams (2010) provides an example from research on African Americans’ self-esteem.
“For example, since the beginning of psychology, White psychologists assumed that African Americans have low self-esteem and proposed several theories for this seemingly obvious fact. However, in 1986 Rosenberg ironically pointed out that “everything stands solidly in support of this conclusion except the facts.” Since then, decades of research have shown that African Americans have the same or even higher self-esteem than White Americans (Twenge & Crocker, 2002). ”
Well, that‘s first-semester wisdom. That‘s why we have a peer-review system, right? Do you really claim that all psychological research is biased (without providing specific proof) unless representatives of the investigated sample were asked for advice? Guilty until proven innocent?