In 1998, Anthony G. Greenwald and colleagues introduced the Implicit Association Test. Since then, Implicit Association Tests have been used in thousands of studies with millions of participants to study stereotypes and attitudes. The most prominent and controversial use of the race IAT that has been used to argue that many White Americans have more negative attitudes towards African Americans than they admit to others or even to themselves.
The popularity of IATs can be attributed to the use of IATs on the Project Implicit website that provides visitors of the website with the opportunity to take an IAT and to receive feedback about their performance. Over 1 million visitors have received feedback about their performance on the race IAT (Howell, Gaither, & Ratliff, 2015).
Providing participants with performance feedback can be valuable and educational. Coaches provide feedback to athletes so that they can improve their performance, and professors provide feedback about performance during midterms so that students can improve their performance on finals. However, the value of feedback depends on the accuracy of the feedback. As psychological researchers know, providing participants with false feedback is unethical and requires extensive debriefing to justify the use of false feedback in research. it is therefore crucial to examine the accuracy of performance feedback on the race IAT.
At face value, IAT feedback is objective and reflects participants’ responses to the stimuli that were presented during an IAT. However, this performance feedback should come with a warning that performance could vary across repeated administration of a test. For example, the retest reliability of performance on the race IAT has been estimated to be between r = .2 and r = .5. Even using a value of r = .5 implies that there is only a 75% probability that somebody with a score above average receives a score above average again on a second test (Rosenthal and Rubin, 1982).
However, the Project Implicit website gives the false impression that performance on IATs is rather consistent, while avoiding quantitative information about reliability.
Unreliability is not the only reason why performance feedback on the Project Implicit website could be misleading. Another problem is that visitors may be given the impression that performance on the race IAT reveals something about themselves that goes beyond performance on this specific task. One possible interpretation of race IAT scores is that they reveal implicit attitudes or evaluations of Black and White Americans. These implicit attitudes can be different from attitudes that individuals think they have that are called explicit attitudes. In fact, Greenwald et al. (1998) introduced IATs as a method that can detect implicit attitudes that can differ from explicit attitudes and this dual-attitude model has fueled interest in IATs.
The Project Implicit website does not provide a clear explanation of what Implicit association Tests test. Regarding the race IAT, visitors are told that it is not a measure of prejudice, but that it does measure their biases, even if these biases are not endorsed or contradict conscious beliefs.
However, other frequently asked question implies that IATs measure implicit stereotypes and attitudes. One question is how IATs measure implicit attitudes, implying that it can measure implicit attitudes (and that implicit attitudes exist).
Another one implies that performance on the race IAT reveals implicit attitudes that reflect cultural biases.
In short, while Project Implicit may not provide a clear explanation of what is being tested with an Implicit Association Test, it is strongly implied that test performance reveals something about participants’ racial biases that may contradict their self-perceptions.
An article by Howell, Gaither, and Ratliff (2015) makes this assumption explicit. This article examines how visitors of the Project Implicit website respond to performance feedback on the race IAT. The key claim of this article is that “people are generally defensive in response to feedback indicating that their implicit attitudes differ from their explicit attitudes” (p. 373). This statement rests on two assumptions. First, it makes the assumption of dual-attitude models that there are explicit and implicit attitudes, as suggested by Greenwald et al. (1998). Second, it implies that performance on a single race IAT provides highly valid information about implicit attitudes. These assumptions are necessary to place researchers in the position of an expert that know individuals’ implicit attitudes, just like a psychoanalyst is in a superior position to understand the true meaning of a dream. If test takers reject the truth, they are considered defensive because they are unwilling to accept the truth.
To measure defensiveness, Howell et al. (2015) used answers to three questions after visitors of the Project Implicit website received performance feedback on the race IAT, namely
(a) the IAT does not reflect anything about my thoughts or feelings unconscious or otherwise,
(b) whether I like my IAT score or not, it captures something important about me (reversed)
(c) the IAT reflects something about my automatic thoughts and feelings concerning this topic (reversed). Responses were made on a 1 = strongly disagree to 4 = strongly agree. On this scale, a score of 2.5 would imply neither agreement nor disagreement with the aforementioned statements.
There was hardly any difference in defensiveness scores between White (M = 2.31, SD = 0.68) Black (M = 2.38, SD = 0.74) or biracial (M = 2.33, SD = 0.73) participants. For White participants, a larger pro-White discrepancy was correlated with higher defensiveness scores, partial r = .16. The same result was found for Black participants, partial r = .13. A similar trend emerged for biracial participants. While these correlations are weak, they suggest that all three racial groups were less likely to believe in the accuracy of the feedback when the IAT scores showed a stronger pro-White bias than the self-ratings implied.
Howell et al. (2015) interpret these results as evidence of defensiveness. Accordingly, “White individuals want to avoid appearing racist (O’Brien et al., 2010) and Black individuals value pro-Black bias (Sniderman & Piazza, 2002)” (p. 378). However, this interpretation of the results rests on the assumption that the race IAT is an unbiased measure of racial attitudes. Howell et al. (2015) ignore a plausible alternative explanation of their results. The alternative explanation is that performance feedback on the race IAT is biased in favor of pro-White attitudes. One source of this bias could be the scoring of IATs which relies on the assumption that neutral attitudes correspond to a zero score. This assumption has been challenged in numerous articles (e.g., Blanton, Jaccard, Strauts, Mitchell, & Tetlock, 2015). It is also noteworthy that other implicit measures of racial attitudes show different results than the race IAT (Judd et al., 1995; Schimmack & Howard, 2021). Another problem is that there is little empirical support for dual-attitude models (Schimmack, 2021). Thus, it is impossible for IAT scores to provide truthful information that is discrepant from individuals’ self-knowledge (Schimmack, 2021).
Of course, people are defensive when they are confronted with unpleasant information and inconvenient truths. A prime example of defensiveness is the response of the researchers behind Project Implicit to valid scientific criticism of their interpretation of IAT scores.
Despite several inquires about questionable or even misleading statements on the frequently asked question page, Project Implicit visitors are not informed that the wider scientific community has challenged the interpretation of performance feedback on the race IAT as valid information about individuals implicit attitudes. The simple fact that a single IAT score provides insufficient information to make valid claims about an individuals’ attitudes or behavioral tendencies is missing. Visitors should be informed that the most plausible and benign reason for a discrepancy between their test scores and their beliefs is that test scores could be biased. However, Project Implicit is unlikely to provide visitors with this information because the website is used for research purposes and willingness to participate in research might decrease when participants are told the truth about the mediocre validity of IATs.
Proponents of IATs often argue that taking an IAT can be educational. However, Howell et al. (2015) point out that even this alleged benefit is elusive because individuals are more likely to believe themselves than the race IAT feedback. Thus, rejection of IAT feedback, whether it is based on defensiveness or valid concerns about the validity of the test results, might undermine educational programs that aim to reduce actual racial biases. It is therefore problematic to use the race IAT in education and intervention programs.