The Implicit Association Test (IAT) is the most widely used indirect measure of attitudes, including attitudes towards groups (European Americans (Caucasians), African Americans, Asian Americans, etc.).

Psychologists disagree about the ability of the IAT to measure individuals’ racial attitudes (Singal, 2017). Thus, it provides another opportunity to examine Gilovich, Keltner, Chen, and Nisbett’s (2019) promise to be “scrupulous about noting when the evidence about a given point is mixed” (p. 57)

The IAT is introduced in Chapter 11 on stereotypes, prejudice, and discrimination. Students are told that the IAT is a technique for “revealing, subtle, nonconscious prejudices, even among those who sincerely believe they are bias-free” (p. 365). This statement makes it clear that the authors consider the IAT a valid measure of unconscious attitudes, that is evaluations outside of individuals’ awareness. Somebody may think that they are prejudice, but they are actually not; or vice versa. Students are encouraged to visit the IAT website to see whether they “hold any implicit stereotypes or prejudice towards a variety of groups” (p. 367).

The difference between response latencies in the critical IAT conditions is presented as a “nonconscious attitude” (p. 367).

Students are informed that

“both young and older individuals show a pronounced prejudice in favor of the young over the old” (p. 368)

“and about two-thirds of white respondents show a strong to moderate prejudice for white over black” (p. 368)

Criticism of the IAT is countered with a claim that the race IAT shows convergent validity with other indirect measures of prejudice.

Although the test has its critics (Blanton & Jaccard, 2008; Oswald, Mitchell, Blanton, Jaccard, & Tetlock, 2013; Roddy, Stewart, & Barnes-Holmes, 2010), there’s evidence that IAT responses correlate with other indirect measures of prejudice (Lane, Banaji, Nosek, & Greenwald, 2007; Rudman & Ashmore, 2007). (p. 368)

The authors also mention a brain-imaging study that showed correlations between IAT scores and some correlates in the brain (Phelps et al. 2000).

The authers then discuss the importance of predictive validity. They cite a single study as evidence that the IAT has predictive validity (McConnell & Leibold, 2009).

In conclusion, the IAT is introduced as a measure of unconscious prejudice that people cannot know about without taking the IAT. The test is presented as having convergent validity with other indirect measures and as being predictive of actual behavior.

This presentation of the IAT is one-sided and misleading. A major problem of social psychology textbooks is to present evidence without quantifying effect sizes. Surely, it matters whether the IAT correlates with other implicit measures with r = .1, r = .3, r = .5, or r = .7. Even a replicable correlation of r = .1 with other indirect measures would not justify the conclusion that the IAT has good convergent validity. Validity is typically assessed in comparison to a standard of r = .7 as a minimum for acceptable validity if a test is used for personality assessment.

The same holds for predictive validity. A test that explains only 1% or 10% of the variance in actual behavior may be scientifically interesting, but for individuals who are given feedback on an online test of their hidden prejudices, it would be important to know that most of their actual behaviors is influenced by other factors.

The textbook does mention criticism of the IAT, but it does not mention what the criticism is. The key criticisms are that the IAT has low convergent validity, low predictive validity, and no evidence of discriminant validity; that is, there is no evidence that it measures some nonconsicous prejudices “even among those who sincerely believe they are bias-free” (p. 368).

Lane, Banaji, Nosek and Greenwald (2007)

The key claim about converent validity is based on Lane, Banaji, Nosek, and Greenwald’s chapter, which includes a section on “Convergent Validity: The Relationship of the IAT to Other Implicit Measures”

The first article mentioned is Bosson et al.’s (2000) study of the IAT as a measure of implicit self-esteem.

“Although the IAT was uncorrelated with the other implicit measures,
it was not alone in failing to converge” (Lane et al., 2007, p. 72).

The next set of studies examined convergent validity between the IAT and priming tasks and found no significant correlation.

“IATs and priming tasks measuring implicit attitudes toward smoking (Sherman, Rose, Koch, Presson, & Chassin, 2003) and condom use (Marsh, Johnson, & Scott-Sheldon, 2001) were unrelated” (Lane et al., 2007, p. 72).

Fazio and Olson (2003) reported that across four studies in their lab,
priming tasks and IATs measuring racial attitudes did not correlate (p. 73).

Other studies found statistically significant correlations, but the chapter doesn’t mention the magnitude of these correlations. Moreover, they did not examine the convergent validity of the IAT as a measure of prejudice.

Stereotypes about gender authority (associations between female and low-status jobs), as measured by the IAT, were correlated with three indices of attitudes toward female authorities derived from the priming task (Rudman & Kilianski, 2000).  (Lane et al., 2007, p. 73). 

Participants who tended to show strong implicit gender stereotypes (on the IAT) also showed more positive attitudes toward women on the priming measure, and political attitudes measured by a priming task and the IAT were reliably related (Nosek & Hansen, 2004). (Lane et al., 2007, p. 73). 

The chapter than attributes low correlations to low reliability of implicit measures.

These findings suggest that when reliability is accounted for, implicit measures are more likely to be related.

This is important because students of the textbook are encouraged to receive feedback about their nonconscious prejudice based on a single visit to the IAT website. However, a single IAT measure has low reliability and as a result low validity. These results should not be considered measures of prejudice, conscious or unconscious.

The chapter concludes with the hope that future research will shed more light on the low convergent validity of implicit prejudice measures. However, there has been little research on this fundamental question and the 2019 textbook draws on the 2007 chapter to support the claim that the IAT is a valid measure of nonconscious prejudice.

Students could just trust the textbook authors, but fact checking is better. As it turns out, the chapter provides no evidence to support the claim that a visit to the IAT website can reveal some hidden secrets about their prejudices.

PREDICTIVE VALIDITY

The textbook cites Rudman and Ashmore’s 2007 article as evidence for convergent validity, but they fail to note that the study also examined and failed to find evidence for predictive validity.

Table 3 shows the correlations of the IAT with three behavioral measures. Importantly, the typical IAT is the Attitude IAT, while the Stereotype IAT was uniquely designed for this study. The Attitude IAT shows only one significant correlation with verbal behavior and two non-significant correlations with defensive and offensive behaviors (Table 3). Moreover, in a regression analysis the Attitude IAT did not explain unique variance over the other attitude measures.

Instead, they present a study by McConnell and Leibold from 2001 with a sample size of N = 42 participants. The results are shown in Table 3 below.

It is easy to find other studies that failed to replicate this finding. For example, a more recent study, Axt, Ebersole, and Nosek (2016) found that IAT was not a significant predictor of discrimination in a hypothetical selection task. After the IAT failed in Study 1, it was replaced with a different, although similar task in the following studies. This shows low trust in the IAT by one of the developers of the online IAT. Moreover, the set of studies showed that White participants showed an overall pro-Black bias in their behaviors. This finding raises doubts about the claim that the most White Americans behavior is influenced by nonconscious prejudice even if they try not to be prejudice.

CONCLUSION

In conclusion, the textbook does not provide an accurate and balanced account of the controversies surrounding the IAT. It repeats the controversial claim that the IAT is a valid measure of some hidden prejudices that can only be revealed by the IAT. This is a strong claim that is not supported by evidence. Students deserve a better textbook, but I am not aware of a social psychology textbook that is more scientific in the introduction of the IAT.