Expression of Concern

http://pss.sagepub.com/content/19/3/302.abstract
doi: 10.1111/j.1467-9280.2008.02084.x

Lawrence E. Williams and
John A. Bargh

Williams and Bargh (2008) published the article “Keeping One’s Distance: The Influence of Spatial Distance Cues on Affect and Evaluation” in Psychological Science (doi: 10.1111/j.1467-9280.2008.02084.x)

As of August, 2015, the article has been cited 98 times in Web of Science.

The article reports four studies that appear to support the claim that priming individuals with the concept of spatial distance produced “greater enjoyment of media depicting embarrassment (Study 1), less emotional distress from violent media (Study 2), lower estimates of the number of calories in unhealthy food (Study 3), and weaker reports of emotional attachments to family members and hometowns (Study 4)”

However, a closer examination of the evidence suggests that the results of these studies were obtained with the help of questionable research methods that inflate effect sizes and the strength of evidence against the null-hypothesis (priming has no effect).

The critical test in the four studies was an Analysis of Variance that compared three experimental conditions.

The critical tests were:
F(2,67) = 3.14, p = .049, z = 1.96
F(2,39) = 4.37, p = .019, z = 2.34
F(2,56) = 3.36, p = .042, z = 2.03
F(2,81) = 4.97, p = .009, z = 2.60

The p-values can be converted into z-scores (norm.inv(1 – p/2)). The z-scores of independent statistical tests should follow a normal distribution and have a variance of 1. Insufficient variation in z-scores suggests that the results of the four studies are influenced by questionable research practices.

The variance of z-scores is Var(z) = 0.08. A chi-square test against the expected variance of 1 is significant, Chi-Square(df = 3) = .26, left-tailed p = .033.
The article reports 100% significant results, but median observed power is only 59%. With an inflation of 41%, the Replicability-Index is 59-41 = 18.

An R-Index of 18 is lower than the R-Index of 22, which would be obtained if the null-hypothesis were true and only significant results are reported. Thus, after correcting for inflation, the data provide no support for the alleged effect.

It is therefore not surprising that multiple replication attempts have failed to replicate the reported results. http://www.psychfiledrawer.org/chart.php?target_article=2

In conclusion, there is no credible empirical support for the theoretical claims in Williams and Bargh (2008) and the article should not be quoted as providing evidence for these claims.