Introduction

In 2015, Science published the results of the first empirical attempt to estimate the reproducibility of psychology.   One key finding was that out of 97 attempts to reproduce a significant result, only 36% of attempts succeeded.

This finding fueled debates about a replication crisis in psychology.  However, there have been few detailed examinations of individual studies to examine why a particular result could be replicated or not.  The main reason is probably that it is a daunting task to conduct detailed examinations of all studies. Another reason is that replication failures can be caused by several factors.  Each study may have a different explanation.  This means it is important to take an ideographic (study-centered) perspective.

The conclusions of these ideographic reviews will be used for a nomothetic research project that aims to predict actual replication outcomes on the basis of the statistical results reported in the original article.  These predictions will only be accurate if the replication studies were close replications of the original study.  Otherwise, differences between the original study and the replication study may explain why replication studies failed.

Summary of Original Article

The article “Nonconscious goal pursuit in novel environments: The case of implicit learning”  was published in the journal Psychological Science.

The article has been cited 41 times and is cited with low frequency in recent years.

Study 1

51 students participated in Study 1.  The experiment was a social priming study, where participants were presented with achievement words or words not related to achievement in the control condition.  In a supposedly unrelated task, participants worked on a hypothetical management task (of a sugar factory).  The authors report that achievement primes significantly enhanced performance in the managerial task, t(45) = 2.1.

Study 2 

The study chosen for the replication attempt was Study 2 with 93 participants.  Once more, achievement-primed participants showed better learning of the managerial task, t(84) = 2.09.

Replication Study 

The replication study had a larger sample size (N = 158).  Nevertheless, it failed to reproduce a significant result, t(156) = 1.32 (mean difference in the opposite direction).

Importantly, both studies reported just significant results, which is statistically unlikely for two independent samples.  Thus, the replication failure may be due to inflated effect sizes in the original studies that is produced by selection for significance.

Moreover, other social priming studies have also failed to replicate and social priming has been names “the poster child” of the replication crisis in social psychology.

Thus, the replication failure is not surprising.