Replicability Rankings of 120 Psychology Journals (2010-2019)

Uncategorized

The individual z-curve plots can be found by clicking on the journal names.

Table 1 shows the results for the expected replication rate. The values are high because the analysis includes manipulation checks and the ERR assumes exact replications. Actual replication attempts of focal hypothesis are likely to produce lower success rates. The ERR is still useful to examine differences between journals and changes over time.

Table 1 reports the individual results for the past three years, aggregated results for 2017-2019, 2014-2016, and 2010-2013. These aggregates produce more stable estimates, especially for journals with fewer test statistics. Correlations among the aggregates are r = .53 for 17-19 with 14-16, r = .65 for 14-16 with 10-13, and r = .32 for 17-19 with 10-13. The lower stability over the longer time period indicates that rank order changes are not just random measurement error.

The Change column is the difference for the average in the last three years (17-19) minus the first four years (2010-2014). The results give some indication whether replicability increased, but scores are still subject to sampling error. Individual time-trend analysis showed statistically significant changes for 13 journals. Statistical significance is indicated by adding a + and printing the change score in bold. Some journals benefit from low sampling error and can show significance with a small increase. Others show larger increases, but sampling error is too large to show a clear linear trend. As more data become available, persistent trends will become significant.

Rank   Journal 2019 2018 2017 17-19 14-16 10-13 Change
1 Journal of Religion and Health 89 75 74 79 81 79 0
2 Journal of Individual Differences 88 69 83 80 66 76 4
3 Journal of Business and Psychology 87 72 86 82 73 77 5
4 Journal of Research in Personality 87 78 78 81 78 72 +9
5 Journal of Happiness Studies 85 69 57 70 79 80 -10
6 Journal of Occupational Health Psychology 85 65 55 68 72 68 0
7 Journal of Youth and Adolescence 85 66 70 74 80 75 -1
8 Journal of Nonverbal Behavior 84 79 88 84 70 68 +16
9 Journal of Research on Adolescence 84 64 66 71 67 73 -2
10 Cognitive Psychology 83 82 72 79 76 76 3
11 Evolution & Human Behavior 83 78 73 78 76 64 +14
12 Psychology of Men and Masculinity 83 69 57 70 69 82 -12
13 Developmental Psychology 82 78 76 79 74 68 +11
14 Evolutionary Psychology 82 78 78 79 78 75 4
15 Experimental Psychology 82 78 74 78 73 72 6
16 Journal of Anxiety Disorders 82 77 78 79 73 75 4
17 Psychological Science 82 76 71 76 67 63 +13
18 Attention, Perception and Psychophysics 81 80 79 80 73 76 4
19 Cognition 81 77 76 78 73 74 4
20 Journal of Behavioral Decision Making 81 76 70 76 75 68 8
21 Journal of Organizational Psychology 81 68 73 74 71 69 5
22 Aggressive Behavior 80 78 72 77 71 68 9
23 Cognitive Development 80 73 79 77 75 70 7
24 Consciousness and Cognition 80 78 77 78 70 70 8
25 Depression & Anxiety 80 82 75 79 74 84 -5
26 European Journal of Personality 80 73 76 76 76 72 4
27 Judgment and Decision Making 80 72 81 78 77 72 6
28 Memory and Cognition 80 80 79 80 76 77 3
29 Psychology and Aging 80 72 79 77 79 74 3
30 Psychonomic Bulletin and Review 80 79 75 78 79 76 2
31 Journal of Applied Psychology 79 67 78 75 74 70 5
32 Journal of Cross-Cultural Psychology 79 78 76 78 78 76 2
33 Journal of Experimental Psychology – General 79 78 78 78 74 70 +8
34 Journal of Memory and Language 79 78 80 79 78 75 4
35 Journal of Occupational and Organizational Psychology 79 82 71 77 76 72 5
36 Journal of Positive Psychology 79 71 82 77 71 68 9
37 Journal of Sex Research 79 80 81 80 78 80 0
38 Journal of Social Psychology 79 79 75 78 70 72 6
39 Law and Human Behavior 79 80 76 78 68 76 2
40 Personality and Individual Differences 79 76 76 77 77 72 +5
41 Perception 79 73 76 76 76 86 -10
42 Acta Psychologica 78 71 77 75 75 74 1
43 Asian Journal of Social Psychology 78 80 68 75 74 70 5
44 Journal of Child and Family Studies 78 75 73 75 70 73 2
45 Journal of Counseling Psychology 78 85 69 77 76 75 2
46 J. of Exp. Psychology – Learning, Memory & Cognition 78 78 79 78 78 76 2
47 Journal of Experimental Social Psychology 78 75 71 75 64 56 +19
48 Memory 78 74 74 75 78 81 -6
49 British Journal of Social Psychology 77 70 75 74 63 63 11
50 Cognitive Therapy and Research 77 74 75 75 67 71 4
51 European Journal of Social Psychology 77 64 71 71 70 64 7
52 Social Psychological and Personality Science 77 80 75 77 63 59 +18
53 Archives of Sexual Behavior 76 72 78 75 78 80 -5
54 Emotion 76 73 73 74 70 70 4
55 Journal of Affective Disorders 76 75 75 75 82 77 -2
56 J. of Exp. Psychology – Human Perception and Performance 76 78 76 77 76 76 1
57 Journal of Pain 76 79 68 74 77 72 2
58 Personal Relationships 76 83 76 78 69 64 +14
59 Psychology of Religion and Spirituality 76 82 69 76 79 69 7
60 Appetite 75 76 77 76 67 72 4
61 Group Processes & Intergroup Relations 75 65 70 70 68 65 5
62 Journal of Cognition and Development 75 82 74 77 70 65 12
63 Journal of Cognitive Psychology 75 79 75 76 76 77 -1
64 Journal of Experimental Psychology – Applied 75 75 80 77 70 70 7
65 JPSP-Personality Processes and Individual Differences 75 79 64 73 72 66 7
66 Political Psychology 75 88 76 80 74 63 17
67 Psychopharmacology 75 68 73 72 74 72 0
68 Psychophysiology 75 79 78 77 74 74 3
69 Quarterly Journal of Experimental Psychology 75 79 76 77 75 74 3
70 Animal Behavior 74 71 77 74 70 72 2
71 Behaviour Research and Therapy 74 75 70 73 74 71 2
72 British Journal of Developmental Psychology 74 77 72 74 72 76 -2
73 Frontiers in Psychology 74 74 76 75 74 72 3
74 Journal of Abnormal Psychology 74 71 69 71 64 70 1
75 Journal of Applied Social Psychology 74 75 73 74 73 72 2
76 Journal of Consumer Behaviour 74 81 72 76 76 79 -3
77 Journal of Health Psychology 74 81 63 73 75 72 1
78 JPSP-Attitudes & Social Cognition 74 80 79 78 66 58 20
79 Journal of Social and Personal Relationships 74 74 71 73 67 73 0
80 Psychology and Marketing 74 68 70 71 70 68 3
81 Behavioral Neuroscience 73 66 73 71 69 70 1
82 Canadian Journal of Experimental Psychology 73 87 73 78 77 76 2
83 Cognition and Emotion 73 75 66 71 72 82 -11
84 European Journal of Developmental Psychology 73 90 85 83 75 71 12
85 Journal of Child Psychology and Psychiatry and Allied Disciplines 73 62 68 68 66 64 4
86 Journal of Educational Psychology 73 73 79 75 71 77 -2
87 Organizational Behavior and Human Decision Processes 73 70 68 70 70 69 1
88 Psychological Medicine 73 76 73 74 75 73 1
89 Sex Roles 73 83 81 79 76 75 4
90 Social Psychology 73 83 73 76 73 71 5
91 Behavioural Brain Research 72 69 71 71 70 72 -1
92 British Journal of Psychology 72 79 76 76 79 74 2
93 Developmental Science 71 67 73 70 67 69 1
94 Journal of Personality 71 79 77 76 70 67 9
95 Behavior Therapy 70 67 71 69 71 72 -3
96 Child Development 70 73 66 70 71 72 -2
97 International Journal of Psychophysiology 70 71 73 71 67 68 3
98 Journal of Consulting and Clinical Psychology 70 71 77 73 64 65 8
99 Journal of Experimental Child Psychology 70 73 71 71 75 73 -2
100 Motivation and Emotion 70 62 73 68 65 70 -2
101 Frontiers in Behavioral Neuroscience 69 72 74 72 70 70 2
102 Journal of Comparative Psychology 69 68 67 68 75 70 -2
103 JPSP-Interpersonal Relationships and Group Processes 69 72 68 70 67 58 +12
104 Frontiers in Human Neuroscience 68 73 71 71 74 75 -4
105 Journal of Consumer Research 68 68 65 67 60 59 8
106 Journal of Family Psychology 68 76 70 71 68 66 5
107 Journal of Vocational Behavior 68 83 75 75 78 80 -5
108 Personality and Social Psychology Bulletin 68 71 74 71 65 61 +10
109 Self and Identity 68 60 67 65 66 72 -7
110 Hormones & Behavior 66 69 61 65 63 63 2
111 Psychoneuroendocrinology 66 68 65 66 64 62 +4
112 Annals of Behavioral Medicine 65 74 70 70 69 74 -4
113 Biological Psychology 65 68 63 65 68 66 -1
114 Cognitive Behavioral Therapy 65 67 75 69 73 71 -2
115 Health Psychology 63 77 70 70 64 65 5
116 Infancy 62 58 61 60 62 64 -4
117 Journal of Consumer Psychology 62 66 57 62 62 62 0
118 Psychology of Music 62 74 81 72 74 79 -7
119 Developmental Psychobiology 60 66 63 63 67 69 -6
120 Social Development 55 80 73 69 73 72 -3

9 thoughts on “Replicability Rankings of 120 Psychology Journals (2010-2019)”

  1. Thank you very much for this very interesting analysis. However, I cannot find a comprehensible explanation how the expected replication rate was calculated. The link to the manuscript “the lost decades…” is not really helpful because the text does not explain how to read and understand the z-curve-graphs and the embedded indices. Therefore, it would be of invaluable help if you can provide such an explanation of the graphs and the calcuations of the indices.
    Thank you very much in advance.
    Best regards,
    Manfred Holodynski

    Reply
  3. Pingback: Research Opportunity Program 2020: Preliminary Results | Replicability-Index
  4. Pingback: “Psychological Science” in 2020 | Replicability-Index
  5. Pingback: Deceptive Research Practices in Relationship Psychology: Illustrating Epistemological and Pragmatic Considerations With the Case of Relationship Science | Replicability-Index
  6. Pingback: Replicability Rankings 2010-2020 | Replicability-Index
  7. Pingback: What makes science credible? Psychology edition.

Leave a Reply