The first measures of the Big Five used a large number of items to measure personality. This made it difficult to include personality measures in studies as the assessment of personality would take up all of the survey time. Over time, shorter scales became available. One important short Big Five measure is the BFI-S (Lang et al., 2011). This 15-item measure has been used in several national representative, longitudinal studies such as the German Socio-Economic Panel (Schimmack, 2019a). These results provide unique insights into the stability of personality (Schimmack, 2019b) and the relationship of personality with other constructs such as life-satisfaction (Schimmack, 2019c). Some of these results overturn textbook claims about personality. However, critics argue that these results cannot be trusted because the BFI-S is an invalid measure of personality.
Thus, it is is critical importance to evaluate the validity of the BFI-S. Here I use Gosling and colleagues data to examine the validity of the BFI-S. Previously, I fitted a measurement model to the full 44-item BFI (Schimmack, 2019d). It is straightforward to evaluate the validity of the BFI-S by examining the correlation of the 3-item BFI-S scale scores with the latent factors based on all 44 BFI items. For comparison purposes, I also show the correlations for the BFI scale scores. The complete results for individual items are shown in the previous blog post (Schimmack, 2019d).
The measurement model for the BFS has seven independent factors. Five factors represent the Big Five and two factors represent method factors. One factor represents acquiescence bias. The other factor represents evaluative bias that is present in all self-ratings of personality (Anusic et al., 2009). As all factors are independent, the squared coefficients can be interpreted as the amount of variance that a factor explains in a scale score.
The results show that the BFI-S scales are nearly as valid as the longer BFI scales (Table 1).
Scale
#Items
N
E
O
A
C
EVB
ACQ
N-BFI
8
0.79
-0.08
-0.01
-0.05
-0.02
-0.42
0.05
N-BFI-S
3
0.77
-0.13
-0.05
0.07
-0.04
-0.29
0.07
E-BFI
8
-0.02
0.83
0.04
-0.05
0.00
0.44
0.06
E-BFI-S
3
0.05
0.82
0.00
0.04
-0.07
0.32
0.07
O-BFI
10
0.04
-0.03
0.76
-0.04
-0.05
0.36
0.19
O-BFI-S
3
0.09
0.00
0.66
-0.04
-0.10
0.32
0.25
A-BFI
9
-0.07
0.00
-0.07
0.78
0.03
0.44
0.04
A-BFI-S
3
-0.03
-0.06
0.00
0.75
0.00
0.33
0.09
C-BFI
9
-0.05
0.00
-0.05
0.04
0.82
0.42
0.03
C-BFI-S
3
-0.09
0.00
-0.02
0.00
0.75
0.44
0.06
For example, the factor-scale correlations for neuroticism, extraversion, and agreeableness are nearly identical. The biggest difference was observed for openness with a correlation of r = .76 for the BFI-scale and r = .66 for the BFI-S scale. The only other notable systematic variance in scales is the evaluative bias influence which tends to be stronger for the longer scales with the exception of conscientiousness. In the future, measurement models with an evaluative bias factor can be used to select items with low loadings on the evaluative bias factor to reduce the influence of this bias on scale scores. Given these results, one would expect that the BFI and BFI-S produce similar results. The next analyses tested this prediction.
Gender Differences
I examined gender differences three ways. First, I examined standardized mean differences at the level of latent factors in a model with scalar invariance (Schimmack, 2019d). Second, I computed standardized mean differences with the BFI scales. Finally, I computed standardized mean differences with the BFI-S scales. Table 2 shows the results. Results for the BFI and BFI-S scales are very similar. The latent mean differences show somewhat larger differences for neuroticism and agreeablness because these mean differences are not attenuated by random measurement error. The latent means also show very small gender differences for the method factors. Thus, mean differences based on scale scores are not biased by method variance.
Table 2. Standardized Mean Differences between Men and Women
N
E
O
A
C
EVB
ACQ
Factor
0.64
0.17
-0.18
0.31
0.15
0.09
0.16
BFI
0.45
0.14
-0.10
0.20
0.14
–
–
BFI-S
0.48
0.21
-0.03
0.18
0.12
–
–
Note. Positive values indicate higher means for women than for men.
In short, there is no evidence that using 3-item scales invalidates the study of gender differences.
Age Differences
I demonstrated measurement invariance for different age groups (Schimmack, 2019d). Thus, I used simple correlations to examine the relationship between age and the Big Five. I restricted the age range from 17 to 70. Analyses of the full dataset suggest that older respondents have higher levels of conscientiousness and agreeableness (Soto, John, Gosling, & Potter, 2011).
Table 3 shows the results. The BFI and the BFI-S both show the predicted positive relationship with conscientiousness and the effect size is practically identical. The effect size for the latent variable model is stronger because the relationship is not attenuated by random measurement error. Other relationships are weaker and also consistent across measures except for Openness. The latent variable model reveals the reason for the discrepancies. Three items (#15 ingenious, #l35 like routine work, and #10 sophisticated in art) showed unique relationships with age. The art-related items showed a unique relationship with age. The latent factor does not include the unique content of these items and shows a positive relationship between openness and age. The scale scores include this content and show a weaker relationship. The positive relationship of openness with age for the latent factor is rather surprising as it is not found in nationally representative samples (Schimmack, 2019b). One possible explanation for this relationship is that older individuals who take an online personality test are more open.
N
E
O
A
C
EVB
ACQ
Factor
-0.08
-0.02
0.18
0.12
0.33
0.01
-0.11
BFI
-0.08
-0.01
0.08
0.09
0.26
–
–
BFI-S
-0.08
-0.04
-0.02
0.08
0.25
–
–
In sum, the most important finding is that the 3-item BFI-S conscientiousness scale shows the same relationship with age as the BFI-scale and the latent factor. Thus, the failure to find aging effects in the longitudinal SOEP data with the BFI-S cannot be attributed to the use of an invalid short measure of conscientiousness. The real scientific question is why the cross-sectional study by Soto et al. (2011) and my analysis of the longitudinal SOEP data show divergent results.
Conclusion
Science has changed since researchers are able to communicate and discuss research findings on social media. I strongly believe that open science outside of peer-controlled journals is beneficial for the advancement of science. However, the downside of social media of open science is that it becomes more difficult to evaluate expertise of online commentators. True experts are able to back up their claims with scientific evidence. This is what I did here. I showed that Brenton Wiernik’s comment has as much scientific validity as a Donald Trump tweet. Whatever the reason for the lack of personality change in the SOEP data will be, it is not the use of the BFI-S to measure the Big Five.
In one of the worst psychometric articles every published (although the authors still have a chance to retract their in press article before it is actually published), Hussey and Hughes argue that personality psychologists intentionally fail to test the validity of personality measures. They call this practice validity-hacking. They also conduct some psychometric tests of popular personality measures and claim that they fail to demonstrate structural validity.
I have demonstrated that this claim is blatantly false and that the authors failed to conduct a proper test of structural validity (Schimmack, 2019a). That is, the authors fitted a model to the data that is known to be false. Not surprisingly, they found that their model didn’t meet standard criteria of model fit. This is exactly what should happen when a false model is subjected to a test of structural validity. Bad models should not fit the data. However, a real test of structural validity requires fitting a plausible model to the data. I already demonstrated with several Big Five measures that these measures have good structural validity and that scale scores can be used as reasonable measures of the latent constructs (Schimmack, 2019b). Here I examine the structural validity of the Big Five Inventory (Oliver John) that was used by Hussay and Hughes.
While I am still waiting to receive the actual data that were used by Hussay and Hughes, I obtained a much larger and better dataset from Sam Gosling that includes data from 1 million visitors to a website that provides personality feedback (https://www.outofservice.com/bigfive/).
For the present analyses I focused on the subgroup of Canadian visitors with complete data (N = 340,000). Subsequent analyses can examine measurement invariance with the US sample and samples from other nations. To examine the structure of the BFI, I fitted a structural equation model. The model has seven factors. Five factors represent the Big Five personality traits. The other two factores represent rating biases. One bias is an evaluative bias and the other bias is acquiescence bias. Initially, loadings on the method factors were fixed. This basic model was then modified in three ways. First, item loadings on the evaluative bias factor were relaxed to allow for some items to show more or less evaluative bias. Second, secondary loadings were added to allow for some items to be influenced by more than one factor. Finally, items of the same construct were allowed to covary to allow for similar wording or shared meaning (e.g., three arts items from the openness factor were allowed to covary). The final model and the complete results can be found on OSF (https://osf.io/23k8v/).
Model fit was acceptable, CFI = .953, RMSEA = .030, SRMR = .032. In contrast, fitting a simple structure without method factors produced unacceptable fit for all three fit indices, CFI = .734, RMSEA = .068, SRMR = .110. This shows that the model specification by Hussey and Hughes accounted for the bad fit. It has been known for over 20 years that a simple structure does not fit Big Five data (McCrae et al., 1996). Thus, Hussay and Hughes claim that the BFI lacks validity is based on an outdated and implausible measurement model.
Table 1 shows the factor loading pattern for the 44 BFI items on the Big Five factors and the two method factors. It also shows the contribution of the seven factors to the scale scores that are used to provide visitors with personality feedback and in many research articles that use scale scores as proxies for the latent constructs.
Item#
N
E
O
A
C
EVB
ACQ
Neuroticism
depressed/blue
4
0.33
-0.15
0.20
-0.48
0.06
relaxed
9
-0.72
0.23
0.18
tense
14
0.51
-0.25
0.20
worry
19
0.60
-0.08
0.07
-0.21
0.17
emotionally stable
24
-0.61
0.27
0.18
moody
29
0.43
-0.33
0.18
calm
34
-0.58
-0.04
-0.14
-0.12
0.25
0.20
nervous
39
0.52
-0.25
0.17
SUM
0.79
-0.08
-0.01
-0.05
-0.02
0.42
0.05
Extraversion
talkative
1
0.13
0.70
-0.07
0.23
0.18
reserved
6
-0.58
0.09
-0.21
0.18
full of energy
11
0.34
-0.11
0.58
0.20
generate enthusiasm
16
0.07
0.44
0.11
0.50
0.20
quiet
21
-0.81
0.04
-0.21
0.17
assertive
26
-0.09
0.40
0.14
-0.24
0.18
0.24
0.19
shy and inhibited
31
0.18
0.64
-0.22
0.17
outgoing
36
0.72
0.09
0.35
0.18
SUM
-0.02
0.83
0.04
-0.05
0.00
0.44
0.06
Openness
original
5
0.53
-0.11
0.38
0.21
curious
10
0.41
-0.07
0.31
0.24
ingenious
15
0.57
0.09
0.21
active imagination
20
0.13
0.53
-0.17
0.27
0.21
inventive
25
-0.09
0.54
-0.10
0.34
0.20
value art
30
0.12
0.46
0.09
0.16
0.18
like routine work
35
-0.28
0.10
0.13
-0.21
0.17
like reflecting
40
-0.08
0.58
0.27
0.21
few artistic interests
41
-0.26
-0.09
0.15
sophisticated in art
44
0.07
0.44
-0.06
0.10
0.16
SUM
0.04
-0.03
0.76
-0.04
-0.05
0.36
0.19
Agreeableness
find faults w. others
2
0.15
-0.42
-0.24
0.19
helpful / unselfish
7
0.44
0.10
0.29
0.23
start quarrels
12
0.13
0.20
-0.50
-0.09
-0.24
0.19
forgiving
17
0.47
-0.14
0.24
0.19
trusting
22
0.15
0.33
0.26
0.20
cold and aloof
27
-0.19
0.14
-0.46
-0.35
0.17
considerate and kind
32
0.04
0.62
0.29
0.23
rude
37
0.09
0.12
-0.63
-0.13
-0.23
0.18
like to cooperate
42
0.15
-0.10
0.44
0.28
0.22
SUM
-0.07
0.00
-0.07
0.78
0.03
0.44
0.04
Conscientiousness
thorough job
3
0.59
0.28
0.22
careless
8
-0.17
-0.51
-0.23
0.18
reliable worker
13
-0.09
0.09
0.55
0.30
0.24
disorganized
18
0.15
-0.59
-0.20
0.16
lazy
23
-0.52
-0.45
0.17
persevere until finished
28
0.56
0.26
0.20
efficient
33
-0.09
0.56
0.30
0.23
follow plans
38
0.10
-0.06
0.46
0.26
0.20
easily distracted
43
0.19
0.09
-0.52
-0.22
0.17
SUM
-0.05
0.00
-0.05
0.04
0.82
0.42
0.03
Most of the secondary loadings are very small, although they are statistically highly significant in this large sample. Most items also have the highest loading on the primary factor. Exceptions are the items blue/depressed, full of engery, and generate enthusiasm that have higher loadings on the evaluative bias factor. Except for two openness items, all items also have loadings greater than .3 on the primary factor. Thus, the loadings are consistent with the intended factor structure.
The most important results are the loadings of the scale scores on the latent factors. As the factors are all independent, squaring these coefficients shows the amount of explained variance by each factor. By far the largest variance component is the intended construct with correlations ranging from .76 for openness to .83 for extraversion. Thus, the lion share of the reliable variance in scale scores reflects the intended construct. The next biggest contributor is evaluative bias with correlations ranging from .36 for openness to .44 for extraversion. Although this means only 15 to 20 percent of the total variance in scale scores reflects evaluative bias, this systematic variance can produce spurious correlations when scale scores are used to predict other self-report measures (e.g., life satisfaction, Schimmack, 2019c).
In sum, a careful psychometric evaluation of the BFI shows that the BFI has good structural validity. The key problem is the presence of evaluative bias in scale scores. Although this requires caution in the interpretation of results obtained with BFI scales, it doesn’t justify the conclusion that the BFI is invalid.
Measurement Invariance
Hussey and Hughes also examined measurement invariance across age-groups and the two largest gender groups. They claimed that the BFI lacks measurement invariance, but this claim was based on a cunning misrepresentation of the results (Schimmack, 2019a). The claim is based on the fact that the simple-structure model does not fit in any group. However, fit did not decrease when measurement invariance was imposed on different groups. Thus, all groups showed the same structure and fit did not increase when measurement invariance was imposed, but this fact was hidden in the supplementary results.
I replicated their analyses with the current dataset. First, I fitted the model for the whole sample separately to the male and female samples. Fit for the male sample was acceptable, CFI = .949, RMSEA = .029, SRMR = .033. So was fit for the female sample, CFI = .947, RMSEA = .030, SRMR = .037.
Table 2 shows the results side by side. There are no notable differences between the parameter estimates for males and females (m/f). This finding replicates results with other Big Five measures (Schimmack, 2019a).
Item#
N
E
O
A
C
EVB
ACQ
Neuroticism
depressed/blue
4
.33/ .30
-.18/-.11
.19/ .20
-.45/-.50
.07/.05
relaxed
9
-.71/-.72
.24/ .23
.19/.18
tense
14
.52/ .49
-.17/-.14
.11/ .13
-.27/-.32
.20/ .20
worry
19
.58/ .57
-.10/-.08
.05/ .07
-.22/-.22
.17/ .17
emotionally stable
24
-.58/-.58
.10/ .06
.25/ .30
.19/ .17
moody
29
.41/ .38
-.26/-.25
-.30/-.38
.18/ .18
calm
34
-.55/-.59
-.02/-.03
.14/ .13
.12/ .13
-.27/-.24
.21/ .19
nervous
39
.51/ .49
-.21/.26
-.10/-.10
.08/ .08
-.11/-.11
-.27/-.25
.18/ .17
SUM
.78/ .77
-.09/-.08
-.01/-.01
-.07/-.05
-.02/-.02
-.42-.46
.05/ .04
Extraversion
talkative
1
.09/ .11
.69/ .70
-.10/-.08
.24/ .24
.19/ .18
reserved
6
-.55/-.60
.08/.10
.21/ .22
.19/ .18
full of energy
11
.33/ .32
-.09/-.04
.56/ .59
.21/ .20
generate enthusiasm
16
.04/ .03
.44/ .43
.12/ .13
.48/ .50
.20/ .20
quiet
21
-.79/-.82
.03/ .04
-.22/-.21
.17/ .16
assertive
26
-.08/-.10
.39/ .40
.12/ .14
-.23/-.25
.18/ .17
.26/ .24
.20/ .18
shy and inhibited
31
.19/ .15
.61/ .66
.23/ .22
.18/ .17
outgoing
36
.71/ .71
.10/ .07
.35/ .38
.18/ .18
SUM
-.02/-.02
.82/ .82
.04/ .05
-.04-.06
.00/ .00
.45/ .44
.07/ .06
Openness
original
5
.50/ .54
-.12/-.12
.40/ .39
.22/ .20
curious
10
.40/ .42
-.05/-.08
.32/ .30
.25/ .23
ingenious
15
0.00/0.00
.60/ .56
.18/ .16
.10/ .04
.22/ .20
active imagination
20
.50/ .55
-.07/-.06
-.17/-.18
.29/ .26
.23/ .21
inventive
25
-.07/ -.08
.51/ .55
-.12/-.10
.37/ .34
.21/ .19
value art
30
.10/ .03
.43/ .52
.08/ .07
.17/ .14
.18/ .19
like routine work
35
-.27/-.27
.10/ .10
.09/ .15
-.22/-.21
.17/ .16
like reflecting
40
-.09/-.08
.58/ .58
.28/ .26
.22/ .20
few artistic interests
41
-.25/-.29
-.10/-.09
.16/ .15
sophisticated in art
44
.03/ .00
.42/ .49
-.08/-.08
.09/ .09
.16/ .16
SUM
.01/ -.01
-.01/-.01
.74/ .78
-.05/-.05
-.03/-.06
.38/ .34
.20/ .19
Agreeableness
find faults w. others
2
.14/ .17
-.42/-.42
-.24/-.24
.19/ .19
helpful / unselfish
7
.45/ .43
.09/.11
.29/ .29
.23/ .23
start quarrels
12
.12/ .16
.23/ .18
-.49/-.49
-.07/-.08
-.24/-.24
.19/ .19
forgiving
17
.49/ .46
-.14/-.13
.25/ .24
.20/ .19
trusting
22
-.14/-.16
.38/ .32
.27/ .25
.21/ .19
cold and aloof
27
-.20/-.18
.14/ .12
.44/ .46
-.34/-.37
.18/ .17
considerate and kind
32
.02/.01
.62/.61
.28/ .30
.22/ .23
rude
37
.10/ .12
.12/ .12
-.62/-.62
-.13/-.08
-.23/-.23
.19/ .18
like to cooperate
42
.18/ .11
-.09/-.10
.43/ .45
.28/ .29
.23/ .22
SUM
-.07/-.08
.00/ .00
-.07/-.07
.78/ .77
.03/ .03
.43/ .44
.04/ .04
Conscientiousness
thorough job
3
.58/ .59
.29/ .28
.23/ .22
careless
8
-0.16
-.49/-.51
.24/ .23
.19/ .18
reliable worker
13
-.10/-.09
.09/ .10
.55/ .55
.30/ .31
.24/ .24
disorganized
18
.13/ .16
-.58/-.59
-.21/-.20
.17/ .15
lazy
23
-.52/-.51
-.45/-.45
.18/ .17
persevere until finished
28
.54/ .58
.27/ .25
.21/ .19
efficient
33
-.11/-.07
.52/ .58
.30/ .29
.24/ .23
follow plans
38
.00/ .00
-.06/-.07
.45/ .44
.27/ .26
.21/ .20
easily distracted
43
.17/ .19
.07/ .06
-.53/-.53
-.22/-.22
.18/ .17
SUM
-.05/-.05
-.01/-.01
-.05/-.06
.04/ .04
.81/ .82
.43/ .41
.03/ .03
I then fitted a multi-group model with metric invariance. Despite the high similarity between the individual models, model fit decreased, CFI = .925, RMSEA = .033, SRMR = .062. Although RMSEA and SRMR were still good, the decrease in fit might be considered evidence that the invariance assumption is violated. Table 2 shows that it is insufficient to examine changes in global fit indices. What matters is whether the decrease in fit has any substantial meaning. Given the results in Table 2, this is not the case.
The next model imposed scalar invariance. Before presenting the results, it is helpful to know what scalar invariance implies. Take extraversion as an example. Assume that there are no notable gender differences in extraversion. However, extraversion has multiple facets that are represented by items in the BFI. One facet is assertiveness and the BFI includes an assertiveness item. Scalar invariance implies that there cannot be gender differences in assertiveness if there are no gender differences in extraversion. It is obvious that this is an odd assumption because gender differences can occur at any level in the hierarchy of personality traits. Thus, evidence that scalar invariance is violated does not imply that we can not examine gender differences in personality. Rather, it would require further examination of the pattern of mean differences at the level of the factors and the item residuals.
However, imposing scalar invariance did not produce a decrease in fit, CFI = .921, RMSEA = .034, SRMR = .063. Inspection of the modification indices showed the highest modification index for item O6 “valuing art” with an implied mean difference of 0.058. This implies that there are no notable gender differences at the item-level. The pattern of mean differences at the factor level is consistent with previous studies, showing higher levels of neuroticism (d = .64) and agreeableness (d = .31), although the difference in agreeableness is relatively small compared to some other studies.
In sum, the results show that the BFI can be used to examine gender differences in personality and that the pattern of gender differences observed with the BFI is not a measurement artifact.
Age Differences
Hussey and Hughes used a median split to examine invariance across age-groups. The problem with a median split is that online samples tend to be very young. Figure 1 shows the age distribution for the Canadian sample. The median age is 22.
To create two age-groups, I split the sample into a group of under 30 and 30+ participants. The unequal sample size is not a problem because both groups are large given the large overall sample size (young N = 221,801, old N = 88,713). A published article examined age differences in the full sample, but the article did not use SEM to test measurement invariance (Soto, John, Gosling, & Potter, 2011). Given the cross-sectional nature of the data, it is not clear whether age differences are cohort differences or aging effects. Longitudinal studies suggest that age differences may reflect generational changes rather than longitudinal changes over time (Schimmack, 2019d). In any case, the main point of the present analyses is to examine measurement invariance across different age groups.
Fit for the model with metric invariance was similar to the fit for the gender model, CFI = .927, RMSEA = .033, SRMR = .062. Fit for the model with scalar invariance was only slightly weaker for CFI and better for RMSEA. More important, inspection of the modification indices showed the largest difference for O10 “sophisticated in art” with a standardized mean difference of .068. Thus, there were no notable differences between the two age groups at the item level.
The results at the factor level reproduced the finding with scale scores by Soto et al. (2011). The older group had a higher level of conscientiousness (d = .61) than the younger group. Differences for the other personalty dimensions were statistically small. There were no notable differences in response styles.
In sum, the results show that the BFI shows reasonable measurement invariance across age groups. Contrary to the claims by Hussey and Hughes, this finding is consistent with the results reported in Hussay and Hughes’s supplementary materials. These results suggest that BFI scale scores provide useful information about personality and that published articles that used scale scores produced meaningful results.
Conclusion
Hussey and Hughes accused personality researchers of validity hacking. That is, they do not report results of psychometric tests because these tests would show that personality measures are invalid. This is a strong claim that requires strong evidence. However, closer inspection of this claim shows that the authors used an outdated measurement model and misrepresented the results of their invariance analyses. Here I showed that the BFI has good structural validity and shows reasonable invariance across gender and age groups. Thus Hussay and Hughes’s claims are blatantly false.
So far, i have only examined the BFI, but I have little confidence in the authors’ conclusions about other measures like Rosenberg’s self-esteem scale. I am still waiting for the authors to share all of their data to examine all of their claims. At present, there is no evidence of v-hacking. Of course, this does not mean that self-ratings of personality are perfectly valid. As I showed, self-ratings of the Big Five are contaminated with evaluative bias. I presented a measurement model that can test for the presence of these biases and that can be used to control for rating biases. Future validation studies might benefit from using this measurement model as a basis for developing better measures and better measurement models. Substantive articles might also benefit from using a measurement model rather than scale scores, especially when the BFI is used as a predictor of other self-report measures to control for shared rating biases.
[corrected 8/6/2019 5.29pm – there was a mistake in the model for worry]
After behaviorism banned emotions as scientific constructs and cognitivism viewed humans as computers, the 1980s witnessed the affective revolution. Finally, psychologists were again allowed to study feelings.
The 1980s also were a time where personality psychologists agreed on the Big Five as a unified model of personality traits. Accordingly, personality can be efficiently summarized by individuals’ standing on five dimensions: Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness.
Not surprisingly, the 1980s also produced a model of personalty, emotions (affect), and well-being that has survived until today. The model was first proposed by Costa and McCrae in 1980 (see Schimmack, 2019, for details). This model assumed that extraversion is a disposition to experience more positive affect, neuroticism is a disposition to experience more negative affect, and the balance of positive and negative affect is a major determinant of life-satisfaction. As extraversion and neuroticism are independent dimensions, the model also assumed that positive affect and negative affect are independent, which led to the creation of the widely used Positive Affect and Negative Affect Schedule (Watson et al., 1988) as a measure of well-being.
The model also assumed that general affective dispositions account for most of the stability in well-being over time, while environmental factors produce only momentary and short-lived fluctuations around dispositional levels of well-being (Diener, 1984; Lykken & Tellgen, 1996). This model dominated well-being research in psychology for 20 years (see Diener, Suh, Lucas, & Smith, 1999, for a review).
However, when Positive Psychology emerged at the beginning of the new millenium, psychologists focus shifted from the influence of stable dispositions to factors that could be changed with interventions to boost individuals’ wellbeing (Seligman & Csikszentmihalyi, 2000) and some articles even questioned the influence of dispositions on well-being (Diener, Lucas, & Scollon, 1996). As a result, the past 20 years have seen very little new research on dispositional influences on well-being. The last major article is a meta-analysis that showed positive correlations of extraversion and neuroticism with several well-being indicators (Steel, Schmidt, & Shultz, 2008).
Revisiting the Evidence
There is robust evidence for the influence of neuroticism on wellbeing. Most important, this relationship has been demonstrated in multi-method studies that control for shared method variance when self-ratings of personality are correlated with self-ratings of well-being (McCrae & Costa, 1991; Schimmack, Oishi, Funder, & Furr, 2004). However, the relationship between extraversion and well-being is not as strong or consistent as one would expect based on Costa and McCrae’s (1980) model. For example, McCrae and Costa failed to find evidence for this relationship in a multi-method study, and other studies that controlled for response styles also failed to find the predicted effect (Schimmack, Schupp, & Wagner, 2008).
Taking a closer look at Costa and McCrae’s (1980) article, we see that they did not include life-satisfaction measures in their study. The key empirical finding supporting their model is that extraversion facets like sociabilty measured at time 1 predict positive affect and hedonic balance (positive affect minus negative affect) concurrently and longitudinally and that these correlations remain fairly stable over time. This suggests that personality is stable and contributes to the stable variance in the affect measures. However, the effect size is small (r = .22 to .24). This suggests that extraversion accounts for about 5% of the variance in affect. This finding hardly supports the claim that extraversion accounts for half of the stable variance in well-being.
It is symptomatic of psychology that subsequent articles run with the story while ignoring gaps in the actual empirical evidence. As longitudinal studies in psychology are rare, there have been few attempts to replicate Costa and McCrae’s findings.
Headey and Wearing (1989) replicated and extended Costa and McCrae’s study by including life-satisfaction measures as an indicator of wellbeing. They replicated the key findings and showed that personality also predicts future life-satisfaction. However, the effect size for extraversion was again fairly small; as was the effect of neuroticism, suggesting that most of the stable variance in life-satisfaction is not explained by extraversion and neuroticism.
A key limitation of both studies is that they do not take shared method variance into account. Although method variance may be transient, it is also possible that it is stable over time (Anusic et al., 2009). Thus, even the already modest effect sizes may still be inflated by shared method variance.
New Evidence
Data and Model
Fortunately, better data are now available to revisit the longitudinal relationships between personality and life-satisfaction. I used the data from the German Socio-Economic Panel (SOEP). The SOEP measured the Big Five personality traits on four occasions (waves) spanning a period of 12 years (2005, 2009, 2013, 2017). Personality was measured with the 15-item BFI-S. I created a measurement model for the BFI-S that shows measurement invariance across the four occasions (Schimmack, 2019a). I also related personality to the single-item life-satisfaction rating in the SOEP (Schimmack, 2019b). Here, I extend this analysis by taking advantage of the fourth measurement of personality in 2017, which makes it possible to separate trait and state variance in personality and well-being.
The SOEP measures life-satisfaction in two ways. First, it includes several domain-satisfaction items (health, finances, recreation, housing). Second, it includes a global life-satisfaction item. In a different post (Schimmack, 2019c), I examined the relationship between these items and found that global items are influenced by a general disposition factor and satisfaction with finances and health, while the other two domains are relatively unimportant. Based on this finding and related evidence (Zou, Schimmack, & Gere, 2013), I averaged the domain satisfaction judgments and used it as an indicator of life-satisfaction. This makes it possible to remove random measurement error from the measurement of life-satisfaction on a single occasion. I then fitted latent-trait-state (LST) models to the personality factors and the well-being factor. These models separate the longitudinal correlations into two components. A stable trait component and a changing state component. A third parameter estimates how stable state variance is over time.
There are several ways to relate personality to life-satisfaction in this model. I chose to predict life-satisfaction variance on each occasion to the personality variances on the same occasion. The model indirect function can then be used to examine how much of the variance is due to stable personality traits or due to personality states.
The availability of four waves of data also makes it possible to model stability of the residual variances in personality items. Typically, these residuals are allowed to correlate to allow for item-specific stability, but the use of correlated residuals makes it impossible to relate this variance to other constructs. With four waves, it is possible to fit an LST model to item-residuals. Exploration of the data showed that the neuroticism item “worry” showed consistent relationships with well-being. Thus, I fitted an LST model to this item and allowed for an influence of worry on life-satisfaction.
The synatax and the complete results are posted on OSF (SOEP.4W.B5.DSX.LS).
Results
Overall model fit was acceptable, CFI = .967, RMSEA = .019, SRMR = .030.
Trait Variance and Stability of State Variance
Table 1 shows the amount of trait variance and the stability of state variance in the personality predictor variables. A more detailed discussion of the implications of these results for personality research can be found elsewhere (Schimmack, 2019a). The results for the Big Five serve as a comparison for the trait variance in life-satisfaction.
Trait
Stability
1Y-Stability
Neuroticism
0.69
0.38
0.79
0.56
Extraversion
0.74
0.38
0.78
0.51
Openness
0.71
0.34
0.76
0.53
Agreeableness
0.68
0.20
0.67
0.57
Conscientiousness
0.60
0.29
0.73
0.64
Halo
0.60
0.36
0.78
0.63
Acquiescence
0.34
0.11
0.58
0.81
Worry
0.64
0.52
0.85
0.60
Table 2 shows how life-satisfaction at each time point is related to personality predictors. For model identification purposes, it is necessary to fix one relationship to zero. I used openness because meta-analysis show that it is the weakest predictor of life-satisfaction (Steel et. al., 2008). I did not impose constraints across the four waves.
LS-T1
LS-T2
LS-T3
LS-T4
Neuroticism
-0.29
-0.27
-0.27
-0.26
Extraversion
0.08
0.07
0.08
0.09
Openness
–
–
–
Agreeableness
0.08
0.05
0.04
0.03
Conscientiousness
0.04
0.04
0.04
0.04
Halo
0.19
0.28
0.24
0.26
Acquiescence
0.18
0.08
0.12
0.16
Worry
-0.35
-0.34
-0.35
-0.33
The results show that out of the Big Five, neuroticism is the only notable predictor of life-satisfaction with a moderate effect size (r = -.26 to -.29). A notable finding is that extraversion is a weak predictor of life-satisfaction (r = .07 to .09). This finding is inconsistent with Costa and McCrae’s (1980) model. The results for agreeableness and conscientiousness are also weak. This finding is inconsistent with meta-analysis and with McCrae and Costa’s (1991) suggestion that high agreeableness and conscientiousness are also instrumental for higher life-satisfaction. Both halo and acquiescence bias are stronger predictors of life-satisfaction judgments than extraversion, agreeableness, and conscientiousness. Another notable finding is that the worry-facet of neuroticism is the strongest personality predictor; even stronger than the neuroticism factor (rs = -.33 to -.35). This finding is consistent with previous studies that facets of neuroticism and extraversion are better predictors of life-satisfaction than the global factors (Schimmack, Oishi, Funder, & Furr, 2004).
Table 3 shows how much of the variance in life-satisfaction is explained by trait factors that remain stable over time.
LS-T1
LS-T2
LS-T3
LS-T4
Neuroticism
0.05
0.05
0.05
0.05
Extraversion
0.00
0.00
0.00
0.01
Openness
–
–
–
–
Agreeableness
0.00
0.00
0.00
0.00
Conscientiousness
0.00
0.00
0.00
0.00
Halo
0.02
0.04
0.03
0.04
Acquiescence
0.01
0.00
0.00
0.01
Worry
0.08
0.08
0.08
0.07
Unexplained
0.38
0.38
0.38
0.38
Total
0.55
0.56
0.56
0.55
Given the weak effects of extraversion, agreeableness, and conscientiousness, it is not surprising that these Big Five traits explain less than 1% of the variance in life-satisfaction judgments. The only notable predictor is neuroticism, which explains 5-6% of the variance. In addition, the worry facet of neuroticism is an even stronger predictor of trait variance in life-satisfaction. This finding shows that more specific traits below the Big Five add to the prediction of life-satisfaction (Schimmack, Oishi, Furr, & Funder, 2004). Halo adds only 2% and acquiescence only 1%. By far the largest portion of the trait variance was unexplained with 41% of the variance. Combined this implies that approximately half of the variance in life-satisfaction is trait variance. This finding is consistent with estimates in a meta-analysis and other analyses of the SOEP data (Anusic & Schimmack, 2016; Schimmack, Krupp, Wagner & Schupp, 2010). The estimate of 55% trait variance is also smaller than the estimate of 70% trait variance in the Big Five personality traits. This finding is also consistent with meta-analytic comparison of personality and well-being measures (Anusic & Schimmack, 2016).
Table 4 shows the results for the state-predictors of life-satisfaction. Once more extraversion, agreeableness, and conscientiousness predict less than 1% of the variance. This time, neuroticism and worry are also relatively weak predictors because most of the relationship for this traits stems from the stable component. However, the results suggest that some changes in neuroticism and worry are related to changes in life-satisfaction. However, most of the state variance in life-satisfaction is not explained by the personality predictors (33% out of 44%).
LS-T1
LS-T2
LS-T3
LS-T4
Neuroticism
0.02
0.02
0.02
0.02
Extraversion
0.00
0.00
0.00
0.00
Openness
–
–
–
–
Agreeableness
0.00
0.00
0.00
0.00
Conscientiousness
0.00
0.00
0.00
0.00
Halo
0.01
0.03
0.02
0.03
Acquiescence
0.02
0.00
0.01
0.02
Worry
0.04
0.04
0.04
0.04
Unexplained
0.33
0.34
0.34
0.34
Total
0.44
0.44
0.44
0.45
Conclusion
These results challenge Costa and McCrae’s (1980) model of personality and well-being in several ways. First, extraversion is not a strong predictor of the stable variance in life-satisfaction. Second, even the influence of neuroticism accounts for only 10% of the stable trait variance in life-satisfaction. Adding other Big Five predictors also does not help because they have negligible relationships with life-satisfaction. Thus, most of the trait variance in life-satisfaction remains unexplained. It is either explained by more specific personality traits than the Big Five (facets) or by stable environmental factors (e.g., income). The SOEP data provide ample opportunity to look for additional predictors of trait variance. Also, researchers should conduct studies with broader personality questionnaires to find additional predictors of life-satisfaction. Searching for these predictors is an important area of research in an area that has stagnated over the past two decades.
Costa and McCrae’s model also underestimated the importance of state-factors. State factors are highly stable over fairly long periods of times and account for 50% of the reliable variance in life-satisfaction. As the Big Five mostly reflect stable traits, they cannot account for this important variance in life-satisfaction. Schimmack and Lucas (2010) argued that these factors are environmental factors because changes in life-satisfaction are shared between spouses. Thus, changes in actual life-circumstances may contribute to state variance in life-satisfaction. Consistent with this model, spouses were more similar in domains that are shared (housing, income) than in domains that are less shared (health, recreation).
Evidently, the conclusions are based on a single German sample. As impressive as these data are, it is important to compare results across samples from different populations. At least regarding the influence of extraversion, the present results are consistent with other studies that suggest the influence of extraversion on life-satisfaction (Kim, Schimmack, & Tsutsui, 2019). The idea that extraverts are happier has been exaggerated by Costa and McCrae’s model, while their own empirical results did not warrant this claim. The reason is that psychologists often ignore effect sizes.
Implications
The present results also have implications for developmental theories of personality. The idea of development is a process with an ideal outcome. For humans, the outcome is an adult human being with optimal capabilities. A collective of personality psychologists suggested that optimal personality development results in a personality type with optimal personality characteristics. I criticized this idea and argued that there is no such thing as an optimal personality. Just like there is no optimal height as the end-goal of human growth, there is no optimal level of extraversion or conscientiousness. In clinical psychology, the key criterion of mental health is that an intervention is beneficial for a patients’ well-being. Thus, we could argue that an optimal personality is a personality that maximizes individuals’ well-being. Meta-analyses suggests that extraveted, agreeable, and conscientious people have higher well-being. Thus, it might be beneficial for individuals to become more extraverted, agreeable, and conscientious. However, the present results challenge this view. After removing the evaluative aspect of personality from the Big Five only neuroticism remains a notable predictor of well-being. Thus, the key personality trait for self-improvement is neuroticism. Not surprisingly, this is also the key aspect that is targeted in self-help books and well-being programs. Until we have a better understanding of the relationship between personality and well-being, it seems premature to propose interventions that are aimed at changing individuals’ personality. Just like personality psychologists no longer endorse conversion therapy for sexual orientation, I urge for caution in submitting individuals who are carefree and impulsive to a conscientiousness conversion program. You never know when acting on the spur of a moment is the best course of action.
Studying personality stability and change is easy and hard. It is easy because the method is straightforward. Administer a valid measure of personality to a group of participants and repeat the measurement several times. Describing the method takes a sentence or two compared to pages that describe an intricate laboratory experiment with an elaborate deception. It is hard because it requires time and participants may drop out of a study. Meanwhile there is nothing to publish while a researcher is waiting for the next retest. In our fast paced world of academic publishing where researchers are expected to publish 5 or more articles a year, there is no place for slow research. As a result, evidence on personality change is scarce. The best evidence so far comes from a meta-analysis that patched together small studies with different measures, populations, and small samples. Although this meta-analysis is the best evidence available, it cannot be trusted because the evidence is inconclusive.
Psychologists have to thank economists and sociologists who are used to collaborate on big data collections. One of these collaborations is the German Socio-Economic Panel (SOEP). The SOEP is an ongoing longitudinal study with a representative sample that started in 1984. In 2005, the SOEP included the BFI-S; a 15-item personality measure that assesses the Big Five. Since then, the BFI-S has been administered in four-year intervals in 2009, 2013, and 2017. Thus, we now have longitudinal data spanning 12-years with four waves of data. This makes it possible to revisit the question of personality stability with much better data than a meta-analysis of heterogeneous studies can provide. Surely, the results are based on a German sample, but there is little evidence that personality development varies across cultures.
Method
One drawback of the SOEP is that each personality dimension is measured with just three items. This makes scale scores unreliable and scale scores can be contaminated with method variance (e.g., evaluative bias, acquiescence bias). To avoid these problems and to examine measurement invariance, it is better to analyze the data with a measurement model that examines personality change at the level of latent variables that correct for measurement error. I developed a measurement model for the SOEP (Schimmack, 2019a) and I already demonstrated invariance across the first three waves of the SOEP (Schimmack, 2019b). Here I added the fourth wave of data from 2017 to the dataset to produce even better information about long-term changes in personality.
To analyze the data, I first fitted the measurement model for the BFI-S to the data from each wave and imposed equality constraints to ensure measurement invariance. The longitudinal stability of personality was examined using the latent-trait-state (LTS) model that decomposes stability over time into two components; (a) a stable trait component that never changes and (b) a changing state component. The changing state component allows for factors that influence personalty to change over time and to change personality. These changing factors may produce changes that last a long time or changes that are more temporary. The time course of changes in personality is modeled with an autoregressive parameter that reflects how many of the changes at time 1 are still present at time 2.
The LTS model is typically fitted without modeling mean level changes. However, the model can also be used to model the mean structure in the data. In latent variable models, changes in personality are assumed to occur at the level of the latent traits, while item means (intercepts) are assumed to be constant over time. As the latent trait is stable, it cannot be used to model mean-level changes. Thus, one option is to free the means of the state factors. However, the influence of the state factors decreases over time, which is inconsistent with the idea of lasting changes in personality. Thus, a better option is to let the means of the occasion specific factors to vary freely, even if the occasion specific variance is zero. Although this model may lack realism, it would show the pattern of mean level changes in the data without imposing some model on the data (e.g., a linear trend).
The model specification and the complete results can be found on OSF (ttps://osf.io/vpcfd/). The overall model fit was acceptable, CFI = .971, RMSEA = .019, SRMR = .031.
Rank-order Stability and Change
A study of the first three waves in the SOEP replicated earlier findings of high retest stability in personality with stabilities over .9 over a one-year period (Conley, 1984; Schimmack, 2019c). However, three ways are insufficient to separate trait variance from state variance, and few studies with four waves of personality are available. Anusic and Schimmack (2016) used a meta-analytic approach to do so on the basis of smaller studies. Their model suggested that about 70% of the reliable variance in personality is trait variance and that the remaining 30% state variance are rather unstable with a low annual stability of .3. This would suggest that any changes in personality do not last long and individuals quickly revert back to their trait level of personality.
Table 1 shows the results for the SOEP data.
Trait
Stability
1Y-Stability
Neuroticism
0.67
0.38
0.79
Extraversion
0.74
0.36
0.77
Openness
0.71
0.38
0.79
Agreeableness
0.69
0.18
0.65
Conscientiousness
0.64
0.24
0.70
Halo
0.64
0.31
0.75
Acquiescence
0.32
0.10
0.56
The results show a similar split between trait and state variance as the meta-analysis, with about two-thirds of the variance being trait variance and one-third being state variance. A new finding is that the halo factor, an evaluative bias in personality ratings, also has 60% trait variance. Thus, this response style can also be considered a stable trait. In contrast, acquiescence bias has less trait variance and seems to be more influenced by momentary factors that are inconsistent over time.
The results for the stability of the state variance are different from the meta-analysis. The SOEP data suggest that changes in personality are more persistent than the meta-analysis suggested. The annual stability estimates are around .7. Thus, any changes that are evident at time 2 would still be evident over the next years. The stability over 4-years is around .3. These results are more encouraging for researchers who are interested in personality change than the meta-analytic results in Anusic and Schimmack, 2016). Nevertheless, the relatively small amount of state variance and the high stability of the state variance imply that it takes time to find even small changes in personality. Not surprisingly, it has been difficult to uncover predictors of personality change even in large samples like the SOEP (Specht et al., 2011).
In sum, the results confirm that personality ratings are highly stable over extended periods of time and that a large portion of this stability is caused by stable factors that ensure persistent individual differences in personality over the life span.
Mean Levels
Table 2 shows the results for the mean levels. Means in the first year, 2005, are used as the reference group. The results provide little evidence for personalty change in adulthood. None of the Big Five dimensions shows a consistent trend over time. The results for conscientiousness are most important because a meta-analysis suggested that conscientiousness increases substantially throughout adulthood. There is no evidence for such a trend in the SOEP.
N
E
O
A
C
2005
0.00
0.00
0.00
0.00
0.00
2009
-0.13
-0.12
-0.16
-0.18
-0.07
2013
-0.18
-0.03
-0.04
-0.08
-0.06
2017
-0.16
-0.06
-0.05
-0.11
-0.26
The general pattern of decreases for all five dimensions suggests that acquiescence bias might have changed over time. Thus, I also fitted a model with free means for acquiescence bias but the results did not change. Thus, it does not account for the small decrease in the Big Five. Adding means for the halo factor, instead, reduces changes for most scales, but would suggest a stronger decrease in neuroticism. However, the pattern is never a gradual change, but a drop from time 1 to time 2 with no major changes afterwards. This suggests that some panel effect or period effects have small effects on personality ratings, but there is no evidence to support the claim that personality systematically changes throughout adulthood.
Conclusion
Personality research was attacked by situationists who claimed that personality is a mere social construction. In the 1980s, personality researchers had presented evidence that personality traits are real and stable using twin studies, multi-rater studies, and longitudinal studies. However, two meta-analysis by Roberts and colleagues suggested that personality exists but is less stable than personality psychologists assumed. These meta-analysis had a strong influence on personality psychology in the 2000s. They are featured in personality textbooks and often cited as evidence that personality still develops throughout adulthood. However, more recent evidence are more consistent with the view of personality as mostly stable throughout adulthood. Costa and McCrae famously compared personality to plaster. While it can be shaped and molded early on, it finally sets into a shape that can not be altered. Yes, there may be cracks here and there, but the overall shape is set. While this image may be too rigid, it is consistent with the evidence that even major life-events that occur during adulthood seem to have very little influence on personality (Specht et al., 2011).
The idea of personalty change is often coupled with the notion that personality develops and that there can be personal growth in adulthood. The problem with these notions is that it implies that there is a normative or desirable direction of personality change. For example, an increase in conscientiousness is seen as evidence of growing maturity. However, the measurement model that I used distinguishes between the denotative and connotative aspects of personality. Lazy is both descriptive and evaluative. However, evaluations are rooted in cultural norms and values. Why is it good to work as much as possible, to avoid mistakes at any costs? Should education and policies try to increase conscientiousness levels? Is there an optimal level? These are all very difficult questions that go well beyond the existing science of personality. Once we focus on the denotative aspect of personality, we see that some people work harder than others or that some people are more creative than others, and that these differences are fairly stable, without any evidence what causes this stability. Just like people differ in personality, they differ in other characteristics that have received more attention. Current culture aims towards greater acceptance of differences in sexual orientation, gender identity, body types, religion, etc. Maybe we should also include personalty traits there and let introverts be proud introverts and disagreeable people be proud disagreeable people. Maybe personality differences only exist because they were not a problem during human evolution or diversity is even an advantage that allows humans as a group to adapt to different circumstances. Thus, the strong evidence of personality stability is not necessary a problem that needs to be solved because there is normal personality. There is only normal variation in personality.
The German Socio-Economic-Panel (SOEP) is a longitudinal study of German households. The core questions address economic issues, work, health, and well-being. However, additional questions are sometimes added. In 2005, the SOEP included a 15-item measure of the Big Five; the so-called BFI-S (Lang et al., 2011). As each personality dimension is measured with only three items, scale scores are rather unreliable measures of the Big Five. A superior way to examine personality in the SOEP is to build a measurement model that relates observed item scores to latent factors that represent the Big Five.
Anusic et al. (2009) proposed a latent variable model for an English version of the BFI-S.
The most important feature of this model is the modeling of method factors in personality ratings. An acquiescence factor accounts for general response tendencies independent of item content. In addition, a halo factor accounts for evaluative bias that inflates correlations between two desirable or two undesirable items and attenuates correlations between a desirable and an undesirable item. The Figure shows that the halo factor is bias because it correlates highly with evaluative bias in ratings of intelligence and attractiveness.
The model also includes a higher-order factor that accounts for a correlation between extraversion and openness.
Since the article was published I have modified the model in two ways. First, the Big Five are conceptualized as fully independent which is in accordance with the original theory. Rather than allowing for correlations among Big Five factors, secondary loadings are used to allow for relationships between extraversion and openness items. Second, halo bias is modeled as a characteristic of individual items rather than the Big Five. This approach is preferable because some items have low loadings on halo.
Figure 2 shows the new model.
I fitted this model to the 2005 data using MPLUS (syntax and output: https://osf.io/vpcfd/ ). The model had acceptable fit to the data, CFI = .962, RMSEA = .035, SRMR = .029.
Table 1 shows the factor loadings. It also shows the correlation of the sum scores with the latent factors.
Item#
N
E
O
A
C
EVB
ACQ
Neuroticism
worried
5
0.49
-0.02
0.19
nervous
10
0.64
-0.31
0.18
relaxed
15
-0.55
0.35
0.21
SUM
0.75
0.00
0.00
0.00
0.00
-0.30
0.09
Extraversion
talkative
2
0.60
0.13
0.40
0.23
sociable
8
0.64
0.37
0.22
reserved
12
-0.52
0.20
-0.11
0.19
SUM
0.00
0.75
0.00
-0.10
0.05
0.36
0.09
Openess
original
4
0.26
0.41
-0.33
0.38
0.22
artistic
9
0.15
0.36
0.29
0.17
imaginative
14
0.30
0.55
0.22
0.21
SUM
0.00
0.30
0.57
-0.13
0.00
0.39
0.26
Agreeableness
rude
3
0.12
-0.51
-0.32
0.19
forgiving
6
0.23
0.32
0.24
considerate
13
0.49
0.48
0.29
SUM
0.00
-0.07
0.00
0.58
0.00
0.50
0.11
Conscientiousness
thorough
1
0.71
0.35
0.30
lazy
7
-0.16
-0.41
-0.35
0.20
efficient
11
0.39
0.48
0.28
SUM
0.00
0.00
0.00
0.09
0.64
0.51
0.11
The results show that all items load on their primary factor although some loadings are very small (e.g., forgiving). Secondary loadings tend to be small (< .2), although they are highly significant in the large sample. All items load on the evaluative bias factor, with some fairly large loadings for considerate, efficient, and talkative. Reserved is the most evaluatively neutral item. Acquiescence bias is rather weak.
The scale scores are most strongly related to the intended latent factor. The relationship is fairly strong for neuroticism and extraversion, suggesting that about 50% of the variance in scale scores reflects the intended construct. However, for the other three dimensions, correlations suggest that less than 50% of the variance reflects the intended construct. Moreover, the remaining variance is not just random measurement error. Evaluative bias contributes from 10% up to 25% of additional variance. Acquiescence bias plays a minor role because most scales have a reverse scored item. Openness is an exception and acquiescence bias contributes 10% of the variance in scores on the Openness scale.
Given the good fit of this model, I recommend it for studies that want to examine correlates of the Big Five or that want to compare groups. Using this model will produce better estimates of effect sizes and control for spurious relationships due to method factors.
The German Socio-Economic Panel (SOEP) is a unique and amazing project. Since 1984, representative samples of German families have been surveyed annually. This project has produced a massive amount of data and hundreds of publications. The traditional journal publications make it difficult to keep track with developments and to find related articles. A better way to make use of these data may be open science where researchers can quickly share information.
In 2005, the SOEP included a brief, 15-item, measure of the Big Five personality traits. These data were used for cross-sectional studies that related personality to other variables measured in the SOEP such as well-being (Rammstedt, 2007). In 2009, the SOEP repeated the measurement of the Big Five. This provided longitudinal data for analyses of stability and change of personality. Researchers rushed to analyze the data and to report their findings. JPSP published two independent articles based on the same data (Lucas & Donnellan, 2011; Specht, Egloff, Schmukle, 2011). Both articles examined age-differences across birth-cohorts and over time. Ideally age-effects would show up in both analyses and produce similar trends in the data. Both articles also paid little attention to cohort differences in personality (i.e., Germans born in 1920 who grew up during Nazi times might differ from Germans born in 1950 who grew up during the revolutionary 60s).
In 2017, the Big Five questions were administered again, which makes it easier to spot age-trends and to distinguish age-effects from cohort effects. Recently, the first article based on the three-waves of data was published in JPSP (Wagner, Lüdtke, & Robitzsch, 2019). The article focused on retest correlations (consistency of individual differences over time), and did not examine mean levels of personality. The article does not mention cohort effects.
Cohort/Culture Effects
Like many Western countries, German culture has changed tremendously during the 20st century. In addition, German culture has been shaped by unique historical events such as the rise and fall of Hitler, the second world war, followed by the Wirschaftswunder, the division of the country into a democratic and a socialist country and the unification of Germany after the fall of the Berlin Wall. The SOEP data provide a unique opportunity to examine whether personality is shaped by culture.
So far, studies of cultural influences on personality have mostly relied on cross-cultural comparisons of Western cultures with non-Western cultures. The main finding of these studies is that citizens of modern, individualistic nations tend to be more extraverted and open to experiences than citizens in traditional, collectivistic cultures.
Based on these findings, one might expect higher levels of extraversion and openness in younger generations of Germans who grew up in a more individualistic culture than their parents and grandparents.
Method
The data are the Big Five ratings for the three waves in the SOEP (vp, zp, & bdp). Data were prepared and analyzed using R (see OSF for R-code). The three items for each of the Big Five scales were summed and analyzed as a function of 7 cohorts spanning 10 years (born 1978 to 1988 age 17-27 to age 77 to 87) and three waves (2005, 2009, 2013). The overall mean was subtracted from each of the 21 means and the mean differences were divided by the pooled standard deviation. This way, mean differences in the figures are standardized mean differences to ease interpretation of effect sizes.
Results
Openness to Experience
Openness to experience showed a clear cohort effect (Figure 1) with the lowest scores for the oldest cohort (1918-28) and the highest scores for the youngest cohort (1978 to 1988). The difference between the youngest and oldest cohorts is d = .72, which is considered a large effect size. In comparison, there is no clear age trend in Figure 1. While, scores decrease from t1 to t2, they increase from t2 to t3. All differences between t1 and t2 are small, |d| < .2.
Extraversion
Extraversion also shows a cohort effect in the predicted direction, but the effect size is smaller, d = .34.
In contrast, there are no age effects and the overall difference between 2005 and 2013 is d = -0.01.
Conscientiousness
I next examined conscientiousness because studies of age effects tend to show the largest age effects for this Big Five dimension. Regarding cohort effects, one might expect a decrease because older generations worked very hard to rebuild post-war Germany.
Consistent with the developmental literature, the youngest age-cohort shows an increase in conscientiousness from 2005 to 2013, although the effect size is small (d = .21). The other age-cohorts show very small decreases in conscientiousness except for the oldest age-cohort that shows a small decrease, d = -.22. Regarding cohort effects, there is no general tend, but the youngest cohort shows very low levels of conscientiousness even in 2013 when they are 25 to 35 years old.
Agreeableness
Developmental studies suggest that agreeableness increases as people get older. However, the SOEP data do not confirm this trend.
Within each cohort, agreeableness scores decrease although the effect sizes are very small. The overall decrease from 2005 to 2013 is d = -.09. In contrast, there is a clear cohort effect with agreeableness being the highest in the oldest generation. The decrease tends to level of for the last three generations. The effect size is moderate, d = -.38.
Neuroticism
The main result for neuroticism is that there is neither a pronounced cohort effect, d = -.09, nor age effect, d = -.13.
Conclusion
Previous analysis of personality data in the SOEP have focused on age effects and interpreted cross-sectional differences between older and younger Germans as age effects. However, these analyses were based on only two waves of data, which makes it difficult to interpret changes in personality scores over time. The third wave shows that some of the trends did not continue and suggest that there are no notable effects of aging in the SOEP data. The only age-effect consistent with the literature is an increase in conscientiousness in the youngest cohort of 17 to 27-year olds.
However, the data are consistent with cohort effects that are consistent with cross-cultural studies. The more individualistic a culture becomes, the more open and extraverted individuals become. Deeper analysis might help to elucidate which factors contribute to these changes (e.g., education level). The results also suggested that agreeableness decreased which might be another consequence of increasing individualism.
Overall, the results suggest that personality is influenced by cultural factors during adolescence and early adulthood, but that personality remains fairly stable throughout adulthood. This conclusion is also supported by other longitudinal studies (e.g., MIDUS) that show little changes in Big Five scores over time. Maybe Costa and McCrae were not entirely wrong when they compared personality to plaster that can be shaped while it is setting, but remains stable after it is dried.
You suggested that personality traits are actionable targets for public policy (Bleidorn et al., 2019). I was surprised and actually shocked by this proposal. I have taught personality psychology for over a decade and I always emphasize that individual differences are normal and should be celebrated like we celebrate other aspects of human diversity in culture and in sports. Therefore I don’t think personality interventions are needed or desirable. Maybe there is some fundamental misunderstanding, but reading your article suggests that you are really proposing that public policy should target personality traits.
This idea is not new.
Socialistic governments and fascist governments had ideals of the model
citizen and aimed to fit their citizens into this mold.
In marked contrast, democracies and market economies are built
on the idea that citizens’ well-being is maximized by personal choice. The rule
of governments is mainly to protect the safety of citizens and to regulate
conflict when individual preferences are in conflict. Well-being surveys consistently show that
free and individualistic societies produce higher well-being than societies
that impose ideological or religious norms on their citizens.
The history of psychology also casts a shadow on attempts to shape individuals’ personality. When homosexuality was a taboo, the Diagnostic and Statistical Manual of Mental Discorders included homosexuality as a mental illness. Today most psychologists consider it a success that homosexuality is considered an expression of personal preferences and conversion therapy to cure homosexuals from some invented illness is considered unethical. More generally, mental illness has been defined in terms of patients’ suffering and concerns about patients’ well-being rather than in terms of cultural norms of acceptable or unacceptable characteristics.
New insights into biological influences on many illnesses (e.g.,
cancer) have given rise to personalized medicine which is based on the idea
that the same treatment can have different effects for different individuals. Rather than trying to fit patients to treatments,
personalized medicine aims to fit treatments to patients.
Given these general trends one could argue that modern societies need personality psychology because a better understanding of individual differences is needed to create policies that respect individual freedom and creates opportunities for individuals to pursue their own well-being and to maximize their own potential. The call to shape personality, however, seems to suggest the opposite. In fact, the call for governments to regulate personality development seems so absurd that it is seems improbable that a group of modern, presumably liberal leaning, psychologists would argue for it. Does this mean I misunderstood your article? I hope so, but reading it didn’t help me to understand your position.
We agree that personality traits as enduring factors (a.k.a. causes, dispositions) within an individual that influence their thoughts, feelings, and behaviors. You propose that governments should influence personality traits because personality traits influence life outcomes. For example, personality traits influence divorce. If governments want to reduce the divorce rates, they could target the personality traits that lead to divorce. Another advantage of changing personality traits is that they are broad dispositions that influence a range of diverse behaviors. For example, conscientiousness influences class attendance, health behaviors, and making your bed every morning. Instead of having different interventions for each behavior, making individuals more conscientious would influence all three behaviors.
Most of the article discusses empirical research whether it is actually possible to change personality traits. I am not going to quibble with you about the evidence here because it is irrelevant to the main question that your article brings up: if it were possible to change personality, should governments role out interventions that shape personality? As the article focused on the Big Five traits, the question is whether governments should make citizens more or less neurotic, extraverted, agreeableness, conscientious, or open to experience?
“Our most general assertion is that personality
traits are both stable and changeable, which makes personality trait change a powerful
and hitherto relatively underused resource for policy makers”
You appear to be so convinced that government interventions that target personality are desirable that you ask only when to intervene, what intervention to use, who to target, and how to intervene. You never stop to wonder whether interventions are a good idea in the first place.
For example, you suggest that increasing conscientiousness in adolescence is a desirable policy goal because “it could elicit a cascade of positive outcomes” (p. 19). And decreasing neuroticism is good because it “could significantly reduce one’s likelihood of experiencing negative life events” (p. 19).
In passing you mention the main problem of your proposal to regulate personality. “This is not to say that there are optimal trait levels that should be universally promoted in all people” However, you do not reconcile this observation with your call for personality policies. If there are no optimal levels, then what should be the target of personality policies? And are the previous examples not evidence that you consider higher conscientiousness and lower neuroticism as optimal? If they are not considered more optimal, why should governments develop interventions to increase conscientiousness and to reduce neuroticism?
You end with the conclusion that “personality traits are ideal targets for interventions designed to improve life success,” which once more begs the question what the goal of personality interventions should be. What is life success? We know the answer is 42 (h/t Hitchhiker’s Guide to the Galaxy), but we don’t really understand the question very well.
To end on a more positive note, I do think that governments can play a role in helping individuals to have better lives with higher well-being, and national rankings of quality of life and well-being show that some governments are doing a better job than others. One main indicator of a good life is a healthy and long life, and health care is both a major contributor to GDP and a major policy agenda. Good health includes physical health and mental health. Prevention and treatment of mental health problems such as anxiety, depression, or addiction are important. Unlike personality, health can be defined in terms of optimal functioning and we can evaluate policies in terms of their effectiveness to maximize optimal functioning. Addressing those concerns is an important policy agenda and psychologists can play an important role in addressing these issues. But I prefer to leave normal variation in personality alone. As you noted yourself, there are no optimal personality traits. The best personality policy is to create free societies that let individuals pursue their own happiness in the way they want to pursue it.
