12 thoughts on “When Personality Psychologists are High”

1. Pingback: What lurks beneath the Big Five? | Replicability-Index

2. Uli, I’ve always had respect for your work, but at the same time I’m wary of engaging with you in a public forum because you’re a bit challenged in the civility department. Nonetheless, I’ll make an effort, and anyway I would never use your abrasive interpersonal style to attempt to discredit your intellectual arguments. Before I get to those arguments (in my next comment) let me make a few observations and requests about the style of your post.

   First, would you please change the spelling of my name throughout so that it’s correct? The initial “d” should be capitalized, “DeYoung.”

   Second, would you please change the title to something that doesn’t insinuate that someone who comes to a different conclusion than you must be on drugs?

   Third, please cite Musek in the following sentence of yours: “Meanwhile, another article built on the 2002 model that did not control for rating biases and proposed that the correlation between the two higher-order factors implies that there is an even higher level in the hierarchy.” I agree with you that his approach and conclusion were misguided from the start, but that doesn’t mean you should fail to cite him: Musek, J. (2007). A general factor of personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality, 41(6), 1213-1233.

   You should probably also cite the following article that’s a meta-analysis of the existence of the two higher-order factors in multi-informant studies. I appreciate the positive things you’ve said about my 2006 article, but this one provided much more extensive evidence to address most of the same questions: Chang, L., Connelly, B. S., & Geeza, A. A. (2012). Separating method factors and higher order traits of the Big Five: A meta-analytic multitrait–multimethod approach. Journal of personality and social psychology, 102(2), 408.

   Fourth, a minor quibble: as a reviewer, I certainly never had the power to “kill” one of your manuscripts. If the editor killed it, perhaps it was because he agreed with whatever arguments I made in my review. Further, I don’t remember reviewing one of your papers that was ultimately rejected. I remember reviewing the paper that eventually turned into your 2009 JPSP “Halo-Alpha-Beta Model” paper, which you still seem pleased with. In fact, you may recall that I was the one who suggested something like that model in the first place. Perhaps you were referring to my “killing” the initial version of that manuscript, which used an odd cluster-based approach with dichotomized Big Five variables to make a similar argument about the presence of halo bias in personality assessments. In response to that manuscript, I suggested that a better approach would be to model halo bias and Digman’s two higher-order factors simultaneously. You know what? This will be easier if I just post my reviews of both your original submission and your revision. Then readers of your blog will be able to see exactly what this “battle” with me was like:

   Review of your original submission:

   “This article investigated patterns of correlation among the Big Five. The typological approach taken here is quite interesting. As the authors note, the loss of information due to dichotomous scoring is probably compensated by the ability to examine interactive effects among traits more easily and to determine whether positive biases are more common than negative biases. Nonetheless, the main results (that the two evaluatively consistent types, Neoac and nEOAC, are over-represented) do not seem very different from a demonstration that there is a large first factor within single-informant ratings of the Big Five, on which all five traits load (e.g., Musek, 2007, JRP), or that the higher-order factors of the Big Five are correlated in single-informant ratings (e.g., DeYoung et al., 2002), or that in a multi-informant model variance in the Big Five unique to each informant is best represented by strongly correlated factors (DeYoung, 2006). All of these findings point to the halo effect. This article may be valuable as another demonstration of that effect – the addition of the bias measure in Study 4 is a nice touch – but demonstrating the halo effect is not novel, and, as the authors rightly note in their introduction, their work does not rule out the possibility that some genuine correlations exist among the Big Five alongside the halo effect. I doubt that this article is likely to have a strong impact on the field.

   “Additionally, there is a discrepancy between the relatively modest goal, set out in the introduction, of examining the relative effect sizes of different contributors to covariance among the Big Five, and the relatively strong conclusion that their findings “undermine Digman’s (1997) hypothesis of two broader personality traits.” Even to begin to make such a claim, the authors would need to test and reject an additional model, specifying that both the higher-order factors and the halo effect are operating together. Digman’s model was not exactly accurate for single-informant ratings, in which the higher-order factors are always correlated, but it appears to be accurate for multiple informants. I am aware of two articles which have examined Digman’s two factors using CFA in multi-informant data, one by DeYoung (2006) and another which I recently reviewed for this journal. Both of these indicated that the halo effect contributed to correlations among the Big Five in individual informant ratings, but also that the higher-order factors were present and uncorrelated. The latter article even found a significant genetic basis for the higher-order factors. (It is true that Biesanz and West (2005) did not find correlated Big Five factors at the latent level in a multi-informant design, but their sample showed considerably lower interrater agreement than either of the two papers just mentioned, which presumably attenuated their latent correlations.)

   “The introduction is considerably too long, dwelling in detail on points that could be summarized much more succinctly.

   “Additional points:

   “p. 5. DeYoung (2006) did not “dismiss the halo model,” but in fact explicitly mentioned that his findings were consistent with the hypothesis that a general bias toward evaluative consistency contributed some (but not all) of the covariance of the Big Five in single informant ratings.

   “p. 8. The author’s assertion that “it is incorrect to assume that Digman’s (1997) rotated factor solution provides a valid account of the structure of Big Five correlations because the unrotated factor structure provides an equally plausible account of the structure” is itself incorrect because rotated and unrotated factor solutions are equivalent in their fit to the data. Perhaps they meant that it is incorrect to assume that Digman’s solution is the ONLY valid account – but this would be a straw man, as no one with any knowledge of factor analysis is likely to make this assumption.

   “p. 10. Here the authors present a false choice: “…it becomes a theoretical question whether it makes more sense to postulate causal mechanisms that produce a positive correlation between Digman’s higher order factors, or to postulate a halo model and to allow for an additional factor that influences E and O.” Another option is that both higher-order factors reflect causal mechanisms, but the halo effect is also at work and produces a correlation between them. This interpretation is supported by the articles discussed above.

   “p. 10. In relation to DeYoung’s (2006) results, the authors note “the two factors are substantially correlated in confirmatory factor analyses of the same-method correlations among the Big Five (based on our own analyses of the covariance matrices)” – strange that they would mention their own analyses but not the fact that DeYoung reported these same results. This rhetorical strategy contributes to my impression that this article is attempting to create conflict where none exists.

   “p. 11. A relatively minor point: correlations should not be squared to determine the amount of variance explained, when the measures in question stand in for latent traits (Ozer, 1985, Psych Bull). [I later realized the flaw in this argument, although I do agree with Ozer’s later argument that correlation is a better measure of effect size because the importance of effects scales with the correlation, not the squared correlation.]

   “p. 12. Their fifth conclusion is not quite accurate because multi-informant models do not reject the independence of the higher-order factors (DeYoung, 2006). Of course, the halo effect is still modeled in those models, but as correlations among the uniquenesses for individual raters.

   “p. 20. The authors don’t have the patterns quite right for their Saints and Superheroes model. Paulhus and John’s (1998) factor analysis showed that bias in N is more strongly related to bias in E and O than to bias in A and C. N should be either left out of their model 4 or grouped with E and O.

   “p. 22. Cohen’s rule of thumb about effect sizes appears to be wrong (Hemphill, 2003, AP; Richard, Bond, & Stokes-Zoota, 2003, RGP). It would be better to cite empirical guidelines for effect sizes.

   “p. 40. The authors “think it is worthwhile to examine the robustness of our findings with finer categories in future research. Evidently, sample sizes will have to be large given the exponential increase in patterns (e.g., 35 = 243).” Why did they not try finer categories in their Study 3? They certainly have the sample size to examine 243 patterns.”

   Review of your revised submission:

   “I have reviewed previous versions of this paper, and the current version is a great improvement. The decision to use structural models is welcome, and the concept of a model that explicitly separates variance in the Big Five due to evaluative consistency (halo) from variance due to the metatraits is innovative and promising. The manner in which the HAB model integrates previous findings is elegant, as well. These analyses support the existence of the metatraits, and further support my hypothesis (DeYoung, 2006) that the metatraits should be more easily found in better instruments. The finding that the halo factor is not consistent across raters but correlates strongly with a latent measure of bias (the shared variance among self-ratings of intelligence, attractiveness, athleticism, and trivia knowledge) is compelling as an argument that halo reflects bias rather than substantive personality variance. The association of halo with self-esteem is sensible conceptually, but the complexities of the theory and method in that study leave some doubts about the interpretation of the result (discussed below). The last study is nice as another replication and an extension to longitudinal data.

   “A central assumption of the authors’ model is that they can impose equality constraints on all loadings on halo. Halo results from a bias toward evaluative consistency; thus the more socially desirable is a trait, the more it should be influenced by halo. The equality constraint implies that all traits are equally socially desirable. The accumulated research on social desirability would suggest otherwise. Is desirability close enough to being equivalent across the Big Five that the HAB model can get away with making this assumption? One way to find out would be for the authors to fit a variant of the HAB model with the halo loadings unconstrained and test whether there is a significant improvement in fit. If there is, then the model has a problem, particularly in Study 3, where this assumption becomes central to testing their hypothesis regarding self-esteem.

   “More on Study 3: I am ready to believe that self-esteem correlates with halo because it inclines people to rate themselves generally positively or negatively, but I’m not sure that the distinction between a bias in “ratings” and a bias in “perceptions of personality traits” functions well theoretically. The halo bias produces evaluative consistency, such that one is likely to view oneself positively or negatively across the board. How is this not a bias in the perception of personality traits? Perhaps the authors would assert that the halo bias only renders one more likely to “rate” oneself positively or negatively across the board, without influencing one’s perception of oneself, but I know of no evidence that would encourage such an assumption, nor am I sure the distinction even makes sense. I like Study 3, but its hypothesis might be too ambitious. Why not simply use the latent Big Five, metatrait, and halo variables to predict observed self-esteem, in the SEM equivalent of multiple regression, and ditch the latent self-esteem variable? (Using a self-esteem measure with multiple items would have rendered latent self-esteem a more useful entity.) It would be interesting to see the degree to which self-esteem reflects real personality over and above halo. I’m skeptical about the authors’ ability to sort out in which direction causality runs (and in what proportion) between self-esteem and halo.

   “My other quibble with this paper has to do with the failure to test alternative models. In their discussion the authors write, “Thus, an important avenue for future research is to test the HAB model against alternative models.” In fact, this is an important avenue for their own research as well: they need to provide direct comparisons of fit for the HAB and CAB models in all of their datasets. If the CAB models fit better, the authors might want to explain why HAB could nonetheless be desirable in some situations.

   “I agree that the HAB model is more pleasing conceptually than the CAB model, but it does require more assumptions and constraints than the CAB model. The CAB model implicitly models halo bias, as a correlation between Alpha and Beta. If one uses the metatraits as simultaneous predictors, the CAB model therefore eliminates the effect of halo, and one gets relatively unbiased estimates of the association of each metatrait with a criterion of interest.

   “One possibility that would require fewer constraints than the authors’ HAB model would be to fit halo as a single higher-order factor influencing Alpha and Beta. Here the two loadings on halo could be constrained to be equal with less difficulty, since the relative loadings for two markers of a latent factor are arbitrary, and the Alpha loadings would not need to be constrained. One might wonder whether it makes sense conceptually to think of halo as influencing Alpha and Beta directly and the Big Five indirectly, but it would be easy enough to calculate the amount of variance in each of the Big Five due to the metatraits vs. due to halo, just as we calculate how much variance in a cognitive test is due to g vs. a lower-order ability factor like verbal ability.

   “Finally, while we’re considering at what level of the hierarchy to fit the halo factor, the authors might discuss why, in Study 2, they have halo influencing latent Big Five traits directly, rather than influencing the observed ratings at the item level. Isn’t it most conceptually accurate to depict halo as a direct influence on items?

   “Despite some concerns with various aspects of this paper, on the whole I think it is excellent, and it seems likely that the authors will be able to address my concerns.

   “Additional comments:

   “p. 3, last par. “these five factors are not at the basic level at personality description” should read “…of personality description”

   “p. 5, par 1. “in Goldberg’s IPIP warmth is a facet of A and impulsivity is contained within E” – to the best of my knowledge, impulsivity is not contained in E in Goldberg’s IPIP. In the IPIP AB5C scales (the largest collection of Big Five facets from the IPIP), “Impulse Control” is a facet of N, and other impulsivity related content is mainly within C.

   “p. 5., last par. The “D” in “deYoung” should be capitalized. (This problem recurs sporadically throughout the manuscript – e.g., pp. 13, 20, 25).

   “p. 6., par 2. “A major limitation of the previous studies is the exclusive reliance on ratings by a single rater,” – somewhat misleading because one of “the previous studies” discussed was DeYoung, 2006, a multi-method study that also analyzed single ratings separately. Perhaps this could read “A major limitation of many previous studies…”

   “p. 6, last par. “while any leftover variance that is not shared between informants is assumed to reflect biases unique to single informants.” When the authors introduce this assumption, they must point out that it is known to be flawed; multiple raters often provide incremental validity (Fiedler, Oltmanns, & Turkheimer, 2004; Mount, Barrick, & Strauss, 1994; Vazire & Mehl, in press, JPSP), which means that different raters detect different valid variance. Some unique variance is therefore valid. MTMM with multiple informants can still be useful, but we must acknowledge that nonshared variance is not all bias.

   “p. 8, end of par. 1. “and found that on average 60-80% of the observed correlations in ratings by a single rater were due to rater-specific biases.” Because of the point just raised, the authors need to note that this is probably an overestimate.

   “p. 11. “An unconstrained HAB model would only have one degree of freedom.”
   It would appear to me that an unconstrained HAB model would have no degrees of freedom. 5 would be spent estimating the loadings of the Big Five on Halo, the other 5 on Alpha and Beta. What am I missing?

   “p. 16. “In order to obtain more accurate measures of the Big Five, we adjusted the observed variances by the reliability coefficients (i.e., Cronbach’s alphas)” – I think it would be helpful for many readers to have a more thorough explanation of how this is accomplished.

   “p. 20, end of par. 1. “superheroes and saints that consider” should be “superheroes and saints who consider”

   “p. 22. “One explanation for the low internal consistency is that the correlation between the directly scored and the reverse scored items are attenuated by both evaluative bias and acquiescence bias.”
   I fail to see how evaluative bias would be likely to attenuate the correlations between the two TIPI items for each of the Big Five. Given that each of the Big Five has one desirable pole, evaluative bias should inflate the correlations of the two items. An internal consistency between .2 and .6 is not low for a scale with two items.

   “p. 22. “Internal consistency for the five personality dimensions was low (range .56 to
   .74)” – Again, this is not particularly low for 3-item scales.

   “p. 24. “However, in lieu of low correlations among the scale items” – I suspect the authors mean “However, in light of…” “In lieu” means “instead.”

   “p. 23. A couple comments about the models in Study 2. First, the uniqueness for self-rated intelligence should be allowed to correlate with O because descriptions of intelligence fall within Openness/Intellect in the Big Five (e.g., DeYoung, Quilty, & Peterson, 2007, JPSP; Hofstee, Goldberg, & de Raad, 1992, JPSP), and many studies have shown that self-rated intelligence does share at least a small amount of variance with true intelligence (i.e., with ability tests), indicating that it is not all bias. Self-rated trivia knowledge also seems likely to be validly related to Openness/Intellect. Second, here again the equality constraint for loadings on halo seems potentially problematic. I’m wondering whether this constraint could prevent the emergence of an Alpha factor. The authors should discuss whether alpha could be found with differently constrained models.

   “p. 27, par. 2. It would be preferable to use the phrase “regression path” instead of “causal path.” SEM cannot directly test causality, any more than standard regression can. All it can do (like standard regression) is to provide support for models that are consistent with causal hypotheses. Provide longitudinal data and maybe you can make stronger causal claims.

   “p. 30, par. 2. “and there independent” should be “and three independent.”

   “p. 32. “The main finding was that self-esteem was highly correlated with halo even after removing halo biases in ratings of self-esteem. This finding suggests that halo biases are not just mere rating biases, but actually reflect biases in self-perceptions.”
   The first of these sentences is true only if self-esteem truly loads equally or less strongly on halo than the Big Five, which the authors have not tested, nor could they. As noted above, it would help their case to test first whether the equal loading constraint is valid even for the Big Five. If it is not, then perhaps they could constrain the self-esteem loading to be similar to the strongest loading among the Big Five, as a conservative estimate. Regarding the second sentence, I’m still not sure that a rating bias and a self-perception bias can be sensibly distinguished. Do we imagine that people are attempting to keep their ratings evaluatively similar through some process that does not involve biased self-perception? Surely, no one is looking over the ratings thinking “If I gave myself a 4 on “friendly,” I should give myself a 4 on “diligent” to be consistent” – I would be inclined to imagine that halo works precisely through biases in perception. I might be wrong, but I think the authors should hedge their interpretation a bit and acknowledge the difficulties with it.

   “Table 1 would be clearer if spaces for non-existent informants, in the first and last three studies, were left blank, retaining the dashes only for parameters that were fixed at zero.

   “Figure 4. Shouldn’t the correlation of True SE with N be negative?

   “Sincerely,
   Colin DeYoung”

   So, there you have it. My reviews don’t seem quite so unreasonable as you make me out to be in this post. Rather than a battle, this was perhaps more like a constructive exchange in which I suggested a method that you then published and have subsequently used extensively. This is already quite long for a comment on a blog, so I’ll post my response to the substance of your argument in another one.

   Loading...

   Reply

   1. Thanks for sharing the reviews. Yes, you killed our original approach to look at the structure of personality by means of simply looking at the frequency of the 32 types that you can create with a median split of the Big Five scales. As you noted, this allows to look for interactions that bivariate correlations cannot reveal. Given how much nonsense has been published (Digman, 19987, Musek, 2007, your 2001 article, for example), this would not have hurt personality psychology. The main finding of course that there are no cool interactions and that there only two very prevalent types. Good people and bad people. Not novel, so reject. The big finding was that there were no types, but you missed this because we only publish sexy results. That is exactly the problem with psychology. Digman invented the Big Two and now they cannot die because psychologist don’t know how to kill bad theories, they only know how to kill papers they don’t like in the review process.

      Loading...

      Reply

3. Okay, on to the scientific questions. As you acknowledge here, and as shown in the meta-analysis by Chang et al. that I mentioned in my previous comment, correlations among the Big Five in a variety of measures are somewhat attenuated but still present even when modeling the Big Five as the shared variance of scores from multiple raters. These studies show that Digman’s higher-order factors are real but relatively weak factors. I think these factors are of interest conceptually, but I also agree that it is generally more valuable empirically to study levels of the trait hierarchy below the Big Five. None of this gets at the real substance of your critique, however, which is that you think the correlations among Big Five scales are likely to be artifactual for reasons other than individual rater biases, namely cross-loadings.

   Let’s talk first about whether it makes sense to study higher-order factors, given that the Big Five were conceived as orthogonal, then we’ll move on to the issue of cross-loadings. In my 2006 JPSP paper that you cite extensively here, I wrote:

   “The traditional conception of the Big Five as orthogonal is partly a historical accident resulting from the methods used in their discovery. Had more of the factor analyses that originally validated the Big Five model been performed with oblique rotations that allow correlations among factors, instead of orthogonal rotations that artificially prevent any correlations among factors (at the expense of explaining less variance), we might never have had to debate the reality of correlations among the Big Five.”

   In general, I prefer oblique rotation because it entails one less assumption being imposed on the analysis. Why should we assume that simple structure should lead to orthogonal factors? If we use oblique rotation, then we’ll find out how correlated the factors are. They might well be nearly uncorrelated, which would be a more informative outcome than forcing them to be uncorrelated using orthogonal rotation. You wrote, “One important feature of PCA is that the components are independent (orthogonal)” but of course this is only true if one does not rotate the components or uses an orthogonal rotation.

   Using an oblique rotation in an adjective dataset like those used to demonstrate the Big Five and then factor-analyzing the resulting factor correlation matrix would be an obvious way to test the existence of the higher-order factors. Further, this method would also deal with the problem of cross-loadings because in PCA/EFA all cross-loadings are included. You suggest that I ignored the paper by Ashton et al. (2009) arguing that cross-loadings accounted for the higher-order factors. This is not true. I briefly discussed the problems with their conclusions in this paper (https://www.frontiersin.org/articles/10.3389/fnhum.2013.00762/full), and I commented more extensively on the problems in a letter I wrote to Jack Block in reply to a letter he sent me after Ashton et al.’s paper was published. I’ll come back to this as an appendix (see next comment) but first I want to show you some data because I think that’s the best argument.

   I conducted common factor analysis with oblimin rotation (delta = 0) in a sample of 700 participants in the Eugene-Springfield community sample (publicly available from the Harvard Dataverse). The variables were 395 of the adjective most commonly used to describe people, which were a subset that remained after I excluded adjectives that seemed purely evaluative (“awful”), purely or substantially physical (“tall,” “attractive”), demographic (“rich,” “educated”), or external (“lucky”) from the full list of 525 adjectives described by Saucier (1997, JPSP). A five-factor solution yielded the standard Big Five factors (I’ll paste this in another comment). Further, these factors were correlated in the following manner (note these are the correlations among the factors themselves, not among factor score estimates):

   A 1.00
   E .021 1.00
   N -.230 .087 1.00
   C .214 -.003 -.259 1.00
   O .144 .236 -.065 .123 1.00

   This correlation matrix yields the following two-factor solution:

   N -.542 .094
   C .485 .021
   A .435 .131
   O .247 .599
   E -.099 .414

   Structure Matrix (oblimin, delta = 0), factor correlation = .051

   In other words, one can account for every possible cross-loading, and the higher-order factors of the Big Five are still alive and well (and uncorrelated!). Long live Stability and Plasticity. Now, you might argue that the problem with this approach is that it doesn’t model the halo or acquiescence biases, and I grant you that those biases are likely to be distorting correlations among items in this analysis to some extent. However, the optimal solution to this problem requires data with multiple raters. I appreciate the efforts you have made to develop a model that includes these two bias factors for individual raters, and it’s interesting to consider your results, but I’m afraid your model is inescapably flawed because the halo factor will always confound substance with style, given that differences in self-esteem (strongly negatively associated with depression; indeed, low self-worth is often measured as an element of depression), which is a substantive trait, are strongly related to halo. Also, Pettersson, Turkheimer, Horn, & Menatti (2011, EJP) found that items related to depression were so strongly evaluative that they concluded one can’t measure depression adequately. I draw a different conclusion, which is that depression is so undesirable that it is seriously confounded with the evaluative factor. For these reasons, extracting halo in single-informant ratings inappropriately removes substantive variance related to self-esteem and depression. This is probably why you sometimes had trouble identifying Alpha/Stability in your 2009 paper.

   Another flaw is that your model is a bifactor model (i.e., one with general factors as well as narrower factors), and simulation evidence is accumulating that bifactor models often fit better than other models even when they are not the generating model (e.g., Greene et al., 2019, Journal of Abnormal Psychology). The general factors may soak up variance that should be modeled in the other factors. (Nonetheless, I’ll note that one way you could improve your model is that, instead of setting all halo loadings to 1 by default, you could do what Pettersson et al., 2011, EJP, have done and fix the loadings according to ratings of desirability made by other subjects.) At any rate, I don’t think you’ll be able to provide better evidence on this question unless you have self- and peer ratings on an unbiased selection of personality descriptors. Using self- and peer ratings of items from purpose-built Big Five instruments won’t work because they are often carefully designed to reduce correlations among the Big Five.

   For now at least, given the convergence between multi-rater studies of Big Five scales and the factor analysis of adjectives I presented above, it seems safe to conclude that the higher-order factors of the Big Five are real. In one of your recent blog posts, you asked, “how would it be possible that personality items that were created to measure a personality attribute and that were selected to show internal consistency and convergent validity with informant ratings do not reflect a personality trait?” I agree that this isn’t possible, and I’d like to point out that I identified a set of items that correlate substantially and similarly with the three Big Five traits (rated by self and peers) within Stability and another set that correlates with both Extraversion and Openness (https://www.tandfonline.com/doi/full/10.1080/10478401003648674). They have good internal consistency and clearly are personality traits by your standards, whether or not you want to acknowledge them as higher-order factors of the Big Five.

   Loading...

   Reply

   1. By all means do an oblique rotation, but please make sure that your item sample is representative and that you control for rating biases like halo that you acknowledge influence correlations among items. As you know in multi-method studies, many of the cross-factor-cross-rater correlations are close to zero.

      Loading...

      Reply

4. Appendix: Some thoughts on Ashton et al. (2009), originally delivered to Jack Block:

   “First of all, they acknowledge what we all know: that personality traits do not have simple structure. But this means that a strict higher-order factor model starting from distinct facets is doomed to fit poorly. It does; no surprise there. Then they compare it to an orthogonal factor model in which they use the five largest modification indices to choose which cross loadings to allow. By definition, these are the five cross loadings that maximally improve the fit of the model in this particular sample and instrument (so this is hardly a “confirmatory” factor analysis). Then they crow about the fact that the cross-loading model fits better than the higher-order factor model.

   “I’m afraid I’m not impressed, and not just because, given the lack of simple structure in personality and given their approach, the conclusion was foregone. Of the 5 cross loadings they allowed, 3 of them cross from alpha- to beta-related traits or vice versa. It is explicit in their argument that they are identifying a type of artifact that explains the existence of the consistent, replicable, higher-order factor structure of the Big Five. For their explanation to be credible, they would have to show that the important cross-loadings involve traits from within the same higher-order factor. Neuroticism shows no cross loadings here, and its subscales receive only one cross loading – from Extraversion. How does this help to explain why N is related to A and C sufficiently to form the Alpha/Stability factor, even when controlling for the biases of individual raters or examining genetic correlations?”

   And a note from the same letter about the implausibility of Ashton et al.’s explanations for why there would exist just the pattern of cross-loadings that would be necessary to reliably produce the standard higher-order factors: “They argue that same-sign blends are more common than opposite-sign blends, especially among the Stability traits, but they fail to mention that there are so many descriptors located at the intersection of low Emotional Stability and high Agreeableness (e.g., sensitivity, sentimentality, femininity) that they constitute the primary markers of one of the HEXACO factors! Ashton himself acknowledges that his Emotionality and Agreeableness factors are 45-degree rotations of Big Five N and A, with Emotionality a combination of high N and high A.” Their explanations for why some cross-loadings would exist but not others are just-so stories, and one can easily think up reasons why other patterns (including some that actually exist, as just described in the quoted passage) would be likely that would not lead to the same higher-order factors.

   Loading...

   Reply

   1. I like Block’s critic of the Big Five (1995) and some of his other work, but he is not known to be an expert on psychometrics or SEM. I don’t care about Ashton’s specific model, I just wanted to give them credit for pointing out that we can fit data in two ways. With a higher-order factor or with secondary loadings. Simply fitting only a higher-order model and ignoring the secondary loading model is not good science. I think the best way is to see how many secondary loadings there are that are consistent with the idea of a higher-order model. If E and O share a common higher order factor MOST E items should load on O and most O items should load on E. We shouldn’t see A LOT of E items that only load on E or O items that only load on O. Moreover, the sign of the loadings should be consistent with the predictions from the higher order model. Another approach would be to identify actual causes of higher-order factors and then show that they relate to E and O items. So, we can only test the theory with a proper measurement model at the item level. Once we sum items into scales the critical information is lost. So, I am proposing a methodological way forward. I personally don’t care whether we find evidence for alpha and beta or not. I just don’t like research that aims to confirm a theory rather than testing it.

      Loading...

      Reply

5. For the truly curious. Big Five factor loadings for 395 adjectives.

   A E N C O
   Kind-hearted .779 -.126 -.240 .302 -.155
   Warm-hearted .761 -.050 -.139 .240 -.143
   Warm .751 .098 -.184 .160 -.205
   Kind .716 -.187 -.318 .355 -.162
   Friendly .707 .080 -.328 .244 -.214
   Caring .703 -.107 -.148 .301 -.213
   Giving .702 -.003 -.184 .256 -.165
   Cheerful .686 .152 -.356 .212 -.140
   Compassionate .680 -.162 -.151 .253 -.192
   Loving .674 .000 -.159 .216 -.240
   Understanding .655 -.192 -.219 .299 -.234
   Lovable .655 .093 -.198 .124 -.198
   Pleasant .651 -.116 -.305 .302 -.251
   Encouraging .651 .055 -.199 .207 -.240
   Good-humored .649 .102 -.296 .181 -.213
   Helpful .636 -.099 -.317 .435 -.216
   Joyful .634 .252 -.350 .162 -.156
   Good-natured .632 -.062 -.283 .164 -.104
   Considerate .626 -.174 -.344 .492 -.191
   Supportive .626 -.059 -.168 .285 -.255
   Laughing .622 .310 -.082 -.007 -.172
   Glad .619 .202 -.329 .213 -.132
   Affectionate .616 .145 -.114 .127 -.255
   Appreciative .614 -.044 -.237 .413 -.092
   Delighted .603 .273 -.124 .115 -.278
   Sympathetic .598 -.188 -.152 .235 -.115
   Likeable .598 .051 -.279 .293 -.267
   Sincere .594 -.113 -.282 .484 -.215
   Gentle .593 -.300 -.189 .174 -.062
   Nice .581 -.165 -.187 .258 -.124
   Grateful .578 .055 -.160 .311 -.053
   Generous .578 .073 -.101 .169 -.180
   Gracious .576 .015 -.174 .326 -.137
   Cooperative .575 -.107 -.262 .391 -.116
   Enthusiastic .570 .405 -.188 .142 -.336
   Thankful .562 .084 -.192 .334 .020
   Thoughtful .555 -.103 -.234 .435 -.379
   Agreeable .553 -.176 -.281 .244 -.101
   Forgiving .552 -.112 -.225 .192 -.100
   Polite .548 -.071 -.270 .505 -.049
   Unfriendly -.544 .050 .479 -.284 .073
   Happy .533 .226 -.422 .206 -.124
   Pleased .532 .283 -.269 .205 -.178
   Hopeful .526 .154 -.171 .205 -.157
   Trusting .521 .080 -.232 .176 .049
   Sensitive .518 -.155 -.040 .150 -.174
   Romantic .512 .136 -.061 .069 -.212
   Open .511 .123 -.181 .095 -.311
   Courteous .511 -.079 -.212 .481 -.086
   Genuine .504 -.041 -.226 .470 -.261
   Neighborly .503 .114 -.161 .192 -.083
   Devoted .501 .085 -.125 .373 -.008
   Optimistic .500 .218 -.361 .163 -.226
   Eager .498 .276 -.122 .191 -.384
   Fair .493 -.068 -.222 .432 -.208
   Lively .491 .459 -.188 .148 -.293
   Cold -.489 .091 .438 -.146 .097
   Unsympathetic -.489 .244 .349 -.292 .115
   Sociable .488 .295 -.272 .184 -.114
   Flexible .478 -.044 -.230 .114 -.293
   Tolerant .477 -.222 -.266 .014 -.189
   Social .473 .361 -.271 .187 -.146
   Peaceful .471 -.131 -.390 .163 .002
   Playful .468 .345 .005 -.031 -.312
   Excited .462 .444 .003 .101 -.358
   Expressive .459 .327 -.051 .136 -.423
   Comfortable .458 .148 -.374 .344 -.120
   Interested .456 -.001 -.198 .333 -.422
   Sentimental .452 .031 .069 .112 .096
   Passionate .447 .260 .026 .033 -.272
   Concerned .446 -.012 .042 .247 -.196
   Spontaneous .429 .286 -.004 -.041 -.372
   Inspirational -.429 .300 .403 -.230 .078
   Reasonable .424 -.196 -.312 .421 -.263
   Protective .419 .019 .042 .324 -.075
   Easygoing .418 -.036 -.263 .008 -.062
   Insensitive .416 .205 -.059 .096 -.374
   Popular .412 .392 -.297 .214 -.276
   Humorous .404 .286 .041 -.028 -.307
   Charming .393 .292 -.175 .144 -.277
   Patient .364 -.235 -.326 .154 -.093
   Down-to-earth .356 -.080 -.159 .256 -.010
   Attentive .355 -.082 -.245 .313 -.243
   Busy .350 .177 -.147 .344 -.253
   Stingy -.346 .088 .345 -.143 .120
   Emotional .342 .093 .310 -.022 -.024
   Idealistic .332 .008 -.011 .028 -.207
   Familiar .325 .233 -.019 .054 -.116
   Carefree .308 .222 -.253 -.052 -.095
   Surprised .300 .134 .113 -.030 .075
   Fashionable .299 .250 -.068 .154 -.024
   Laid-back .259 -.103 -.127 -.120 -.073
   Casual .241 -.099 -.049 .015 -.125
   Religious .226 .010 -.039 .176 .178
   Innocent .213 -.072 .035 .059 .200
   Informal .169 -.168 -.039 .079 -.124
   Aggressive -.055 .586 .023 .138 -.251
   Bold .054 .542 .052 .037 -.445
   Cocky -.108 .536 .261 -.180 -.308
   Assertive .145 .522 -.134 .261 -.413
   Powerful .110 .522 -.047 .098 -.480
   Loud .000 .519 .289 -.208 -.082
   Forward .191 .515 -.032 .076 -.294
   Outgoing .475 .509 -.220 .136 -.219
   Noisy .076 .508 .251 -.233 -.045
   Dominant -.111 .485 .199 .143 -.307
   Extraverted .342 .468 -.118 .001 -.260
   Rowdy .005 .468 .259 -.251 -.194
   Entertaining .426 .464 -.110 .043 -.415
   Controlling -.133 .449 .323 .099 -.229
   Bossy -.085 .439 .403 .044 -.151
   Hilarious .234 .430 .066 -.155 -.236
   Egotistical -.191 .430 .229 -.075 -.326
   Intimidating -.216 .428 .329 .032 -.349
   Quiet -.030 -.422 .018 .172 .006
   Competitive .000 .416 -.030 .187 -.219
   Talkative .402 .410 -.004 .062 -.240
   Colorful .308 .407 .114 -.097 -.347
   Rough -.300 .406 .395 -.229 -.118
   Soft-spoken .039 -.405 -.008 .056 .055
   Uninhibited .219 .399 -.023 -.046 -.309
   Comical .279 .383 .089 -.142 -.238
   Conceited -.173 .382 .262 -.141 -.259
   Opinionated -.079 .381 .311 .059 -.263
   Demanding -.170 .379 .301 .140 -.289
   Amusing .331 .378 .039 -.071 -.344
   Stuck-up -.347 .377 .375 -.150 -.113
   Seductive .187 .369 .069 -.079 -.279
   Controversial -.083 .363 .344 -.158 -.342
   Energetic .350 .352 -.253 .275 -.277
   Shy -.069 -.351 .316 -.147 .108
   Funny .337 .339 .043 -.075 -.325
   Ambitious .282 .305 -.130 .270 -.211
   Impulsive .205 .283 .231 -.216 -.212
   Spoiled -.094 .275 .274 -.111 -.030
   Materialistic -.109 .238 .160 .068 -.019
   Sad -.225 -.113 .710 -.222 .077
   Irritated -.293 .084 .710 -.131 .023
   Troubled -.168 -.043 .707 -.210 .030
   Insecure -.123 -.149 .703 -.360 .130
   Confused -.121 -.001 .680 -.417 .108
   Crabby -.325 .222 .678 -.138 .079
   Angry -.255 .146 .678 -.195 -.038
   Disappointed -.157 -.003 .673 -.176 .019
   Frustrated -.067 -.004 .672 -.147 -.002
   Upset -.167 .061 .662 -.156 .045
   Grumpy -.319 .171 .659 -.088 .054
   Frustrating -.221 .128 .654 -.218 .000
   Worried -.027 -.109 .651 -.131 .122
   Annoyed -.227 .121 .646 -.036 .021
   Disturbed -.183 .007 .643 -.299 .110
   Unhappy -.322 -.138 .637 -.233 .151
   Moody -.155 .046 .633 -.057 .021
   Afraid .006 -.142 .633 -.242 .178
   Grouchy -.330 .243 .629 -.106 .012
   Hostile -.335 .282 .624 -.248 -.036
   Difficult -.271 .304 .623 -.210 -.093
   Complaining -.215 .136 .620 -.206 .110
   Irritable -.283 .254 .619 -.098 -.034
   Uncomfortable -.210 -.105 .617 -.290 .101
   Scared -.015 -.103 .614 -.218 .142
   Depressed -.158 -.136 .610 -.200 .053
   Fretful -.062 .007 .609 -.131 .077
   Annoying -.244 .262 .609 -.287 -.025
   Aggravating -.281 .282 .600 -.213 -.068
   Ashamed -.115 .023 .593 -.294 .109
   Unsure -.087 -.288 .591 -.374 .223
   Tense -.026 -.027 .588 -.064 .058
   Lonesome -.167 -.153 .583 -.218 .000
   Bored -.265 -.036 .578 -.221 .152
   Awkward -.024 -.066 .577 -.308 .044
   Temperamental -.175 .238 .572 -.113 .003
   Furious -.225 .234 .571 -.218 -.006
   Bitter -.262 .012 .568 -.253 .027
   Unstable -.280 .062 .564 -.404 .038
   Irritating -.280 .233 .564 -.213 -.057
   Touchy -.090 .102 .560 .041 .030
   Lonely -.155 -.121 .556 -.234 .007
   Well-adjusted .424 .114 -.553 .414 -.257
   Cranky -.274 .241 .546 -.113 .063
   Selfish -.338 .236 .539 -.308 -.079
   Embarrassing -.138 .206 .538 -.307 .019
   Anxious .020 .045 .535 -.068 .131
   Needy -.073 .120 .534 -.275 .016
   Rude -.356 .335 .532 -.372 -.056
   Humiliated -.138 .056 .527 -.236 .134
   Withdrawn -.289 -.260 .526 -.234 .056
   Irrational -.098 .246 .525 -.397 .040
   Offensive -.253 .344 .521 -.276 -.067
   Confusing -.116 .118 .516 -.390 -.043
   Suspicious -.193 .060 .515 -.068 .009
   Immature -.116 .129 .513 -.405 .105
   Obnoxious -.253 .419 .513 -.381 .043
   Self-assured .284 .354 -.509 .354 -.314
   Defensive -.092 .100 .509 -.055 .081
   Envious -.096 .041 .507 -.224 .146
   Embarrassed -.081 -.096 .507 -.216 .184
   Obsessive -.166 .257 .504 -.119 -.138
   Exhausted -.046 -.059 .502 -.156 .085
   Short-tempered -.260 .261 .502 .060 .074
   Heartbroken -.012 -.007 .499 -.160 .024
   Arrogant -.270 .437 .497 -.208 -.255
   Greedy -.239 .217 .495 -.260 -.017
   Unreasonable -.331 .224 .490 -.358 .178
   Inconsiderate -.435 .172 .490 -.402 .088
   Harsh -.300 .335 .489 -.054 -.025
   Insulting -.392 .380 .489 -.247 -.013
   Hot-tempered -.250 .317 .487 -.028 -.001
   Confident .303 .361 -.482 .397 -.394
   Cruel -.316 .338 .480 -.334 .086
   Argumentative -.193 .381 .476 -.101 -.200
   Foolish .001 .171 .471 -.380 .010
   Nervous .053 -.074 .471 -.088 .146
   Guilty -.040 .047 .469 -.266 .097
   Preoccupied -.100 .027 .468 -.117 -.094
   Jealous -.068 .217 .462 -.141 .052
   Thoughtless -.345 .192 .461 -.329 .106
   Weak -.124 -.158 .460 -.282 .302
   Mean -.347 .340 .458 -.209 -.012
   Abusive -.271 .403 .457 -.280 .053
   Secure .257 .234 -.455 .258 -.040
   Dumb -.109 .204 .455 -.319 .236
   Self-conscious -.023 -.187 .452 -.066 .107
   Self-confident .238 .430 -.450 .343 -.376
   Satisfied .359 .117 -.450 .210 -.001
   Helpless -.025 -.073 .447 -.286 .198
   Tired -.073 -.202 .446 -.127 .120
   Rebellious -.117 .229 .444 -.237 -.392
   Indecisive -.054 -.231 .439 -.315 .243
   Violent -.224 .356 .438 -.166 -.070
   Wishy-washy -.092 -.124 .438 -.427 .231
   Self-centered -.316 .266 .435 -.197 -.114
   Reckless -.130 .384 .430 -.415 -.120
   Inexperienced -.060 -.067 .420 -.262 .262
   Impatient -.137 .272 .420 -.001 -.080
   Critical -.214 .149 .414 .100 -.150
   Impolite -.317 .196 .412 -.366 .001
   Childish .046 .287 .409 -.367 -.088
   Unfair -.327 .196 .403 -.380 .186
   Ignorant -.060 .092 .396 -.371 .194
   Disobedient -.196 .171 .391 -.337 -.235
   Sarcastic -.237 .279 .390 -.116 -.102
   Phony -.213 .151 .386 -.355 .134
   Corrupt -.202 .242 .382 -.348 .059
   Bashful -.041 -.328 .381 -.101 .158
   Relaxed .270 .012 -.380 .080 -.088
   Compulsive -.045 .220 .379 -.182 -.049
   Possessive -.053 .243 .372 .058 .009
   Snobbish -.246 .276 .371 -.181 -.122
   Clumsy .063 -.027 .367 -.199 .093
   Narrow-minded -.279 .127 .365 -.071 .311
   Calm .259 -.113 -.362 .204 -.143
   Shallow -.262 .099 .358 -.338 .334
   Boring -.280 -.137 .350 -.149 .266
   Cheap -.243 .026 .348 -.204 .134
   Dangerous -.227 .345 .348 -.229 -.150
   Stupid -.081 .166 .343 -.282 .219
   Superstitious .005 .150 .342 -.216 .019
   Ungraceful -.153 -.103 .339 -.227 .045
   Nosy .053 .210 .339 -.153 -.018
   Stubborn -.105 .195 .335 .063 -.073
   Closed-minded -.318 .165 .332 -.023 .286
   Naive .148 -.074 .329 -.140 .245
   Frightening -.256 .276 .327 -.223 -.130
   Rushed .014 .099 .310 .031 -.075
   Prejudiced -.248 .160 .309 -.053 .106
   Active .266 .277 -.305 .224 -.219
   Forgetful .035 -.043 .300 -.263 .142
   Unenvious .059 -.024 -.207 .104 -.160
   Responsible .415 -.078 -.225 .671 -.253
   Practical .248 -.071 -.214 .648 -.113
   Reliable .399 -.050 -.214 .636 -.167
   Conscientious .400 -.043 -.244 .634 -.251
   Thorough .265 .019 -.190 .627 -.283
   Sensible .305 -.110 -.209 .616 -.139
   Efficient .167 .075 -.243 .614 -.187
   Level-headed .375 -.037 -.282 .610 -.206
   Trustworthy .429 -.110 -.252 .593 -.136
   Consistent .207 .049 -.263 .592 -.067
   Dependable .327 -.066 -.191 .587 -.160
   Organized .088 .126 -.219 .584 -.090
   Alert .325 .071 -.320 .582 -.340
   Competent .310 .062 -.287 .577 -.470
   Honest .366 -.169 -.291 .564 -.096
   Realistic .130 -.054 -.250 .560 -.156
   Irresponsible -.209 .127 .428 -.552 .089
   Careless -.149 .159 .488 -.546 -.012
   Systematic .022 .022 -.094 .544 -.142
   Inefficient -.123 -.025 .464 -.539 .166
   Disorganized -.002 -.017 .389 -.535 -.005
   Productive .271 .145 -.203 .532 -.346
   Respectful .510 -.119 -.214 .528 -.017
   Trusted .496 -.126 -.276 .526 -.173
   Loyal .502 -.091 -.167 .522 -.075
   Stable .381 -.040 -.407 .521 -.129
   Unreliable -.260 .135 .380 -.519 .170
   Firm .125 .257 -.172 .517 -.156
   Undependable -.179 .155 .340 -.514 .066
   Sloppy -.178 .041 .455 -.509 -.024
   Mature .320 -.064 -.244 .507 -.172
   Logical .147 -.073 -.125 .502 -.338
   Truthful .398 -.161 -.205 .494 -.053
   Dedicated .422 .119 -.127 .489 -.248
   Incompetent -.160 .035 .460 -.482 .294
   Ethical .300 -.201 -.192 .476 -.128
   Rational .160 -.117 -.225 .475 -.235
   Determined .360 .222 -.124 .472 -.403
   Effective .247 .202 -.239 .471 -.430
   Decent .441 -.154 -.202 .467 -.167
   Punctual .113 -.039 -.132 .461 .001
   Prompt .065 -.072 -.173 .458 .025
   Certain .292 .282 -.314 .452 -.281
   Unfaithful -.251 .164 .327 -.449 .001
   Traditional .187 -.032 -.063 .442 .220
   Faithful .394 -.153 -.247 .437 .011
   Careful .241 -.248 -.029 .433 .062
   Disrespectful -.287 .172 .420 -.432 -.046
   Wasteful -.106 .152 .351 -.425 .029
   Hard-working .269 .106 -.187 .423 -.277
   Strict .016 .163 .062 .419 .078
   Straightforward .174 .087 -.202 .418 -.369
   Businesslike .039 .088 -.118 .415 -.110
   Proper .303 -.032 -.137 .410 .103
   Self-sufficient .193 .078 -.275 .399 -.263
   Messy -.020 .048 .278 -.392 -.185
   Believable .327 -.167 -.175 .383 -.143
   Persistent .212 .263 -.031 .378 -.365
   Proud .325 .278 -.111 .372 -.103
   Direct .153 .207 -.133 .368 -.357
   Serious -.012 -.159 .129 .367 -.151
   Independent .162 .169 -.170 .361 -.358
   Unpredictable -.066 .265 .304 -.352 -.222
   Observant .196 .032 -.175 .336 -.331
   Cautious .198 -.254 .063 .310 .221
   Dishonest -.203 .135 .305 -.308 -.011
   Predictable .164 -.176 -.101 .287 .269
   Modest .213 -.249 .008 .255 .147
   Procrastinating .113 -.105 .212 -.252 -.023
   Private -.032 -.205 .091 .225 -.125
   Lazy -.123 -.027 .038 -.167 .089
   Smart .188 .053 -.191 .289 -.687
   Intelligent .170 -.048 -.195 .271 -.685
   Exceptional .189 .357 -.077 .078 -.674
   Clever .222 .293 -.114 .096 -.664
   Brilliant .024 .298 -.019 .004 -.664
   Knowledgeable .205 .063 -.202 .318 -.649
   Excellent .275 .224 -.160 .228 -.648
   Extraordinary .168 .426 -.043 .014 -.648
   Intellectual .098 .082 -.100 .125 -.642
   Bright .157 .034 -.159 .196 -.609
   Outstanding .129 .377 -.026 .125 -.605
   Skilled .161 .133 -.185 .285 -.581
   Creative .211 .145 -.028 .044 -.568
   Wise .293 .072 -.220 .285 -.566
   Skillful .090 .119 -.208 .333 -.563
   Deep .068 .055 .141 .011 -.563
   Complex -.036 .112 .195 .038 -.562
   Imaginative .258 .054 -.069 .105 -.556
   Original .270 .208 -.004 .096 -.554
   Fascinating .308 .431 -.096 -.004 -.553
   Thinking .225 -.077 -.021 .330 -.553
   Complicated -.077 .149 .323 -.046 -.524
   Exciting .379 .489 -.124 .071 -.519
   Persuasive .217 .337 -.078 .235 -.518
   Accomplished .230 .158 -.222 .300 -.511
   Ordinary .083 -.210 .040 .044 .495
   Unintellectual -.158 .020 .277 -.177 .493
   Perceptive .300 -.007 -.185 .277 -.492
   Brave .245 .332 -.069 .144 -.479
   Outspoken .082 .449 .083 .114 -.464
   Uncreative -.154 -.061 .269 -.072 .459
   Resourceful .303 .107 -.223 .432 -.459
   Curious .312 .029 -.009 .157 -.459
   Philosophical .243 -.045 .003 .036 -.453
   Courageous .268 .305 -.095 .223 -.449
   Experienced .274 .181 -.256 .303 -.443
   Amazing .151 .365 .083 -.054 -.436
   Valuable .348 .185 -.108 .297 -.430
   Constructive .344 .017 -.137 .313 -.430
   Daring .048 .409 .067 -.081 -.424
   Open-minded .414 -.027 -.190 .006 -.419
   Artistic .200 .056 .041 -.044 -.410
   Tough -.010 .338 .124 .114 -.409
   Unforgettable .170 .234 .029 -.065 -.400
   Sophisticated .108 .192 -.027 .048 -.392
   Witty .256 .374 .047 .005 -.391
   Distinguished .131 .343 -.103 .088 -.387
   Radical -.034 .186 .317 -.354 -.375
   Adventurous .227 .353 -.067 .037 -.372
   Conservative -.084 -.029 -.006 .301 .351
   Liberal .246 -.025 .079 -.231 -.314
   Harmless .104 -.138 -.025 .017 .230
   Old-fashioned .092 -.072 .080 .180 .209

   Loading...

   Reply

   1. Awesome!! Thanks

      Loading...

      Reply

6. just now JRP published an article that confirms my results with several datasets.

   https://www.sciencedirect.com/science/article/abs/pii/S0092656618301971

   Loading...

   Reply
7. Pingback: Testing Hierarchical Models of Personality with Confirmatory Factor Analysis | Replicability-Index
8. Pingback: 2019 Blogs | Replicability-Index