- Email: [email protected]
Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory
Author’s Accepted Manuscript Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory A Mokken analysis of the Norwegian general population study for scalability. P. Bech., D. Carrozzino, S.F. Austin, S.B. Møller, O. Vassend www.elsevier.com/locate/jad
PII: DOI: Reference:
S0165-0327(15)31185-X http://dx.doi.org/10.1016/j.jad.2015.12.039 JAD7930
To appear in: Journal of Affective Disorders Received date: 29 October 2015 Revised date: 3 December 2015 Accepted date: 27 December 2015 Cite this article as: P. Bech., D. Carrozzino, S.F. Austin, S.B. Møller and O. Vassend, Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory A Mokken analysis of the Norwegian general population study for scalability., Journal of Affective Disorders, http://dx.doi.org/10.1016/j.jad.2015.12.039 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory. A Mokken analysis of the Norwegian general population study for scalability.
Bech P
1)
Carrozzino D1) Austin SF
1)
Møller SB
1)
Vassend O
2)
Word count: Abstract: 202 Main text: 2197 Tables: 2
Revised Version 01.12.2015
1) Psychiatric Research Unit, Psychiatric Centre North Zealand, University of Copenhagen, Hillerød, Denmark
2) Department of Psychology, Psychological Institute. University of Oslo, 0317 Oslo, Norway.
Corresponding author: Per Bech Psychiatric Research Unit Psychiatric Centre North Zealand, Copenhagen University Hospital Dyrehavevej 48 DK-3400 Hillerød, Denmark [email protected] Tel.: +45 38 64 30 95 Fax: +45 48 26 38 77 Abstract Background
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Whereas the Eysenck Neuroticism Scale only contains items covering negative mental health to measure dysthymia, the NEO Personality Inventory (NEO-PI) contains neuroticism items covering both negative mental health and positive mental health (or euthymia). The consequence of wording items both positively and negatively within the NEO-PI has never been psychometrically investigated. The aim of this study was to perform a validation analysis of the NEO-PI neuroticism scale. Methods Using a Norwegian general population study we examined the structure of the negatively and positively formulated items by principal component analysis (PCA). The scalability of the identified two groups of euthymia versus dysthymia items was examined by Mokken analysis. Results With a response rate of 90%, 1082 individuals with a completed NEO-PI were available. The PCA identified the neuroticism scale as the most distinct where 14 items had acceptable loadings for the euthymia subscale, another 14 items for the dysthymia subscale. However, the Mokken analysis coefficient of homogeneity only found acceptable scalability for the euthymia subscale. Limitations A comparison with the Eysenck Neuroticism Scale was not performed. Conclusion The NEO-PI neuroticism scale contains two subscales consisting of items worded in an opposite direction where only the positive euthymia items have an acceptable scalability.
Key words NEO Personality Inventory; Mokken analysis; Euthymia; Dysthymia. Introduction The hierarchical structure of the NEO-PI (Neuroticism, Extraversion, Openness Personality Inventory) was demonstrated by Costa & McCrae (1985) by use of principal component analysis (PCA). Neuroticism,
1
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Extraversion and Openness (NEO) were found more distinct than Agreeableness or Conscientiousness in the five-factor model behind the NEO-PI (Costa and McCrae, 1995). This is in concordance with the PCA approach used to demonstrate that neuroticism and extraversion are the two distinct factors in the Eysenck Personality Questionnaire (EPQ) (Eysenck and Eysenck, 1976). The NEO-PI Neuroticism factor includes facets such as anxiety, hostility, depression, self-consciousness, impulsivity, and vulnerability, covering all DSM-IV personality disorders clusters: A (e.g. paranoia), B (e.g. antisocial), and C (e.g. anxious avoidant) (Farmer, et al, 2002). In contrast, the neuroticism dimension in EPQ only covers cluster C, referred to by Eysenck and Eysenck (1985) as dysthymia, which includes components of depression, anxiety, and interpersonal sensitivity. In DSM-IV the clinical construct of dysthymia was a co-existence of subclinical chronic depression and personality disturbance. However, in DSM-5 dysthymia indicates a co-existence of chronic major depression and subthreshold depression implying that the term dysthymia has been deleted and the category renamed: “persistent depression disorder”. The clinical construct of dysthymia for the personality dimension of neuroticism was used by Farmer et al (2002) to indicate that a high neuroticism score covers facets of being anxious, worrying, or pessimistic, whereas a low neuroticism score covers facets of being relaxed, stable, or coping well with adversities. These polar opposites have been referred to as dysthymia versus euthymia by Fava and Bech (2016). However, these concepts should not be considered as opposite ends on the same scale because clinimetrically, they refer to two different dimensions, namely ill-being or dysthymia (e.g. easily becoming anxious or helpless), and well-being or euthymia (being able to face problems) (Fava and Bech, 2016; Stochl, et al, 2012). Euthymia can be seen as a positive mental state and it has clinical relevance since studies have shown that attainment of this state after a depression has been associated with significantly lower rates of relapse (Fava and Tomba, 2009; Hardeveld, et al, 2010). From a measurement perspective we have recently evaluated the neuroticism factor in the EPQ (Bech, et al, 2012) using the item response theory model developed by Mokken (1971). This model is a generalisation of the Guttman scaling (Bech, 2012), which evaluates unidimensionality or scalability without reliance on
2
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
the classical psychometrics such as the Cronbach alpha coefficient. In their validation study of the NEO-PI, Vassend and Skrondal (1997; 1995) used confirmative factor analysis which evaluates some measurement properties in contrast to explorative factor analysis, PCA or Cronbach’s alpha. However Vassend and Skrondal (1997; 1995), like McCrae et al (1996) had problems reproducing the NEO structure. When analysing the EPQ we used both PCA and the Mokken analysis (Bech, et al, 2012; Moeller, et al, 2015). To the best of our knowledge the NEO-PI has not been psychometrically validated by the Mokken analysis. When evaluating the scalability of this neuroticism scale to test that high total scores correspond to high levels of neuroticism, or dysthymia, while low scores correspond to euthymia (being relaxed, stable, coping well with adversity) it is psychometrically important to take the basic pattern of the NEO-PI into account, namely that this scale includes reversed scoring, i.e. the items which measure positive mental health are reversed to correspond to the items which measure negative mental health. Consequently, we have followed Stochl et al (2012) in performing the Mokken analysis separately on the items for positive versus negative mental health formulation. With reference to Fava and Bech (2016) we named the NEO-PI Neuroticism subscale measuring positive mental health “euthymia” and the subscale measuring negative mental health “dysthymia”, corresponding to the Eysenck neuroticism scale. On this basis we have reanalysed the NEO-PI combining PCA and Mokken analysis in order to answer the following two questions:
Research question 1 Using PCA analysis, unrotated or with varimax rotation, to what extent is neuroticism the most distinct component in the NEO-PI as indicated by the percentage of items on the first principal component?
Research question 2
3
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
With regard to the euthymia items and the dysthymia items identified by PCA, to what extent are the corresponding two NEO-PI subscales accepted by the Mokken analysis as being scalable, i.e. their total score being a sufficient measure of euthymia and dysthymia, respectively?
Methods Sample The data set is part of the Norwegian general population study performed by Vassend and Skrondal (1997; 1995), where approximately 1200 individuals were contacted and asked to participate in the investigation. A total of 1082 individuals completed the NEO-PI, giving a response rate of 90%, which is considered representative of the non-institutionalised Norwegian population aged 15 years and above. Measures The NEO-PI comprises 180 items of which 144 items cover N, E, and O. The Neuroticism scale consists of 48 items grouped into facets: anxiety, hostility, depression, self-consciousness, impulsiveness, and vulnerability. The Extraversion scale and Openness scale also consist of 48 items, while the Agreeableness and the Conscientiousness scale consist of 18 items each. The item format of NEO-PI consists of five-point Likert answer categories, formulated either positively (positive mental health), see Table 1, or negatively (negative mental health), see Table 2. We have evaluated the manifest item scores by PCA or Mokken without reversals, i.e. evaluated positive mental health and negative mental health separately.
Psychometric analyses Principal Component Analysis (PCA) Without rotations, we focused on the first principal components with an eigenvalue above 1. The correlations of the items with the principal components in the PCA are referred to as loadings, a term borrowed from factor analysis (Dunteman, 1989) . The size of these loadings is with either positive or
4
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
negative signs which reflect that the NEO-PI items are contrasted, covering either positive or negative mental health. Factor loadings of +0.30 or higher were considered, in accordance with Vassend and Skrondal (1997; 1995). In the PCA we reversed the signs of the loadings so that positive loadings corresponded to positive mental health and negative loadings corresponded to negative mental health (Child, 2006) The SAS statistical package (version 9.0.0, 2002) was used.
The Mokken analysis The non-parametric Mokken analysis was used to evaluate the measurement aspects of the NEO-PI for scalability; to what extent the total score covers the components or items of the scale sufficiently (Bech, 2012). The Mokken model (Mokken, 1971) is based on the Guttman cumulative rating scale principle (Bech, 2012; Michell, 1990): that scorings on lower prevalence manifest items must be preceded by scorings on high prevalence items. The scalability is evaluated by use of Loevinger’s coefficient of homogeneity (Bech, 2012) but the rank order of prevalence is evaluated by the mean scores of the manifest items. The items with higher mean scores are considered as more prevalent. Loevingers’ coefficent of homogeneity (H) provides a measure of how often items are found to be relative to one another in a group of individuals responding to a set of items. According to Mokken (1971) or van Schuur (2011) coefficients of homogeneity from 0.20 to 0.29 belong in a questionable zone as to constituting a cumulative scale, from 0.30 to 0.39 they are just acceptable, while a coefficient of homogeneity of 0.40 or higher clearly indicates scalability. The Mokken analysis was performed using the MSP program (Molenaar, et al, 1994).
Results Out of the 180 items in the NEO-PI, PCA identified 49 components with an eigenvalue of 1 or more. However, a scree test focussed on the five components with an eigenvalue of 4 or more. The first principal component obtained an eigenvalue of 15.59 and the second an eigenvalue of 10.84. The third principal component obtained an eigenvalue of 9.23.
5
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Using an item loading level of ± 0.30, the first principal component identified 55 items of which 28 (51%) were neuroticism items. Fourteen of these were positively loaded (measuring positive mental health), and the remaining 14 items were negatively loaded (measuring negative mental health). Furthermore 21 items (38%) belonged to the extraversion group (5 negatively loaded and 16 positively loaded), and 6 items (11%) belonged to the openness group (1 negatively and 5 positively loaded). A varimax rotation verified that the first principal component included 80% neuroticism items, the second principal component included 50% extraversion items, and the third principal component included 63% openness items. Therefore, the answer to our first research question is that neuroticism items dominated on the first principal component (51%), followed by extraversion items (38%), and lastly openness items (11%), fulfilling the NEO rank order. Table 1 shows the 14 neuroticism items from the first principal component with positive loadings of 0.30 or higher, i.e. indicating positive mental health. The loadings without and with varimax rotation are indicated. The coefficient of homogeneity for the total score of these 14 items was 0.34 (just acceptable scalability). In Table 1 the items are rank ordered by the mean scores (prevalence). The coefficient of homogeneity is correspondingly indicated. Only Item nos. 179 and 67 had coefficients of homogeneity just lower than 0.30. The neuroticism facets to which the items belong are also shown. All facets apart from impulsiveness are represented (Table 1).
Table 2 shows the 14 neuroticism items from the first principal component with items indicating negative mental health (negative loadings of -0.30 or lower). The loadings without and with varimax rotation are indicated. The coefficient of homogeneity for the total score was 0.27 (not acceptable for scalability). The neuroticism facets to which the items belonged are shown in Table 2. Again, all facets apart from impulsiveness are represented (Table 2). The coefficient of homogeneity was only 0.26 when combining the 28 items (Table 1 plus Table 2) by reversing the 14 items reflecting positive mental health according to the NEO-PI principle.
6
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Discussion Whereas McCrae et al (1996) used a varimax rotation within PCA to identify the rank order of the NEO subscales, we were able to support this rank order by unrotated PCA. The answer to our first research question was that the items from the neuroticism scale were more representative (51%) on the first principal component than the items from the extraversion scale (38%), and that the items from the openness scale were even less representative (11%). This is in concordance with the work performed by Eysenck and Eysenck (1969) in which the Neuroticism and Extraversion dimensions were identified by PCA. The PCA identified 14 items with positive loadings (Table 1), reflecting positive mental health, and another 14 items with negative loadings (Table 2) reflecting negative mental health. However, the Mokken analysis (research question 2) only accepted the 14 items reflecting positive mental health (Table 1) as scalable, i.e. the total score being a meaningful measure of this dimension. The coefficient of homogeneity was 0.34 which is only just acceptable. The 14 items on the neuroticism scale reflecting negative mental health (Table 2) obtained a coefficient of homogeneity below 0.30. This is in contrast to our findings with the Eysenck Neuroticism Scale where we confirmed the Eysenck neuroticism scale as a measure of dysthymia severity, including the facets of anxiety, depression, and interpersonal sensitivity (Bech, et al, 2012; Moeller, et al, 2015).The facets of hostility and impulsivity within the NEO-PI are to be found in the Eysenck dimension of extraversion or psychoticism. The lower scalability of the neuroticism scale from the NEO-PI compared to the EPQ may indicate that the facets hostility and impulsivity are not well placed on the neuroticism scale. Hostility and impulsivity are not internalising facets; instead they both are externalising in their nature.
The advantage of the Mokken model is that when the criteria of scalability are fulfilled, the simple sum of the items then reflects the magnitude of the dimension being measured. In other words, this simple sum is sufficient to account for inter-item associations in the empirical data. The Mokken model is a measurement 7
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
interpretation and not, as PCA, a test to explore the number of factors identified within an explorative interpretation of the data. The NEO-PI subscale identified by our combination of PCA and Mokken analysis measures positive mental health (Table 1) and can be considered a measure of euthymia (Fava and Bech, 2016). When developing a questionnaire for euthymia Fava and Bech (2016) identified 10 items, of which five items correspond to the WHO-5 Well-being scale and five items cover such components as being able to cope with stress, being resistant to anxiety or depression, and being resilient. In Table 1, lack of vulnerability (items 179, 168, 146, 123, 101) and resilience (items 14, 67, 82, 89) reflect such euthymia items. Concerning the WHO-5, fewer items overlap, e.g. item 67 or 89 with the WHO-5 item of being cheerful and item 14 or 82 with the WHO-5 item of being calm.
In conclusion, the clinical relevance of identifying items that measure levels of euthymia means that recovery from depression goes beyond just identifying the absence of symptoms and also focusses on the attainment of a positive mental state (Fava and Tomba, 2009; Fava and Bech, 2016). This euthymic state has been associated with lower rates of relapse within a population with depression (Fava and Tomba, 2009).
Limitations A comparison with the Eysenck Neuroticism Scale was not performed.
In conclusion, the neuroticism scale within the NEO-PI as conventionally used is much more heterogenic than the Eysenck neuroticism scale. However, within the NEO-PI a neuroticism subscale (Table 1) has been identified which measures euthymia.
8
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
References Bech, P., 2012. Clinical Psychometrics. Wiley Blackwell, Oxford. Bech, P., Lunde, M., Moller, S.B., 2012. Eysenck's Two Big Personality Factors and Their Relationship to Depression in Patients with Chronic Idiopathic Pain Disorder: A Clinimetric Validation Analysis. ISRN Psychiatry. 2012, 140458. Child, D., 2006. The Essentials of Factor Analysis. Continuum, London. Costa, P.T. and Mc Crae, R.R., 1985. The NEO Personality Inventory Manual. Psychological Assessment Resources, Odessa. Costa, P.T.,Jr and McCrae, R.R., 1995. Domains and facets: hierarchical personality assessment using the revised NEO personality inventory. J. Pers. Assess. 64, 21-50. Dunteman, G.H., 1989. Principal Components Analysis. SAGE Publications, Newbury Park. Eysenck, H.J. and Eysenck, S.B.G., 1969. Personality Structure and Measurement. Routledge & Kegan Paul, London. Eysenck, H.J. and Eysenck, S.B.G., 1976. Psychoticism as a Dimension of Personality. Hodder and Stoughton, London. Eysenck, H.J. and Eysenck, M.W., 1985. Personality and Individual Differences. Plenum Press, New York. Farmer, A., McGuffin, P., Williams, J., 2002. Measuring Psychopathology. Oxford University Press, Oxford. Fava, G.A. and Tomba, E., 2009. Increasing psychological well-being and resilience by psychotherapeutic methods. J. Pers. 77, 1903-1934. Fava, G.A. and Bech, P., 2016. The concept of euthymia. Psychotherapy and Psychosomatics. 85, 1-5. Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W.A., Beekman, A.T., 2010. Prevalence and predictors of recurrence of major depressive disorder in the adult population. Acta Psychiatr. Scand. 122, 184-191. McCrae, R.R., Zonderman, A.B., Costa, P.T., Bond, M.H., Paunonon, S.V., 1996. Evaluating replicability of factors in the revised NEO Personality Inventory: confirmatory factor analysis versus Procrustes rotation. Journal of personality and social psychology. 70, 552-566. Michell, J., 1990. An Introduction to the Logic of Psychological Measurement. Psychology Press, New York. Moeller, S.B., Bech, P., Kessing, L., Mortensen, E.L., Austin, S.F., Bukh, J.D., 2015. A psychometric validation analysis of Eysenck's neuroticism and extraversion scales in a sample of first time depressed patients. Depression & Anxiety. In press. DOI: 10.4200/2167-1044.1000202. Mokken, R.J., 1971. Theory and Practice of Scale Analysis. Mouton, Berlin. Molenaar, I.W., Debels, P., Sijtsna, K., 1994. User's Manual MSP, a Program for Mokken Scale Analyses for Polytomous Items (Version 3.0). ProGAMMA, Groeningen, The Netherlands. Stochl, J., Jones, P.B., Croudace, T.J., 2012. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med. Res. Methodol. 12, 74-2288-12-74. van Schuur, R.H., 2011. Ordinal Item Response Theory. Mokken Scale Analysis. SAGE Publications, London. Vassend, O. and Skrondal, A., 1997. Validation of the NEO Personality Inventory and the five-factor model. Can findings from exploratory and confirmatory factor analysis be reconciled? European Journal of Personality. 11, 147-166. Vassend, O. and Skrondal, A., 1995. Factor analytic studies of the neo personality inventory and the fivefactor model: The problem of high structural complexity and conceptual indeterminacy. Personality and Individual Differences. 19, 135-147. Table 1 Mokken analysis of the 14 neuroticism items with positive wording, i.e. measuring positive mental health, from the first principal component (positive loadings ≥ 0.30) Mokken total 0.34
9
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
NEO-PI item nos.
Positive mental health items
89
I am seldom sad or depressed
93
I seldom feel self-conscious when I´m around people
Facets
Mean
Coefficient of homogeneity
First principal component Loadings Unrotated Varimax rotation
depression
2.05
0.36
0.51
0.58
selfconsciousness
2.00
0.38
0.52
0.36
vulnerability
1.87
0.24
0.46
0.34
179
I’m pretty stable emotionally
116
It doesn’t embarrass me too much if people ridicule and tease me
selfconsciousness
1.76
0.39
0.39
0.34
168
When everything seems to be going wrong, I can still make good decisions I keep a cool head in emergencies
vulnerability
1.74
0.31
0.46
0.19
vulnerability
1.72
0.31
0.43
0.24
depression
1.71
0.29
0.43
0.46
selfconsciousness
1.68
0.37
0.43
0.30
123 67 149
I rarely feel lonely or blue I feel comfortable in the presence of my bosses or other authorities
14
I rarely feel fearful or anxious
anxiety
1.54
0.38
0.46
0.47
82
I have fewer fears than most people
anxiety
1.52
0.35
0.33
0.33
146
I can handle myself pretty well in a crisis
vulnerability
1.39
0.37
0.52
0.30
101
I feel I am capable of coping with most of my problems
vulnerability
1.37
0.32
0.55
0.38
I’m an even-tempered person
hostility
1.29
0.35
0.40
0.32
I am not considered a touchy or temperamental person
hostility
1.26
0.30
0.46
0.20
7 74
10
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Table 2 Mokken analysis of the 14 neuroticism items with negative wording, i.e. measuring negative mental health, from the first principal component (negative loadings < -0.30) Mokken total 0.27 NEO-PI item nos.
Negative mental health items
Facets
Mean
Coefficient of homogeneity
First principal component Loadings Unrotated
Varimax rotation
44
I have a low opinion of myself
depression
1.96
0.26
-0.50
-0.57
78
Too often, when things go wrong, I get discouraged and feel like giving up I often worry about things that might go wrong
depression
1.86
0.25
-0.51
-0.63
anxiety
1.79
0.32
-0.36
-0.61
48 138
I often feel inferior to others
selfconsciousness
1.78
0.21
-0.50
-0.61
18
I often get angry at the way people treat me
hostility
1.70
0.28
-0.33
-0.51
56
Sometimes things look pretty bleak and hopeless to me
depression
1.59
0.28
-0.39
-0.64
11
Sometimes I feel completely worthless
depression
1.48
0.30
-0.45
-0.61
157
It’s often hard for me to make up my mind
vulnerability
1.47
0.25
-0.42
-0.54
104
In dealing with other people, I always dread making a social blunder If I have said or done the wrong thing to someone, I can hardly bear to face them again
selfconsciousness
1.47
0.25
-0.45
-0.55
selfconsciousness
1.37
0.31
-0.31
-0.42
vulnerability
1.27
0.32
-0.53
-0.60
anxiety
1.24
0.24
-0.46
-0.58
161
112 3
I often feel helpless and want someone else to solve my problems I often feel tense and jittery
59
Frightening thoughts sometimes come into my head
anxiety
1.19
0.32
-0.37
-0.52
26
I am easily frightened
anxiety
1.18
0.21
-0.34
-0.50
11
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Bech P. et al. Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory. A Mokken analysis of the Norwegian general population study for scalability.
High lights
Principal Component Analysis of the NEO Personality Inventory identified the neuroticism scale as the most distinct and showed that this scale consists of two subscales corresponding to items covering positive versus negative mental health
Mokken analysis showed that the subscale measuring positive mental health, or euthymia has a better scalability than the subscale measuring negative mental health
It is clinically very important to measure euthymia on a scale covering positive mental health and not just as a low cut-off score on a neuroticism or dysthymia scale
12
PII: DOI: Reference:
S0165-0327(15)31185-X http://dx.doi.org/10.1016/j.jad.2015.12.039 JAD7930
To appear in: Journal of Affective Disorders Received date: 29 October 2015 Revised date: 3 December 2015 Accepted date: 27 December 2015 Cite this article as: P. Bech., D. Carrozzino, S.F. Austin, S.B. Møller and O. Vassend, Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory A Mokken analysis of the Norwegian general population study for scalability., Journal of Affective Disorders, http://dx.doi.org/10.1016/j.jad.2015.12.039 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory. A Mokken analysis of the Norwegian general population study for scalability.
Bech P
1)
Carrozzino D1) Austin SF
1)
Møller SB
1)
Vassend O
2)
Word count: Abstract: 202 Main text: 2197 Tables: 2
Revised Version 01.12.2015
1) Psychiatric Research Unit, Psychiatric Centre North Zealand, University of Copenhagen, Hillerød, Denmark
2) Department of Psychology, Psychological Institute. University of Oslo, 0317 Oslo, Norway.
Corresponding author: Per Bech Psychiatric Research Unit Psychiatric Centre North Zealand, Copenhagen University Hospital Dyrehavevej 48 DK-3400 Hillerød, Denmark [email protected] Tel.: +45 38 64 30 95 Fax: +45 48 26 38 77 Abstract Background
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Whereas the Eysenck Neuroticism Scale only contains items covering negative mental health to measure dysthymia, the NEO Personality Inventory (NEO-PI) contains neuroticism items covering both negative mental health and positive mental health (or euthymia). The consequence of wording items both positively and negatively within the NEO-PI has never been psychometrically investigated. The aim of this study was to perform a validation analysis of the NEO-PI neuroticism scale. Methods Using a Norwegian general population study we examined the structure of the negatively and positively formulated items by principal component analysis (PCA). The scalability of the identified two groups of euthymia versus dysthymia items was examined by Mokken analysis. Results With a response rate of 90%, 1082 individuals with a completed NEO-PI were available. The PCA identified the neuroticism scale as the most distinct where 14 items had acceptable loadings for the euthymia subscale, another 14 items for the dysthymia subscale. However, the Mokken analysis coefficient of homogeneity only found acceptable scalability for the euthymia subscale. Limitations A comparison with the Eysenck Neuroticism Scale was not performed. Conclusion The NEO-PI neuroticism scale contains two subscales consisting of items worded in an opposite direction where only the positive euthymia items have an acceptable scalability.
Key words NEO Personality Inventory; Mokken analysis; Euthymia; Dysthymia. Introduction The hierarchical structure of the NEO-PI (Neuroticism, Extraversion, Openness Personality Inventory) was demonstrated by Costa & McCrae (1985) by use of principal component analysis (PCA). Neuroticism,
1
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Extraversion and Openness (NEO) were found more distinct than Agreeableness or Conscientiousness in the five-factor model behind the NEO-PI (Costa and McCrae, 1995). This is in concordance with the PCA approach used to demonstrate that neuroticism and extraversion are the two distinct factors in the Eysenck Personality Questionnaire (EPQ) (Eysenck and Eysenck, 1976). The NEO-PI Neuroticism factor includes facets such as anxiety, hostility, depression, self-consciousness, impulsivity, and vulnerability, covering all DSM-IV personality disorders clusters: A (e.g. paranoia), B (e.g. antisocial), and C (e.g. anxious avoidant) (Farmer, et al, 2002). In contrast, the neuroticism dimension in EPQ only covers cluster C, referred to by Eysenck and Eysenck (1985) as dysthymia, which includes components of depression, anxiety, and interpersonal sensitivity. In DSM-IV the clinical construct of dysthymia was a co-existence of subclinical chronic depression and personality disturbance. However, in DSM-5 dysthymia indicates a co-existence of chronic major depression and subthreshold depression implying that the term dysthymia has been deleted and the category renamed: “persistent depression disorder”. The clinical construct of dysthymia for the personality dimension of neuroticism was used by Farmer et al (2002) to indicate that a high neuroticism score covers facets of being anxious, worrying, or pessimistic, whereas a low neuroticism score covers facets of being relaxed, stable, or coping well with adversities. These polar opposites have been referred to as dysthymia versus euthymia by Fava and Bech (2016). However, these concepts should not be considered as opposite ends on the same scale because clinimetrically, they refer to two different dimensions, namely ill-being or dysthymia (e.g. easily becoming anxious or helpless), and well-being or euthymia (being able to face problems) (Fava and Bech, 2016; Stochl, et al, 2012). Euthymia can be seen as a positive mental state and it has clinical relevance since studies have shown that attainment of this state after a depression has been associated with significantly lower rates of relapse (Fava and Tomba, 2009; Hardeveld, et al, 2010). From a measurement perspective we have recently evaluated the neuroticism factor in the EPQ (Bech, et al, 2012) using the item response theory model developed by Mokken (1971). This model is a generalisation of the Guttman scaling (Bech, 2012), which evaluates unidimensionality or scalability without reliance on
2
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
the classical psychometrics such as the Cronbach alpha coefficient. In their validation study of the NEO-PI, Vassend and Skrondal (1997; 1995) used confirmative factor analysis which evaluates some measurement properties in contrast to explorative factor analysis, PCA or Cronbach’s alpha. However Vassend and Skrondal (1997; 1995), like McCrae et al (1996) had problems reproducing the NEO structure. When analysing the EPQ we used both PCA and the Mokken analysis (Bech, et al, 2012; Moeller, et al, 2015). To the best of our knowledge the NEO-PI has not been psychometrically validated by the Mokken analysis. When evaluating the scalability of this neuroticism scale to test that high total scores correspond to high levels of neuroticism, or dysthymia, while low scores correspond to euthymia (being relaxed, stable, coping well with adversity) it is psychometrically important to take the basic pattern of the NEO-PI into account, namely that this scale includes reversed scoring, i.e. the items which measure positive mental health are reversed to correspond to the items which measure negative mental health. Consequently, we have followed Stochl et al (2012) in performing the Mokken analysis separately on the items for positive versus negative mental health formulation. With reference to Fava and Bech (2016) we named the NEO-PI Neuroticism subscale measuring positive mental health “euthymia” and the subscale measuring negative mental health “dysthymia”, corresponding to the Eysenck neuroticism scale. On this basis we have reanalysed the NEO-PI combining PCA and Mokken analysis in order to answer the following two questions:
Research question 1 Using PCA analysis, unrotated or with varimax rotation, to what extent is neuroticism the most distinct component in the NEO-PI as indicated by the percentage of items on the first principal component?
Research question 2
3
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
With regard to the euthymia items and the dysthymia items identified by PCA, to what extent are the corresponding two NEO-PI subscales accepted by the Mokken analysis as being scalable, i.e. their total score being a sufficient measure of euthymia and dysthymia, respectively?
Methods Sample The data set is part of the Norwegian general population study performed by Vassend and Skrondal (1997; 1995), where approximately 1200 individuals were contacted and asked to participate in the investigation. A total of 1082 individuals completed the NEO-PI, giving a response rate of 90%, which is considered representative of the non-institutionalised Norwegian population aged 15 years and above. Measures The NEO-PI comprises 180 items of which 144 items cover N, E, and O. The Neuroticism scale consists of 48 items grouped into facets: anxiety, hostility, depression, self-consciousness, impulsiveness, and vulnerability. The Extraversion scale and Openness scale also consist of 48 items, while the Agreeableness and the Conscientiousness scale consist of 18 items each. The item format of NEO-PI consists of five-point Likert answer categories, formulated either positively (positive mental health), see Table 1, or negatively (negative mental health), see Table 2. We have evaluated the manifest item scores by PCA or Mokken without reversals, i.e. evaluated positive mental health and negative mental health separately.
Psychometric analyses Principal Component Analysis (PCA) Without rotations, we focused on the first principal components with an eigenvalue above 1. The correlations of the items with the principal components in the PCA are referred to as loadings, a term borrowed from factor analysis (Dunteman, 1989) . The size of these loadings is with either positive or
4
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
negative signs which reflect that the NEO-PI items are contrasted, covering either positive or negative mental health. Factor loadings of +0.30 or higher were considered, in accordance with Vassend and Skrondal (1997; 1995). In the PCA we reversed the signs of the loadings so that positive loadings corresponded to positive mental health and negative loadings corresponded to negative mental health (Child, 2006) The SAS statistical package (version 9.0.0, 2002) was used.
The Mokken analysis The non-parametric Mokken analysis was used to evaluate the measurement aspects of the NEO-PI for scalability; to what extent the total score covers the components or items of the scale sufficiently (Bech, 2012). The Mokken model (Mokken, 1971) is based on the Guttman cumulative rating scale principle (Bech, 2012; Michell, 1990): that scorings on lower prevalence manifest items must be preceded by scorings on high prevalence items. The scalability is evaluated by use of Loevinger’s coefficient of homogeneity (Bech, 2012) but the rank order of prevalence is evaluated by the mean scores of the manifest items. The items with higher mean scores are considered as more prevalent. Loevingers’ coefficent of homogeneity (H) provides a measure of how often items are found to be relative to one another in a group of individuals responding to a set of items. According to Mokken (1971) or van Schuur (2011) coefficients of homogeneity from 0.20 to 0.29 belong in a questionable zone as to constituting a cumulative scale, from 0.30 to 0.39 they are just acceptable, while a coefficient of homogeneity of 0.40 or higher clearly indicates scalability. The Mokken analysis was performed using the MSP program (Molenaar, et al, 1994).
Results Out of the 180 items in the NEO-PI, PCA identified 49 components with an eigenvalue of 1 or more. However, a scree test focussed on the five components with an eigenvalue of 4 or more. The first principal component obtained an eigenvalue of 15.59 and the second an eigenvalue of 10.84. The third principal component obtained an eigenvalue of 9.23.
5
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Using an item loading level of ± 0.30, the first principal component identified 55 items of which 28 (51%) were neuroticism items. Fourteen of these were positively loaded (measuring positive mental health), and the remaining 14 items were negatively loaded (measuring negative mental health). Furthermore 21 items (38%) belonged to the extraversion group (5 negatively loaded and 16 positively loaded), and 6 items (11%) belonged to the openness group (1 negatively and 5 positively loaded). A varimax rotation verified that the first principal component included 80% neuroticism items, the second principal component included 50% extraversion items, and the third principal component included 63% openness items. Therefore, the answer to our first research question is that neuroticism items dominated on the first principal component (51%), followed by extraversion items (38%), and lastly openness items (11%), fulfilling the NEO rank order. Table 1 shows the 14 neuroticism items from the first principal component with positive loadings of 0.30 or higher, i.e. indicating positive mental health. The loadings without and with varimax rotation are indicated. The coefficient of homogeneity for the total score of these 14 items was 0.34 (just acceptable scalability). In Table 1 the items are rank ordered by the mean scores (prevalence). The coefficient of homogeneity is correspondingly indicated. Only Item nos. 179 and 67 had coefficients of homogeneity just lower than 0.30. The neuroticism facets to which the items belong are also shown. All facets apart from impulsiveness are represented (Table 1).
Table 2 shows the 14 neuroticism items from the first principal component with items indicating negative mental health (negative loadings of -0.30 or lower). The loadings without and with varimax rotation are indicated. The coefficient of homogeneity for the total score was 0.27 (not acceptable for scalability). The neuroticism facets to which the items belonged are shown in Table 2. Again, all facets apart from impulsiveness are represented (Table 2). The coefficient of homogeneity was only 0.26 when combining the 28 items (Table 1 plus Table 2) by reversing the 14 items reflecting positive mental health according to the NEO-PI principle.
6
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Discussion Whereas McCrae et al (1996) used a varimax rotation within PCA to identify the rank order of the NEO subscales, we were able to support this rank order by unrotated PCA. The answer to our first research question was that the items from the neuroticism scale were more representative (51%) on the first principal component than the items from the extraversion scale (38%), and that the items from the openness scale were even less representative (11%). This is in concordance with the work performed by Eysenck and Eysenck (1969) in which the Neuroticism and Extraversion dimensions were identified by PCA. The PCA identified 14 items with positive loadings (Table 1), reflecting positive mental health, and another 14 items with negative loadings (Table 2) reflecting negative mental health. However, the Mokken analysis (research question 2) only accepted the 14 items reflecting positive mental health (Table 1) as scalable, i.e. the total score being a meaningful measure of this dimension. The coefficient of homogeneity was 0.34 which is only just acceptable. The 14 items on the neuroticism scale reflecting negative mental health (Table 2) obtained a coefficient of homogeneity below 0.30. This is in contrast to our findings with the Eysenck Neuroticism Scale where we confirmed the Eysenck neuroticism scale as a measure of dysthymia severity, including the facets of anxiety, depression, and interpersonal sensitivity (Bech, et al, 2012; Moeller, et al, 2015).The facets of hostility and impulsivity within the NEO-PI are to be found in the Eysenck dimension of extraversion or psychoticism. The lower scalability of the neuroticism scale from the NEO-PI compared to the EPQ may indicate that the facets hostility and impulsivity are not well placed on the neuroticism scale. Hostility and impulsivity are not internalising facets; instead they both are externalising in their nature.
The advantage of the Mokken model is that when the criteria of scalability are fulfilled, the simple sum of the items then reflects the magnitude of the dimension being measured. In other words, this simple sum is sufficient to account for inter-item associations in the empirical data. The Mokken model is a measurement 7
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
interpretation and not, as PCA, a test to explore the number of factors identified within an explorative interpretation of the data. The NEO-PI subscale identified by our combination of PCA and Mokken analysis measures positive mental health (Table 1) and can be considered a measure of euthymia (Fava and Bech, 2016). When developing a questionnaire for euthymia Fava and Bech (2016) identified 10 items, of which five items correspond to the WHO-5 Well-being scale and five items cover such components as being able to cope with stress, being resistant to anxiety or depression, and being resilient. In Table 1, lack of vulnerability (items 179, 168, 146, 123, 101) and resilience (items 14, 67, 82, 89) reflect such euthymia items. Concerning the WHO-5, fewer items overlap, e.g. item 67 or 89 with the WHO-5 item of being cheerful and item 14 or 82 with the WHO-5 item of being calm.
In conclusion, the clinical relevance of identifying items that measure levels of euthymia means that recovery from depression goes beyond just identifying the absence of symptoms and also focusses on the attainment of a positive mental state (Fava and Tomba, 2009; Fava and Bech, 2016). This euthymic state has been associated with lower rates of relapse within a population with depression (Fava and Tomba, 2009).
Limitations A comparison with the Eysenck Neuroticism Scale was not performed.
In conclusion, the neuroticism scale within the NEO-PI as conventionally used is much more heterogenic than the Eysenck neuroticism scale. However, within the NEO-PI a neuroticism subscale (Table 1) has been identified which measures euthymia.
8
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
References Bech, P., 2012. Clinical Psychometrics. Wiley Blackwell, Oxford. Bech, P., Lunde, M., Moller, S.B., 2012. Eysenck's Two Big Personality Factors and Their Relationship to Depression in Patients with Chronic Idiopathic Pain Disorder: A Clinimetric Validation Analysis. ISRN Psychiatry. 2012, 140458. Child, D., 2006. The Essentials of Factor Analysis. Continuum, London. Costa, P.T. and Mc Crae, R.R., 1985. The NEO Personality Inventory Manual. Psychological Assessment Resources, Odessa. Costa, P.T.,Jr and McCrae, R.R., 1995. Domains and facets: hierarchical personality assessment using the revised NEO personality inventory. J. Pers. Assess. 64, 21-50. Dunteman, G.H., 1989. Principal Components Analysis. SAGE Publications, Newbury Park. Eysenck, H.J. and Eysenck, S.B.G., 1969. Personality Structure and Measurement. Routledge & Kegan Paul, London. Eysenck, H.J. and Eysenck, S.B.G., 1976. Psychoticism as a Dimension of Personality. Hodder and Stoughton, London. Eysenck, H.J. and Eysenck, M.W., 1985. Personality and Individual Differences. Plenum Press, New York. Farmer, A., McGuffin, P., Williams, J., 2002. Measuring Psychopathology. Oxford University Press, Oxford. Fava, G.A. and Tomba, E., 2009. Increasing psychological well-being and resilience by psychotherapeutic methods. J. Pers. 77, 1903-1934. Fava, G.A. and Bech, P., 2016. The concept of euthymia. Psychotherapy and Psychosomatics. 85, 1-5. Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W.A., Beekman, A.T., 2010. Prevalence and predictors of recurrence of major depressive disorder in the adult population. Acta Psychiatr. Scand. 122, 184-191. McCrae, R.R., Zonderman, A.B., Costa, P.T., Bond, M.H., Paunonon, S.V., 1996. Evaluating replicability of factors in the revised NEO Personality Inventory: confirmatory factor analysis versus Procrustes rotation. Journal of personality and social psychology. 70, 552-566. Michell, J., 1990. An Introduction to the Logic of Psychological Measurement. Psychology Press, New York. Moeller, S.B., Bech, P., Kessing, L., Mortensen, E.L., Austin, S.F., Bukh, J.D., 2015. A psychometric validation analysis of Eysenck's neuroticism and extraversion scales in a sample of first time depressed patients. Depression & Anxiety. In press. DOI: 10.4200/2167-1044.1000202. Mokken, R.J., 1971. Theory and Practice of Scale Analysis. Mouton, Berlin. Molenaar, I.W., Debels, P., Sijtsna, K., 1994. User's Manual MSP, a Program for Mokken Scale Analyses for Polytomous Items (Version 3.0). ProGAMMA, Groeningen, The Netherlands. Stochl, J., Jones, P.B., Croudace, T.J., 2012. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med. Res. Methodol. 12, 74-2288-12-74. van Schuur, R.H., 2011. Ordinal Item Response Theory. Mokken Scale Analysis. SAGE Publications, London. Vassend, O. and Skrondal, A., 1997. Validation of the NEO Personality Inventory and the five-factor model. Can findings from exploratory and confirmatory factor analysis be reconciled? European Journal of Personality. 11, 147-166. Vassend, O. and Skrondal, A., 1995. Factor analytic studies of the neo personality inventory and the fivefactor model: The problem of high structural complexity and conceptual indeterminacy. Personality and Individual Differences. 19, 135-147. Table 1 Mokken analysis of the 14 neuroticism items with positive wording, i.e. measuring positive mental health, from the first principal component (positive loadings ≥ 0.30) Mokken total 0.34
9
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
NEO-PI item nos.
Positive mental health items
89
I am seldom sad or depressed
93
I seldom feel self-conscious when I´m around people
Facets
Mean
Coefficient of homogeneity
First principal component Loadings Unrotated Varimax rotation
depression
2.05
0.36
0.51
0.58
selfconsciousness
2.00
0.38
0.52
0.36
vulnerability
1.87
0.24
0.46
0.34
179
I’m pretty stable emotionally
116
It doesn’t embarrass me too much if people ridicule and tease me
selfconsciousness
1.76
0.39
0.39
0.34
168
When everything seems to be going wrong, I can still make good decisions I keep a cool head in emergencies
vulnerability
1.74
0.31
0.46
0.19
vulnerability
1.72
0.31
0.43
0.24
depression
1.71
0.29
0.43
0.46
selfconsciousness
1.68
0.37
0.43
0.30
123 67 149
I rarely feel lonely or blue I feel comfortable in the presence of my bosses or other authorities
14
I rarely feel fearful or anxious
anxiety
1.54
0.38
0.46
0.47
82
I have fewer fears than most people
anxiety
1.52
0.35
0.33
0.33
146
I can handle myself pretty well in a crisis
vulnerability
1.39
0.37
0.52
0.30
101
I feel I am capable of coping with most of my problems
vulnerability
1.37
0.32
0.55
0.38
I’m an even-tempered person
hostility
1.29
0.35
0.40
0.32
I am not considered a touchy or temperamental person
hostility
1.26
0.30
0.46
0.20
7 74
10
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Table 2 Mokken analysis of the 14 neuroticism items with negative wording, i.e. measuring negative mental health, from the first principal component (negative loadings < -0.30) Mokken total 0.27 NEO-PI item nos.
Negative mental health items
Facets
Mean
Coefficient of homogeneity
First principal component Loadings Unrotated
Varimax rotation
44
I have a low opinion of myself
depression
1.96
0.26
-0.50
-0.57
78
Too often, when things go wrong, I get discouraged and feel like giving up I often worry about things that might go wrong
depression
1.86
0.25
-0.51
-0.63
anxiety
1.79
0.32
-0.36
-0.61
48 138
I often feel inferior to others
selfconsciousness
1.78
0.21
-0.50
-0.61
18
I often get angry at the way people treat me
hostility
1.70
0.28
-0.33
-0.51
56
Sometimes things look pretty bleak and hopeless to me
depression
1.59
0.28
-0.39
-0.64
11
Sometimes I feel completely worthless
depression
1.48
0.30
-0.45
-0.61
157
It’s often hard for me to make up my mind
vulnerability
1.47
0.25
-0.42
-0.54
104
In dealing with other people, I always dread making a social blunder If I have said or done the wrong thing to someone, I can hardly bear to face them again
selfconsciousness
1.47
0.25
-0.45
-0.55
selfconsciousness
1.37
0.31
-0.31
-0.42
vulnerability
1.27
0.32
-0.53
-0.60
anxiety
1.24
0.24
-0.46
-0.58
161
112 3
I often feel helpless and want someone else to solve my problems I often feel tense and jittery
59
Frightening thoughts sometimes come into my head
anxiety
1.19
0.32
-0.37
-0.52
26
I am easily frightened
anxiety
1.18
0.21
-0.34
-0.50
11
Measuring euthymia within the Neuroticism Scale 01.12.2015_Revised version
Bech P. et al. Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory. A Mokken analysis of the Norwegian general population study for scalability.
High lights
Principal Component Analysis of the NEO Personality Inventory identified the neuroticism scale as the most distinct and showed that this scale consists of two subscales corresponding to items covering positive versus negative mental health
Mokken analysis showed that the subscale measuring positive mental health, or euthymia has a better scalability than the subscale measuring negative mental health
It is clinically very important to measure euthymia on a scale covering positive mental health and not just as a low cut-off score on a neuroticism or dysthymia scale
12