- Email: [email protected]
Screening for mood and anxiety disorders with the five-item, the three-item, and the two-item Mental Health Inventory
Psychiatry Research 168 (2009) 250–255
Contents lists available at ScienceDirect
Psychiatry Research j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / p s y c h r e s
Screening for mood and anxiety disorders with the five-item, the three-item, and the two-item Mental Health Inventory Pim Cuijpers a,⁎, Niels Smits a, Tara Donker a, Margreet ten Have b, Ron de Graaf b a b
Department of Clinical Psychology, Vrije Universiteit, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands Trimbos Institute, P.O. Box 725, 3500 AH Utrecht, The Netherlands
A R T I C L E
I N F O
Article history: Received 3 July 2007 Received in revised form 2 September 2007 Accepted 26 May 2008 Keywords: Screening Major depression Depressive disorders Anxiety disorders Sensitivity Specificity
A B S T R A C T The Mental Health Inventory (MHI)-5 is an attractive, brief screening questionnaire for depression and anxiety disorders. It has been suggested that the three questions on depression (MHI-d) may be as good as the full MHI-5 in assessing depressive disorders. We examined the validity of the MHI-d and the MHI-a (the remaining two items on anxiety) in a large population-based sample of 7076 adults in the Netherlands. We also examined the validity of the MHI in assessing specific anxiety disorders. The presence of depressive and anxiety disorders in the past month was assessed with the Composite International Diagnostic Interview (CIDI), computerized version 1.1. ROC analyses indicated no significant difference between the MHI-5 (area under the curve of 0.93) and the MHI-d (area under the curve of 0.91) in detecting major depression and dysthymia. There was no difference either between the MHI-5 (area under the curve of 0.73) and the MHI-a (area under the curve 0.73) in detecting anxiety disorders. Both the MHI-5 and the MHI-a also seem to be adequate as a screener for some anxiety disorders (generalized anxiety disorder; panic disorder; obsessivecompulsive disorder), but not others, especially phobias (agoraphobia; social phobia; simple phobia). © 2008 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Depressive and anxiety disorders are highly prevalent (Kessler et al., 1994; Bijl and Ravelli, 2000), have a high incidence (De Graaf et al., 2002), and are associated with huge losses in quality of life in patients and their relatives (Bijl and Ravelli, 2000), with high levels of service use, and huge economic costs (Smit et al., 2006; Cuijpers et al., 2007). Currently, depression is the fourth disorder worldwide in terms of disease burden, and by 2030 will be the illness with the highest disease burden in high-income countries (Mathers and Loncar, 2006). Screening instruments are important tools for identifying depressed and anxious patients, both for research and clinical purposes (Yamazaki et al., 2005). Recent decades have seen the development of a large number of these screening instruments. Although most screening instruments take only a few minutes to complete, many of them list 15 to 30 items. It has been shown, however, that very brief instruments of only a few questions also perform well in detecting depressive disorders (Whooley et al., 1997; Mitchell and Coyne, 2007) and anxiety disorders (Kroenke et al., 2007). One brief screening instrument for the detection of depression is the Mental Health Inventory (MHI-5). The MHI is the mental health subscale of the Medical Outcomes Study (MOS) Short Form Health Survey (Ware and Sherbourne, 1992), and is included in both the 20-item version (SF-20; Ware and Sherbourne, 1992) and the 36-item version (SF-36; Ware and ⁎ Corresponding author. Tel.: +31 20 44 48757; fax: +31 20 44 48758. E-mail address: [email protected] (P. Cuijpers). 0165-1781/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.psychres.2008.05.012
Sherbourne,1992; McHorney et al.,1993,1994). In recent years, the interest in the MHI-5 has grown considerably, not only because it is so short, but also because it is expected that it can be used as a screen for both mood and anxiety disorders (Rumpf et al., 2001). Research has shown that the specificity and sensitivity of the MHI-5 for detecting DSM-IV Axis-I diagnoses in general is good in the general population (Rumpf et al., 2001), as it is for detecting major depression in functionally impaired, community-dwelling elderly (Friedman et al., 2005), and major depression or panic disorder in primary care patients (Means-Christensen et al., 2005). The MHI-5 contains five questions, three of which are aimed at depressive symptoms and psychological well-being, while two questions measure symptoms of anxiety. There is some evidence that removing the two anxiety-related items does not reduce the effectiveness of the MHI in detecting depression (Yamazaki et al., 2005), although this has not been examined in studies in which a formal diagnosis according to DSM criteria was used as a gold standard. This is interesting, because it would reduce the length of the MHI even further from five to three items. Although the quality of the MHI-5 as a screener for anxiety disorders has been examined less well than for depressive disorders, there is some evidence that the MHI is also a good screener for anxiety disorders in general (Rumpf et al., 2001), and for panic disorder (Means-Christensen et al., 2005). The ability of the MHI-5 to detect specific anxiety disorders other than panic disorder has not yet been examined. And just as one could assume that removing the two anxiety-related items would not reduce the ability to detect depression, one could assume that removing the three depression-related
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
items would not reduce the ability to detect anxiety. However, this has not been examined until now. In the current study, we will examine the sensitivity, specificity, and psychometric qualities of the MHI-5 in detecting depressive and anxiety disorders in a large community-based sample. We will also examine the sensitivity and specificity of the three-item depression scale of the MHI (MHI-d), and the two-item anxiety scale of the MHI (MHI-a), and compare these to the sensitivity and specificity of the full MHI-5. 2. Method 2.1. Subjects and procedure The Netherlands Mental Health Survey and Incidence Study (NEMESIS) was based on a multistage, stratified, random sampling procedure (Bijl et al., 1998a,b). In brief, a sample of 90 municipalities was drawn, using urbanisation as the stratification criterion; the sample resulted in an adequate distribution of the respondents over the 12 Dutch provinces. Then a sample of private households was drawn from the postal registers. The selected households were first sent a letter of introduction followed by a telephone contact. In each household, the member with the most recent birthday was selected, on condition that (s)he was between 18 and 64 years and sufficiently fluent in Dutch to be interviewed. To establish contact, the interviewers made a minimum of 10 phone calls or visits to a given address at different times of the day and week. In the initial data collection phase, which is used in the current study, 7076 respondents were interviewed (year of interview: 1996; response rate 69.7%). To correct for the combined effect of initial nonresponse, post-stratification weights were calculated. 2.2. Measures
251
him up, and considered himself to be a happy person). Possible scores on the MHI-d range from 3 to 18 points. The MHI-a consists of the two remaining items of the MHI-5 (asking the respondent how much of the time in the last month he had considered himself to be a very nervous person, and whether he had felt calm and peaceful), with a range of scores from 2 to 12. Higher scores on the MHI-d and MHI-a indicate worse symptoms of depression and anxiety (contrary to the MHI-5 scores). We considered transforming the score on the MHI-d and MHI-a in the same way as the MHI-5 (with a range from 0 to 100, and a higher score indicating better mental health). We decided, however, not to transform the MHI-d and MHI-a in the same way. The MHI-5 is part of an instrument measuring health-related quality of life, and for such an instrument it seems useful to have a higher score indicating better health, and a range from 0 to 100. However, because the MHI-d and the MHI-a are used as a screener for mental disorders, there is no need for such a transformation. In order to keep these screeners as userfriendly as possible and to avoid complicated transformations, we decided to use a simple method for the calculation of the total scores. 2.3. Analyses We first calculated the means and standard deviations of the MHI5, MHI-d and MHI-a for all mood and anxiety disorders separately, and used the Mann–Whitney U-test to establish whether the scores of subjects with each of these disorders differed significantly from the scores of subjects without the disorder. Then we performed Receiver Operating Characteristic (ROC) analyses and examined the Area Under the Curve (AUC) for the MHI-5, MHI-d and Table 1 MHI-scores in subjects with current DSM-III-R mood and anxiety disorders (1 month prevalence): means and standard deviations.a MHIb
Mental disorders were assessed according to DSM-III-R criteria. The instrument used was the Composite International Diagnostic Interview (CIDI, computerized version 1.1; WHO, 1990), Dutch version (Smeets and Dingemans, 1993). The CIDI can be used by trained lay interviewers. It is known to have a high interrater and test–retest reliability (Wittchen et al., 1991). The validity of the CIDI has been found to be good in several studies (Janca et al., 1992a,b; Jordanova et al., 2004), although there are indications that it may be less valid in some anxiety disorders (Komiti et al., 2001; Means-Christensen et al., 2003), and in general it can be concluded that community psychiatric surveys using structured diagnostic interview data must be interpreted cautiously (Brugha et al., 2001). In the current study, we use the diagnoses of the mood disorders (major depression and dysthymia) and anxiety disorders (panic disorder with or without agoraphobia, agoraphobia, social phobia, simple phobia, generalized anxiety disorder, and obsessive-compulsive disorder). Because we were interested in the relationship between the MHI and current depressive and anxiety disorders, we only used past month diagnoses. The Mental Health Inventory (MHI-5) was used as a screener for mood and anxiety disorders (Ware and Sherbourne, 1992). The MHI-5 consists of five items asking the respondent how much of the time in the last month he had considered himself to be a very nervous person, had felt downhearted, had felt calm and peaceful, had felt so down in the dumps that nothing could cheer you up, and considered himself to be a happy person. The answers are scored on six-point scales ranging from all of the time to none of the time. The total score is calculated by reversing the answers to two items (the third and fifth), summing up the scores, and transforming the raw scores to a scale ranging from zero to 100. A higher score indicates better mental health. The MHI-5 was administered after the CIDI during a face-to-face interview. The MHI-d comprises three items of the MHI-5 (asking the respondent how much of the time in the last month he had felt downhearted, had felt so down in the dumps that nothing could cheer
N
MHI-ad
M
S.D.
M
S.D.
Any mood disorder ▪ Major depression ▪ Dysthymia ▪ Bipolar disorder
293 193 132 43
4.14 2.73 1.87 0.61
48.87 46.23 46.70 51.63
20.11 19.88 21.61 20.26
10.34 10.72 10.54 9.42
3.38 3.35 3.58 3.14
7.58 7.67 7.71 7.77
2.32 2.33 2.39 2.44
Any anxiety disorder ▪ GAD ▪ Panic disorder ▪ Agoraphobia ▪ Social phobia ▪ Simple phobia ▪ OCD
710 58 108 79 275 407 20
10.03 0.82 1.53 1.12 3.89 5.75 0.28
66.86 52.97 52.74 58.89 63.23 68.35 42.80
21.81 17.58 22.42 23.89 22.02 22.50 23.00
7.27 9.16 9.29 8.43 7.71 7.04 10.90
3.41 3.05 3.73 3.69 3.43 3.50 4.17
6.02 7.60 7.53 6.85 6.49 5.87 8.40
2.48 2.23 2.34 2.86 2.52 2.57 1.93
6260 88.47 546 7.72 169 2.39 63 0.89 28 0.40 9 0.13 3 0.04
84.04 71.96 55.49 48.33 42.92 32.05 22.89
12.28 18.56 21.36 20.39 17.25 18.75 13.38
4.99 6.61 9.09 9.98 10.87 13.09 14.00
1.91 2.92 3.52 3.76 2.97 3.18 2.00
4.06 5.49 7.12 7.94 8.47 9.55 10.50
1.63 2.24 2.46 1.96 2.27 1.97 1.29
53.88 73.13 42.63
19.77 17.70 19.10
9.47 6.25 11.21
3.30 2.63 3.25
7.03 5.47 8.14
2.38 2.27 2.13
N mood/anxiety disorder ▪0 ▪1 ▪2 ▪3 ▪4 ▪5 ▪6 Combinations ▪ Mood disorder onlye ▪ Anxiety disorder onlyf ▪ Mood + anxiety disorderg
145 564 146
%
MHI-dc
2.05 7.97 2.06
M
S.D.
a The mean scores for the MHI-5, MHI-d and MHI-a differ significantly in all categories of mood and anxiety disorders from the population without these disorders at the P b 0.001 level. b The scores on the MHI-5 range from 0 to 100, with a higher score indicating a better mental health. c The scores on the MHI-d range from 3 to 18, with a higher score indicating more depressive symptoms. d The scores on the MHI-a range from 2 to 12, with a higher score indicating more anxiety symptoms. e Subjects with an anxiety disorder were excluded in these analyses. f Subjects with a mood disorder were excluded in these analyses. g Subjects in this group had at least one mood disorder plus at least one anxiety disorder.
252
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
Fig. 1. ROC curve of the MHI-5, the MHI-d and the MHI-a for the detection of major depressive disorders.
MHI-a with each of the mood and anxiety disorders as a gold standard. We also tested whether the AUC for the MHI-d and the MHI-a was significantly smaller than the AUC for the MHI-5 (or the other way around). Next, we calculated the optimal cut-off points for the MHI-5, the MHI-d, and the MHI-a. In many studies, optimal cut-off scores are determined based on the score for which the sum of sensitivity and specificity has a maximum. As we have illustrated elsewhere, this method is not optimal (Smits et al., 2007). For setting optimal cut-off points, the costs (in terms of money, human suffering, or other units) of the resulting false positives and false negatives should be made explicit. These costs must be weighted by the rates of the matching misclassifications. In that case, the optimal cut-off point is the cut-off with the lowest expected costs (Kraemer,1992; Smits et al., 2007). We decided in this study to calculate the optimal cut-off scores by several methods. First, we used the traditional method (the sum of sensitivity and specificity has a maximum value). Second, we calculated the cutoff score in which the costs of a false negative are fifty times higher than the costs of a false positive (which could be the perspective of the researcher who is looking for participants in a study on depression and anxiety). And third, we calculated the cut-off score in which the costs of a false negative are five times higher than the costs of a false positive (which could be the perspective of a respondent or a professional). The exact methods that were used to calculate these cut-off scores are described in detail elsewhere (Smits et al., 2007). All analyses were conducted in SPSS 12.0.02, except for the ROC analyses which were conducted in STATA/SE 8.2 (which permits testing of the equality of two different measures given a gold standard). 3. Results 3.1. Reliability, means and standard deviations of the MHI-5, MHI-d and MHI-a in subpopulations The reliability (Cronbach's α) of the MHI-5 in our sample was 0.83; and 0.77 for the MHI-d. In the MHI-a, the correlation between the two items was 0.55 (P b 0.001; Cronbach's α: 0.71). The mean scores on the MHI-5, the MHI-d and the MHI-a of subjects with mood and anxiety disorders are presented in Table 1. All mean scores on the MHI-5 of subjects with a mood or anxiety disorder were significantly lower than those of subjects without these dis-
orders (P b 0.001), and the mean scores on the MHI-d and MHI-a were significantly higher than in subjects without these disorders (P b 0.001). As can be seen in Table 1, the scores on the MHI-5 decreased with the number of mental disorders (and increased on the MHI-d and MHI-a). We calculated the Spearman correlation between the number of disorders (as a continuous variable) and the MHI scores, and found that this was 0.31 for the MHI-5, 0.28 for the MHI-d, and 0.30 for the MHI-a (P b 0.001 for all three correlations). Remarkably, the mean scores on the MHI-a for the mood disorder groups were higher than most of the mean scores for the anxiety disorder groups. Similarly, the mood disorder only group has a higher mean on the MHI-a than the anxiety disorder only group, and this difference was significant (P b 0.001).
Table 2 Area under the receiver operating characteristic (ROC) curves for the MHI-5, MHI-d and MHI-a, for mood and anxiety disorders. MHI-5
MHI-d
AUC
95% CI
AUC
95% CI
Pa
AUC
MHI-a
Any mood disorder ▪ Major depression ▪ Dysthymia ▪ Bipolar disorder
0.92 0.93 0.91 0.88
0.90–0.94 0.91–0.95 0.89–0.94 0.82–0.94
0.90 0.91 0.89 0.86
0.88–0.92 0.89–0.94 0.86–0.93 0.80–0.92
** ns ns *
0.87 085–0.89 *** 0.87 0.84–0.90 *** 0.87 0.84–0.90 *** 0.86 0.80–0.93 ns
Any anxiety disorder ▪ GAD ▪ Panic disorder ▪ Agoraphobia ▪ Social phobia ▪ Simple phobia ▪ OCD
0.73 0.90 0.87 0.80 0.77 0.69 0.93
0.71–0.75 0.87–0.93 0.83–0.90 0.75–0.85 0.74–0.80 0.66–0.72 0.89–0.97
0.70 0.85 0.83 0.78 0.74 0.66 0.89
0.68–0.72 0.81–0.90 0.79–0.87 0.73–0.83 0.70–0.77 0.63–0.69 0.82–0.96
*** *** *** ns *** *** ns
0.73 0.88 0.86 0.76 0.76 0.69 0.93
95% CI
0.71–0.75 0.84–0.91 0.82–0.90 0.70–0.83 0.73–0.79 0.66–0.72 0.89–0.97
Pa
ns ns ns * ns ns ns
Combinations 0.84 0.81–0.88 *** ▪ Mood disorder onlyb 0.90 0.87–0.93 0.88 0.84–0.91 * ▪ Anxiety disorder onlyc 0.69 0.66–0.71 0.65 0.63–0.68 *** 0.69 0.66–0.71 ns 0.96 0.94–0.97 0.95 0.93–0.96 * 0.93 0.91–0.95 *** ▪ Mood + anxiety disorderd *P b 0.05; **P b 0.01; ***P b 0.001. a The p-value indicates whether the area under the curve (AUC) for the subscale is significantly smaller than the AUC for the full MHI-5. b Subjects with an anxiety disorders were excluded in these analyses. c Subjects with a mood disorders were excluded in these analyses. d Subjects in this group had at least one mood disorder plus at least one anxiety disorder.
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
3.2. ROC analyses The results of the ROC analyses are presented in Table 2. The AUC of the MHI-5 for mood disorders is high (0.93 for major depression; 0.91 for dysthymia; and 0.88 for bipolar disorder; Kraemer and Kupfer, 2006). The AUC of the MHI-5 for generalized anxiety disorder (0.90), panic disorder (0.87) and obsessive-compulsive disorder (0.93) are also satisfactory, while the AUCs for phobias are still good, but considerably lower (0.80 for agoraphobia; 0.77 for social phobia; 0.69 for simple phobia). For illustrative purposes, we have presented the ROC curve of the MHI-5, the MHI-d and the MHI-a for the detection of major depressive disorders in Fig. 1. The ROC curves of the MHI-5, the MHI-d and the MHI-a for the detection of the other mood and anxiety disorders that we examined are available as supplementary materials in the online version, at doi:10.1016/j.psychres.2008.05.012. The AUC of the MHI-d for major depression and dysthymia was not significantly lower than the AUC of the MHI-5. As expected, the AUC of the MHI-d for most anxiety disorders was significantly lower than the AUC of the MHI-5 (P b 0.001 for generalized anxiety disorder, panic disorder, social phobia, simple phobia), although the difference was not significant for agoraphobia and for obsessive-compulsive disorder. This indicates that the MHI-d is as good as the full MHI-5 in screening for major depression and dysthymia.
253
The AUC of the MHI-a was not significantly lower than the MHI-5 for all anxiety disorders, except agoraphobia. This is an indication that the quality of the MHI-a as a screener for anxiety disorders (except agoraphobia) is as good as that of the full MHI-5. Unexpectedly, the MHI-a was quite good as a screener for depressive disorders. Although the AUC of the MHI-a was significantly lower than the AUC of the full MHI-5, it still had respectable values of 0.87 (for major depression and dysthymia) and 0.86 (for bipolar disorder). 3.3. Optimal cut-off values of the MHI-5, the MHI-d and the MHI-a As indicated earlier, we used three methods to calculate the optimal cut-off values of the full MHI-5, the MHI-d and the MHI-a: the cut-off score for which the sum of sensitivity and specificity has a maximum; the cut-off score for which the costs of a false negative are fifty times higher than the costs of a false positive; and the cut-off score for which the costs of a false negative are five times higher than the costs of a false positive. The results of the cut-off scores for selected disorders are presented in Table 3. As can be seen, the cut-off scores calculated with the first and second methods were the same or almost the same for most of the disorders. The optimal cut-off score on the MHI-5 for major depression and/or dysthymia according to the first and second methods was 74; and 54 according to the third method.
Table 3 Optimal cut-off points for the MHI-5, MHI-d and MHI-a. MHI-5
MDD + DYS
GAD
Panic
OCD
a b c
46 50 54 58 62 66 70 74 78 82 2 6 14 22 30 38 46 54 62 70 18 26 34 42 46 54 62 70 78 82 6 14 22 30 38 46 54 62 70 78
MHI-d Sensitivity
Specificity
0.46 0.53 0.63 0.70 0.74 0.80 0.85 0.90 0.94 0.96 0.00 0.02 0.03 0.07 0.09 0.14 0.31 0.53 0.74 0.83 0.08 0.15 0.23 0.32 0.34 0.48 0.62 0.78 0.89 0.91 0.05 0.15 0.25 0.30 0.40 0.60 0.70 0.75 0.80 1.00
0.98 0.97 0.96 0.94 0.91 0.88 0.84 0.80 0.73 0.63 1.00 1.00 1.00 1.00 0.99 0.98 0.97 0.94 0.89 0.82 1.00 0.99 0.99 0.98 0.97 0.94 0.90 0.83 0.71 0.62 1.00 1.00 1.00 0.99 0.98 0.97 0.94 0.89 0.82 0.71
c
a,b
c
a,b
c
a,b
c
b
a
5 6 7 8 9 10 11 12 13 14 4 6 8 9 10 12 14 16 18 19 6 7 8 9 10 11 12 13 14 15 9 10 11 12 13 14 15 16 17 18
MHI-a Sensitivity
Specificity
0.95 0.92 0.87 0.82 0.74 0.60 0.48 0.39 0.28 0.19 1.00 0.85 0.71 0.62 0.47 0.19 0.05 0.02 0.02 0.00 0.82 0.78 0.65 0.55 0.43 0.31 0.26 0.22 0.15 0.13 0.70 0.70 0.55 0.45 0.30 0.25 0.25 0.15 0.10 0.10
0.50 0.69 0.81 0.87 0.92 0.95 0.98 0.99 0.99 1.00 0.18 0.68 0.85 0.90 0.94 0.98 0.99 1.00 1.00 1.00 0.68 0.80 0.86 0.91 0.94 0.97 0.98 0.99 0.99 0.99 0.90 0.94 0.96 0.98 0.99 0.99 0.99 1.00 1.00 1.00
Cut-off score for which the sum of sensitivity and specificity has a maximum. Cut-off score for which the costs of a false negative are fifty times higher than the costs of a false positive. Cut-off score for which the costs of a false negative are five times higher than the costs of a false positive.
b a
c
a b
c
a,b
c
a b
c
3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12
Sensitivity
Specificity
1.00 0.96 0.88 0.80 0.69 0.48 0.39 0.22 0.10 0.04 1.00 1.00 0.93 0.81 0.64 0.45 0.40 0.21 0.12 0.05 0.99 0.94 0.90 0.81 0.66 0.51 0.35 0.21 0.12 0.04 1.00 1.00 1.00 0.90 0.85 0.70 0.45 0.30 0.15 0.05
0.14 0.40 0.66 0.82 0.88 0.95 0.97 0.99 1.00 1.00 0.14 0.39 0.64 0.80 0.87 0.94 0.96 0.98 0.99 1.00 0.14 0.39 0.65 0.81 0.87 0.94 0.96 0.98 0.99 1.00 0.14 0.39 0.64 0.80 0.87 0.93 0.96 0.98 0.99 1.00
a,b
c
a,b
c
a,b
c
a b
c
254
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
4. Discussion We found that the MHI-5 is a good screener for mood disorders in the general population, with high sensitivity and specificity. In the ROC analyses we found AUCs that were as good as those from more elaborate instruments. This means that screening for depression will take less than a minute per patient. We also found that when patients are screened for major depression and/or dysthymia, the three-item MHI-d is as good as the full MHI-5. This is interesting, because it implies that depression can be screened adequately with only three questions, while many older screening questionnaires contain 15 to 30 questions. This means that the time patients need to spend on screening can be even further reduced. This finding is in agreement with other recent research indicating that short questionnaires can be used to screen for depression (Mitchell and Coyne, 2007) and anxiety disorders (Kroenke et al., 2007). Reduction of instruments is important for several reasons. First, for some respondent groups, such as the elderly, the attention span is short and administering many questions may decrease the quality of the answers. Second, the willingness to participate and/or complete questionnaires may decrease. In addition, faced with a fixed booklet size, longer tests do not allow for the collection of data on other variables. The MHI-5 is also adequate as a screener for some anxiety disorders (generalized anxiety disorder; panic disorder; obsessivecompulsive disorder), but not for others, especially phobias (agoraphobia; social phobia; simple phobia). It is not exactly clear why the performance of the MHI-5 as a screener for phobic disorders is not optimal. Perhaps, these patients do not recognize the general anxiety symptoms of the MHI-5 as representing their phobic symptoms. It is also possible that patients with phobic disorders are better at avoiding their problems than patients with other anxiety disorders. Avoidance is one of the core problems of phobic disorders. We also found that the two-item MHI-a is as good as the full MHI-5 when used for screening on these anxiety disorders, except for agoraphobia. A remarkable finding of this study was that the mean scores on the MHI-a for the mood disorder groups were higher than most of the mean scores for the anxiety disorder groups, and that subjects with a mood disorder (without a comorbid anxiety disorder) scored higher than subjects with an anxiety disorder (without a comorbid mood disorder; P b 0.001). This is certainly caused in part by the low scores in the MHI-a in subjects with a phobic disorder, once more pointing at the fact that the MHI-a is not very good in detecting these patients. However, patients with other anxiety disorders did not score higher on the MHI-a than patients with a depressive disorder, except for patients with an obsessive-compulsive disorder. This finding is difficult to interpret, but stresses the fact that the MHI is better in detecting depressive disorders than anxiety disorders. One possible explanation may be that depressed patients suffering from agitation (one of the symptoms of depression) recognize this in nervousness and not being calm, which makes them score high on the two anxiety questions. This is the first study in which the validity of the MHI-a was examined, and the second study of the MHI-d (Yamazaki et al., 2005). Clearly more research is necessary in other populations and countries. It would be useful to examine the psychometric properties more thoroughly in primary care populations and among patients seeking treatment in specialised mental health care settings. Direct comparisons with other screening questionnaires would also be useful to examine whether the MHI is actually as effective as the others, as is suggested by this study. It would also be helpful to examine whether it might be possible to add one or two questions to the MHI-5 or MHI-a which would improve the qualities of screening for phobic disorders. This study has several limitations. First, the validity of the instrument we used as the gold standard, the CIDI, may not be optimal in all disorders, especially anxiety disorders (Komiti et al., 2001; Means-Christensen et al., 2003). A second limitation of this study is that we cannot rule out the possibility that method variance contributes to the high degree of
concordance between the CIDI and the MHI-5, as they were administered together. And because the MHI was administered after the CIDI, it cannot be ruled out that item sequence effects might have occurred. In general, it is considered to be better when screening tests are presented prior to the diagnostic interview when analysing their psychometric performance. Another limitation of this study is that DSM-III-R diagnoses were used instead of the current DSM-IV diagnoses. Future research should indicate whether our results are also valid for DSM-IV disorders. We can conclude that the MHI-5, the MHI-d and the MHI-a are attractive, very brief, and at the same time very good screening questionnaires for depression and some anxiety disorders in community settings. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2008.05.012. References Bijl, R.V., Ravelli, A., 2000. Current and residual functional disability associated with psychopathology. Psychological Medicine 30, 657–668. Bijl, R.V., Ravelli, A., Van Zessen, G., 1998a. Prevalence of psychiatric disorder in the general population: results of the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Social Psychiatry & Psychiatric Epidemiology 33, 587–595. Bijl, R.V., Van Zessen, G., Ravelli, A., 1998b. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): objectives and design. Social Psychiatry & Psychiatric Epidemiology 33, 581–586. Brugha, T.S., Jenkins, R., Taub, N., Meltzer, H., Bebbington, P.E., 2001. A general population comparison of the Composite International Diagnostic Interview (CIDI) and the Schedules for Clinical Assessment in Neuropsychiatry (SCAN). Psychological Medicine 31 (6), 1001–1013 Aug. Cuijpers, P., Smit, F., Oostenbrink, J., de Graaf, R., ten Have, M., Beekman, A., 2007. Economic costs of minor depression: a population-based study. Acta Psychiatrica Scandinavica 115, 229–236. De Graaf, R., Bijl, R.V., Ravelli, A., Smit, F., Vollebergh, W.A.M., 2002. Predictors of first incidence of DSM-III-R psychiatric disorders in the general population: findings from the Netherlands Mental Health Survey and Incidence Study. Acta Psychiatrica Scandinavica 106, 303–313. Friedman, B., Heisel, M., Delavan, R., 2005. Validity of the SF-36 five-item mental health index for major depression in functionally impaired, community-dwelling elderly patients. Journal of the American Geriatrics Society 53, 1978–1985. Janca, A., Robins, L.N., Bucholz, K.K., Early, T.S., Shayka, J.J., 1992a. Comparison of Composite International Diagnostic Interview and clinical DSM-III-R criteria checklist diagnoses. Acta Psychiatrica Scandinavica 85, 440–443. Janca, A., Robins, L.N., Cottler, L.B., Early, T.S., 1992b. Clinical observation of assessment using the Composite International Diagnostic Interview (CIDI). An analysis of the CIDI Field Trials–Wave II at the St Louis site. British Journal of Psychiatry 160, 815–818. Jordanova, V., Wickramesinghe, C., Gerada, C., Prince, M., 2004. Validation of two survey diagnostic interviews among primary care attendees: a comparison of CIS-R and CIDI with SCAN ICD-10 diagnostic categories. Psychological Medicine 34, 1013–1024. Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S., Wittchen, H.U., Kendler, K.S., 1994. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States. Results from the National Comorbidity Survey. Archives of General Psychiatry 51, 8–19. Komiti, A.A., Jackson, H.J., Judd, F.K., Cockram, A.M., Kyrios, M., Yeatman, R., Murray, G., Hordern, C., Wainwright, K., Allen, N., Singh, B., 2001. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Australian and New Zealand Journal of Psychiatry 35, 224–230. Kraemer, H.C., 1992. Evaluating Medical Tests: Objective and Quantitative Guidelines. Sage Publications, Newbury Park, Ca. Kraemer, H.C., Kupfer, D.J., 2006. Size of treatment effects and their importance to clinical research and practice. Biological Psychiatry 59, 990–996. Kroenke, K., Spitzer, R.L., Williams, J.B.W., Monahan, P.O., Lowe, B., 2007. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine 146, 317–325. Mathers, C.D., Loncar, D., 2006. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine 3 (11), e442. doi:10.1371/journal.pmed.0030442. McHorney, C.A., Ware, J.E., Lu, J.F.R., Sherbourne, C.D., 1994. The MOS 36-item shortform health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care 32, 40–66. McHorney, C.A., Ware, J.E., Raczek, A.E., 1993. The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental constructs. Medical Care 31, 247–263. Means-Christensen, A., Sherbourne, C.D., Roy-Byrne, P., 2003. The Composite International Diagnostic Interview (CIDI-Auto): problems and remedies for diagnosing panic disorder and social phobia. International Journal of Methods in Psychiatric Research 12, 167–181. Means-Christensen, A.J., Arnau, R.C., Tonidandel, A.M., Bramson, R., Meagher, M.W., 2005. An efficient method of identifying major depression and panic disorder in primary care. Journal of Behavioral Medicine 28, 565–572.
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255 Mitchell, A.J., Coyne, J.C., 2007. Do ultra-short screening instruments accurately detect depression in primary care? A pooled analysis and meta-analysis of 22 studies. British Journal of General Practice 57, 144–151. Rumpf, H.-J., Meyer, C., Hapke, U., John, U., 2001. Screening for mental health: validity of the MHI-5 using DSM-IV Axis I psychiatric disorders as gold standard. Psychiatry Research 105, 243–253. Smeets, R.M.W., Dingemans, P.M.A.J., 1993. Composite International Diagnostic Interview (CIDI), Versie 1.1. World Health Organization, Amsterdam. Smit, F., Cuijpers, P., Oostenbrink, J., Batelaan, N., de Graaf, R., Beekman, A., 2006. Excess costs of common mental disorders: population-based cohort study. Journal of Mental Health Policy and Economics 9, 193–200. Smits, N., Smit, F., Cuijpers, P., de Graaf, R., 2007. Using decision theory to derive optimal cut-off scores of screening instruments: an illustration explicating costs and benefits of mental health screening. International Journal of Methods in Psychiatric Research 16, 219–229.
255
Ware, J.E., Sherbourne, C.D., 1992. The MOS Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Medical Care 30, 473–483. Whooley, M.A., Avins, A.L., Miranda, J., Browner, W.S., 1997. Case-finding instruments for depression. Two questions are as good as many. Journal of General Internal Medicine 12, 439–445. Wittchen, H.U., Robins, L.N., Cottler, L.B., 1991. Cross-cultural feasibility, reliability and sources of variance in the Composite International Diagnostic Interview (CIDI). British Journal of Psychiatry 159, 645–653. World Health Organization, 1990. Composite International Diagnostic Interview (CIDI), Version 1.0. WHO, Geneva. Yamazaki, S., Fukuhara, S., Green, J., 2005. Usefulness of five-item and three-item Mental Health Inventories to screen for depressive symptoms in the general population of Japan. Health and Quality of Life Outcomes 3, 48.
Contents lists available at ScienceDirect
Psychiatry Research j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / p s y c h r e s
Screening for mood and anxiety disorders with the five-item, the three-item, and the two-item Mental Health Inventory Pim Cuijpers a,⁎, Niels Smits a, Tara Donker a, Margreet ten Have b, Ron de Graaf b a b
Department of Clinical Psychology, Vrije Universiteit, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands Trimbos Institute, P.O. Box 725, 3500 AH Utrecht, The Netherlands
A R T I C L E
I N F O
Article history: Received 3 July 2007 Received in revised form 2 September 2007 Accepted 26 May 2008 Keywords: Screening Major depression Depressive disorders Anxiety disorders Sensitivity Specificity
A B S T R A C T The Mental Health Inventory (MHI)-5 is an attractive, brief screening questionnaire for depression and anxiety disorders. It has been suggested that the three questions on depression (MHI-d) may be as good as the full MHI-5 in assessing depressive disorders. We examined the validity of the MHI-d and the MHI-a (the remaining two items on anxiety) in a large population-based sample of 7076 adults in the Netherlands. We also examined the validity of the MHI in assessing specific anxiety disorders. The presence of depressive and anxiety disorders in the past month was assessed with the Composite International Diagnostic Interview (CIDI), computerized version 1.1. ROC analyses indicated no significant difference between the MHI-5 (area under the curve of 0.93) and the MHI-d (area under the curve of 0.91) in detecting major depression and dysthymia. There was no difference either between the MHI-5 (area under the curve of 0.73) and the MHI-a (area under the curve 0.73) in detecting anxiety disorders. Both the MHI-5 and the MHI-a also seem to be adequate as a screener for some anxiety disorders (generalized anxiety disorder; panic disorder; obsessivecompulsive disorder), but not others, especially phobias (agoraphobia; social phobia; simple phobia). © 2008 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Depressive and anxiety disorders are highly prevalent (Kessler et al., 1994; Bijl and Ravelli, 2000), have a high incidence (De Graaf et al., 2002), and are associated with huge losses in quality of life in patients and their relatives (Bijl and Ravelli, 2000), with high levels of service use, and huge economic costs (Smit et al., 2006; Cuijpers et al., 2007). Currently, depression is the fourth disorder worldwide in terms of disease burden, and by 2030 will be the illness with the highest disease burden in high-income countries (Mathers and Loncar, 2006). Screening instruments are important tools for identifying depressed and anxious patients, both for research and clinical purposes (Yamazaki et al., 2005). Recent decades have seen the development of a large number of these screening instruments. Although most screening instruments take only a few minutes to complete, many of them list 15 to 30 items. It has been shown, however, that very brief instruments of only a few questions also perform well in detecting depressive disorders (Whooley et al., 1997; Mitchell and Coyne, 2007) and anxiety disorders (Kroenke et al., 2007). One brief screening instrument for the detection of depression is the Mental Health Inventory (MHI-5). The MHI is the mental health subscale of the Medical Outcomes Study (MOS) Short Form Health Survey (Ware and Sherbourne, 1992), and is included in both the 20-item version (SF-20; Ware and Sherbourne, 1992) and the 36-item version (SF-36; Ware and ⁎ Corresponding author. Tel.: +31 20 44 48757; fax: +31 20 44 48758. E-mail address: [email protected] (P. Cuijpers). 0165-1781/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.psychres.2008.05.012
Sherbourne,1992; McHorney et al.,1993,1994). In recent years, the interest in the MHI-5 has grown considerably, not only because it is so short, but also because it is expected that it can be used as a screen for both mood and anxiety disorders (Rumpf et al., 2001). Research has shown that the specificity and sensitivity of the MHI-5 for detecting DSM-IV Axis-I diagnoses in general is good in the general population (Rumpf et al., 2001), as it is for detecting major depression in functionally impaired, community-dwelling elderly (Friedman et al., 2005), and major depression or panic disorder in primary care patients (Means-Christensen et al., 2005). The MHI-5 contains five questions, three of which are aimed at depressive symptoms and psychological well-being, while two questions measure symptoms of anxiety. There is some evidence that removing the two anxiety-related items does not reduce the effectiveness of the MHI in detecting depression (Yamazaki et al., 2005), although this has not been examined in studies in which a formal diagnosis according to DSM criteria was used as a gold standard. This is interesting, because it would reduce the length of the MHI even further from five to three items. Although the quality of the MHI-5 as a screener for anxiety disorders has been examined less well than for depressive disorders, there is some evidence that the MHI is also a good screener for anxiety disorders in general (Rumpf et al., 2001), and for panic disorder (Means-Christensen et al., 2005). The ability of the MHI-5 to detect specific anxiety disorders other than panic disorder has not yet been examined. And just as one could assume that removing the two anxiety-related items would not reduce the ability to detect depression, one could assume that removing the three depression-related
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
items would not reduce the ability to detect anxiety. However, this has not been examined until now. In the current study, we will examine the sensitivity, specificity, and psychometric qualities of the MHI-5 in detecting depressive and anxiety disorders in a large community-based sample. We will also examine the sensitivity and specificity of the three-item depression scale of the MHI (MHI-d), and the two-item anxiety scale of the MHI (MHI-a), and compare these to the sensitivity and specificity of the full MHI-5. 2. Method 2.1. Subjects and procedure The Netherlands Mental Health Survey and Incidence Study (NEMESIS) was based on a multistage, stratified, random sampling procedure (Bijl et al., 1998a,b). In brief, a sample of 90 municipalities was drawn, using urbanisation as the stratification criterion; the sample resulted in an adequate distribution of the respondents over the 12 Dutch provinces. Then a sample of private households was drawn from the postal registers. The selected households were first sent a letter of introduction followed by a telephone contact. In each household, the member with the most recent birthday was selected, on condition that (s)he was between 18 and 64 years and sufficiently fluent in Dutch to be interviewed. To establish contact, the interviewers made a minimum of 10 phone calls or visits to a given address at different times of the day and week. In the initial data collection phase, which is used in the current study, 7076 respondents were interviewed (year of interview: 1996; response rate 69.7%). To correct for the combined effect of initial nonresponse, post-stratification weights were calculated. 2.2. Measures
251
him up, and considered himself to be a happy person). Possible scores on the MHI-d range from 3 to 18 points. The MHI-a consists of the two remaining items of the MHI-5 (asking the respondent how much of the time in the last month he had considered himself to be a very nervous person, and whether he had felt calm and peaceful), with a range of scores from 2 to 12. Higher scores on the MHI-d and MHI-a indicate worse symptoms of depression and anxiety (contrary to the MHI-5 scores). We considered transforming the score on the MHI-d and MHI-a in the same way as the MHI-5 (with a range from 0 to 100, and a higher score indicating better mental health). We decided, however, not to transform the MHI-d and MHI-a in the same way. The MHI-5 is part of an instrument measuring health-related quality of life, and for such an instrument it seems useful to have a higher score indicating better health, and a range from 0 to 100. However, because the MHI-d and the MHI-a are used as a screener for mental disorders, there is no need for such a transformation. In order to keep these screeners as userfriendly as possible and to avoid complicated transformations, we decided to use a simple method for the calculation of the total scores. 2.3. Analyses We first calculated the means and standard deviations of the MHI5, MHI-d and MHI-a for all mood and anxiety disorders separately, and used the Mann–Whitney U-test to establish whether the scores of subjects with each of these disorders differed significantly from the scores of subjects without the disorder. Then we performed Receiver Operating Characteristic (ROC) analyses and examined the Area Under the Curve (AUC) for the MHI-5, MHI-d and Table 1 MHI-scores in subjects with current DSM-III-R mood and anxiety disorders (1 month prevalence): means and standard deviations.a MHIb
Mental disorders were assessed according to DSM-III-R criteria. The instrument used was the Composite International Diagnostic Interview (CIDI, computerized version 1.1; WHO, 1990), Dutch version (Smeets and Dingemans, 1993). The CIDI can be used by trained lay interviewers. It is known to have a high interrater and test–retest reliability (Wittchen et al., 1991). The validity of the CIDI has been found to be good in several studies (Janca et al., 1992a,b; Jordanova et al., 2004), although there are indications that it may be less valid in some anxiety disorders (Komiti et al., 2001; Means-Christensen et al., 2003), and in general it can be concluded that community psychiatric surveys using structured diagnostic interview data must be interpreted cautiously (Brugha et al., 2001). In the current study, we use the diagnoses of the mood disorders (major depression and dysthymia) and anxiety disorders (panic disorder with or without agoraphobia, agoraphobia, social phobia, simple phobia, generalized anxiety disorder, and obsessive-compulsive disorder). Because we were interested in the relationship between the MHI and current depressive and anxiety disorders, we only used past month diagnoses. The Mental Health Inventory (MHI-5) was used as a screener for mood and anxiety disorders (Ware and Sherbourne, 1992). The MHI-5 consists of five items asking the respondent how much of the time in the last month he had considered himself to be a very nervous person, had felt downhearted, had felt calm and peaceful, had felt so down in the dumps that nothing could cheer you up, and considered himself to be a happy person. The answers are scored on six-point scales ranging from all of the time to none of the time. The total score is calculated by reversing the answers to two items (the third and fifth), summing up the scores, and transforming the raw scores to a scale ranging from zero to 100. A higher score indicates better mental health. The MHI-5 was administered after the CIDI during a face-to-face interview. The MHI-d comprises three items of the MHI-5 (asking the respondent how much of the time in the last month he had felt downhearted, had felt so down in the dumps that nothing could cheer
N
MHI-ad
M
S.D.
M
S.D.
Any mood disorder ▪ Major depression ▪ Dysthymia ▪ Bipolar disorder
293 193 132 43
4.14 2.73 1.87 0.61
48.87 46.23 46.70 51.63
20.11 19.88 21.61 20.26
10.34 10.72 10.54 9.42
3.38 3.35 3.58 3.14
7.58 7.67 7.71 7.77
2.32 2.33 2.39 2.44
Any anxiety disorder ▪ GAD ▪ Panic disorder ▪ Agoraphobia ▪ Social phobia ▪ Simple phobia ▪ OCD
710 58 108 79 275 407 20
10.03 0.82 1.53 1.12 3.89 5.75 0.28
66.86 52.97 52.74 58.89 63.23 68.35 42.80
21.81 17.58 22.42 23.89 22.02 22.50 23.00
7.27 9.16 9.29 8.43 7.71 7.04 10.90
3.41 3.05 3.73 3.69 3.43 3.50 4.17
6.02 7.60 7.53 6.85 6.49 5.87 8.40
2.48 2.23 2.34 2.86 2.52 2.57 1.93
6260 88.47 546 7.72 169 2.39 63 0.89 28 0.40 9 0.13 3 0.04
84.04 71.96 55.49 48.33 42.92 32.05 22.89
12.28 18.56 21.36 20.39 17.25 18.75 13.38
4.99 6.61 9.09 9.98 10.87 13.09 14.00
1.91 2.92 3.52 3.76 2.97 3.18 2.00
4.06 5.49 7.12 7.94 8.47 9.55 10.50
1.63 2.24 2.46 1.96 2.27 1.97 1.29
53.88 73.13 42.63
19.77 17.70 19.10
9.47 6.25 11.21
3.30 2.63 3.25
7.03 5.47 8.14
2.38 2.27 2.13
N mood/anxiety disorder ▪0 ▪1 ▪2 ▪3 ▪4 ▪5 ▪6 Combinations ▪ Mood disorder onlye ▪ Anxiety disorder onlyf ▪ Mood + anxiety disorderg
145 564 146
%
MHI-dc
2.05 7.97 2.06
M
S.D.
a The mean scores for the MHI-5, MHI-d and MHI-a differ significantly in all categories of mood and anxiety disorders from the population without these disorders at the P b 0.001 level. b The scores on the MHI-5 range from 0 to 100, with a higher score indicating a better mental health. c The scores on the MHI-d range from 3 to 18, with a higher score indicating more depressive symptoms. d The scores on the MHI-a range from 2 to 12, with a higher score indicating more anxiety symptoms. e Subjects with an anxiety disorder were excluded in these analyses. f Subjects with a mood disorder were excluded in these analyses. g Subjects in this group had at least one mood disorder plus at least one anxiety disorder.
252
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
Fig. 1. ROC curve of the MHI-5, the MHI-d and the MHI-a for the detection of major depressive disorders.
MHI-a with each of the mood and anxiety disorders as a gold standard. We also tested whether the AUC for the MHI-d and the MHI-a was significantly smaller than the AUC for the MHI-5 (or the other way around). Next, we calculated the optimal cut-off points for the MHI-5, the MHI-d, and the MHI-a. In many studies, optimal cut-off scores are determined based on the score for which the sum of sensitivity and specificity has a maximum. As we have illustrated elsewhere, this method is not optimal (Smits et al., 2007). For setting optimal cut-off points, the costs (in terms of money, human suffering, or other units) of the resulting false positives and false negatives should be made explicit. These costs must be weighted by the rates of the matching misclassifications. In that case, the optimal cut-off point is the cut-off with the lowest expected costs (Kraemer,1992; Smits et al., 2007). We decided in this study to calculate the optimal cut-off scores by several methods. First, we used the traditional method (the sum of sensitivity and specificity has a maximum value). Second, we calculated the cutoff score in which the costs of a false negative are fifty times higher than the costs of a false positive (which could be the perspective of the researcher who is looking for participants in a study on depression and anxiety). And third, we calculated the cut-off score in which the costs of a false negative are five times higher than the costs of a false positive (which could be the perspective of a respondent or a professional). The exact methods that were used to calculate these cut-off scores are described in detail elsewhere (Smits et al., 2007). All analyses were conducted in SPSS 12.0.02, except for the ROC analyses which were conducted in STATA/SE 8.2 (which permits testing of the equality of two different measures given a gold standard). 3. Results 3.1. Reliability, means and standard deviations of the MHI-5, MHI-d and MHI-a in subpopulations The reliability (Cronbach's α) of the MHI-5 in our sample was 0.83; and 0.77 for the MHI-d. In the MHI-a, the correlation between the two items was 0.55 (P b 0.001; Cronbach's α: 0.71). The mean scores on the MHI-5, the MHI-d and the MHI-a of subjects with mood and anxiety disorders are presented in Table 1. All mean scores on the MHI-5 of subjects with a mood or anxiety disorder were significantly lower than those of subjects without these dis-
orders (P b 0.001), and the mean scores on the MHI-d and MHI-a were significantly higher than in subjects without these disorders (P b 0.001). As can be seen in Table 1, the scores on the MHI-5 decreased with the number of mental disorders (and increased on the MHI-d and MHI-a). We calculated the Spearman correlation between the number of disorders (as a continuous variable) and the MHI scores, and found that this was 0.31 for the MHI-5, 0.28 for the MHI-d, and 0.30 for the MHI-a (P b 0.001 for all three correlations). Remarkably, the mean scores on the MHI-a for the mood disorder groups were higher than most of the mean scores for the anxiety disorder groups. Similarly, the mood disorder only group has a higher mean on the MHI-a than the anxiety disorder only group, and this difference was significant (P b 0.001).
Table 2 Area under the receiver operating characteristic (ROC) curves for the MHI-5, MHI-d and MHI-a, for mood and anxiety disorders. MHI-5
MHI-d
AUC
95% CI
AUC
95% CI
Pa
AUC
MHI-a
Any mood disorder ▪ Major depression ▪ Dysthymia ▪ Bipolar disorder
0.92 0.93 0.91 0.88
0.90–0.94 0.91–0.95 0.89–0.94 0.82–0.94
0.90 0.91 0.89 0.86
0.88–0.92 0.89–0.94 0.86–0.93 0.80–0.92
** ns ns *
0.87 085–0.89 *** 0.87 0.84–0.90 *** 0.87 0.84–0.90 *** 0.86 0.80–0.93 ns
Any anxiety disorder ▪ GAD ▪ Panic disorder ▪ Agoraphobia ▪ Social phobia ▪ Simple phobia ▪ OCD
0.73 0.90 0.87 0.80 0.77 0.69 0.93
0.71–0.75 0.87–0.93 0.83–0.90 0.75–0.85 0.74–0.80 0.66–0.72 0.89–0.97
0.70 0.85 0.83 0.78 0.74 0.66 0.89
0.68–0.72 0.81–0.90 0.79–0.87 0.73–0.83 0.70–0.77 0.63–0.69 0.82–0.96
*** *** *** ns *** *** ns
0.73 0.88 0.86 0.76 0.76 0.69 0.93
95% CI
0.71–0.75 0.84–0.91 0.82–0.90 0.70–0.83 0.73–0.79 0.66–0.72 0.89–0.97
Pa
ns ns ns * ns ns ns
Combinations 0.84 0.81–0.88 *** ▪ Mood disorder onlyb 0.90 0.87–0.93 0.88 0.84–0.91 * ▪ Anxiety disorder onlyc 0.69 0.66–0.71 0.65 0.63–0.68 *** 0.69 0.66–0.71 ns 0.96 0.94–0.97 0.95 0.93–0.96 * 0.93 0.91–0.95 *** ▪ Mood + anxiety disorderd *P b 0.05; **P b 0.01; ***P b 0.001. a The p-value indicates whether the area under the curve (AUC) for the subscale is significantly smaller than the AUC for the full MHI-5. b Subjects with an anxiety disorders were excluded in these analyses. c Subjects with a mood disorders were excluded in these analyses. d Subjects in this group had at least one mood disorder plus at least one anxiety disorder.
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
3.2. ROC analyses The results of the ROC analyses are presented in Table 2. The AUC of the MHI-5 for mood disorders is high (0.93 for major depression; 0.91 for dysthymia; and 0.88 for bipolar disorder; Kraemer and Kupfer, 2006). The AUC of the MHI-5 for generalized anxiety disorder (0.90), panic disorder (0.87) and obsessive-compulsive disorder (0.93) are also satisfactory, while the AUCs for phobias are still good, but considerably lower (0.80 for agoraphobia; 0.77 for social phobia; 0.69 for simple phobia). For illustrative purposes, we have presented the ROC curve of the MHI-5, the MHI-d and the MHI-a for the detection of major depressive disorders in Fig. 1. The ROC curves of the MHI-5, the MHI-d and the MHI-a for the detection of the other mood and anxiety disorders that we examined are available as supplementary materials in the online version, at doi:10.1016/j.psychres.2008.05.012. The AUC of the MHI-d for major depression and dysthymia was not significantly lower than the AUC of the MHI-5. As expected, the AUC of the MHI-d for most anxiety disorders was significantly lower than the AUC of the MHI-5 (P b 0.001 for generalized anxiety disorder, panic disorder, social phobia, simple phobia), although the difference was not significant for agoraphobia and for obsessive-compulsive disorder. This indicates that the MHI-d is as good as the full MHI-5 in screening for major depression and dysthymia.
253
The AUC of the MHI-a was not significantly lower than the MHI-5 for all anxiety disorders, except agoraphobia. This is an indication that the quality of the MHI-a as a screener for anxiety disorders (except agoraphobia) is as good as that of the full MHI-5. Unexpectedly, the MHI-a was quite good as a screener for depressive disorders. Although the AUC of the MHI-a was significantly lower than the AUC of the full MHI-5, it still had respectable values of 0.87 (for major depression and dysthymia) and 0.86 (for bipolar disorder). 3.3. Optimal cut-off values of the MHI-5, the MHI-d and the MHI-a As indicated earlier, we used three methods to calculate the optimal cut-off values of the full MHI-5, the MHI-d and the MHI-a: the cut-off score for which the sum of sensitivity and specificity has a maximum; the cut-off score for which the costs of a false negative are fifty times higher than the costs of a false positive; and the cut-off score for which the costs of a false negative are five times higher than the costs of a false positive. The results of the cut-off scores for selected disorders are presented in Table 3. As can be seen, the cut-off scores calculated with the first and second methods were the same or almost the same for most of the disorders. The optimal cut-off score on the MHI-5 for major depression and/or dysthymia according to the first and second methods was 74; and 54 according to the third method.
Table 3 Optimal cut-off points for the MHI-5, MHI-d and MHI-a. MHI-5
MDD + DYS
GAD
Panic
OCD
a b c
46 50 54 58 62 66 70 74 78 82 2 6 14 22 30 38 46 54 62 70 18 26 34 42 46 54 62 70 78 82 6 14 22 30 38 46 54 62 70 78
MHI-d Sensitivity
Specificity
0.46 0.53 0.63 0.70 0.74 0.80 0.85 0.90 0.94 0.96 0.00 0.02 0.03 0.07 0.09 0.14 0.31 0.53 0.74 0.83 0.08 0.15 0.23 0.32 0.34 0.48 0.62 0.78 0.89 0.91 0.05 0.15 0.25 0.30 0.40 0.60 0.70 0.75 0.80 1.00
0.98 0.97 0.96 0.94 0.91 0.88 0.84 0.80 0.73 0.63 1.00 1.00 1.00 1.00 0.99 0.98 0.97 0.94 0.89 0.82 1.00 0.99 0.99 0.98 0.97 0.94 0.90 0.83 0.71 0.62 1.00 1.00 1.00 0.99 0.98 0.97 0.94 0.89 0.82 0.71
c
a,b
c
a,b
c
a,b
c
b
a
5 6 7 8 9 10 11 12 13 14 4 6 8 9 10 12 14 16 18 19 6 7 8 9 10 11 12 13 14 15 9 10 11 12 13 14 15 16 17 18
MHI-a Sensitivity
Specificity
0.95 0.92 0.87 0.82 0.74 0.60 0.48 0.39 0.28 0.19 1.00 0.85 0.71 0.62 0.47 0.19 0.05 0.02 0.02 0.00 0.82 0.78 0.65 0.55 0.43 0.31 0.26 0.22 0.15 0.13 0.70 0.70 0.55 0.45 0.30 0.25 0.25 0.15 0.10 0.10
0.50 0.69 0.81 0.87 0.92 0.95 0.98 0.99 0.99 1.00 0.18 0.68 0.85 0.90 0.94 0.98 0.99 1.00 1.00 1.00 0.68 0.80 0.86 0.91 0.94 0.97 0.98 0.99 0.99 0.99 0.90 0.94 0.96 0.98 0.99 0.99 0.99 1.00 1.00 1.00
Cut-off score for which the sum of sensitivity and specificity has a maximum. Cut-off score for which the costs of a false negative are fifty times higher than the costs of a false positive. Cut-off score for which the costs of a false negative are five times higher than the costs of a false positive.
b a
c
a b
c
a,b
c
a b
c
3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12 3 4 5 6 7 8 9 10 11 12
Sensitivity
Specificity
1.00 0.96 0.88 0.80 0.69 0.48 0.39 0.22 0.10 0.04 1.00 1.00 0.93 0.81 0.64 0.45 0.40 0.21 0.12 0.05 0.99 0.94 0.90 0.81 0.66 0.51 0.35 0.21 0.12 0.04 1.00 1.00 1.00 0.90 0.85 0.70 0.45 0.30 0.15 0.05
0.14 0.40 0.66 0.82 0.88 0.95 0.97 0.99 1.00 1.00 0.14 0.39 0.64 0.80 0.87 0.94 0.96 0.98 0.99 1.00 0.14 0.39 0.65 0.81 0.87 0.94 0.96 0.98 0.99 1.00 0.14 0.39 0.64 0.80 0.87 0.93 0.96 0.98 0.99 1.00
a,b
c
a,b
c
a,b
c
a b
c
254
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255
4. Discussion We found that the MHI-5 is a good screener for mood disorders in the general population, with high sensitivity and specificity. In the ROC analyses we found AUCs that were as good as those from more elaborate instruments. This means that screening for depression will take less than a minute per patient. We also found that when patients are screened for major depression and/or dysthymia, the three-item MHI-d is as good as the full MHI-5. This is interesting, because it implies that depression can be screened adequately with only three questions, while many older screening questionnaires contain 15 to 30 questions. This means that the time patients need to spend on screening can be even further reduced. This finding is in agreement with other recent research indicating that short questionnaires can be used to screen for depression (Mitchell and Coyne, 2007) and anxiety disorders (Kroenke et al., 2007). Reduction of instruments is important for several reasons. First, for some respondent groups, such as the elderly, the attention span is short and administering many questions may decrease the quality of the answers. Second, the willingness to participate and/or complete questionnaires may decrease. In addition, faced with a fixed booklet size, longer tests do not allow for the collection of data on other variables. The MHI-5 is also adequate as a screener for some anxiety disorders (generalized anxiety disorder; panic disorder; obsessivecompulsive disorder), but not for others, especially phobias (agoraphobia; social phobia; simple phobia). It is not exactly clear why the performance of the MHI-5 as a screener for phobic disorders is not optimal. Perhaps, these patients do not recognize the general anxiety symptoms of the MHI-5 as representing their phobic symptoms. It is also possible that patients with phobic disorders are better at avoiding their problems than patients with other anxiety disorders. Avoidance is one of the core problems of phobic disorders. We also found that the two-item MHI-a is as good as the full MHI-5 when used for screening on these anxiety disorders, except for agoraphobia. A remarkable finding of this study was that the mean scores on the MHI-a for the mood disorder groups were higher than most of the mean scores for the anxiety disorder groups, and that subjects with a mood disorder (without a comorbid anxiety disorder) scored higher than subjects with an anxiety disorder (without a comorbid mood disorder; P b 0.001). This is certainly caused in part by the low scores in the MHI-a in subjects with a phobic disorder, once more pointing at the fact that the MHI-a is not very good in detecting these patients. However, patients with other anxiety disorders did not score higher on the MHI-a than patients with a depressive disorder, except for patients with an obsessive-compulsive disorder. This finding is difficult to interpret, but stresses the fact that the MHI is better in detecting depressive disorders than anxiety disorders. One possible explanation may be that depressed patients suffering from agitation (one of the symptoms of depression) recognize this in nervousness and not being calm, which makes them score high on the two anxiety questions. This is the first study in which the validity of the MHI-a was examined, and the second study of the MHI-d (Yamazaki et al., 2005). Clearly more research is necessary in other populations and countries. It would be useful to examine the psychometric properties more thoroughly in primary care populations and among patients seeking treatment in specialised mental health care settings. Direct comparisons with other screening questionnaires would also be useful to examine whether the MHI is actually as effective as the others, as is suggested by this study. It would also be helpful to examine whether it might be possible to add one or two questions to the MHI-5 or MHI-a which would improve the qualities of screening for phobic disorders. This study has several limitations. First, the validity of the instrument we used as the gold standard, the CIDI, may not be optimal in all disorders, especially anxiety disorders (Komiti et al., 2001; Means-Christensen et al., 2003). A second limitation of this study is that we cannot rule out the possibility that method variance contributes to the high degree of
concordance between the CIDI and the MHI-5, as they were administered together. And because the MHI was administered after the CIDI, it cannot be ruled out that item sequence effects might have occurred. In general, it is considered to be better when screening tests are presented prior to the diagnostic interview when analysing their psychometric performance. Another limitation of this study is that DSM-III-R diagnoses were used instead of the current DSM-IV diagnoses. Future research should indicate whether our results are also valid for DSM-IV disorders. We can conclude that the MHI-5, the MHI-d and the MHI-a are attractive, very brief, and at the same time very good screening questionnaires for depression and some anxiety disorders in community settings. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.psychres.2008.05.012. References Bijl, R.V., Ravelli, A., 2000. Current and residual functional disability associated with psychopathology. Psychological Medicine 30, 657–668. Bijl, R.V., Ravelli, A., Van Zessen, G., 1998a. Prevalence of psychiatric disorder in the general population: results of the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Social Psychiatry & Psychiatric Epidemiology 33, 587–595. Bijl, R.V., Van Zessen, G., Ravelli, A., 1998b. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): objectives and design. Social Psychiatry & Psychiatric Epidemiology 33, 581–586. Brugha, T.S., Jenkins, R., Taub, N., Meltzer, H., Bebbington, P.E., 2001. A general population comparison of the Composite International Diagnostic Interview (CIDI) and the Schedules for Clinical Assessment in Neuropsychiatry (SCAN). Psychological Medicine 31 (6), 1001–1013 Aug. Cuijpers, P., Smit, F., Oostenbrink, J., de Graaf, R., ten Have, M., Beekman, A., 2007. Economic costs of minor depression: a population-based study. Acta Psychiatrica Scandinavica 115, 229–236. De Graaf, R., Bijl, R.V., Ravelli, A., Smit, F., Vollebergh, W.A.M., 2002. Predictors of first incidence of DSM-III-R psychiatric disorders in the general population: findings from the Netherlands Mental Health Survey and Incidence Study. Acta Psychiatrica Scandinavica 106, 303–313. Friedman, B., Heisel, M., Delavan, R., 2005. Validity of the SF-36 five-item mental health index for major depression in functionally impaired, community-dwelling elderly patients. Journal of the American Geriatrics Society 53, 1978–1985. Janca, A., Robins, L.N., Bucholz, K.K., Early, T.S., Shayka, J.J., 1992a. Comparison of Composite International Diagnostic Interview and clinical DSM-III-R criteria checklist diagnoses. Acta Psychiatrica Scandinavica 85, 440–443. Janca, A., Robins, L.N., Cottler, L.B., Early, T.S., 1992b. Clinical observation of assessment using the Composite International Diagnostic Interview (CIDI). An analysis of the CIDI Field Trials–Wave II at the St Louis site. British Journal of Psychiatry 160, 815–818. Jordanova, V., Wickramesinghe, C., Gerada, C., Prince, M., 2004. Validation of two survey diagnostic interviews among primary care attendees: a comparison of CIS-R and CIDI with SCAN ICD-10 diagnostic categories. Psychological Medicine 34, 1013–1024. Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S., Wittchen, H.U., Kendler, K.S., 1994. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States. Results from the National Comorbidity Survey. Archives of General Psychiatry 51, 8–19. Komiti, A.A., Jackson, H.J., Judd, F.K., Cockram, A.M., Kyrios, M., Yeatman, R., Murray, G., Hordern, C., Wainwright, K., Allen, N., Singh, B., 2001. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Australian and New Zealand Journal of Psychiatry 35, 224–230. Kraemer, H.C., 1992. Evaluating Medical Tests: Objective and Quantitative Guidelines. Sage Publications, Newbury Park, Ca. Kraemer, H.C., Kupfer, D.J., 2006. Size of treatment effects and their importance to clinical research and practice. Biological Psychiatry 59, 990–996. Kroenke, K., Spitzer, R.L., Williams, J.B.W., Monahan, P.O., Lowe, B., 2007. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine 146, 317–325. Mathers, C.D., Loncar, D., 2006. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine 3 (11), e442. doi:10.1371/journal.pmed.0030442. McHorney, C.A., Ware, J.E., Lu, J.F.R., Sherbourne, C.D., 1994. The MOS 36-item shortform health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care 32, 40–66. McHorney, C.A., Ware, J.E., Raczek, A.E., 1993. The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental constructs. Medical Care 31, 247–263. Means-Christensen, A., Sherbourne, C.D., Roy-Byrne, P., 2003. The Composite International Diagnostic Interview (CIDI-Auto): problems and remedies for diagnosing panic disorder and social phobia. International Journal of Methods in Psychiatric Research 12, 167–181. Means-Christensen, A.J., Arnau, R.C., Tonidandel, A.M., Bramson, R., Meagher, M.W., 2005. An efficient method of identifying major depression and panic disorder in primary care. Journal of Behavioral Medicine 28, 565–572.
P. Cuijpers et al. / Psychiatry Research 168 (2009) 250–255 Mitchell, A.J., Coyne, J.C., 2007. Do ultra-short screening instruments accurately detect depression in primary care? A pooled analysis and meta-analysis of 22 studies. British Journal of General Practice 57, 144–151. Rumpf, H.-J., Meyer, C., Hapke, U., John, U., 2001. Screening for mental health: validity of the MHI-5 using DSM-IV Axis I psychiatric disorders as gold standard. Psychiatry Research 105, 243–253. Smeets, R.M.W., Dingemans, P.M.A.J., 1993. Composite International Diagnostic Interview (CIDI), Versie 1.1. World Health Organization, Amsterdam. Smit, F., Cuijpers, P., Oostenbrink, J., Batelaan, N., de Graaf, R., Beekman, A., 2006. Excess costs of common mental disorders: population-based cohort study. Journal of Mental Health Policy and Economics 9, 193–200. Smits, N., Smit, F., Cuijpers, P., de Graaf, R., 2007. Using decision theory to derive optimal cut-off scores of screening instruments: an illustration explicating costs and benefits of mental health screening. International Journal of Methods in Psychiatric Research 16, 219–229.
255
Ware, J.E., Sherbourne, C.D., 1992. The MOS Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Medical Care 30, 473–483. Whooley, M.A., Avins, A.L., Miranda, J., Browner, W.S., 1997. Case-finding instruments for depression. Two questions are as good as many. Journal of General Internal Medicine 12, 439–445. Wittchen, H.U., Robins, L.N., Cottler, L.B., 1991. Cross-cultural feasibility, reliability and sources of variance in the Composite International Diagnostic Interview (CIDI). British Journal of Psychiatry 159, 645–653. World Health Organization, 1990. Composite International Diagnostic Interview (CIDI), Version 1.0. WHO, Geneva. Yamazaki, S., Fukuhara, S., Green, J., 2005. Usefulness of five-item and three-item Mental Health Inventories to screen for depressive symptoms in the general population of Japan. Health and Quality of Life Outcomes 3, 48.