- Email: [email protected]
The panic disorder screener (PADIS): Development of an accurate and brief population screening tool
Psychiatry Research 228 (2015) 72–76
Contents lists available at ScienceDirect
Psychiatry Research journal homepage: www.elsevier.com/locate/psychres
The panic disorder screener (PADIS): Development of an accurate and brief population screening tool Philip J. Batterham a,n, Andrew J. Mackinnon b, Helen Christensen c a National Institute for Mental Health Research, Research School of Population Health, The Australian National University, 63 Eggleston Road, Acton, ACT 2601, Australia b Orygen, The National Centre of Excellence in Youth Mental Health, The University of Melbourne, Melbourne, Australia c Black Dog Institute, The University of New South Wales, Sydney, Australia
art ic l e i nf o
a b s t r a c t
Article history: Received 7 November 2014 Received in revised form 10 April 2015 Accepted 15 April 2015 Available online 23 April 2015
The Panic Disorder Screener (PADIS) was developed as a new screener to identify panic disorder in the community and to assess severity of symptoms. The PADIS was developed to fill a gap in existing screening measures, as there are no brief panic screeners available that assess severity. The current study aimed to test the performance of the screener relative to the Patient Health Questionnaire-panic scale (PHQ-panic). The 4-item PADIS was administered to 12,336 young Australian adults, together with the PHQ-panic. A subsample of 1674 participants also completed a phone-based clinical interview to determine whether they met DSM-IV criteria for panic disorder. The PADIS (77% sensitivity, 84% specificity) had higher sensitivity for identifying panic disorder based on clinical criteria than the PHQ-panic (57% sensitivity, 91% specificity), although with reduced specificity. Administration of the PADIS required a mean of 1.9 items, compared to 4.7 items for the PHQ-panic. Each one-point increase in PADIS score was associated with 69% increased odds of meeting clinical criteria for panic disorder. The PADIS was found to be a valid, reliable and brief panic screener that is freely available for use in research and clinical settings. & 2015 Elsevier Ireland Ltd. All rights reserved.
Keywords: Panic disorder Panic symptoms Anxiety Sensitivity Specificity Measurement
1. Introduction It has been estimated that up to 4.7% of the population experiences panic disorder during their lifetime (Andersson and Carlbring, 2011; Kessler et al., 2005a). Panic disorder accounts for a considerable proportion of disease burden in the general population (Eaton et al., 2008; Gadermann et al., 2012). However, individuals experiencing anxiety disorders rarely seek help (Burgess et al., 2009; Kessler et al., 2006) and panic disorder is frequently unrecognised in primary care (Lowe et al., 2003; Spitzer et al., 1999). In addition, subclinical panic symptoms have a high burden and carry considerable risk for development of agoraphobia and other mental health problems (Andersson and Carlbring, 2011; Bunaciu et al., 2010; Korte et al., 2013). There are several self-report scales available to assess symptoms of panic disorder. Screening scales are particularly useful in primary care, for online programs and for population-based research, where scale and resource constraints demand brevity (Kessler et al., 2002; Williams et al., 2002). However, most panic scales have more than 10 items, making them unsuitable for use as brief screeners in research and clinical settings.
n
Corresponding author. Tel.: þ 61 2 61251031; fax: þ 61 2 61250733. E-mail address: [email protected] (P.J. Batterham).
http://dx.doi.org/10.1016/j.psychres.2015.04.016 0165-1781/& 2015 Elsevier Ireland Ltd. All rights reserved.
Two exceptions are the five-item Patient Health Questionnaire – panic scale (PHQ-panic; Spitzer et al., 1999) and the seven-item selfreport version of the Panic Disorder Severity Scale (PDSS-SR; Houck et al., 2002; Shear et al., 2001). The PHQ-panic scale includes five items that approximately equate to the core symptoms of panic disorder as defined in DSM-IV, although the final item is sometimes presented as a checklist of 11 symptoms. The PHQ-panic scale has been reported to have excellent screening performance by its authors (Lowe et al., 2003), although another group has reported more moderate performance in a primary care sample (Wittkampf et al., 2011). With its use of binary responses, the PHQ-panic scale does not provide a severity rating (Kroenke et al., 2010). The lack of a severity rating limits the use of the scale in identifying individuals experiencing subclinical symptoms of panic disorder, which is responsible for considerable burden above and beyond that of panic disorder. In contrast, the PDSS-SR provides a severity rating, with scores ranging 0–28. However, the PDSS-SR is considerably lengthier than the PHQ-panic (7 items/1108 words in the PDSS-SR vs. 5 items/195 words in PHQ-panic) and has greater literacy demands (Flesch-Kincaid reading level 11.1 vs. 2.4). Its length makes it unsuitable as a screener in many population-based settings. Although the PDSS-SR was designed to assess severity of panic disorder in clinical cases, it has been widely adopted for use as a screening tool (e.g., van Ballegooijen et al., 2011). Consequently, there is only indirect evidence for the accuracy of the PDSS-SR relative to clinical criteria,
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
with its counterpart interviewer-administered version demonstrating 83% sensitivity and 64% specificity relative to clinical criteria (Shear et al., 2001) [the correlation between the PDSS-SR and the interviewerrated version is 0.81 (Houck et al., 2002)]. Consequently, there is a need to develop new screening measures for panic disorder that are validated, are brief and provide a severity rating. The present study reports on the development and screening performance of the Panic Disorder Screener (PADIS), a four-item scale designed to screen for panic disorder in the community setting. The scale uses a vignette to describe the typical symptoms of a panic attack, adapting an approach used previously for screening (Christensen et al., 2011). This introductory vignette is followed by questions about number of panic attacks, worry about panic attacks, worry about consequences of panic attacks and avoidance due to panic attacks. The questions were designed to correspond approximately with each of the DSM-IV criteria for panic disorder within a self-report framework. This process was guided by similar principles to the development of the DSM-5 cross-cutting assessment, although with a sole focus on panic disorder. Specifically, it was designed to be suitable for self-report use by most adults in most settings, providing information for subsequent clinical assessment and appropriate treatment (Narrow et al., 2013). Responses on the PADIS are provided on a frequency scale, to enable assessment of the severity of panic disorder symptoms. The PADIS was designed to be brief (four items, 228 words, reading grade 7.5) to ensure utility in a range of research and clinical settings. This study aimed to assess the screening performance of the PADIS relative to the PHQ-panic scale in identifying panic disorder within a population-based sample. We hypothesised that the PADIS severity score would provide significantly greater sensitivity in screening for panic disorder in a population sample than the PHQ-panic scale, with adequate specificity. 2. Method 2.1. Participants and procedure Data for this study were taken from the screening phase of the iChill study, an intervention trial testing an online cognitive behavioural therapy intervention to prevent new cases of Generalised Anxiety Disorder (Christensen et al., 2014). The trial, conducted between May 2010 and June 2012, is described by Christensen et al. (2010) and registered in Current Controlled Trials (controlled-trials.com, ISRCTN76298775). Invitations and screening surveys were mailed to 120,000 18–30 year olds selected randomly from the Australian electoral roll, with 12,430 (10.4%) responding by returning the paper-based screening survey. Registration on the electoral roll is compulsory in Australia. Data for both the PADIS and the PHQ-panic were available for 12,336 (99.2%) of respondents. Individuals who returned a survey were assessed for eligibility for the intervention trial, with eligible participants being contacted by telephone to complete a Mini International Neuropsychiatric Interview (MINI, v5.0) (Sheehan et al., 1998) within 21 days of survey return. Inclusion and exclusion criteria are shown in Fig. 1. The exclusion criteria were based on the requirements of the trial, which targeted young adults (18–30) who were not currently in treatment and who had GAD symptoms above the median score for this population. Access to the internet was also a requirement. Postgraduate clinical psychology students administered the telephone MINI interviews, and were blind to the participant responses to the survey. Interviewers were given four hours training, including practice interviews, with oversight from a clinical psychologist and the research team. The MINI interviews included the modules on panic (lifetime, current), social anxiety disorder (current), depression (current) and generalised anxiety disorder (lifetime, current). Not all participants could be contacted within the allotted time period, despite up to seven call attempts being made, resulting in a final sample of 1674 participants with both survey data and clinical assessment data available (66.5% of eligible and consenting; 99.2% of clinically assessed participants). No incentives were provided to participants. The study had approval from the ANU Human Research Ethics Committee (Protocol 2008/548). 2.2. Measures The PADIS is four-item screener for panic disorder. The introduction to the scale reads:
73
“A sudden feeling of anxiety, fear, discomfort or uneasiness may indicate a panic episode. A panic episode occurs unexpectedly, peaks within 10 min and includes four or more of the following sensations at the same time: Skipping, racing or pounding heart; Sweating or clammy hands; Trembling or shaking; Shortness of breath or difficulty breathing; A choking feeling or lump in your throat; Chest pain, pressure or discomfort; Nausea or stomach problems; Feeling dizzy, unsteady, lightheaded or faint; Feeling strange, unreal, detached or unfamiliar; Fear that you are losing control or going crazy; Fear that you are dying; Feeling tingling or numbness; Hot flushes or chills”.
The four items are: (1) “Based on this description, about how many panic episodes have you experienced in the past month?” (None/One/Between 2 and 5/Between 6 and 10/11 or more). (2) “In the past month, how often have you been worried about having another panic episode?” (Never/Occasionally/Often/All the time). (3) “In the past month, how often have you been worried about the consequences of having another panic episode?” (Never/Occasionally/Often/All the time). (4) “In the past month, how often did you avoid places, situations or activities because you were afraid of having or triggering a panic episode? (For example, not going to public places alone, not leaving your house, avoiding physical exertion or avoiding stressful activities.)” (Never/Occasionally/Often/All the time). The first item is rated on a 5-point Likert scale scored 0–4, while remaining items are rated on a 4-point Likert scale score 0–3. Severity scores are obtained by summing responses to all items, with scores ranging from 0 to 13. Respondents who respond “None” to the first item do not complete subsequent items as these items are irrelevant, and are consequently scored zero. The PHQ-panic has 5 items, each with responses of “yes” or “no”. Individuals who respond “yes” to every item are deemed to meet criteria for panic disorder (Spitzer et al., 1999). Individuals who respond “no” to an item do not receive subsequent items. For the fifth item, a list of panic symptoms, individuals who endorse fewer than four symptoms are counted as not meeting criteria.
2.3. Analysis The internal consistency of the PADIS was assessed using Cronbach's alpha and the dimensionality of the PADIS items were assessed using an exploratory factor analysis with principal axis factoring. The accuracy of the PADIS score in assessing MINI criteria for current panic disorder was then examined, with comparison to the accuracy of the PHQ-Panic. Analyses examined accuracy (based on sensitivity, specificity, positive predictive value and negative predictive value) of the PADIS and PHQ-Panic based on prescribed cut-offs, with confidence intervals calculated using the software and methods described by Mackinnon (2000). In addition, the PADIS and PHQ-panic were compared based on the area under the Receiver Operating Characteristic curve (AUC) for each scale. As PHQ-panic has only five binary items, AUC was assessed on the basis of number of items endorsed (out of five) but also total symptom count (out of 11). Convergent validity was also assessed by comparing PADIS severity scores with panic item counts from the PHQ-panic, social phobia scores on the SPIN (Connor et al., 2000), generalised anxiety scores on the GAD-7 (Spitzer et al., 2006) and depression scores on the Patient Health Questionnaire-9 (Spitzer et al., 1999). Analyses were conducted in SPSS v20 (IBM Corp, Chicago Ill), except for comparisons of AUC, which were conducted in SAS v9.4 (SAS Institute Inc, Cary NC).
3. Results 3.1. Response and sample characteristics In the full screener sample (n ¼12,336), the PADIS had high internal consistency (Cronbach α ¼0.86). Based on the first item, 8740 (70.8%) participants reported no panic episodes in the past month, with 1524 (12.3%) reporting one episode, 1634 (13.2) reporting 2–5 episodes, 325 (2.6%) reporting 6–10 episodes and 111 (0.9%) reporting 11 or more episodes. Endorsement rates for remaining items are shown in Table 1. In items 2–4, endorsement of each response category was similar, with 12–15% of participants endorsing each item at any level. Participants who reported no panic episodes in the first item were not administered the remaining items and scored as zero. This skip pattern meant 8740 participants were administered a single item, while the
74
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
120,000 surveys sent • 579 (4.7%) not in appropriate
107,570 surveys not returned
age range (18-30)
12,430 (10.4%) surveys returned
• 627 (5.0%) currently undergoing cognitive behavioural therapy
• 1,229 (9.9%) currently seeing a psychiatrist or psychologist
• 1,414 (33.6%) no consent to follow up • 274 (6.5%) insufficient contact information
4,205 (33.8%) eligible participants
2,517 (59.9%) consenting participants
1,687 (67.0%) completed MINI
• 139 (1.1%) no internet access at home or work
• 6,884 (55.4%) few symptoms of generalized anxiety (GAD -7 score <5)
830 (33.0%) unreachable by phone for MINI interview within 21 days
Fig. 1. CONSORT-based flow diagram of participant inclusion.
Table 1 Rates of endorsement for PADIS items in the general population sample (n¼ 12,361). Occasionally Worry about another panic episode Worry about consequences of another panic episode Avoided places/situations/activities
1399 1208 996
Often (11.4%) (9.8%) (8.1%)
346 355 344
All the time (2.8%) (2.9%) (2.8%)
113 114 106
(0.9%) (0.9%) (0.9%)
Responses of “Not at all” are excluded from the table; participants who reported no panic episodes were classified in the “not at all” category.
remainder answered all four items, resulting in a mean of 1.9 items administered across all participants. By comparison, participants were administered a mean of 4.7 items on the PHQ-panic (or 2.3 items if the symptom checklist is counted as a single item). Factor analysis revealed a single factor with an eigenvalue of 3.04 accounting for 78.1% of the variance in the items. Subsequent factors had eigenvalues of 0.43 and 0.39, indicating that the scale was strongly unidimensional. All items loaded on the single factor at 0.73 or higher (loadings for the four items were 0.75, 0.92, 0.90 and 0.73 respectively). Sample characteristics for the participants who completed the clinical assessment are presented in Table 2. Severity scores on the PADIS were significantly higher among participants who met criteria for panic disorder on the PHQ-panic (F1, 1660 ¼789.3, p o0.001) or MINI (F1, 1660 ¼430.4, p o0.001), separated/ widowed/divorced participants (F3, 1650 ¼4.6, p ¼0.003), participants with lower levels of education (F3, 1657 ¼7.9, p o0.001), and participants who were unemployed (F3, 1649 ¼ 4.2, p ¼0.006). There were no significant differences on the basis of age, gender or language. PADIS scores were strongly correlated with panic symptom counts from the PHQ-panic, indicating convergent validity, but only moderately correlated with social phobia, generalised anxiety and depression scores, indicative of divergent validity while also reflecting high comorbidity between these disorders. Likewise, the magnitude of the difference in PADIS scores was greater among those who met MINI caseness for panic disorder than for those who met criteria for generalised anxiety, social phobia or depression. 3.2. Comparison to clinical assessment Based on the MINI assessment, 107 (6.4%) of the 1674 participants who were assessed met clinical criteria for current panic disorder, including 27 males (6.7%) and 80 females (6.3%). A cut point of the PADIS was determined based on Youden indices, with
scores of 4 or higher on the PADIS found to be most accurate for distinguishing MINI panic cases from non-cases. The area under the curve was 0.87 for the PADIS, significantly greater than that for PHQ-panic symptom count [based on 11 symptoms, AUC ¼0.83, χ2 (1)¼5.63, p ¼0.02] and for PHQ-panic item endorsement [based on the 5 items, AUC¼0.74, χ2 (1)¼33.81, p o0.001]. Based on univariate logistic regression, each one-point increase in PADIS score was associated with 68.6% increase in the odds of meeting clinical criteria for panic disorder on the MINI. The sensitivity, specificity, positive predictive value and negative predictive value of the PADIS and PHQ-panic, relative to MINI clinical criteria for current panic disorder, are shown in Table 3. The chosen cut point of 4 is compared to alternative cut points in the Table. McNemar's test demonstrated a significant differences between the PADIS and the PHQ-panic, with the PADIS having significantly higher sensitivity [χ2 (1)¼ 16.3, p o0.001] and lower specificity [χ2 (1) ¼47.7, po 0.001] compared to the PHQ-panic at prescribed cut-points. Using a cut-off of 3 or higher on the PADIS also showed accurate classification, while a cut-off of 5 or higher was associated with commensurate specificity with the PHQ-panic [χ2 (1)¼ 1.8, p¼ 0.12] and significantly higher sensitivity [χ2 (1)¼ 3.3, p ¼0.04]. As sensitivity o0.7 on either measure was deemed to be suboptimal, the cut point of 4 was retained.
4. Discussion This study found that the PADIS performs with greater sensitivity in detecting cases of current panic disorder than the PHQpanic, with comparable response burden. For screening purposes, a high probability of detecting true cases (high sensitivity) and a high probability that negative test results indicate non-caseness (negative predictive value) are the imperative. The PADIS outperformed the PHQ-panic scale on both of these indices, using considerably fewer items, while retaining adequate specificity for
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
screening purposes. Indeed using a cut-off with similarly high specificity to the PHQ-panic scale (PADIS Z5), sensitivity remained significantly higher for the PADIS than the PHQ-panic. For popuTable 2 Sample characteristics for participants who received a clinical assessment (n¼ 1674). n (%) or Mean (S.D.) PADIS score PADIS criteria ( Z4) Yes No PHQ-panic criteria Yes No MINI current panic criteria Yes No MINI current social phobia Yes No MINI current GAD Yes No MINI current depression Yes No Age Gender Female Male Marital status Married De facto Separated/divorced/ widowed Single, never married Education Did not complete high school Completed high school Bachelor's degree Higher degree Employment status Employed full-time Employed PT/casual Unemployed, looking for work Not in the labour force Language spoken at home English English and other Other PHQ-panic symptom count SPIN score GAD-7 score PHQ-9 score
Mean (S.D.) PADIS score
r
–
–
331 1330
(19.9%) – (80.1%) –
– –
209 1479
(12.4%) 5.36 (2.86)n (87.6%) 1.13 (1.89)
–
107 1581
(6.4%) 5.95 (3.34)n (93.7%) 1.37 (2.10)
–
161 1501
(9.7%) 2.94 (3.03)n (90.3%) 1.53 (2.36)
–
267 1395
(16.1%) 3.09 (3.17)n (83.9%) 1.39 (2.21)
–
210 (12.6%) 3.02 (3.18)n 1451 (87.4%) 1.47 (2.28) 25.35 (3.38) –
–
1270 404
(75.9%) 1.70 (2.47) (24.1%) 1.53 (2.44)
–
238 375 17
(14.3%) 1.37 (2.49)n (22.5%) 1.69 (2.48) (1.0%) 3.59 (3.18)
–
1036
(62.2%) 1.69 (2.44)
1.66 (2.47)
92
(5.5%)
2.62 (2.95)n
532 817 232
(31.8%) 1.85 (2.59) (48.8%) 1.51 (2.40) (13.9%) 1.36 (2.06)
976 473 118
(58.6%) 1.56 (2.36)n (28.3%) 1.71 (2.58) (7.1%) 2.40 (2.89)
98
(5.9%)
1.58 (2.24)
1383 259 32 3.56 19.36 9.22 8.56
(82.6%) (15.5%) (1.9%) (4.59) (12.46) (3.98) (5.19)
1.62 (2.43) 1.81 (2.57) 2.35 (3.06) – – – –
0.03
–
–
–
0.66n 0.27n 0.34n 0.32n
SOPHS: Social Phobia Screener; SPIN: Social Phobia Inventory; GAD: Generalised Anxiety Disorder; PHQ: Patient Health Questionnaire; MINI: Mini International Neuropsychiatric Interview. n po 0.05 for difference between categories based on one-way ANOVA or Pearson correlation coefficient (r).
75
lation-based research and clinical trials, a cut-off of 4 on the PADIS is recommended, while in clinical settings, a cut-off of 3 may be preferable to ensure that potential cases are not missed. The PHQpanic scale had poor sensitivity for detecting panic disorder using the prescribed criteria. In addition, the PADIS provides a metric for assessing severity of panic symptoms in addition to caseness, unlike the PHQ-panic. Using symptom count from PHQ-panic led to significantly less precision to the PADIS in identifying cases, based on area under the curve. However, the use of symptom count is a non-standard method for scoring the PHQ-panic scale, and administration of the PHQ-panic scale does not typically involve collection of data on individual symptoms (Spitzer et al., 1999). This finding suggests that researchers or clinicians who use the PHQ-panic scale might be advised to collect data on specific symptoms in order to approach the precision of the PADIS. The assessment of functional impairment within the PADIS is an additional advantage of the screener. The PADIS demonstrated convergent validity through strong correlations with PHQ-panic symptoms and weaker although moderate correlations with symptom measures for diverging anxiety disorders. As reflected in previous epidemiological research (Andrade et al., 1994; Kessler et al., 2006; Von Korff et al., 1985), severity of panic symptoms was higher in participants who were separated, divorced or widowed, those with less education, and those who were unemployed. Participants meeting MINI criteria for social anxiety, depression and generalised anxiety also had higher PADIS scores than those not meeting criteria, indicative of comorbidity between the common mental disorders (Kessler et al., 2005b; Teesson et al., 2009). The lack of gender differences in both PADIS scores and MINI panic disorder caseness are in contrast to previous population-based research (Kessler et al., 2006), and may reflect selection bias, although gender differences for panic disorder tend to be less pronounced than for many other anxiety disorders (Slade et al., 2009). The lack of age effects is likely related to the restricted age range of the sample. Although the present study found strong support for the use of the PADIS to screen panic disorder in the community, further validation of the PADIS in clinical populations may provide additional evidence for its use. Such validation research should also extend to use of DSM-5 criteria as a comparator, although DSM-IV criteria for panic disorder are virtually identical to DSM-5 criteria. Validation with older adults and in primary care settings may also be beneficial, as the current sample comprised young adults with slightly elevated symptoms of generalised anxiety. The restriction of the study to those with elevated anxiety symptoms may have biased detection of non-cases, potentially resulting in conservative estimates of specificity across both the PADIS and PHQ-9. As the median age of onset for panic disorder is 24 (Kessler et al., 2005a) and may be bimodally distributed with variability in outcomes (Katerndahl and Talamantes, 2000), further investigation across the life span may be warranted. In addition, testing in clinical populations over time is required to evaluate whether the PADIS is sensitive to change. Data from the present study indicate that the PADIS provides a brief, valid and reliable way to screen for panic disorder in
Table 3 Screening performance indices for the PADIS at various cut points, compared to the PHQ-panic, in identifying MINI clinical caseness for current panic disorder.
PADIS Z 3 PADIS Z 4 PADIS Z 5 PHQ-panic
Sensitivity (95% CI)
Specificity (95% CI)
PPV (95% CI)
NPV (95% CI)
0.849 (0.767, 0.911) 0.774 (0.682, 0.849) 0.670 (0.572, 0.758) 0.570 (0.417, 0.665)
0.758 0.840 0.894 0.906
0.193 (0.158, 0.232) 0.248 (0.202, 0.298) 0.301 (0.243, 0.364) 0.292 (0.231, 0.359)
0.987 (0.978, 0.992) 0.982 (0.973, 0.988) 0.976 (0.966, 0.983) 0.969 (0.959, 0.977)
(0.736, 0.779) (0.821, 0.858) (0.878, 0.909) (0.891, 0.920)
PADIS: Panic Disorder Screener; PHQ: Patient Health Questionnaire; PPV: positive predictive value; NPV: negative predictive value.
76
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
population-based settings. The PADIS is freely available for screening panic disorder in a range of clinical and research settings, potentially including primary care, hospital settings, internet programs, clinical research and population research.
Acknowledgements This study was funded by National Health and Medical Research Council (NHMRC), Australia Project Grant 525419. PB is supported NHMRC Early Career Fellowship 1035262. HC is supported by NHMRC John Cade Fellowship 1056964. The authors declare no conflict of interest. References Andersson, G., Carlbring, P., 2011. Panic Disorder, Stepped Care and e-Health. Springer, New York, pp. 61–76. Andrade, L., Eaton, W.W., Chilcoat, H., 1994. Lifetime comorbidity of panic attacks and major depression in a population-based study. Symptom profiles. British Journal of Psychiatry 165, 363–369. Bunaciu, L., Feldner, M.T., Blumenthal, H., Babson, K.A., Schmidt, N.B., SachsEricsson, N., 2010. A test of the relations between avoidance of panic-related situations and substance use, substance dependence, and depression among subclinical panickers. Behaviour Research and Therapy 48, 158–163. Burgess, P.M., Pirkis, J.E., Slade, T.N., Johnston, A.K., Meadows, G.N., Gunn, J.M., 2009. Service use for mental health problems: findings from the 2007 National Survey of Mental Health and Wellbeing. Australian and New Zealand Journal of Psychiatry 43, 615–623. Christensen, H., Batterham, P., Mackinnon, A., Griffiths, K., Kalia Hehir, K., Kenardy, J., Gosling, J., Bennett, K., 2014. Prevention of Generalized Anxiety Disorder Using a Web Intervention, iChill: randomized Controlled Trial. Journal of Medical Internet Research 16, e199. Christensen, H., Batterham, P.J., Grant, J.B., Griffiths, K.M., Mackinnon, A.J., 2011. A population study comparing screening performance of prototypes for depression and anxiety with standard scales. BMC Medical Research Methodolology 11, 154. Christensen, H., Griffiths, K.M., Mackinnon, A.J., Kalia, K., Batterham, P.J., Kenardy, J., Eagleson, C., Bennett, K., 2010. Protocol for a randomised controlled trial investigating the effectiveness of an online e health application for the prevention of Generalised Anxiety Disorder. BMC Psychiatry 10, 25. Connor, K.M., Davidson, J.R., Churchill, L.E., Sherwood, A., Foa, E., Weisler, R.H., 2000. Psychometric properties of the Social Phobia Inventory (SPIN). New selfrating scale. British Journal of Psychiatry 176, 379–386. Eaton, W.W., Martins, S.S., Nestadt, G., Bienvenu, O.J., Clarke, D., Alexandre, P., 2008. The burden of mental disorders. Epidemiologic Reviews 30, 1–14. Gadermann, A.M., Alonso, J., Vilagut, G., Zaslavsky, A.M., Kessler, R.C., 2012. Comorbidity and disease burden in the National Comorbidity Survey Replication (NCS-R). Depression and Anxiety 29, 797–806. Houck, P.R., Spiegel, D.A., Shear, M.K., Rucci, P., 2002. Reliability of the self-report version of the panic disorder severity scale. Depression and Anxiety 15, 183–185. Katerndahl, D.A., Talamantes, M., 2000. A comparison of persons with early-versus late-onset panic attacks. Journal of Clinical Psychiatry 61, 422–427. Kessler, R.C., Andrews, G., Colpe, L.J., Hiripi, E., Mroczek, D.K., Normand, S.L., Walters, E.E., Zaslavsky, A.M., 2002. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine 32, 959–976.
Kessler, R.C., Berglund, P., Demler, O., Jin, R., Merikangas, K.R., Walters, E.E., 2005a. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62, 593–602. Kessler, R.C., Chiu, W.T., Demler, O., Merikangas, K.R., Walters, E.E., 2005b. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62, 617–627. Kessler, R.C., Chiu, W.T., Jin, R., Ruscio, A.M., Shear, K., Walters, E.E., 2006. The epidemiology of panic attacks, panic disorder, and agoraphobia in the National Comorbidity Survey Replication. Archives of General Psychiatry 63, 415–424. Korte, K.J., Brown, M.A., Schmidt, N.B., 2013. Anxiety sensitivity and impairment: evidence for a direct association and partial mediation by subclinical anxiety symptoms. Journal of Affective Disorders 151, 875–881. Kroenke, K., Spitzer, R.L., Williams, J.B., Lowe, B., 2010. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. General Hospital Psychiatry 32, 345–359. Lowe, B., Grafe, K., Zipfel, S., Spitzer, R.L., Herrmann-Lingen, C., Witte, S., Herzog, W., 2003. Detecting panic disorder in medical and psychosomatic outpatients: comparative validation of the Hospital Anxiety and Depression Scale, the Patient Health Questionnaire, a screening question, and physicians' diagnosis. Journal of Psychosomatic Research 55, 515–519. Mackinnon, A., 2000. A spreadsheet for the calculation of comprehensive statistics for the assessment of diagnostic tests and inter-rater agreement. Computers in Biology and Medicine 30, 127–134. Narrow, W.E., Clarke, D.E., Kuramoto, S.J., Kraemer, H.C., Kupfer, D.J., Greiner, L., Regier, D.A., 2013. DSM-5 field trials in the United States and Canada, Part III: development and reliability testing of a cross-cutting symptom assessment for DSM-5. American Journal of Psychiatry 170, 71–82. Shear, M.K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., Houck, P., Wang, T., 2001. Reliability and validity of the Panic Disorder Severity Scale: replication and extension. Journal of Psychiatric Research 35, 293–296. Sheehan, D.V., Lecrubier, Y., Sheehan, K.H., Amorim, P., Janavs, J., Weiller, E., Hergueta, T., Baker, R., Dunbar, G.C., 1998. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry 59 (Suppl 20), S22–S33, quiz 34-57. Slade, T., Johnston, A., Oakley Browne, M.A., Andrews, G., Whiteford, H., 2009. 2007 National Survey of Mental Health and Wellbeing: methods and key findings. Australian & New Zealand Journal of Psychiatry 43, 594–605. Spitzer, R.L., Kroenke, K., Williams, J.B., 1999. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA 282, 1737–1744. Spitzer, R.L., Kroenke, K., Williams, J.B., Lowe, B., 2006. A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of Internal Medicine 166, 1092–1097. Teesson, M., Slade, T., Mills, K., 2009. Comorbidity in Australia: findings of the 2007 National Survey of Mental Health and Wellbeing. Australian & New Zealand Journal of Psychiatry 43, 606–614. van Ballegooijen, W., Riper, H., van Straten, A., Kramer, J., Conijn, B., Cuijpers, P., 2011. The effects of an Internet based self-help course for reducing panic symptoms – Don't Panic Online: study protocol for a randomised controlled trial. Trials 12, 75. Von Korff, M.R., Eaton, W.W., Keyl, P.M., 1985. The epidemiology of panic attacks and panic disorder. Results of three community surveys. American Journal of Epidemiology 122, 970–981. Williams Jr., J.W., Pignone, M., Ramirez, G., Perez Stellato, C., 2002. Identifying depression in primary care: a literature synthesis of case-finding instruments. General Hospital Psychiatry 24, 225–237. Wittkampf, K.A., Baas, K.D., van Weert, H.C., Lucassen, P., Schene, A.H., 2011. The psychometric properties of the panic disorder module of the Patient Health Questionnaire (PHQ-PD) in high-risk groups in primary care. Journal of Affective Disorders 130, 260–267.
Contents lists available at ScienceDirect
Psychiatry Research journal homepage: www.elsevier.com/locate/psychres
The panic disorder screener (PADIS): Development of an accurate and brief population screening tool Philip J. Batterham a,n, Andrew J. Mackinnon b, Helen Christensen c a National Institute for Mental Health Research, Research School of Population Health, The Australian National University, 63 Eggleston Road, Acton, ACT 2601, Australia b Orygen, The National Centre of Excellence in Youth Mental Health, The University of Melbourne, Melbourne, Australia c Black Dog Institute, The University of New South Wales, Sydney, Australia
art ic l e i nf o
a b s t r a c t
Article history: Received 7 November 2014 Received in revised form 10 April 2015 Accepted 15 April 2015 Available online 23 April 2015
The Panic Disorder Screener (PADIS) was developed as a new screener to identify panic disorder in the community and to assess severity of symptoms. The PADIS was developed to fill a gap in existing screening measures, as there are no brief panic screeners available that assess severity. The current study aimed to test the performance of the screener relative to the Patient Health Questionnaire-panic scale (PHQ-panic). The 4-item PADIS was administered to 12,336 young Australian adults, together with the PHQ-panic. A subsample of 1674 participants also completed a phone-based clinical interview to determine whether they met DSM-IV criteria for panic disorder. The PADIS (77% sensitivity, 84% specificity) had higher sensitivity for identifying panic disorder based on clinical criteria than the PHQ-panic (57% sensitivity, 91% specificity), although with reduced specificity. Administration of the PADIS required a mean of 1.9 items, compared to 4.7 items for the PHQ-panic. Each one-point increase in PADIS score was associated with 69% increased odds of meeting clinical criteria for panic disorder. The PADIS was found to be a valid, reliable and brief panic screener that is freely available for use in research and clinical settings. & 2015 Elsevier Ireland Ltd. All rights reserved.
Keywords: Panic disorder Panic symptoms Anxiety Sensitivity Specificity Measurement
1. Introduction It has been estimated that up to 4.7% of the population experiences panic disorder during their lifetime (Andersson and Carlbring, 2011; Kessler et al., 2005a). Panic disorder accounts for a considerable proportion of disease burden in the general population (Eaton et al., 2008; Gadermann et al., 2012). However, individuals experiencing anxiety disorders rarely seek help (Burgess et al., 2009; Kessler et al., 2006) and panic disorder is frequently unrecognised in primary care (Lowe et al., 2003; Spitzer et al., 1999). In addition, subclinical panic symptoms have a high burden and carry considerable risk for development of agoraphobia and other mental health problems (Andersson and Carlbring, 2011; Bunaciu et al., 2010; Korte et al., 2013). There are several self-report scales available to assess symptoms of panic disorder. Screening scales are particularly useful in primary care, for online programs and for population-based research, where scale and resource constraints demand brevity (Kessler et al., 2002; Williams et al., 2002). However, most panic scales have more than 10 items, making them unsuitable for use as brief screeners in research and clinical settings.
n
Corresponding author. Tel.: þ 61 2 61251031; fax: þ 61 2 61250733. E-mail address: [email protected] (P.J. Batterham).
http://dx.doi.org/10.1016/j.psychres.2015.04.016 0165-1781/& 2015 Elsevier Ireland Ltd. All rights reserved.
Two exceptions are the five-item Patient Health Questionnaire – panic scale (PHQ-panic; Spitzer et al., 1999) and the seven-item selfreport version of the Panic Disorder Severity Scale (PDSS-SR; Houck et al., 2002; Shear et al., 2001). The PHQ-panic scale includes five items that approximately equate to the core symptoms of panic disorder as defined in DSM-IV, although the final item is sometimes presented as a checklist of 11 symptoms. The PHQ-panic scale has been reported to have excellent screening performance by its authors (Lowe et al., 2003), although another group has reported more moderate performance in a primary care sample (Wittkampf et al., 2011). With its use of binary responses, the PHQ-panic scale does not provide a severity rating (Kroenke et al., 2010). The lack of a severity rating limits the use of the scale in identifying individuals experiencing subclinical symptoms of panic disorder, which is responsible for considerable burden above and beyond that of panic disorder. In contrast, the PDSS-SR provides a severity rating, with scores ranging 0–28. However, the PDSS-SR is considerably lengthier than the PHQ-panic (7 items/1108 words in the PDSS-SR vs. 5 items/195 words in PHQ-panic) and has greater literacy demands (Flesch-Kincaid reading level 11.1 vs. 2.4). Its length makes it unsuitable as a screener in many population-based settings. Although the PDSS-SR was designed to assess severity of panic disorder in clinical cases, it has been widely adopted for use as a screening tool (e.g., van Ballegooijen et al., 2011). Consequently, there is only indirect evidence for the accuracy of the PDSS-SR relative to clinical criteria,
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
with its counterpart interviewer-administered version demonstrating 83% sensitivity and 64% specificity relative to clinical criteria (Shear et al., 2001) [the correlation between the PDSS-SR and the interviewerrated version is 0.81 (Houck et al., 2002)]. Consequently, there is a need to develop new screening measures for panic disorder that are validated, are brief and provide a severity rating. The present study reports on the development and screening performance of the Panic Disorder Screener (PADIS), a four-item scale designed to screen for panic disorder in the community setting. The scale uses a vignette to describe the typical symptoms of a panic attack, adapting an approach used previously for screening (Christensen et al., 2011). This introductory vignette is followed by questions about number of panic attacks, worry about panic attacks, worry about consequences of panic attacks and avoidance due to panic attacks. The questions were designed to correspond approximately with each of the DSM-IV criteria for panic disorder within a self-report framework. This process was guided by similar principles to the development of the DSM-5 cross-cutting assessment, although with a sole focus on panic disorder. Specifically, it was designed to be suitable for self-report use by most adults in most settings, providing information for subsequent clinical assessment and appropriate treatment (Narrow et al., 2013). Responses on the PADIS are provided on a frequency scale, to enable assessment of the severity of panic disorder symptoms. The PADIS was designed to be brief (four items, 228 words, reading grade 7.5) to ensure utility in a range of research and clinical settings. This study aimed to assess the screening performance of the PADIS relative to the PHQ-panic scale in identifying panic disorder within a population-based sample. We hypothesised that the PADIS severity score would provide significantly greater sensitivity in screening for panic disorder in a population sample than the PHQ-panic scale, with adequate specificity. 2. Method 2.1. Participants and procedure Data for this study were taken from the screening phase of the iChill study, an intervention trial testing an online cognitive behavioural therapy intervention to prevent new cases of Generalised Anxiety Disorder (Christensen et al., 2014). The trial, conducted between May 2010 and June 2012, is described by Christensen et al. (2010) and registered in Current Controlled Trials (controlled-trials.com, ISRCTN76298775). Invitations and screening surveys were mailed to 120,000 18–30 year olds selected randomly from the Australian electoral roll, with 12,430 (10.4%) responding by returning the paper-based screening survey. Registration on the electoral roll is compulsory in Australia. Data for both the PADIS and the PHQ-panic were available for 12,336 (99.2%) of respondents. Individuals who returned a survey were assessed for eligibility for the intervention trial, with eligible participants being contacted by telephone to complete a Mini International Neuropsychiatric Interview (MINI, v5.0) (Sheehan et al., 1998) within 21 days of survey return. Inclusion and exclusion criteria are shown in Fig. 1. The exclusion criteria were based on the requirements of the trial, which targeted young adults (18–30) who were not currently in treatment and who had GAD symptoms above the median score for this population. Access to the internet was also a requirement. Postgraduate clinical psychology students administered the telephone MINI interviews, and were blind to the participant responses to the survey. Interviewers were given four hours training, including practice interviews, with oversight from a clinical psychologist and the research team. The MINI interviews included the modules on panic (lifetime, current), social anxiety disorder (current), depression (current) and generalised anxiety disorder (lifetime, current). Not all participants could be contacted within the allotted time period, despite up to seven call attempts being made, resulting in a final sample of 1674 participants with both survey data and clinical assessment data available (66.5% of eligible and consenting; 99.2% of clinically assessed participants). No incentives were provided to participants. The study had approval from the ANU Human Research Ethics Committee (Protocol 2008/548). 2.2. Measures The PADIS is four-item screener for panic disorder. The introduction to the scale reads:
73
“A sudden feeling of anxiety, fear, discomfort or uneasiness may indicate a panic episode. A panic episode occurs unexpectedly, peaks within 10 min and includes four or more of the following sensations at the same time: Skipping, racing or pounding heart; Sweating or clammy hands; Trembling or shaking; Shortness of breath or difficulty breathing; A choking feeling or lump in your throat; Chest pain, pressure or discomfort; Nausea or stomach problems; Feeling dizzy, unsteady, lightheaded or faint; Feeling strange, unreal, detached or unfamiliar; Fear that you are losing control or going crazy; Fear that you are dying; Feeling tingling or numbness; Hot flushes or chills”.
The four items are: (1) “Based on this description, about how many panic episodes have you experienced in the past month?” (None/One/Between 2 and 5/Between 6 and 10/11 or more). (2) “In the past month, how often have you been worried about having another panic episode?” (Never/Occasionally/Often/All the time). (3) “In the past month, how often have you been worried about the consequences of having another panic episode?” (Never/Occasionally/Often/All the time). (4) “In the past month, how often did you avoid places, situations or activities because you were afraid of having or triggering a panic episode? (For example, not going to public places alone, not leaving your house, avoiding physical exertion or avoiding stressful activities.)” (Never/Occasionally/Often/All the time). The first item is rated on a 5-point Likert scale scored 0–4, while remaining items are rated on a 4-point Likert scale score 0–3. Severity scores are obtained by summing responses to all items, with scores ranging from 0 to 13. Respondents who respond “None” to the first item do not complete subsequent items as these items are irrelevant, and are consequently scored zero. The PHQ-panic has 5 items, each with responses of “yes” or “no”. Individuals who respond “yes” to every item are deemed to meet criteria for panic disorder (Spitzer et al., 1999). Individuals who respond “no” to an item do not receive subsequent items. For the fifth item, a list of panic symptoms, individuals who endorse fewer than four symptoms are counted as not meeting criteria.
2.3. Analysis The internal consistency of the PADIS was assessed using Cronbach's alpha and the dimensionality of the PADIS items were assessed using an exploratory factor analysis with principal axis factoring. The accuracy of the PADIS score in assessing MINI criteria for current panic disorder was then examined, with comparison to the accuracy of the PHQ-Panic. Analyses examined accuracy (based on sensitivity, specificity, positive predictive value and negative predictive value) of the PADIS and PHQ-Panic based on prescribed cut-offs, with confidence intervals calculated using the software and methods described by Mackinnon (2000). In addition, the PADIS and PHQ-panic were compared based on the area under the Receiver Operating Characteristic curve (AUC) for each scale. As PHQ-panic has only five binary items, AUC was assessed on the basis of number of items endorsed (out of five) but also total symptom count (out of 11). Convergent validity was also assessed by comparing PADIS severity scores with panic item counts from the PHQ-panic, social phobia scores on the SPIN (Connor et al., 2000), generalised anxiety scores on the GAD-7 (Spitzer et al., 2006) and depression scores on the Patient Health Questionnaire-9 (Spitzer et al., 1999). Analyses were conducted in SPSS v20 (IBM Corp, Chicago Ill), except for comparisons of AUC, which were conducted in SAS v9.4 (SAS Institute Inc, Cary NC).
3. Results 3.1. Response and sample characteristics In the full screener sample (n ¼12,336), the PADIS had high internal consistency (Cronbach α ¼0.86). Based on the first item, 8740 (70.8%) participants reported no panic episodes in the past month, with 1524 (12.3%) reporting one episode, 1634 (13.2) reporting 2–5 episodes, 325 (2.6%) reporting 6–10 episodes and 111 (0.9%) reporting 11 or more episodes. Endorsement rates for remaining items are shown in Table 1. In items 2–4, endorsement of each response category was similar, with 12–15% of participants endorsing each item at any level. Participants who reported no panic episodes in the first item were not administered the remaining items and scored as zero. This skip pattern meant 8740 participants were administered a single item, while the
74
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
120,000 surveys sent • 579 (4.7%) not in appropriate
107,570 surveys not returned
age range (18-30)
12,430 (10.4%) surveys returned
• 627 (5.0%) currently undergoing cognitive behavioural therapy
• 1,229 (9.9%) currently seeing a psychiatrist or psychologist
• 1,414 (33.6%) no consent to follow up • 274 (6.5%) insufficient contact information
4,205 (33.8%) eligible participants
2,517 (59.9%) consenting participants
1,687 (67.0%) completed MINI
• 139 (1.1%) no internet access at home or work
• 6,884 (55.4%) few symptoms of generalized anxiety (GAD -7 score <5)
830 (33.0%) unreachable by phone for MINI interview within 21 days
Fig. 1. CONSORT-based flow diagram of participant inclusion.
Table 1 Rates of endorsement for PADIS items in the general population sample (n¼ 12,361). Occasionally Worry about another panic episode Worry about consequences of another panic episode Avoided places/situations/activities
1399 1208 996
Often (11.4%) (9.8%) (8.1%)
346 355 344
All the time (2.8%) (2.9%) (2.8%)
113 114 106
(0.9%) (0.9%) (0.9%)
Responses of “Not at all” are excluded from the table; participants who reported no panic episodes were classified in the “not at all” category.
remainder answered all four items, resulting in a mean of 1.9 items administered across all participants. By comparison, participants were administered a mean of 4.7 items on the PHQ-panic (or 2.3 items if the symptom checklist is counted as a single item). Factor analysis revealed a single factor with an eigenvalue of 3.04 accounting for 78.1% of the variance in the items. Subsequent factors had eigenvalues of 0.43 and 0.39, indicating that the scale was strongly unidimensional. All items loaded on the single factor at 0.73 or higher (loadings for the four items were 0.75, 0.92, 0.90 and 0.73 respectively). Sample characteristics for the participants who completed the clinical assessment are presented in Table 2. Severity scores on the PADIS were significantly higher among participants who met criteria for panic disorder on the PHQ-panic (F1, 1660 ¼789.3, p o0.001) or MINI (F1, 1660 ¼430.4, p o0.001), separated/ widowed/divorced participants (F3, 1650 ¼4.6, p ¼0.003), participants with lower levels of education (F3, 1657 ¼7.9, p o0.001), and participants who were unemployed (F3, 1649 ¼ 4.2, p ¼0.006). There were no significant differences on the basis of age, gender or language. PADIS scores were strongly correlated with panic symptom counts from the PHQ-panic, indicating convergent validity, but only moderately correlated with social phobia, generalised anxiety and depression scores, indicative of divergent validity while also reflecting high comorbidity between these disorders. Likewise, the magnitude of the difference in PADIS scores was greater among those who met MINI caseness for panic disorder than for those who met criteria for generalised anxiety, social phobia or depression. 3.2. Comparison to clinical assessment Based on the MINI assessment, 107 (6.4%) of the 1674 participants who were assessed met clinical criteria for current panic disorder, including 27 males (6.7%) and 80 females (6.3%). A cut point of the PADIS was determined based on Youden indices, with
scores of 4 or higher on the PADIS found to be most accurate for distinguishing MINI panic cases from non-cases. The area under the curve was 0.87 for the PADIS, significantly greater than that for PHQ-panic symptom count [based on 11 symptoms, AUC ¼0.83, χ2 (1)¼5.63, p ¼0.02] and for PHQ-panic item endorsement [based on the 5 items, AUC¼0.74, χ2 (1)¼33.81, p o0.001]. Based on univariate logistic regression, each one-point increase in PADIS score was associated with 68.6% increase in the odds of meeting clinical criteria for panic disorder on the MINI. The sensitivity, specificity, positive predictive value and negative predictive value of the PADIS and PHQ-panic, relative to MINI clinical criteria for current panic disorder, are shown in Table 3. The chosen cut point of 4 is compared to alternative cut points in the Table. McNemar's test demonstrated a significant differences between the PADIS and the PHQ-panic, with the PADIS having significantly higher sensitivity [χ2 (1)¼ 16.3, p o0.001] and lower specificity [χ2 (1) ¼47.7, po 0.001] compared to the PHQ-panic at prescribed cut-points. Using a cut-off of 3 or higher on the PADIS also showed accurate classification, while a cut-off of 5 or higher was associated with commensurate specificity with the PHQ-panic [χ2 (1)¼ 1.8, p¼ 0.12] and significantly higher sensitivity [χ2 (1)¼ 3.3, p ¼0.04]. As sensitivity o0.7 on either measure was deemed to be suboptimal, the cut point of 4 was retained.
4. Discussion This study found that the PADIS performs with greater sensitivity in detecting cases of current panic disorder than the PHQpanic, with comparable response burden. For screening purposes, a high probability of detecting true cases (high sensitivity) and a high probability that negative test results indicate non-caseness (negative predictive value) are the imperative. The PADIS outperformed the PHQ-panic scale on both of these indices, using considerably fewer items, while retaining adequate specificity for
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
screening purposes. Indeed using a cut-off with similarly high specificity to the PHQ-panic scale (PADIS Z5), sensitivity remained significantly higher for the PADIS than the PHQ-panic. For popuTable 2 Sample characteristics for participants who received a clinical assessment (n¼ 1674). n (%) or Mean (S.D.) PADIS score PADIS criteria ( Z4) Yes No PHQ-panic criteria Yes No MINI current panic criteria Yes No MINI current social phobia Yes No MINI current GAD Yes No MINI current depression Yes No Age Gender Female Male Marital status Married De facto Separated/divorced/ widowed Single, never married Education Did not complete high school Completed high school Bachelor's degree Higher degree Employment status Employed full-time Employed PT/casual Unemployed, looking for work Not in the labour force Language spoken at home English English and other Other PHQ-panic symptom count SPIN score GAD-7 score PHQ-9 score
Mean (S.D.) PADIS score
r
–
–
331 1330
(19.9%) – (80.1%) –
– –
209 1479
(12.4%) 5.36 (2.86)n (87.6%) 1.13 (1.89)
–
107 1581
(6.4%) 5.95 (3.34)n (93.7%) 1.37 (2.10)
–
161 1501
(9.7%) 2.94 (3.03)n (90.3%) 1.53 (2.36)
–
267 1395
(16.1%) 3.09 (3.17)n (83.9%) 1.39 (2.21)
–
210 (12.6%) 3.02 (3.18)n 1451 (87.4%) 1.47 (2.28) 25.35 (3.38) –
–
1270 404
(75.9%) 1.70 (2.47) (24.1%) 1.53 (2.44)
–
238 375 17
(14.3%) 1.37 (2.49)n (22.5%) 1.69 (2.48) (1.0%) 3.59 (3.18)
–
1036
(62.2%) 1.69 (2.44)
1.66 (2.47)
92
(5.5%)
2.62 (2.95)n
532 817 232
(31.8%) 1.85 (2.59) (48.8%) 1.51 (2.40) (13.9%) 1.36 (2.06)
976 473 118
(58.6%) 1.56 (2.36)n (28.3%) 1.71 (2.58) (7.1%) 2.40 (2.89)
98
(5.9%)
1.58 (2.24)
1383 259 32 3.56 19.36 9.22 8.56
(82.6%) (15.5%) (1.9%) (4.59) (12.46) (3.98) (5.19)
1.62 (2.43) 1.81 (2.57) 2.35 (3.06) – – – –
0.03
–
–
–
0.66n 0.27n 0.34n 0.32n
SOPHS: Social Phobia Screener; SPIN: Social Phobia Inventory; GAD: Generalised Anxiety Disorder; PHQ: Patient Health Questionnaire; MINI: Mini International Neuropsychiatric Interview. n po 0.05 for difference between categories based on one-way ANOVA or Pearson correlation coefficient (r).
75
lation-based research and clinical trials, a cut-off of 4 on the PADIS is recommended, while in clinical settings, a cut-off of 3 may be preferable to ensure that potential cases are not missed. The PHQpanic scale had poor sensitivity for detecting panic disorder using the prescribed criteria. In addition, the PADIS provides a metric for assessing severity of panic symptoms in addition to caseness, unlike the PHQ-panic. Using symptom count from PHQ-panic led to significantly less precision to the PADIS in identifying cases, based on area under the curve. However, the use of symptom count is a non-standard method for scoring the PHQ-panic scale, and administration of the PHQ-panic scale does not typically involve collection of data on individual symptoms (Spitzer et al., 1999). This finding suggests that researchers or clinicians who use the PHQ-panic scale might be advised to collect data on specific symptoms in order to approach the precision of the PADIS. The assessment of functional impairment within the PADIS is an additional advantage of the screener. The PADIS demonstrated convergent validity through strong correlations with PHQ-panic symptoms and weaker although moderate correlations with symptom measures for diverging anxiety disorders. As reflected in previous epidemiological research (Andrade et al., 1994; Kessler et al., 2006; Von Korff et al., 1985), severity of panic symptoms was higher in participants who were separated, divorced or widowed, those with less education, and those who were unemployed. Participants meeting MINI criteria for social anxiety, depression and generalised anxiety also had higher PADIS scores than those not meeting criteria, indicative of comorbidity between the common mental disorders (Kessler et al., 2005b; Teesson et al., 2009). The lack of gender differences in both PADIS scores and MINI panic disorder caseness are in contrast to previous population-based research (Kessler et al., 2006), and may reflect selection bias, although gender differences for panic disorder tend to be less pronounced than for many other anxiety disorders (Slade et al., 2009). The lack of age effects is likely related to the restricted age range of the sample. Although the present study found strong support for the use of the PADIS to screen panic disorder in the community, further validation of the PADIS in clinical populations may provide additional evidence for its use. Such validation research should also extend to use of DSM-5 criteria as a comparator, although DSM-IV criteria for panic disorder are virtually identical to DSM-5 criteria. Validation with older adults and in primary care settings may also be beneficial, as the current sample comprised young adults with slightly elevated symptoms of generalised anxiety. The restriction of the study to those with elevated anxiety symptoms may have biased detection of non-cases, potentially resulting in conservative estimates of specificity across both the PADIS and PHQ-9. As the median age of onset for panic disorder is 24 (Kessler et al., 2005a) and may be bimodally distributed with variability in outcomes (Katerndahl and Talamantes, 2000), further investigation across the life span may be warranted. In addition, testing in clinical populations over time is required to evaluate whether the PADIS is sensitive to change. Data from the present study indicate that the PADIS provides a brief, valid and reliable way to screen for panic disorder in
Table 3 Screening performance indices for the PADIS at various cut points, compared to the PHQ-panic, in identifying MINI clinical caseness for current panic disorder.
PADIS Z 3 PADIS Z 4 PADIS Z 5 PHQ-panic
Sensitivity (95% CI)
Specificity (95% CI)
PPV (95% CI)
NPV (95% CI)
0.849 (0.767, 0.911) 0.774 (0.682, 0.849) 0.670 (0.572, 0.758) 0.570 (0.417, 0.665)
0.758 0.840 0.894 0.906
0.193 (0.158, 0.232) 0.248 (0.202, 0.298) 0.301 (0.243, 0.364) 0.292 (0.231, 0.359)
0.987 (0.978, 0.992) 0.982 (0.973, 0.988) 0.976 (0.966, 0.983) 0.969 (0.959, 0.977)
(0.736, 0.779) (0.821, 0.858) (0.878, 0.909) (0.891, 0.920)
PADIS: Panic Disorder Screener; PHQ: Patient Health Questionnaire; PPV: positive predictive value; NPV: negative predictive value.
76
P.J. Batterham et al. / Psychiatry Research 228 (2015) 72–76
population-based settings. The PADIS is freely available for screening panic disorder in a range of clinical and research settings, potentially including primary care, hospital settings, internet programs, clinical research and population research.
Acknowledgements This study was funded by National Health and Medical Research Council (NHMRC), Australia Project Grant 525419. PB is supported NHMRC Early Career Fellowship 1035262. HC is supported by NHMRC John Cade Fellowship 1056964. The authors declare no conflict of interest. References Andersson, G., Carlbring, P., 2011. Panic Disorder, Stepped Care and e-Health. Springer, New York, pp. 61–76. Andrade, L., Eaton, W.W., Chilcoat, H., 1994. Lifetime comorbidity of panic attacks and major depression in a population-based study. Symptom profiles. British Journal of Psychiatry 165, 363–369. Bunaciu, L., Feldner, M.T., Blumenthal, H., Babson, K.A., Schmidt, N.B., SachsEricsson, N., 2010. A test of the relations between avoidance of panic-related situations and substance use, substance dependence, and depression among subclinical panickers. Behaviour Research and Therapy 48, 158–163. Burgess, P.M., Pirkis, J.E., Slade, T.N., Johnston, A.K., Meadows, G.N., Gunn, J.M., 2009. Service use for mental health problems: findings from the 2007 National Survey of Mental Health and Wellbeing. Australian and New Zealand Journal of Psychiatry 43, 615–623. Christensen, H., Batterham, P., Mackinnon, A., Griffiths, K., Kalia Hehir, K., Kenardy, J., Gosling, J., Bennett, K., 2014. Prevention of Generalized Anxiety Disorder Using a Web Intervention, iChill: randomized Controlled Trial. Journal of Medical Internet Research 16, e199. Christensen, H., Batterham, P.J., Grant, J.B., Griffiths, K.M., Mackinnon, A.J., 2011. A population study comparing screening performance of prototypes for depression and anxiety with standard scales. BMC Medical Research Methodolology 11, 154. Christensen, H., Griffiths, K.M., Mackinnon, A.J., Kalia, K., Batterham, P.J., Kenardy, J., Eagleson, C., Bennett, K., 2010. Protocol for a randomised controlled trial investigating the effectiveness of an online e health application for the prevention of Generalised Anxiety Disorder. BMC Psychiatry 10, 25. Connor, K.M., Davidson, J.R., Churchill, L.E., Sherwood, A., Foa, E., Weisler, R.H., 2000. Psychometric properties of the Social Phobia Inventory (SPIN). New selfrating scale. British Journal of Psychiatry 176, 379–386. Eaton, W.W., Martins, S.S., Nestadt, G., Bienvenu, O.J., Clarke, D., Alexandre, P., 2008. The burden of mental disorders. Epidemiologic Reviews 30, 1–14. Gadermann, A.M., Alonso, J., Vilagut, G., Zaslavsky, A.M., Kessler, R.C., 2012. Comorbidity and disease burden in the National Comorbidity Survey Replication (NCS-R). Depression and Anxiety 29, 797–806. Houck, P.R., Spiegel, D.A., Shear, M.K., Rucci, P., 2002. Reliability of the self-report version of the panic disorder severity scale. Depression and Anxiety 15, 183–185. Katerndahl, D.A., Talamantes, M., 2000. A comparison of persons with early-versus late-onset panic attacks. Journal of Clinical Psychiatry 61, 422–427. Kessler, R.C., Andrews, G., Colpe, L.J., Hiripi, E., Mroczek, D.K., Normand, S.L., Walters, E.E., Zaslavsky, A.M., 2002. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine 32, 959–976.
Kessler, R.C., Berglund, P., Demler, O., Jin, R., Merikangas, K.R., Walters, E.E., 2005a. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62, 593–602. Kessler, R.C., Chiu, W.T., Demler, O., Merikangas, K.R., Walters, E.E., 2005b. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62, 617–627. Kessler, R.C., Chiu, W.T., Jin, R., Ruscio, A.M., Shear, K., Walters, E.E., 2006. The epidemiology of panic attacks, panic disorder, and agoraphobia in the National Comorbidity Survey Replication. Archives of General Psychiatry 63, 415–424. Korte, K.J., Brown, M.A., Schmidt, N.B., 2013. Anxiety sensitivity and impairment: evidence for a direct association and partial mediation by subclinical anxiety symptoms. Journal of Affective Disorders 151, 875–881. Kroenke, K., Spitzer, R.L., Williams, J.B., Lowe, B., 2010. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. General Hospital Psychiatry 32, 345–359. Lowe, B., Grafe, K., Zipfel, S., Spitzer, R.L., Herrmann-Lingen, C., Witte, S., Herzog, W., 2003. Detecting panic disorder in medical and psychosomatic outpatients: comparative validation of the Hospital Anxiety and Depression Scale, the Patient Health Questionnaire, a screening question, and physicians' diagnosis. Journal of Psychosomatic Research 55, 515–519. Mackinnon, A., 2000. A spreadsheet for the calculation of comprehensive statistics for the assessment of diagnostic tests and inter-rater agreement. Computers in Biology and Medicine 30, 127–134. Narrow, W.E., Clarke, D.E., Kuramoto, S.J., Kraemer, H.C., Kupfer, D.J., Greiner, L., Regier, D.A., 2013. DSM-5 field trials in the United States and Canada, Part III: development and reliability testing of a cross-cutting symptom assessment for DSM-5. American Journal of Psychiatry 170, 71–82. Shear, M.K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., Houck, P., Wang, T., 2001. Reliability and validity of the Panic Disorder Severity Scale: replication and extension. Journal of Psychiatric Research 35, 293–296. Sheehan, D.V., Lecrubier, Y., Sheehan, K.H., Amorim, P., Janavs, J., Weiller, E., Hergueta, T., Baker, R., Dunbar, G.C., 1998. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry 59 (Suppl 20), S22–S33, quiz 34-57. Slade, T., Johnston, A., Oakley Browne, M.A., Andrews, G., Whiteford, H., 2009. 2007 National Survey of Mental Health and Wellbeing: methods and key findings. Australian & New Zealand Journal of Psychiatry 43, 594–605. Spitzer, R.L., Kroenke, K., Williams, J.B., 1999. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA 282, 1737–1744. Spitzer, R.L., Kroenke, K., Williams, J.B., Lowe, B., 2006. A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of Internal Medicine 166, 1092–1097. Teesson, M., Slade, T., Mills, K., 2009. Comorbidity in Australia: findings of the 2007 National Survey of Mental Health and Wellbeing. Australian & New Zealand Journal of Psychiatry 43, 606–614. van Ballegooijen, W., Riper, H., van Straten, A., Kramer, J., Conijn, B., Cuijpers, P., 2011. The effects of an Internet based self-help course for reducing panic symptoms – Don't Panic Online: study protocol for a randomised controlled trial. Trials 12, 75. Von Korff, M.R., Eaton, W.W., Keyl, P.M., 1985. The epidemiology of panic attacks and panic disorder. Results of three community surveys. American Journal of Epidemiology 122, 970–981. Williams Jr., J.W., Pignone, M., Ramirez, G., Perez Stellato, C., 2002. Identifying depression in primary care: a literature synthesis of case-finding instruments. General Hospital Psychiatry 24, 225–237. Wittkampf, K.A., Baas, K.D., van Weert, H.C., Lucassen, P., Schene, A.H., 2011. The psychometric properties of the panic disorder module of the Patient Health Questionnaire (PHQ-PD) in high-risk groups in primary care. Journal of Affective Disorders 130, 260–267.