- Email: [email protected]
Anxiety screening tools in people with epilepsy: A systematic review of validated tools
YEBEH-06392; No of Pages 6 Epilepsy & Behavior 99 (2019) 106392
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Review
Anxiety screening tools in people with epilepsy: A systematic review of validated tools Zilin Wang a,1, Zhiwen Luo a,b,1, Shunhua Li a,1, ，, Zhiwen Luo a,b,⁎,1, Zilin Wang a,⁎⁎,1 a b
Queen Marry School of Nanchang University, Nanchang, Jiangxi 330006, People's Republic of China Affiliated Eye Hospital of Nanchang University, Jiangxi, Research Institute of Ophthalmology and Visual Science, Nanchang, People's Republic of China
a r t i c l e
i n f o
Article history: Received 22 May 2019 Revised 18 June 2019 Accepted 27 June 2019 Available online xxxx Keywords: Epilepsy Anxiety disorder Mental health Comorbidity Screening tools Diagnostic accuracy Measurement Scale
a b s t r a c t Objective: Anxiety is a common neurological condition often comorbid with epilepsy, with approximately 20% of patients with epilepsy exhibiting symptoms of anxiety. Despite this prevalence, accurate and efficacious tools designed to screen for anxiety specifically in patients with epilepsy have not yet been developed. The purpose of this study is to systematically review the literature and better understand this relationship. Methods: Ovid MEDLINE, EMBASE, and PsyclNFO were searched until April 22nd, 2019 without language restrictions. We extracted abstracts, data abstraction, and full-text reviews in duplicate and chose the studies that included measures for anxiety screening in patients with epilepsy. The Quality Assessment of Diagnostic Accuracy Studies Version was used to assess study quality. We used the medians and ranges to calculate the accuracy of the tools. Results: We screened 4758 abstracts and selected 11 articles dealing with anxiety. The most common validated anxiety screening tools were the Generalized Anxiety Disorder (GAD-7) and Hospital Anxiety and Depression Scale-A (HADS-A). The Mini International Neuropsychiatric Interview (MINI) was the most common reference standard used. Significance: Many studies have validated depression screening tools rather than anxiety. This lack of data has left much uncertainty about the relationship of epilepsy to anxiety, as well as diagnostic inconsistencies. The effectiveness of these assessments in practice may be overestimating the prevalence, as the cutpoints are usually chosen after the fact, based on the study sample. © 2019 Elsevier Inc. All rights reserved.
1. Introduction Anxiety is the second most common psychiatric disorder in people with epilepsy (PWE) [1], with prevalence ranging from 5% to 25% [2]. Meanwhile, there are several types of anxiety disorders, including separation anxiety disorder, selective mutism, specific phobias, social anxiety disorder (SAD), panic disorder (PD), agoraphobia, and generalized anxiety disorder (GAD) [3]. It has been reported that PWE were more likely to develop GAD compared with those without epilepsy [4,8]. While various anxiety symptoms may be associated with epilepsy, GAD is most common, which is characterized by disability and persistent free-floating concerns. In particular, GAD, which occurs in the
⁎ Correspondence to: Z. Luo, Queen Marry School of Nanchang University, Nanchang, Jiangxi 330006, People's Republic of China. ⁎⁎ Corresponding author. E-mail addresses: [email protected] (Z. Luo), [email protected] (Z. Wang). 1 Contributed equally to this work.
context of epilepsy, is often associated with fear of future seizures, disease progression, or specific complications. As far as we are concerned, GAD is nearly 80% in clinical with epilepsy compared with other anxiety subtype [8]. The early detection and proper treatment of anxiety should be highlighted because of increased suicidal ideation, poorer seizure outcome, higher rate of hospital admission, impaired quality of life, and heavier economic burden [5,6]. However, the diagnosis and treatment of anxiety in PWE is frequently neglected, as there are no validated epilepsy-specific treatments for anxiety available. While efficacious treatments are rare, many methods are available for anxiety screening. The Hospital Anxiety and Depression Scale (HADS) proves to be an effective and reliable assessment tool in identifying anxiety [7]. The Generalized Anxiety Disorder (GAD)7, a screening questionnaire to detect GAD, was validated on Chinese, Korean, and French PWE [8,9]. Both HADS and GAD-7 could minimize the occurrence of false positives because they do not probe somatic, cognitive, or underlying neurological symptoms associated with seizure disorders and epilepsy [10,11]. The Mini International Neuropsychiatric Interview (MINI) is used as a gold
https://doi.org/10.1016/j.yebeh.2019.06.035 1525-5050/© 2019 Elsevier Inc. All rights reserved.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
2
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
2.5. Risk of bias/quality assessment
standard for psychiatric diagnoses, as it is designed to handle the substantial variability present in targeted populations evaluated with psychometric instruments [12]. Considering the clinical and social impact of anxiety in patients with epilepsy, the development of a reliable anxiety screening tool for PWE is urgently needed. The ideal psychometric tool is a highly sensitive and specific self-questionnaire, which could be easily implemented in a clinical setting. The purpose of this study is to systematically and comprehensively evaluate the effectiveness of anxiety screening tools in PWE.
The Quality Assessment of Diagnostic Studies-2 (QUADAS-2) tool was used to assess the risk of bias [13,14], which was examined throughout patient selection, index test, reference standard, and flow/ timing. Each domain contained multiple questions for assessment. When multiple domains showed high or uncertain risk of bias, the overall classification was considered to be a high risk for bias. Additionally, an overall classification of low risk was judged only when all the domains were low risk. (See Box 1.)
2. Method
2.6. Data synthesis and analysis
All findings were reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards (Appendix S1) [1]. The aim of this study was to investigate the diagnostic validity of screening instruments to identify anxiety in PWE. The standard for anxiety-screening tools was calibrated according to a known reference standard and defined by criterion accuracy. Standard validation is performed using the diagnostic accuracy measures of the report (as defined below).
Median, range, and frequency were used to summarize the results of all the included studies. Because of research heterogeneity (e.g., different cutpoints for evaluation, reference standards used, etc.), we could not perform the meta-analysis. Medical estimation only considers the studies with commonly used gold standards. If there is more than one standard, we only chose the best standard as the reference. The “best” reference criterion is based on its known clinical usefulness for diagnosing anxiety, and whether it is widely referred to as the gold standard tool in the literature. If the same version of the screening tool at the same cutpoint was evaluated by two or more studies, we chose the median and range to summarize Se, Sp, PPV, NPV, and AUC estimated values. For example, three studies validated the anxiety screening tool GAD-7 with a cutpoint N 7, so we then applied the median and range to assess these estimates. Regardless of the cutpoint of the assessment, all estimates are included in Appendix S4. What is more, where feasible, the median and range were estimated for each QUADAS-2 domain with high or ambiguous risk of bias, and for each QUADAS-2 domain with low risk of bias, respectively (Appendix S5). When researchers published a paper that lacked sufficient details, we attempted to contact the research author to acquire the necessary information.
2.1. Eligibility criteria We included validation studies with original data (e.g., review papers were not involved). We considered all research demonstrating the diagnostic accuracy of any anxiety screening tool (such as gold standards, other screening tools, clinical diagnostic interviews, etc.) in PWE. We did not limit eligibility based on the reference criteria used for validation, but only the gold standards used in the statistical analysis. We excluded abstracts and conference proceedings because of the lack of comprehensive studies from these sources. 2.2. Search strategy We used Ovid MEDLINE, EMBASE, and PsyclNFO to conduct a systematic review of the published literature and searched up until April 22nd, 2019. The search terms were epilepsy, anxiety, and validity (Search Strategy, Appendix S2). We researched reference lists of previously published reviews and studies in these reviews to make sure no papers were missed. There were no restrictions on language or country of publication. 2.3. Study selection After removing the same study from different databases, three reviewers (LSH, LZW, or WZL) assessed the titles and abstracts of initial papers independently and determined the final inclusion eligibility. Then, either reviewer identified the full text of all possibly relevant records and included all subtype of PWE with anxiety disorders. Differences were settled by consensus. 2.4. Data abstraction The data were abstracted in a standardized manner. The demographics and study information of patients were recorded, including study region, sample size, population age range, and language of dissemination. In addition, screening tools, reference standards, prevalence of anxiety based on the reference standard, and relative measures of diagnostic accuracy were extracted. Diagnostic accuracy (when given) included sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), area under the receiver operating characteristic (AUC), and any recommended cutpoints.
Box 1 Description of included anxiety index tests. 1. Hospital Anxiety and Depression Scale (HADS-Anxiety n = 4). The HADSTotal is a tool with 14 scales including 7 Depression Scales and 7 Anxiety Scales. So, researchers can use it to assess anxiety in many different populations [41,42]. 2. Generalized Anxiety Disorder-7 (GAD-7 n = 3). The GAD-7 consists of a self-report questionnaire that allows rapid detection of GAD. Subjects are asked to be bothered by anxiety-related problems over the past two weeks by answering seven items on a 4-point scale [9,24,43]. 3. The State–Trait Anxiety Inventory (STAI n = 2) is a self-evaluation scale, commonly used as an instrument that measures state and trait of anxiety. It consists of a 40-item questionnaire, divided into 20 items that refer to state anxiety (STAI-S) and evaluate how participants feel about anxiety “right now, at this moment” through four scales, and 20 items that refer to trait anxiety (STAI-T) and assess how people “generally feel” about anxiety [44–47]. 4. Quality of Life in Neurological Disorders; Neuro-QOL system (NQOL n = 1) comprises a set of 14 self-report instruments of health-related quality of life for adults and children with neurological disorders. It contains eight common symptoms of anxiety, which can be used to assess anxiety over the past 7 days [15]. 5. World Health Organization index for psychological wellbeing (WHO-5 n = 1). A 5-item questionnaire that uses positive statements to measure an individual's mental wellbeing over the preceding 2 weeks. It has been validated for detecting depression in a number of clinical populations [47,48]. 6. Hamilton Anxiety Rating Scale (HAMA, n = 1). The HAMA should be jointly examined by two trained assessors, usually by conversation and observation. After the examination, the two assessors score independently. It is mainly used to assess the severity of anxiety symptoms in neurosis and other patients, but it is not suitable for estimating anxiety in various mental illnesses [49].
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
3
Fig. 1. PRISMA flow diagram. After screening all the 4758 articles, 11 studies conform to all the conditions that we set and finally were chosen to carry out this study.
3. Results
3.3. Screening tools and reference standards
3.1. Results of the search
Effectiveness was evaluated in seven screening tools based on six reference criteria, see Appendix S4 for a description of all filtering tools. Most studies validated each tool against a reference standard multiple times or validated different versions of the filtering tool. The most commonly validated screening tools were the HADS-Anxiety Questions (HADS-A; n = 4 validations [7,13,14, 19]), GAD-7 (n = 3 validations [6,8,9]), State–Trait Anxiety Inventory Trait (STAI-T; n = 2 validations [7,14]), and Hamilton Anxiety Rating Scale (HAMA; n = 1 validation [17]). The HADS-A was validated in 4 languages including Arabic [19], Polish [7,17], Brazilian [14], and English [13]. The various gold and/or reference standards used to validate the screening tools were the MINI (n = 1, multiple versions in different languages, including the MINI-Plus [16]), Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Text Revision (DSM-IV-TR) criteria, and the International League Against Epilepsy (ILAE) Commission of Psychobiology classification (n = 1 validation [14]), Composite International Diagnostic Interview (CIDI; n = 1 validation [19]), and Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders (SCIDI; n = 1 validation [15]).
We identified 4762 unique abstracts. Of these abstracts, 38 of them were fully assessed, and 11 passed all eligibility criteria. Fig. 1 lists the reasons for full-text article exclusions.
3.2. Description of studies The 11 studies that passed all eligibility criteria came out between 2011 and 2019 in 9 different countries (Appendix S4); all of the 11 studies were published in English [13]. All studies included males and females, and sample sizes ranged between 51 and 213 (median 121). Ten studies included participants ≥ 18 years old [6–9,11,13–18,20] and one study examined individuals N16 years old [19]. All studies identified samples from outpatient departments. The median prevalence of anxiety was 22.8% (range 12.1% to 45%) based on reference criteria. These 11 articles all contained the subtype-GAD, some of them also include PD, agoraphobia and SAD which were not usual in PWE with anxiety.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
4
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
The GAD-7 was validated in 3 languages, Korean [9], French [8], and Chinese [6] with reference standards including MINI (n = 2) and MINI-Plus 5.0.0 (n = 1). Mini international neurologic and psychiatric interview (Mini) is a kind of short of diagnosis of structured interview, developed in clinician-rated (MINI-CR) and patient-rated (MINI-PR) formats, for 17 kinds of Diagnostic and Statistical Manual (DSM)-III-R Axis I psychiatric disorders [50]. 3.4. Performance characteristics Two screening tools, the GAD-7 and HADS-A, had 3 studies that validated the same cutpoint (Table 1). Their recommended cutpoints were 4, 6, and 7 respectively (Appendix S4). According to the highest combination of Se and Sp values, we recommended a cutpoint N5 for the GAD7 (Se, 88.1; Sp, 87.9) and 9 for the HADS-A (Se, 75.6; Sp, 76.6). Other screening tools, such as the STAI-T, have been validated by several studies, but the median and range were not reported and did not verify the same cutpoint. We became increasingly concerned that many studies did not report the same cutpoint because this value was predetermined. The median value of all calculations and the estimated range of diagnostic accuracy are shown in Table 1. The GAD-7 has the greatest number of validated cutpoints (n = 4), between 5 and 8, and the most validations of GAD-7 is at a cutpoint N 6 (n = 3). At the GAD-7 cutpoint N7, the median Se (n = 3) was 88.1 (82.4–95.9); Sp (n = 3) was 87.9 (76–92.7); PPV (n = 3) was 95.0 (67.1–87.3); NPV (n = 3) was 96.1 (95.2–97.3), and the AUC value of the cutpoint 7 (n = 2) was 0.89 (0.873–0.899). According to the highest value of the combination of Se and Sp, we recommended a cutpoint for HADS-A of 9 (Se, 75.6; Sp, 76.6). The HADS-A had several assessed cutpoints (n = 5); however, the most validations were at a cutpoint of 8 (n = 3), where the median Se (n = 3) was 75.88 (61– 84.85), Sp (n = 3) 74.68 (57.8–91.25), PPV (n = 2) 32.25 (20.5–44), and NPV (n = 2) 99.6 (96.2–103). In order to better understand the connection between the median Se and the median Sp at different cutpoints in GAD-7, the median and the cutpoints were plotted (Fig. 2). It seems that the best balance between Se and Sp occurs at the cutpoint of N7. Based on a single estimate of the highest combination of Se and Sp, we suggested the cutpoint for GAD-7 would be N5 (Se 97.05; Sp 84.2). Only when two or more of the same validation tools were used to compute the same cutpoint were AUC values able to be estimated. These estimates are shown in Table 1. The median AUC of the overall screening tool was estimated for GAD-7 (n = 2) (AUC 91.5%, range 0.89–0.94%) and HADS-A (n = 1) (AUC 77.7%). This verified that GAD-7 study had the highest total AUC estimate based on the reference criteria applied. The MINI version was used as a reference standard in all three studies that calculated the GAD-7 total AUC. Data analysis based on research quality assessment is only applicable to GAD-7 (Appendix S5). However, instead of a high risk, we only
Fig. 2. Median sensitivities and specificities for different GAD-7 cutpoints. In this figure, it is obvious to see that the cutpoint N7 of GAD-7 may be the best point to detect anxiety in patients with epilepsy because this point appears to have the best balance between Se and Sp because these two curves.
have a low risk group, so we cannot compare the data between low and high risk of bias studies. 3.5. Risk of bias assessment The risk of bias in at least one of the four categories was ambiguous in four studies out of a total of 11 categories, while the risk of bias in at least one of the four QUADSA-2 rating system categories was high in two studies (Appendix S3). Moreover, there are 7 studies with a low risk of bias among all varieties. Generally, this undefined or high-risk bias is classified as an “index test.” Specifically, whether the screening tool could be shown with no reference standard results is unknown. 4. Discussion This systematic review examined whether the current literature contains valid methods for screening for anxiety in patients with epilepsy. In order to properly treat patients with epilepsy, it is essential that the right anxiety screening tool is implemented to improve the prognosis [21,24]. To allow clinicians to make an informed decision, these anxiety screening tools must be assessed in a comprehensive manner. An effective tool for assessing anxiety will not only identify anxiety as accurately as possible, but also minimize potential harmful adverse effects of treatment if the instrument is too sensitive. The recommended cutpoint in HADS-A when used to detect anxiety in a “normal” population is 9, and this has been consistently validated
Table 1 Summary of diagnostic accuracy estimates. When n ≥ 2, the average value was calculated to show the tendency. When n = 1, only the data of the most recommendable cutpoint were shown. Tool
GAD-7
HADS-A NQOL-A NDDI-E WHO-5 STAI-T STAI-S HAMA
Cutpoint
Sensitivity median (range) n
Specificity median (range) n
PPV median (range) n
NPV median (range) n
AUC median (range) n
N5 N6 N7 N8 7 8 9 10 ≥24 N12 b50 ≥52 ≥52 ≥17
97.05 (96.1–98.0) n = 2 94.7 (92.2–95.9) n = 3 88.1 (82.4–95.9) n = 3 74.1 (60–85.7) n = 3 92.42 (90.9–93.94) n = 2 75.88 (61–84.85) n = 3 75.55 (63.6–81.3) n = 3 75.5 (69.7–81.3) n = 2 0.77 n = 1 0.800 n = 1 0.800 n = 1 81.3 n = 1 56.3 n = 1 68.8 n = 1
84.2 (84.0–84.4) n = 2 83.4 (69.8–91.4) n = 3 87.9 (76–92.7) n = 3 90.9 (80.2–98.8) n = 3 64.7 (46.9–0.825) n = 2 74.68 (57.8–91.25) n = 3 76.6 (62.5–97.5) n = 3 83.75 (70–97.5) n = 2 0.82 n = 1 0.798 n = 1 0.917 n = 1 77.5 n = 1 86.3 n = 1 87.5 n = 1
63.65 (62–65.3) n = 2 69.5 (62.3–77) n = 3 95.0 (67.1–87.3) n = 3 79.2 (68.9–93.8) n = 3 – 32.25 (20.5–44) n = 2 – – 0.81 n = 1 0.343 n = 1 0.571 n = 1 41.9 n = 1 45.0 n = 1 52.4 n = 1
99.05 (98.8–99.3) n = 2 98.1 (97.7–98.5) n = 3 96.1 (95.2–97.3) n = 3 91.1 (88–93.7) n = 3 – 99.6 (96.2–103) n = 2 – – 0.78 n = 1 0.967 n = 1 0.971 n = 1 95.4 n = 1 90.8 n = 1 93.3 n = 1
– 0.94 (0.906–0.974) n = 2 0.89 (0.873–0.899) n = 2 – – 0.777 (0.68–0.951) n = 3 – – 0.932 n = 1 0.916 n = 1 0.847 n = 1 0.704 n = 1 0.811 n = 1
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
[22]. Several studies focus on using this questionnaire in PWE. Indeed, HADS-A is a useful tool in evaluating patients with epilepsy for anxiety as it does not include possible overlap of somatic symptoms with the side effects of pharmacological treatments or the disease itself [23]. The GAD-7 is also highly validated, undoubtedly because it was created to rapidly detect patients with epilepsy with anxiety symptoms; it is composed of a self-report questionnaire and can efficiently detect GAD [24,25]. In this review, the GAD-7 cutpoints N6 and N 7 were the most validated cutpoints. We found that the cutpoint N7 had the highest PPV, Se, and Sp for identifying an anxiety disorder. This result reveals that the optimal cutpoint for detection, with this tool, is 7 [24]. Although several screening instruments such as HADS-A, STAI-T, and HAMA are useful to identify anxiety, there are no validated instruments of anxiety for PWE. The GAD-7 was recently developed in the USA as a valuable screening tool for detecting GAD in PWE [9]. Meanwhile, The ILAE also recommends GAD-7 as the best screening tools for PWE with anxiety, which is in line with our findings [26]. The GAD-7 cutpoint of N7 is recommended because this point appears to have the best balance between Se and Sp, as these two curves, when plotted, converge at this point. Our results show that the GAD-7 cutpoint of N 6 and N 7 have higher Se than Sp [(N6: Sp, 83.4; Se, 94.7) (N 7: Sp, 87.9; Se, 88.1)]. In the 3 studies of GAD-7 that we selected, all used MINI as the reference standard, which can avoid under- or overrepresenting true positive cases, and results in a similar threshold for GAD-7 [27]. In addition, most of the GAD-7 using non-English were verified for accuracy. GAD-7 was translated directly from English into the corresponding language and then reverse-translated back to English to ensure reliability between the two versions, also known as Brislin technology [28]. While this step increases the reliability of the data compared, it must be noted that these different language validations are combined in determining the median and range of diagnostic accuracy estimates [29]. The most common reference standard is the MINI, which has previously tested for SCID in patients with epilepsy [30,36]. The SCID is widely accepted as the gold standard for measuring anxiety in research studies [31,37]; however, it may not be the best tool to use in a clinical setting because it is time-consuming and should be managed by mental health professionals. Although the MINI is generally considered a short screening tool rather than a true “gold standard,” it is understood that the MINI is widely used because it can be managed by trained staff, has face validity, and is highly consistent with SCID [32]. Most importantly, when assessing the diagnostic accuracy of a screening tool, it is important to use an accepted reference or gold standard (e.g., SCID or MINI) rather than another screening tool, as this can lead to inaccurate assessments of anxiety. An important gap in the literature is the lack of studies validating anxiety screening tools for epilepsy in children and adolescents, as no studies out of the 11 that met our criteria included individuals younger than 16. However, the prevalence of anxiety in children with epilepsy can range from 13% to 49% [33]. Future research should develop novel screening tools focused on adolescents and children or modify the traditional adult screening tools, such as the GAD-7, for younger individuals. It is promising that the clinical application of these tools could lead to better detection of anxiety in young patients with epilepsy. In this review, we found cutoff points for screening tools, substantial variability in the language and reference criteria used to evaluate screening tools, and standardized reports lacking diagnostic accuracy estimates, thereby excluding them from the meta-analysis. In addition, AUC statistics and corresponding accuracy estimates are reported more frequently in basic studies; however, these estimates provide less clinical utility because they are more difficult to interpret and apply in clinical practice [34]. Another problem is the lack of reported variability within the estimates themselves. Notably, Se and Sp were reported in 11 studies, of which only 4 reported corresponding confidence intervals; PPV and NPV were reported in 9 studies, and only 5 studies
5
reported corresponding confidence intervals. Additionally, many studies only reported PPV and NPV cutpoints with the best Se and Sp, not with all of the assessed cutpoints. Base rates were also applied for predictive value with the original test characteristics, including Se and Sp, resulting in less generalizable values. Based on these findings, incorporating adequate standards of reporting, including the variability of the estimates, is significant and will lead to higher data quality. This will allow researchers to conduct an in-depth critical appraisal of these studies and provide a greater contribution to the field [35]. The value of research that assesses diagnostic accuracy of screening tools needs to improve, and manuscripts should follow published guidelines to encourage this practice, such as the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement [36]. A comprehensive list of essential items, which should be included in a published manuscript, can be found in the STARD statement. This work used a wide range of search strategies in three large databases without the limitations of publication language or date. We also used a well-documented methodology to search through the entries, including PRISMA standards for reporting and QUADAS-2 for assessing the risk of bias and applicability of included studies [37,38]. As such, we did observe a high risk of bias for the index test domain. In many cases, when the assessment of the screening tool and reference criteria results were blinded, we considered this an issue of unclear reporting rather than a problem of the method itself. Regardless of the outcome of the QUADAS-2 evaluation, all articles were included in the review. We believe that the inclusion of all studies in our analysis will not affect the final conclusion because of the fact that the median estimates of diagnostic accuracy were not associated with a low versus high risk of bias. Although many studies have included epileptic subtypes in their overall sample, they have not stratified the results of these subtypes (e.g., events and epidemics, temporal and frontal lobe, etc.). Future research should focus on the effectiveness of anxiety screening questionnaires on different subtypes of epilepsy. Another potential issue found in this study is the choice to use postmortem cutpoints according to the sample size. By using this method based on sample size to select cutoff points, the tool's effectiveness may be overestimated because of selective reporting, partly due to sampling variability, resulting in better performance [39]. Because of this, the effectiveness of these tools may be overestimated in the studies themselves compared to how they would perform in a clinical setting. Therefore, we recommend that researchers report all of the cutpoints evaluated in their studies, not just the high performing ones. The difficulty in making a recommendation on the “perfect” cutpoint for various screening instruments is another limitation of this analysis. Unfortunately, while previous methods used to conduct systematic reviews were well-organized and thoughtful, comprehensive estimates are still difficult to provide, and consequently, concrete suggestions are unable to be made [39,40].
5. Conclusion To identify a superior anxiety-screening instrument for PWE, many factors should be taken into consideration, including the properties of the tool itself and the requirements of the clinician and healthcare system. Although we lack the evidence to make a definitive recommendation for the best screening tool, our analysis suggests that GAD-7 is likely the most useful screening tool because of it being freely available, relatively simple to implement, and available in many languages. We suggest that individuals choose a PWE anxiety screening tool based on factors known to be important, namely a high Sp, Se, and PPV. Future studies should validate anxiety screening tools against the best possible reference criteria stated by STARD. Moreover, screening tools developed for different age ranges also need to be explored.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
6
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
Funding This research did not receive any specific grant from funding agencies in the public, or not-for-profit sectors. Declaration of Competing Interest The authors declare that they have on conflict of interest. Supplementary data to this article can be found online at https://doi. org/10.1016/j.yebeh.2019.06.035. References [1] Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. [2] Gaitatzis A, Carroll K, Majeed A, Sander J. The epidemiology of the comorbidity of epilepsy in the general population. Epilepsia 2004;45(12):1613–22. [3] Craske MG, Stein MB. Anxiety. Lancet 2016:3048–59. [4] Wiglusz M, Landowski J, Cubała W. Prevalence of anxiety disorders in epilepsy. Epilepsy Behav 2018;79:1–3. [5] Kanner AM, Schachter SC, Barry JJ, Hersdorffer DC, Mula M, Trimble M, et al. Depression and epilepsy: epidemiologic and neurobiologic perspectives that may explain their high comorbid occurrence. Epilepsy Behav 2012;24:156–68. [6] Tong X, An D, McGonigal A, Park S-P, Zhou D. Validation of the Generalized Anxiety Disorder-7 (GAD-7) among Chinese people with epilepsy. Epilepsy Res 2016;120:31–6. [7] Wiglusz MS, Landowski J, Cubala WJ. Validation of the Polish version of the Hospital Anxiety and Depression Scale for anxiety disorders in patients with epilepsy. Epilepsy Behav 2018;84:162–5. [8] Micoulaud-Franchi J-A, Lagarde S, Barkate G, Dufournet B, Besancon C, TrebuchonDa Fonseca A, et al. Rapid detection of generalized anxiety disorder and major depression in epilepsy: validation of the GAD-7 as a complementary tool to the NDDI-E in a French sample. Epilepsy Behav 2016;57:211–6. [9] Seo J-G, Cho YW, Lee S-J, Lee J-J, Kim J-E, Moon H-J, et al. Validation of the Generalized Anxiety Disorder-7 in people with epilepsy: a MEPSY study. Epilepsy Behav 2014;35:59–63. [10] Kanner AM. Anxiety disorders in epilepsy: the forgotten psychiatric comorbidity. Epilepsy Curr 2011;11(3):90–1. [11] Mumford DB. Hospital anxiety and depression scale[J]. Br J Psychiatry 1991;159(159):729. [12] Wiglusz MS, Landowski J, Cubała WJ. Psychometric properties of the Polish version of the Hamilton Anxiety Rating Scale in patients with epilepsy with and without comorbid anxiety disorder. Epilepsy Behav 2019;94:9–13. [13] Gandy M, Sharpe L, Perry KN, Miller L, Thayer Z, Boserio J, et al. Anxiety in epilepsy: a neglected disorder. J Psychosom Res 2015;78:149–55. [14] Zingano BL, Guarnieri R, Diaz AP, Schwarzbold ML, Wolf P, Lin K, et al. Hospital Anxiety and Depression Scale-Anxiety subscale (HADS-A) and The State-Trait Anxiety Inventory (STAI) accuracy for anxiety disorders detection in drug-resistant mesial temporal lobe epilepsy patients. J Affect Disord 2019;246:452–7. [15] De Marco AP, Mahoney JJ, Aduen PA, Langer J, Bajo SD, Broshek DK. The relationship between the neuro-quality of life depression and anxiety measures and the personality assessment inventory in persons with epilepsy. Epilepsy Behav 2017;70:145–9. [16] Hansen CP, Amiri M. Combined detection of depression and anxiety in epilepsy patients using the neurological disorders depression inventory for epilepsy and the World Health Organization well-being index. Seizure 2015;33:41–5. [17] Wiglusz MS, Landowski J. Psychometric properties and diagnostic utility of the state–trait anxiety inventory in epilepsy with and without comorbid anxiety disorder. Epilepsy Behav 2019;92:221–5. [18] Thombs BD, Benedetti A, Kloda LA, Levis B, Azar M, Riehm KE, et al. Diagnostic accuracy of the Depression subscale of the Hospital Anxiety and Depression Scale (HADSD) for detecting major depression: protocol for a systematic review and individual patient data meta-analyses. BMJ Open 2016;6(4):e011913. [19] Al-Asmi A, Dorvlo AS, Burke DT, Al-Adawi S, Al-Zaabi A, Al-Zadjali HA, et al. The detection of mood and anxiety in people with epilepsy using two-phase designs: experiences from a tertiary care centre in Oman. Epilepsy Res 2012;98:174–81.
[20] Kanner Andres M. Anxiety disorders in epilepsy: the forgotten psychiatric comorbidity. Epilepsy Curr 2011;11:90–1. [21] Kleijnen J, Antes G. Systematic reviews in health care. Meta-analysis in context. Int J Epidemiol 2002;31(3):206–7. [22] Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. J Psychosom Res 2002;52(2):69–77. [23] Stern AF. The Hospital Anxiety and Depression Scale. Occup Med (Chic Ill) 2014;67(6):361. [24] Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166(10):1092–7. [25] Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 2008;46(3):266–74. [26] Parkerson HA, Thibodeau MA, Brandt CP, Zvolensky MJ, Asmundson GJG. Culturalbased biases of the GAD-7. J Anxiety Disord 2015;31:38–42. [27] Seo JG, Park SP. Validation of the Generalized Anxiety Disorder-7 (GAD-7) and GAD2 in patients with migraine. J Headache Pain 2015;16(1):97. [28] Brislin RW. Back-translation for cross-cultural research. J Cross-Cult Psychol 1970;1 (3):185–216. [29] Swinson RP. The GAD-7 scale was accurate for diagnosing generalised anxiety disorder. Evid Based Med 2006;11(6):184. [30] Jones JE, Hermann BP, Barry JJ, Gilliam F, Kanner AM, Meador KJ. Clinical assessment of Axis I psychiatric morbidity in chronic epilepsy: a multicenter investigation. J Neuropsychiatry Clin Neurosci 2014;17(2):172–9. [31] Brown TA, Campbell LA, Lehman CL, Grisham JR, Mancill RB. Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. J Abnorm Psychol 2001;110(4):585. [32] Sheehan DV, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Keskiner A, et al. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry 1997;12(5):232–41. [33] Jones JE, Austin JK, Caplan R, Dunn D, Plioplys S, Salpekar JA. Psychiatric disorders in children and adolescents who have epilepsy. Pediatr Rev 2008;29(2):e9-14. [34] Egger M, Smith GD, Altman DG. Systematic reviews in health care: meta-analysis in context. 2nd ed.; 2008. [35] Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Clin Chem 2015;2015:h5527. [36] Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016;6(11):e012799. [37] Bias AR. QUADAS 2. Am J Gastroenterol 2012;107:1354–60. [38] Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;465. [39] Plummer F, Manea L, Trepel D, McMillan D. Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis. Gen Hosp Psychiatry 2016;39:24–31. [40] Gill SJ, Lukmanji S, Fiest KM, Patten SB, Wiebe S, Jetté N. Depression screening tools in persons with epilepsy: a systematic review of validated tools. Epilepsia 2017;58 (5):695–705. [41] Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67(6):361–70. [42] Skapinakis P. Hospital Anxiety and Depression Scale (HADS) Encycl Qual Life WellBeing Res; 2014. [43] Williams N. The GAD-7 questionnaire. Occup Med (Chic Ill) 2014;64(3):224. [44] STAI. Handb. Dis Burdens Qual Life Meas; 2009. [45] Shahid A, Wilkinson K, Marcu S, Shapiro CM. State-Trait Anxiety Inventory (STAI). STOP, THAT one hundred other sleep scales; 2012; 367–8. [46] Spielberger CD. Manual for the State-Trait Anxiety Inventory (STAI form Y); 1983. [47] Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom 2015;84(3). [48] Calderone FA. World Health Organization. Food Drug Cosmet Law Q; 1947. [49] Hamilton M. Hamilton Anxiety Rating Scale (HAM-A). Br J Med Psychol 1959;32:50–5. [50] Amorim P, Lecrubier Y, Weiller E, et al. DSM-IH-R psychotic disorders: procedural validity of the Mini International Neuropsychiatric Interview (MINI). Concordance and causes for discordance with the CIDI. Eur Psychiatry 1998;13:26–34.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Review
Anxiety screening tools in people with epilepsy: A systematic review of validated tools Zilin Wang a,1, Zhiwen Luo a,b,1, Shunhua Li a,1, ，, Zhiwen Luo a,b,⁎,1, Zilin Wang a,⁎⁎,1 a b
Queen Marry School of Nanchang University, Nanchang, Jiangxi 330006, People's Republic of China Affiliated Eye Hospital of Nanchang University, Jiangxi, Research Institute of Ophthalmology and Visual Science, Nanchang, People's Republic of China
a r t i c l e
i n f o
Article history: Received 22 May 2019 Revised 18 June 2019 Accepted 27 June 2019 Available online xxxx Keywords: Epilepsy Anxiety disorder Mental health Comorbidity Screening tools Diagnostic accuracy Measurement Scale
a b s t r a c t Objective: Anxiety is a common neurological condition often comorbid with epilepsy, with approximately 20% of patients with epilepsy exhibiting symptoms of anxiety. Despite this prevalence, accurate and efficacious tools designed to screen for anxiety specifically in patients with epilepsy have not yet been developed. The purpose of this study is to systematically review the literature and better understand this relationship. Methods: Ovid MEDLINE, EMBASE, and PsyclNFO were searched until April 22nd, 2019 without language restrictions. We extracted abstracts, data abstraction, and full-text reviews in duplicate and chose the studies that included measures for anxiety screening in patients with epilepsy. The Quality Assessment of Diagnostic Accuracy Studies Version was used to assess study quality. We used the medians and ranges to calculate the accuracy of the tools. Results: We screened 4758 abstracts and selected 11 articles dealing with anxiety. The most common validated anxiety screening tools were the Generalized Anxiety Disorder (GAD-7) and Hospital Anxiety and Depression Scale-A (HADS-A). The Mini International Neuropsychiatric Interview (MINI) was the most common reference standard used. Significance: Many studies have validated depression screening tools rather than anxiety. This lack of data has left much uncertainty about the relationship of epilepsy to anxiety, as well as diagnostic inconsistencies. The effectiveness of these assessments in practice may be overestimating the prevalence, as the cutpoints are usually chosen after the fact, based on the study sample. © 2019 Elsevier Inc. All rights reserved.
1. Introduction Anxiety is the second most common psychiatric disorder in people with epilepsy (PWE) [1], with prevalence ranging from 5% to 25% [2]. Meanwhile, there are several types of anxiety disorders, including separation anxiety disorder, selective mutism, specific phobias, social anxiety disorder (SAD), panic disorder (PD), agoraphobia, and generalized anxiety disorder (GAD) [3]. It has been reported that PWE were more likely to develop GAD compared with those without epilepsy [4,8]. While various anxiety symptoms may be associated with epilepsy, GAD is most common, which is characterized by disability and persistent free-floating concerns. In particular, GAD, which occurs in the
⁎ Correspondence to: Z. Luo, Queen Marry School of Nanchang University, Nanchang, Jiangxi 330006, People's Republic of China. ⁎⁎ Corresponding author. E-mail addresses: [email protected] (Z. Luo), [email protected] (Z. Wang). 1 Contributed equally to this work.
context of epilepsy, is often associated with fear of future seizures, disease progression, or specific complications. As far as we are concerned, GAD is nearly 80% in clinical with epilepsy compared with other anxiety subtype [8]. The early detection and proper treatment of anxiety should be highlighted because of increased suicidal ideation, poorer seizure outcome, higher rate of hospital admission, impaired quality of life, and heavier economic burden [5,6]. However, the diagnosis and treatment of anxiety in PWE is frequently neglected, as there are no validated epilepsy-specific treatments for anxiety available. While efficacious treatments are rare, many methods are available for anxiety screening. The Hospital Anxiety and Depression Scale (HADS) proves to be an effective and reliable assessment tool in identifying anxiety [7]. The Generalized Anxiety Disorder (GAD)7, a screening questionnaire to detect GAD, was validated on Chinese, Korean, and French PWE [8,9]. Both HADS and GAD-7 could minimize the occurrence of false positives because they do not probe somatic, cognitive, or underlying neurological symptoms associated with seizure disorders and epilepsy [10,11]. The Mini International Neuropsychiatric Interview (MINI) is used as a gold
https://doi.org/10.1016/j.yebeh.2019.06.035 1525-5050/© 2019 Elsevier Inc. All rights reserved.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
2
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
2.5. Risk of bias/quality assessment
standard for psychiatric diagnoses, as it is designed to handle the substantial variability present in targeted populations evaluated with psychometric instruments [12]. Considering the clinical and social impact of anxiety in patients with epilepsy, the development of a reliable anxiety screening tool for PWE is urgently needed. The ideal psychometric tool is a highly sensitive and specific self-questionnaire, which could be easily implemented in a clinical setting. The purpose of this study is to systematically and comprehensively evaluate the effectiveness of anxiety screening tools in PWE.
The Quality Assessment of Diagnostic Studies-2 (QUADAS-2) tool was used to assess the risk of bias [13,14], which was examined throughout patient selection, index test, reference standard, and flow/ timing. Each domain contained multiple questions for assessment. When multiple domains showed high or uncertain risk of bias, the overall classification was considered to be a high risk for bias. Additionally, an overall classification of low risk was judged only when all the domains were low risk. (See Box 1.)
2. Method
2.6. Data synthesis and analysis
All findings were reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards (Appendix S1) [1]. The aim of this study was to investigate the diagnostic validity of screening instruments to identify anxiety in PWE. The standard for anxiety-screening tools was calibrated according to a known reference standard and defined by criterion accuracy. Standard validation is performed using the diagnostic accuracy measures of the report (as defined below).
Median, range, and frequency were used to summarize the results of all the included studies. Because of research heterogeneity (e.g., different cutpoints for evaluation, reference standards used, etc.), we could not perform the meta-analysis. Medical estimation only considers the studies with commonly used gold standards. If there is more than one standard, we only chose the best standard as the reference. The “best” reference criterion is based on its known clinical usefulness for diagnosing anxiety, and whether it is widely referred to as the gold standard tool in the literature. If the same version of the screening tool at the same cutpoint was evaluated by two or more studies, we chose the median and range to summarize Se, Sp, PPV, NPV, and AUC estimated values. For example, three studies validated the anxiety screening tool GAD-7 with a cutpoint N 7, so we then applied the median and range to assess these estimates. Regardless of the cutpoint of the assessment, all estimates are included in Appendix S4. What is more, where feasible, the median and range were estimated for each QUADAS-2 domain with high or ambiguous risk of bias, and for each QUADAS-2 domain with low risk of bias, respectively (Appendix S5). When researchers published a paper that lacked sufficient details, we attempted to contact the research author to acquire the necessary information.
2.1. Eligibility criteria We included validation studies with original data (e.g., review papers were not involved). We considered all research demonstrating the diagnostic accuracy of any anxiety screening tool (such as gold standards, other screening tools, clinical diagnostic interviews, etc.) in PWE. We did not limit eligibility based on the reference criteria used for validation, but only the gold standards used in the statistical analysis. We excluded abstracts and conference proceedings because of the lack of comprehensive studies from these sources. 2.2. Search strategy We used Ovid MEDLINE, EMBASE, and PsyclNFO to conduct a systematic review of the published literature and searched up until April 22nd, 2019. The search terms were epilepsy, anxiety, and validity (Search Strategy, Appendix S2). We researched reference lists of previously published reviews and studies in these reviews to make sure no papers were missed. There were no restrictions on language or country of publication. 2.3. Study selection After removing the same study from different databases, three reviewers (LSH, LZW, or WZL) assessed the titles and abstracts of initial papers independently and determined the final inclusion eligibility. Then, either reviewer identified the full text of all possibly relevant records and included all subtype of PWE with anxiety disorders. Differences were settled by consensus. 2.4. Data abstraction The data were abstracted in a standardized manner. The demographics and study information of patients were recorded, including study region, sample size, population age range, and language of dissemination. In addition, screening tools, reference standards, prevalence of anxiety based on the reference standard, and relative measures of diagnostic accuracy were extracted. Diagnostic accuracy (when given) included sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), area under the receiver operating characteristic (AUC), and any recommended cutpoints.
Box 1 Description of included anxiety index tests. 1. Hospital Anxiety and Depression Scale (HADS-Anxiety n = 4). The HADSTotal is a tool with 14 scales including 7 Depression Scales and 7 Anxiety Scales. So, researchers can use it to assess anxiety in many different populations [41,42]. 2. Generalized Anxiety Disorder-7 (GAD-7 n = 3). The GAD-7 consists of a self-report questionnaire that allows rapid detection of GAD. Subjects are asked to be bothered by anxiety-related problems over the past two weeks by answering seven items on a 4-point scale [9,24,43]. 3. The State–Trait Anxiety Inventory (STAI n = 2) is a self-evaluation scale, commonly used as an instrument that measures state and trait of anxiety. It consists of a 40-item questionnaire, divided into 20 items that refer to state anxiety (STAI-S) and evaluate how participants feel about anxiety “right now, at this moment” through four scales, and 20 items that refer to trait anxiety (STAI-T) and assess how people “generally feel” about anxiety [44–47]. 4. Quality of Life in Neurological Disorders; Neuro-QOL system (NQOL n = 1) comprises a set of 14 self-report instruments of health-related quality of life for adults and children with neurological disorders. It contains eight common symptoms of anxiety, which can be used to assess anxiety over the past 7 days [15]. 5. World Health Organization index for psychological wellbeing (WHO-5 n = 1). A 5-item questionnaire that uses positive statements to measure an individual's mental wellbeing over the preceding 2 weeks. It has been validated for detecting depression in a number of clinical populations [47,48]. 6. Hamilton Anxiety Rating Scale (HAMA, n = 1). The HAMA should be jointly examined by two trained assessors, usually by conversation and observation. After the examination, the two assessors score independently. It is mainly used to assess the severity of anxiety symptoms in neurosis and other patients, but it is not suitable for estimating anxiety in various mental illnesses [49].
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
3
Fig. 1. PRISMA flow diagram. After screening all the 4758 articles, 11 studies conform to all the conditions that we set and finally were chosen to carry out this study.
3. Results
3.3. Screening tools and reference standards
3.1. Results of the search
Effectiveness was evaluated in seven screening tools based on six reference criteria, see Appendix S4 for a description of all filtering tools. Most studies validated each tool against a reference standard multiple times or validated different versions of the filtering tool. The most commonly validated screening tools were the HADS-Anxiety Questions (HADS-A; n = 4 validations [7,13,14, 19]), GAD-7 (n = 3 validations [6,8,9]), State–Trait Anxiety Inventory Trait (STAI-T; n = 2 validations [7,14]), and Hamilton Anxiety Rating Scale (HAMA; n = 1 validation [17]). The HADS-A was validated in 4 languages including Arabic [19], Polish [7,17], Brazilian [14], and English [13]. The various gold and/or reference standards used to validate the screening tools were the MINI (n = 1, multiple versions in different languages, including the MINI-Plus [16]), Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Text Revision (DSM-IV-TR) criteria, and the International League Against Epilepsy (ILAE) Commission of Psychobiology classification (n = 1 validation [14]), Composite International Diagnostic Interview (CIDI; n = 1 validation [19]), and Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders (SCIDI; n = 1 validation [15]).
We identified 4762 unique abstracts. Of these abstracts, 38 of them were fully assessed, and 11 passed all eligibility criteria. Fig. 1 lists the reasons for full-text article exclusions.
3.2. Description of studies The 11 studies that passed all eligibility criteria came out between 2011 and 2019 in 9 different countries (Appendix S4); all of the 11 studies were published in English [13]. All studies included males and females, and sample sizes ranged between 51 and 213 (median 121). Ten studies included participants ≥ 18 years old [6–9,11,13–18,20] and one study examined individuals N16 years old [19]. All studies identified samples from outpatient departments. The median prevalence of anxiety was 22.8% (range 12.1% to 45%) based on reference criteria. These 11 articles all contained the subtype-GAD, some of them also include PD, agoraphobia and SAD which were not usual in PWE with anxiety.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
4
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
The GAD-7 was validated in 3 languages, Korean [9], French [8], and Chinese [6] with reference standards including MINI (n = 2) and MINI-Plus 5.0.0 (n = 1). Mini international neurologic and psychiatric interview (Mini) is a kind of short of diagnosis of structured interview, developed in clinician-rated (MINI-CR) and patient-rated (MINI-PR) formats, for 17 kinds of Diagnostic and Statistical Manual (DSM)-III-R Axis I psychiatric disorders [50]. 3.4. Performance characteristics Two screening tools, the GAD-7 and HADS-A, had 3 studies that validated the same cutpoint (Table 1). Their recommended cutpoints were 4, 6, and 7 respectively (Appendix S4). According to the highest combination of Se and Sp values, we recommended a cutpoint N5 for the GAD7 (Se, 88.1; Sp, 87.9) and 9 for the HADS-A (Se, 75.6; Sp, 76.6). Other screening tools, such as the STAI-T, have been validated by several studies, but the median and range were not reported and did not verify the same cutpoint. We became increasingly concerned that many studies did not report the same cutpoint because this value was predetermined. The median value of all calculations and the estimated range of diagnostic accuracy are shown in Table 1. The GAD-7 has the greatest number of validated cutpoints (n = 4), between 5 and 8, and the most validations of GAD-7 is at a cutpoint N 6 (n = 3). At the GAD-7 cutpoint N7, the median Se (n = 3) was 88.1 (82.4–95.9); Sp (n = 3) was 87.9 (76–92.7); PPV (n = 3) was 95.0 (67.1–87.3); NPV (n = 3) was 96.1 (95.2–97.3), and the AUC value of the cutpoint 7 (n = 2) was 0.89 (0.873–0.899). According to the highest value of the combination of Se and Sp, we recommended a cutpoint for HADS-A of 9 (Se, 75.6; Sp, 76.6). The HADS-A had several assessed cutpoints (n = 5); however, the most validations were at a cutpoint of 8 (n = 3), where the median Se (n = 3) was 75.88 (61– 84.85), Sp (n = 3) 74.68 (57.8–91.25), PPV (n = 2) 32.25 (20.5–44), and NPV (n = 2) 99.6 (96.2–103). In order to better understand the connection between the median Se and the median Sp at different cutpoints in GAD-7, the median and the cutpoints were plotted (Fig. 2). It seems that the best balance between Se and Sp occurs at the cutpoint of N7. Based on a single estimate of the highest combination of Se and Sp, we suggested the cutpoint for GAD-7 would be N5 (Se 97.05; Sp 84.2). Only when two or more of the same validation tools were used to compute the same cutpoint were AUC values able to be estimated. These estimates are shown in Table 1. The median AUC of the overall screening tool was estimated for GAD-7 (n = 2) (AUC 91.5%, range 0.89–0.94%) and HADS-A (n = 1) (AUC 77.7%). This verified that GAD-7 study had the highest total AUC estimate based on the reference criteria applied. The MINI version was used as a reference standard in all three studies that calculated the GAD-7 total AUC. Data analysis based on research quality assessment is only applicable to GAD-7 (Appendix S5). However, instead of a high risk, we only
Fig. 2. Median sensitivities and specificities for different GAD-7 cutpoints. In this figure, it is obvious to see that the cutpoint N7 of GAD-7 may be the best point to detect anxiety in patients with epilepsy because this point appears to have the best balance between Se and Sp because these two curves.
have a low risk group, so we cannot compare the data between low and high risk of bias studies. 3.5. Risk of bias assessment The risk of bias in at least one of the four categories was ambiguous in four studies out of a total of 11 categories, while the risk of bias in at least one of the four QUADSA-2 rating system categories was high in two studies (Appendix S3). Moreover, there are 7 studies with a low risk of bias among all varieties. Generally, this undefined or high-risk bias is classified as an “index test.” Specifically, whether the screening tool could be shown with no reference standard results is unknown. 4. Discussion This systematic review examined whether the current literature contains valid methods for screening for anxiety in patients with epilepsy. In order to properly treat patients with epilepsy, it is essential that the right anxiety screening tool is implemented to improve the prognosis [21,24]. To allow clinicians to make an informed decision, these anxiety screening tools must be assessed in a comprehensive manner. An effective tool for assessing anxiety will not only identify anxiety as accurately as possible, but also minimize potential harmful adverse effects of treatment if the instrument is too sensitive. The recommended cutpoint in HADS-A when used to detect anxiety in a “normal” population is 9, and this has been consistently validated
Table 1 Summary of diagnostic accuracy estimates. When n ≥ 2, the average value was calculated to show the tendency. When n = 1, only the data of the most recommendable cutpoint were shown. Tool
GAD-7
HADS-A NQOL-A NDDI-E WHO-5 STAI-T STAI-S HAMA
Cutpoint
Sensitivity median (range) n
Specificity median (range) n
PPV median (range) n
NPV median (range) n
AUC median (range) n
N5 N6 N7 N8 7 8 9 10 ≥24 N12 b50 ≥52 ≥52 ≥17
97.05 (96.1–98.0) n = 2 94.7 (92.2–95.9) n = 3 88.1 (82.4–95.9) n = 3 74.1 (60–85.7) n = 3 92.42 (90.9–93.94) n = 2 75.88 (61–84.85) n = 3 75.55 (63.6–81.3) n = 3 75.5 (69.7–81.3) n = 2 0.77 n = 1 0.800 n = 1 0.800 n = 1 81.3 n = 1 56.3 n = 1 68.8 n = 1
84.2 (84.0–84.4) n = 2 83.4 (69.8–91.4) n = 3 87.9 (76–92.7) n = 3 90.9 (80.2–98.8) n = 3 64.7 (46.9–0.825) n = 2 74.68 (57.8–91.25) n = 3 76.6 (62.5–97.5) n = 3 83.75 (70–97.5) n = 2 0.82 n = 1 0.798 n = 1 0.917 n = 1 77.5 n = 1 86.3 n = 1 87.5 n = 1
63.65 (62–65.3) n = 2 69.5 (62.3–77) n = 3 95.0 (67.1–87.3) n = 3 79.2 (68.9–93.8) n = 3 – 32.25 (20.5–44) n = 2 – – 0.81 n = 1 0.343 n = 1 0.571 n = 1 41.9 n = 1 45.0 n = 1 52.4 n = 1
99.05 (98.8–99.3) n = 2 98.1 (97.7–98.5) n = 3 96.1 (95.2–97.3) n = 3 91.1 (88–93.7) n = 3 – 99.6 (96.2–103) n = 2 – – 0.78 n = 1 0.967 n = 1 0.971 n = 1 95.4 n = 1 90.8 n = 1 93.3 n = 1
– 0.94 (0.906–0.974) n = 2 0.89 (0.873–0.899) n = 2 – – 0.777 (0.68–0.951) n = 3 – – 0.932 n = 1 0.916 n = 1 0.847 n = 1 0.704 n = 1 0.811 n = 1
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
[22]. Several studies focus on using this questionnaire in PWE. Indeed, HADS-A is a useful tool in evaluating patients with epilepsy for anxiety as it does not include possible overlap of somatic symptoms with the side effects of pharmacological treatments or the disease itself [23]. The GAD-7 is also highly validated, undoubtedly because it was created to rapidly detect patients with epilepsy with anxiety symptoms; it is composed of a self-report questionnaire and can efficiently detect GAD [24,25]. In this review, the GAD-7 cutpoints N6 and N 7 were the most validated cutpoints. We found that the cutpoint N7 had the highest PPV, Se, and Sp for identifying an anxiety disorder. This result reveals that the optimal cutpoint for detection, with this tool, is 7 [24]. Although several screening instruments such as HADS-A, STAI-T, and HAMA are useful to identify anxiety, there are no validated instruments of anxiety for PWE. The GAD-7 was recently developed in the USA as a valuable screening tool for detecting GAD in PWE [9]. Meanwhile, The ILAE also recommends GAD-7 as the best screening tools for PWE with anxiety, which is in line with our findings [26]. The GAD-7 cutpoint of N7 is recommended because this point appears to have the best balance between Se and Sp, as these two curves, when plotted, converge at this point. Our results show that the GAD-7 cutpoint of N 6 and N 7 have higher Se than Sp [(N6: Sp, 83.4; Se, 94.7) (N 7: Sp, 87.9; Se, 88.1)]. In the 3 studies of GAD-7 that we selected, all used MINI as the reference standard, which can avoid under- or overrepresenting true positive cases, and results in a similar threshold for GAD-7 [27]. In addition, most of the GAD-7 using non-English were verified for accuracy. GAD-7 was translated directly from English into the corresponding language and then reverse-translated back to English to ensure reliability between the two versions, also known as Brislin technology [28]. While this step increases the reliability of the data compared, it must be noted that these different language validations are combined in determining the median and range of diagnostic accuracy estimates [29]. The most common reference standard is the MINI, which has previously tested for SCID in patients with epilepsy [30,36]. The SCID is widely accepted as the gold standard for measuring anxiety in research studies [31,37]; however, it may not be the best tool to use in a clinical setting because it is time-consuming and should be managed by mental health professionals. Although the MINI is generally considered a short screening tool rather than a true “gold standard,” it is understood that the MINI is widely used because it can be managed by trained staff, has face validity, and is highly consistent with SCID [32]. Most importantly, when assessing the diagnostic accuracy of a screening tool, it is important to use an accepted reference or gold standard (e.g., SCID or MINI) rather than another screening tool, as this can lead to inaccurate assessments of anxiety. An important gap in the literature is the lack of studies validating anxiety screening tools for epilepsy in children and adolescents, as no studies out of the 11 that met our criteria included individuals younger than 16. However, the prevalence of anxiety in children with epilepsy can range from 13% to 49% [33]. Future research should develop novel screening tools focused on adolescents and children or modify the traditional adult screening tools, such as the GAD-7, for younger individuals. It is promising that the clinical application of these tools could lead to better detection of anxiety in young patients with epilepsy. In this review, we found cutoff points for screening tools, substantial variability in the language and reference criteria used to evaluate screening tools, and standardized reports lacking diagnostic accuracy estimates, thereby excluding them from the meta-analysis. In addition, AUC statistics and corresponding accuracy estimates are reported more frequently in basic studies; however, these estimates provide less clinical utility because they are more difficult to interpret and apply in clinical practice [34]. Another problem is the lack of reported variability within the estimates themselves. Notably, Se and Sp were reported in 11 studies, of which only 4 reported corresponding confidence intervals; PPV and NPV were reported in 9 studies, and only 5 studies
5
reported corresponding confidence intervals. Additionally, many studies only reported PPV and NPV cutpoints with the best Se and Sp, not with all of the assessed cutpoints. Base rates were also applied for predictive value with the original test characteristics, including Se and Sp, resulting in less generalizable values. Based on these findings, incorporating adequate standards of reporting, including the variability of the estimates, is significant and will lead to higher data quality. This will allow researchers to conduct an in-depth critical appraisal of these studies and provide a greater contribution to the field [35]. The value of research that assesses diagnostic accuracy of screening tools needs to improve, and manuscripts should follow published guidelines to encourage this practice, such as the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement [36]. A comprehensive list of essential items, which should be included in a published manuscript, can be found in the STARD statement. This work used a wide range of search strategies in three large databases without the limitations of publication language or date. We also used a well-documented methodology to search through the entries, including PRISMA standards for reporting and QUADAS-2 for assessing the risk of bias and applicability of included studies [37,38]. As such, we did observe a high risk of bias for the index test domain. In many cases, when the assessment of the screening tool and reference criteria results were blinded, we considered this an issue of unclear reporting rather than a problem of the method itself. Regardless of the outcome of the QUADAS-2 evaluation, all articles were included in the review. We believe that the inclusion of all studies in our analysis will not affect the final conclusion because of the fact that the median estimates of diagnostic accuracy were not associated with a low versus high risk of bias. Although many studies have included epileptic subtypes in their overall sample, they have not stratified the results of these subtypes (e.g., events and epidemics, temporal and frontal lobe, etc.). Future research should focus on the effectiveness of anxiety screening questionnaires on different subtypes of epilepsy. Another potential issue found in this study is the choice to use postmortem cutpoints according to the sample size. By using this method based on sample size to select cutoff points, the tool's effectiveness may be overestimated because of selective reporting, partly due to sampling variability, resulting in better performance [39]. Because of this, the effectiveness of these tools may be overestimated in the studies themselves compared to how they would perform in a clinical setting. Therefore, we recommend that researchers report all of the cutpoints evaluated in their studies, not just the high performing ones. The difficulty in making a recommendation on the “perfect” cutpoint for various screening instruments is another limitation of this analysis. Unfortunately, while previous methods used to conduct systematic reviews were well-organized and thoughtful, comprehensive estimates are still difficult to provide, and consequently, concrete suggestions are unable to be made [39,40].
5. Conclusion To identify a superior anxiety-screening instrument for PWE, many factors should be taken into consideration, including the properties of the tool itself and the requirements of the clinician and healthcare system. Although we lack the evidence to make a definitive recommendation for the best screening tool, our analysis suggests that GAD-7 is likely the most useful screening tool because of it being freely available, relatively simple to implement, and available in many languages. We suggest that individuals choose a PWE anxiety screening tool based on factors known to be important, namely a high Sp, Se, and PPV. Future studies should validate anxiety screening tools against the best possible reference criteria stated by STARD. Moreover, screening tools developed for different age ranges also need to be explored.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035
6
Z. Wang et al. / Epilepsy & Behavior 99 (2019) 106392
Funding This research did not receive any specific grant from funding agencies in the public, or not-for-profit sectors. Declaration of Competing Interest The authors declare that they have on conflict of interest. Supplementary data to this article can be found online at https://doi. org/10.1016/j.yebeh.2019.06.035. References [1] Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. [2] Gaitatzis A, Carroll K, Majeed A, Sander J. The epidemiology of the comorbidity of epilepsy in the general population. Epilepsia 2004;45(12):1613–22. [3] Craske MG, Stein MB. Anxiety. Lancet 2016:3048–59. [4] Wiglusz M, Landowski J, Cubała W. Prevalence of anxiety disorders in epilepsy. Epilepsy Behav 2018;79:1–3. [5] Kanner AM, Schachter SC, Barry JJ, Hersdorffer DC, Mula M, Trimble M, et al. Depression and epilepsy: epidemiologic and neurobiologic perspectives that may explain their high comorbid occurrence. Epilepsy Behav 2012;24:156–68. [6] Tong X, An D, McGonigal A, Park S-P, Zhou D. Validation of the Generalized Anxiety Disorder-7 (GAD-7) among Chinese people with epilepsy. Epilepsy Res 2016;120:31–6. [7] Wiglusz MS, Landowski J, Cubala WJ. Validation of the Polish version of the Hospital Anxiety and Depression Scale for anxiety disorders in patients with epilepsy. Epilepsy Behav 2018;84:162–5. [8] Micoulaud-Franchi J-A, Lagarde S, Barkate G, Dufournet B, Besancon C, TrebuchonDa Fonseca A, et al. Rapid detection of generalized anxiety disorder and major depression in epilepsy: validation of the GAD-7 as a complementary tool to the NDDI-E in a French sample. Epilepsy Behav 2016;57:211–6. [9] Seo J-G, Cho YW, Lee S-J, Lee J-J, Kim J-E, Moon H-J, et al. Validation of the Generalized Anxiety Disorder-7 in people with epilepsy: a MEPSY study. Epilepsy Behav 2014;35:59–63. [10] Kanner AM. Anxiety disorders in epilepsy: the forgotten psychiatric comorbidity. Epilepsy Curr 2011;11(3):90–1. [11] Mumford DB. Hospital anxiety and depression scale[J]. Br J Psychiatry 1991;159(159):729. [12] Wiglusz MS, Landowski J, Cubała WJ. Psychometric properties of the Polish version of the Hamilton Anxiety Rating Scale in patients with epilepsy with and without comorbid anxiety disorder. Epilepsy Behav 2019;94:9–13. [13] Gandy M, Sharpe L, Perry KN, Miller L, Thayer Z, Boserio J, et al. Anxiety in epilepsy: a neglected disorder. J Psychosom Res 2015;78:149–55. [14] Zingano BL, Guarnieri R, Diaz AP, Schwarzbold ML, Wolf P, Lin K, et al. Hospital Anxiety and Depression Scale-Anxiety subscale (HADS-A) and The State-Trait Anxiety Inventory (STAI) accuracy for anxiety disorders detection in drug-resistant mesial temporal lobe epilepsy patients. J Affect Disord 2019;246:452–7. [15] De Marco AP, Mahoney JJ, Aduen PA, Langer J, Bajo SD, Broshek DK. The relationship between the neuro-quality of life depression and anxiety measures and the personality assessment inventory in persons with epilepsy. Epilepsy Behav 2017;70:145–9. [16] Hansen CP, Amiri M. Combined detection of depression and anxiety in epilepsy patients using the neurological disorders depression inventory for epilepsy and the World Health Organization well-being index. Seizure 2015;33:41–5. [17] Wiglusz MS, Landowski J. Psychometric properties and diagnostic utility of the state–trait anxiety inventory in epilepsy with and without comorbid anxiety disorder. Epilepsy Behav 2019;92:221–5. [18] Thombs BD, Benedetti A, Kloda LA, Levis B, Azar M, Riehm KE, et al. Diagnostic accuracy of the Depression subscale of the Hospital Anxiety and Depression Scale (HADSD) for detecting major depression: protocol for a systematic review and individual patient data meta-analyses. BMJ Open 2016;6(4):e011913. [19] Al-Asmi A, Dorvlo AS, Burke DT, Al-Adawi S, Al-Zaabi A, Al-Zadjali HA, et al. The detection of mood and anxiety in people with epilepsy using two-phase designs: experiences from a tertiary care centre in Oman. Epilepsy Res 2012;98:174–81.
[20] Kanner Andres M. Anxiety disorders in epilepsy: the forgotten psychiatric comorbidity. Epilepsy Curr 2011;11:90–1. [21] Kleijnen J, Antes G. Systematic reviews in health care. Meta-analysis in context. Int J Epidemiol 2002;31(3):206–7. [22] Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. J Psychosom Res 2002;52(2):69–77. [23] Stern AF. The Hospital Anxiety and Depression Scale. Occup Med (Chic Ill) 2014;67(6):361. [24] Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166(10):1092–7. [25] Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 2008;46(3):266–74. [26] Parkerson HA, Thibodeau MA, Brandt CP, Zvolensky MJ, Asmundson GJG. Culturalbased biases of the GAD-7. J Anxiety Disord 2015;31:38–42. [27] Seo JG, Park SP. Validation of the Generalized Anxiety Disorder-7 (GAD-7) and GAD2 in patients with migraine. J Headache Pain 2015;16(1):97. [28] Brislin RW. Back-translation for cross-cultural research. J Cross-Cult Psychol 1970;1 (3):185–216. [29] Swinson RP. The GAD-7 scale was accurate for diagnosing generalised anxiety disorder. Evid Based Med 2006;11(6):184. [30] Jones JE, Hermann BP, Barry JJ, Gilliam F, Kanner AM, Meador KJ. Clinical assessment of Axis I psychiatric morbidity in chronic epilepsy: a multicenter investigation. J Neuropsychiatry Clin Neurosci 2014;17(2):172–9. [31] Brown TA, Campbell LA, Lehman CL, Grisham JR, Mancill RB. Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. J Abnorm Psychol 2001;110(4):585. [32] Sheehan DV, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Keskiner A, et al. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry 1997;12(5):232–41. [33] Jones JE, Austin JK, Caplan R, Dunn D, Plioplys S, Salpekar JA. Psychiatric disorders in children and adolescents who have epilepsy. Pediatr Rev 2008;29(2):e9-14. [34] Egger M, Smith GD, Altman DG. Systematic reviews in health care: meta-analysis in context. 2nd ed.; 2008. [35] Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Clin Chem 2015;2015:h5527. [36] Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016;6(11):e012799. [37] Bias AR. QUADAS 2. Am J Gastroenterol 2012;107:1354–60. [38] Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;465. [39] Plummer F, Manea L, Trepel D, McMillan D. Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis. Gen Hosp Psychiatry 2016;39:24–31. [40] Gill SJ, Lukmanji S, Fiest KM, Patten SB, Wiebe S, Jetté N. Depression screening tools in persons with epilepsy: a systematic review of validated tools. Epilepsia 2017;58 (5):695–705. [41] Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67(6):361–70. [42] Skapinakis P. Hospital Anxiety and Depression Scale (HADS) Encycl Qual Life WellBeing Res; 2014. [43] Williams N. The GAD-7 questionnaire. Occup Med (Chic Ill) 2014;64(3):224. [44] STAI. Handb. Dis Burdens Qual Life Meas; 2009. [45] Shahid A, Wilkinson K, Marcu S, Shapiro CM. State-Trait Anxiety Inventory (STAI). STOP, THAT one hundred other sleep scales; 2012; 367–8. [46] Spielberger CD. Manual for the State-Trait Anxiety Inventory (STAI form Y); 1983. [47] Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom 2015;84(3). [48] Calderone FA. World Health Organization. Food Drug Cosmet Law Q; 1947. [49] Hamilton M. Hamilton Anxiety Rating Scale (HAM-A). Br J Med Psychol 1959;32:50–5. [50] Amorim P, Lecrubier Y, Weiller E, et al. DSM-IH-R psychotic disorders: procedural validity of the Mini International Neuropsychiatric Interview (MINI). Concordance and causes for discordance with the CIDI. Eur Psychiatry 1998;13:26–34.
Please cite this article as: Z. Wang, Z. Luo, S. Li, et al., Anxiety screening tools in people with epilepsy: A systematic review of validated tools, Epilepsy & Behavior, https://doi.org/10.1016/j.yebeh.2019.06.035