- Email: [email protected]
Attributing seizures to TBI: Validation of a brief patient questionnaire
Epilepsy & Behavior 57 (2016) 141–144
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Attributing seizures to TBI: Validation of a brief patient questionnaire Martin Salinsky a,b,⁎, Karen Parko c, Paul Rutecki d, Eilis Boudreau a,b, Daniel Storzbach a a
Portland Veterans Affairs Medical Center, Portland, OR, United States Oregon Health & Sciences University, Portland, OR, United States c San Francisco Veterans Affairs Medical Center, San Francisco, CA, United States d William S. Middleton Veterans Affairs Medical Center, Madison, WI, United States b
a r t i c l e
i n f o
Article history: Received 16 November 2015 Revised 2 February 2016 Accepted 5 February 2016 Available online 5 March 2016 Keywords: Psychogenic seizures Epilepsy TBI Veterans
a b s t r a c t Purpose: Traumatic brain injury (TBI) is an important cause of epilepsy and has also been associated with psychogenic nonepileptic seizures (PNES). We designed a brief questionnaire assessing patient beliefs regarding TBI as the cause of their seizures (Patient Seizure Etiology Questionnaire; PSEQ). This study reports content validity for the PSEQ. Methods: Ninety Veterans undergoing comprehensive evaluation at 3 VA epilepsy centers completed the PSEQ, a series of questions regarding possible causes for their seizures, including TBI. The PSEQ was scored as YES vs. NO for TBI as the proposed cause of seizures. For each patient, two expert reviewers independently completed a structured chart review to determine whether TBI was the proposed cause of seizures (n = 180 reviews). Kappa statistic was used to assess agreement between the PSEQ and each chart review and between the PSEQ and combined chart reviews where both reviewers agreed on a TBI seizure etiology. Results: The PSEQ scored higher overall rates for a TBI seizure etiology than did expert chart reviews (40% vs. 28%; p b 0.001). The PSEQ agreed with 82% of 180 independent chart reviews (sensitivity 88%; specificity 79%). Kappa statistic for agreement was 0.60. The two reviewers agreed on a probable TBI seizure etiology for 83% of chart reviews. The PSEQ sensitivity increased to 100% when both reviewers were in agreement. Conclusion: The PSEQ provides a direct, standardized measure of patient beliefs regarding TBI as the cause of their seizures and has moderate–substantial agreement with expert chart reviews. Published by Elsevier Inc.
1. Rationale Traumatic Brain Injury (TBI) is a leading cause of epilepsy and is of particular importance in military settings because of high long-term seizure rates [1–4]. Traumatic brain injury has also been associated with psychogenic nonepileptic seizures (PNES). In civilians ultimately diagnosed with PNES, seizures were attributed to TBI in 20–36% of cases [5–8]. In Veterans, TBI was the proposed cause for seizures in 56% of patients subsequently diagnosed with PNES [9]. Identifying patient beliefs regarding the cause of their seizures is of particular importance with PNES, as these beliefs may be part of the process resulting in seizures. For example, a pathway of mild TBI moderated by posttraumatic stress disorder (PTSD) has been associated with the development of PNES in Veterans [9].
⁎ Corresponding author at: Portland VAMC Epilepsy Center of Excellence, 3710 SW US Veterans Hospital Road (P3ECOE), Portland, OR 97239, United States. Tel.: +1 503 494 5682; fax: +1 503 494 6658. E-mail address: [email protected] (M. Salinsky).
http://dx.doi.org/10.1016/j.yebeh.2016.02.003 1525-5050/Published by Elsevier Inc.
Studies investigating the link between TBI and PNES have generally relied on patient history as reflected in chart documentation [5–8]. However, chart documentation of TBI and the relationship of TBI to seizures can be inconsistent. Specific questions regarding TBI and seizures may not have been asked, may have been asked in a variety of ways, and may not have been clearly documented. Assigning a ‘principal’ cause for seizures can be difficult when multiple risk factors are documented. Also, it can be difficult to separate patient beliefs from the opinion of the provider. A review of primary TBI documentation is an alternative, but the information is often not available. This is particularly the case for Veterans where military TBI documentation cannot be routinely accessed. To obtain a more consistent measure of patient beliefs regarding TBI as the cause of their seizures, we designed a patient-based seizure etiology questionnaire (PSEQ). The PSEQ was developed for use in patientbased PNES research studies where beliefs regarding TBI as a cause of seizures could be a factor influencing seizure development [9]. The current study was performed to validate PSEQ content against chart documentation in Veterans undergoing epilepsy monitoring unit (EMU) evaluation for seizures. We also measured agreement between two independent expert reviewers.
142
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144
2. Methods
2.4. Statistical analysis
2.1. Patients
Scores from the PSEQ and the chart reviews were entered into a SAS dataset for analysis. The planned primary analysis was the extent of agreement between the PSEQ scores (YES; NO) and the expert chart reviews (YES; NO). Agreement was measured in two ways: (1) comparison of PSEQ scores to each expert chart review (N = 180 independent chart reviews for 90 patients) and (2) comparison of PSEQ scores to chart reviews for which reviewer responses were combined (N = 90). For the combined comparison, only those patients for whom both reviewers agreed on a YES response were coded as YES. Otherwise, the combined reviewer responses were coded as NO. Agreement between the PSEQ and chart review scores, and between the two reviewers, was assessed by Kappa statistic [10,11]. Fisher's exact test was used to compare the frequency of YES responses in the two methods.
Patients were evaluated in the EMUs of the Portland Oregon, Madison Wisconsin, and San Francisco California Veterans Affairs Medical Centers. Thirty Veterans were evaluated at each site (N = 90). All were participating in a multicenter study of seizures in Veterans. Evaluations included the PSEQ. The first 30 patients completing the PSEQ at each site were selected for study. Patient selection was not contingent upon the EMU discharge diagnosis or other clinical characteristics. The EMU discharge diagnoses were classified as: (1) epileptic seizures only; (2) psychogenic nonepileptic seizures only; (3) mixed epileptic and psychogenic seizures; (4) other nonepileptic events; and (5) inconclusive [8]. 2.2. PSEQ The PSEQ was completed by patients independently after brief instruction. The questionnaire asked patients to rate how strongly they believed each of 11 possible seizure etiologies was the original cause of their seizures. A 5-point Likert scale for each item included (0) ‘No’, (1) ‘unlikely’, (2) ‘possibly’, (3) ‘probably’, and (4) ‘definitely’ (Appendix A). Although the questionnaire covered a broad range of etiologies, it was designed to evaluate TBI as the proposed cause of seizures. Traumatic brain injury was covered in two items, one for ‘military TBI’ and one for ‘civilian TBI’. The overall PSEQ was scored as ‘YES’ (TBI was the primary cause for the seizures) vs. ‘NO’ using criteria developed from a previous pilot study. ‘YES’ required a score of ‘probably’ or ‘definitely’ for either the ‘civilian TBI’ or ‘military TBI’ questions, with no other etiology score higher than the highest TBI score. If the PSEQ did not meet criteria for ‘YES’, it was scored ‘NO’. We divided ‘YES’ scores into those generated by the military TBI question, those generated by the civilian TBI question, and those where the military and civilian questions tied. 2.3. Chart reviews At each site, two providers specializing in epilepsy care independently reviewed the EMU admission chart notes and additional outpatient notes as necessary. A reviewer response form asked two questions: whether chart notes indicated that the patient believed TBI to be the primary cause of the seizures and whether the attending provider's formulation indicated that TBI was the primary cause of the seizures. There were four possible responses to each question: (1) YES (TBI was the probable cause for the seizures); (2) NO (no cause was specified); (3) NO (another cause was specified); and (4) NO (TBI was among the possibilities but not sufficient for assigning probable cause). For statistical analyses, the three ‘NO’ categories were combined. A YES response required specifying whether the TBI was civilian, military, or both. Reviewers did not have access to the PSEQ results and were asked not to discuss their impressions with the other reviewer at their site. There were two independent reviews of 30 charts at each of the three sites (180 independent chart reviews). To assure consistent evaluations, each reviewer underwent training prior to reviewing patient charts. Reviewers read several brief case scenarios with explanations of the response form completion. Each reviewer was then required to pass an examination, completing response forms for 10 additional fictional cases (with ≥ 90% correct scoring). Scoring results from the chart reviews were simplified to YES (TBI was thought to be the primary cause of seizures) or NO for each of the 2 response questions. To simplify analysis, a YES response to either of the two questions was considered an overall YES response for that chart review.
2.5. Protocol approval, patient consent This study was approved by the VA Central Institutional Review Board. All patients provided informed consent. 3. Results 3.1. EMU diagnoses The EMU discharge diagnoses for the 90 patients are provided in Fig. 1. The distribution of diagnoses was nearly identical to that of a 2013–14 national survey of VA Epilepsy Center EMU discharge diagnoses (N = 705) [12]. 3.2. PSEQ results Individual item responses for the PSEQ are presented in Table 1. The PSEQ scored 36/90 (40%) YES responses for a TBI seizure etiology and 54 (60%) NO responses (Fig. 2). Twenty-one YES scores were for a military TBI, 10 were for a civilian TBI, and 5 were for both. Five of 26 patients (19%) with an EMU discharge diagnosis of epileptic seizures scored YES on the PSEQ; and 7/18 patients (39%) with an EMU diagnosis of PNES scored YES (ns; Fisher's exact test). 3.3. Chart review results There were 2 independent chart reviews per patient (180 reviews for 90 patients). Reviewers scored 51 charts (28%) as YES (probable TBI seizure etiology) and the remainder as NO. Fifty of the YES scores were based on the patient's report as reflected in the chart notes (33 reviews) or both the patient's report and the formulation of the admitting clinician (17 reviews). In only one case was the YES response based
Epileptic 29%
PNES Mixed
42%
Other NES Inconclusive 20% 8% 1% Fig. 1. EMU discharge diagnoses for the 90 patients. PNES — psychogenic nonepileptic seizures. Other NES — nonepileptic seizures of physiologic origin.
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144 Table 1 Individual item response summary for the PSEQ (n = 90 patients). Each number is the percentage of patients providing the response.
Table 2 Agreement between the PSEQ scores and chart review scores. Each score assessed TBI as the primary seizure etiology.
PSEQ item TBI military TBI civilian Stress Neurological Medication Family Poison Alcohol Premature Surgery Infection
143
PSEQ score No
Unlikely
Possibly
Probably
Definitely
43.3 50.0 24.4 53.3 58.9 73.3 55.6 74.4 86.6 91.1 81.1
8.9 17.8 25.6 10.0 16.7 15.6 14.4 14.4 6.7 2.2 8.9
18.9 15.6 28.9 24.4 17.8 6.7 22.2 7.8 4.4 6.7 10.0
11.1 11.1 16.7 6.7 6.7 2.2 6.7 3.3 1.1 0.0 0.0
17.8 5.6 4.4 5.6 0.0 2.2 1.1 0.0 1.1 0.0 0.0
Chart review score
NO
YES
Total
NO Row percent Column percent YES Row percent Column percent Total
102 79.1 94.4 6 11.8 5.6 108
27 20.9 37.5 45 88.2 62.3 72
129
51
180
Table 3 Agreement between the combined chart review scores and PSEQ scores. PSEQ score
solely on the attending clinician's formulation. The most common reason for a NO response was that no seizure etiology was specified in the chart notes (55% of all NO responses). The 2 reviewers agreed on the YES vs. NO score for 75 of 90 charts (83%; 18 YES responses, 57 NO responses). The Kappa coefficient was 0.59 (moderate–substantial agreement). The chart reviewers disagreed on chart scoring for nine patients categorized as having a TBI seizure etiology on the PSEQ and six patients categorized as not having a TBI seizure etiology on the PSEQ. 3.4. Agreement between PSEQ and chart reviews (primary analysis) 3.4.1. PSEQ vs. individual chart reviews (N = 180) YES scores were more frequent on the PSEQ as opposed to chart reviews (40% vs. 28%; p b 0.001). The overall agreement between the PSEQ scoring (YES:NO) and chart review scoring (YES:NO) is provided in Table 2. The table includes all 180 independent reviews for the 90 patients. The PSEQ agreed with the chart review scoring for 147/180 (81.7%) reviews, including 45/51 YES responses (sensitivity 88.2%) and 102 of 129 NO responses (specificity 79.1%). The false positive rate was 37.5% and the false negative rate 5.6%. The Kappa coefficient for agreement was 0.60. 3.4.2. PSEQ vs. combined reviewers scoring (N = 90) For this analysis, we considered a chart review score as YES (probable TBI seizure etiology) only if both reviewers scored the patient as YES. Otherwise, the patient was scored as NO. Results for agreement with the PSEQ are shown in Table 3. The PSEQ agreed with the combined reviewer's scoring 75% of the time (72/90). Agreement was 100% (sensitivity) for the combined reviewer YES responses and 60% (specificity) for NO responses. The false positive rate was 50% and the false negative rate 0%. Kappa coefficient was 0.55. 4. Conclusions The PSEQ provides a direct measure of a patient's beliefs regarding TBI as the primary cause of their seizures. The results demonstrate
90 patients
36 TBI 21 Military TBI
10 Civilian TBI
54 NO TBI
5 Both
Fig. 2. PSEQ scoring for all patients.
Chart review score (reviewers combined)
NO
YES
Total
NO Row percent Column percent YES Row percent Column percent Total
54 75.0 100.0 0 0.0 0.0 54
18 25.0 50.0 18 100.0 50.0 36
72
18
90
that the PSEQ will generally agree with expert chart review in assigning a probable TBI etiology for a patient's seizures. Agreement was found for 82% of 180 chart reviews with a sensitivity of 88%. The PSEQ sensitivity increased to 100% when two independent chart reviewers agreed on a TBI seizure etiology. The overall agreement rate was similar to that found between the two expert reviewers at each site (83%). Our study design has limitations. We chose to compare the PSEQ with expert review of chart documentation since chart review has been used in studies of the relationship between TBI and PNES [6,7,9]. However, chart documentation is not an ideal comparator. Chart documentation of TBI and its relationship to seizures primarily reflects history provided by the patient, as noted by our reviewers. However, patient interviews are generally not structured. Questions about TBI may or may not have been asked and may have been framed in a variety of ways. Patient responses may be misinterpreted by the interviewer prior to documentation or may not be documented at all, leading to the impression that the patient did not implicate TBI as the cause of their seizures. It is due to these concerns that we developed the PSEQ. A structured interview covering each of the PSEQ items would have provided a more reliable standard for validation of the PSEQ. The PSEQ may be particularly useful for research studies in which patient attribution of a TBI seizure etiology is of interest. Civilian studies have shown that mild TBI is commonly cited as the cause for seizures later proven to be PNES [6,7]. In Veterans, the well-known association between TBI and the development of epilepsy may facilitate the development of PNES following mild TBI, particularly in those suffering from PTSD [9,13]. In the present study, Veterans diagnosed with PNES were twice as likely to indicate that TBI was the cause of their seizures as compared with Veterans with epileptic seizures, results similar to those of a previous retrospective study [9]. The PSEQ may potentially be useful in clinical settings as well, providing a standardized method of determining patient beliefs regarding the cause of their seizures. Conflict of interest None of the authors report any direct or indirect conflicts of interest regarding this research.
144
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144
Acknowledgment The authors thank S. Chen FNP, R. Kotloski MD PhD, and C. Evrard FNP for their participation in this project. This work was supported by grant # 5101CX000721-04 from the Department of Veterans Affairs.
Appendix A. Patient Seizure Etiology Questionnaire (PSEQ)
References [1] Raymont V, Salizar AM, Lipsky R, Goldman D, Tasick G, Grafman J. Correlates of posttraumatic epilepsy 35 years after combat head injury. Neurology 2010;75: 224–9. [2] Annegers JF. The epidemiology of epilepsy. In: Wylie E, editor. The treatment of epilepsy: principles and practice. Baltimore, Md: Williams and Wilkins; 1996. p. 166–72. [3] Pugh MJ, Orman JA, Jaramillo CA, Salinsky MC, Eapen BC, Towne AR, et al. The prevalence of epilepsy and association with traumatic brain injury in Veterans of the Afghanistan and Iraq wars. J Head Trauma Rehabil 2015;30:29–37. [4] Mahler B, Carlsson S, Andersson T, Adelow C, Ahlbom A, Tomson T. Unprovoked seizures after traumatic brain injury: a population-based case–control study. Epilepsia 2015;56:1438–44. [5] LaFrance Jr WC, Deluca M, Machan JT, Fava JL. Traumatic brain injury and psychogenic nonepileptic seizures yield worse outcomes. Epilepsia 2013;54:718–25. [6] Westbrook LE, Devinsky O, Geocadin R. Nonepileptic seizures after head injury. Epilepsia 1998;39:978–82. [7] Barry E, Krumholz A, Bergey GK, Chatha H, Alemayehu S, Grattan G. Nonepileptic posttraumatic seizures. Epilepsia 1998;39:427–31. [8] Salinsky MC, Spencer D, Ferguson F, Boudreau E. Psychogenic seizures in U.S. Veterans. Neurology 2011;77:945–50. [9] Salinsky MC, Storzbach D, Goy E, Evrard C. Traumatic brain injury and psychogenic seizures in Veterans. J Head Trauma Rehabil 2015;30:e65–70. [10] Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of Kappa and weighted Kappa. Psychol Bull 1969;72:323–7. [11] Landis JR, Kochen S. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [12] Epilepsy Centers of Excellence Annual Report 2014; 2014. http://www.epilepsy.va. gov/About_the_ECoEs.asp. [13] Salinsky MC, Evrard C, Storzbach D, Pugh MJ. Psychiatric comorbidity in Veterans with psychogenic seizures. Epilepsy Behav 2012;25:345–9.
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Attributing seizures to TBI: Validation of a brief patient questionnaire Martin Salinsky a,b,⁎, Karen Parko c, Paul Rutecki d, Eilis Boudreau a,b, Daniel Storzbach a a
Portland Veterans Affairs Medical Center, Portland, OR, United States Oregon Health & Sciences University, Portland, OR, United States c San Francisco Veterans Affairs Medical Center, San Francisco, CA, United States d William S. Middleton Veterans Affairs Medical Center, Madison, WI, United States b
a r t i c l e
i n f o
Article history: Received 16 November 2015 Revised 2 February 2016 Accepted 5 February 2016 Available online 5 March 2016 Keywords: Psychogenic seizures Epilepsy TBI Veterans
a b s t r a c t Purpose: Traumatic brain injury (TBI) is an important cause of epilepsy and has also been associated with psychogenic nonepileptic seizures (PNES). We designed a brief questionnaire assessing patient beliefs regarding TBI as the cause of their seizures (Patient Seizure Etiology Questionnaire; PSEQ). This study reports content validity for the PSEQ. Methods: Ninety Veterans undergoing comprehensive evaluation at 3 VA epilepsy centers completed the PSEQ, a series of questions regarding possible causes for their seizures, including TBI. The PSEQ was scored as YES vs. NO for TBI as the proposed cause of seizures. For each patient, two expert reviewers independently completed a structured chart review to determine whether TBI was the proposed cause of seizures (n = 180 reviews). Kappa statistic was used to assess agreement between the PSEQ and each chart review and between the PSEQ and combined chart reviews where both reviewers agreed on a TBI seizure etiology. Results: The PSEQ scored higher overall rates for a TBI seizure etiology than did expert chart reviews (40% vs. 28%; p b 0.001). The PSEQ agreed with 82% of 180 independent chart reviews (sensitivity 88%; specificity 79%). Kappa statistic for agreement was 0.60. The two reviewers agreed on a probable TBI seizure etiology for 83% of chart reviews. The PSEQ sensitivity increased to 100% when both reviewers were in agreement. Conclusion: The PSEQ provides a direct, standardized measure of patient beliefs regarding TBI as the cause of their seizures and has moderate–substantial agreement with expert chart reviews. Published by Elsevier Inc.
1. Rationale Traumatic Brain Injury (TBI) is a leading cause of epilepsy and is of particular importance in military settings because of high long-term seizure rates [1–4]. Traumatic brain injury has also been associated with psychogenic nonepileptic seizures (PNES). In civilians ultimately diagnosed with PNES, seizures were attributed to TBI in 20–36% of cases [5–8]. In Veterans, TBI was the proposed cause for seizures in 56% of patients subsequently diagnosed with PNES [9]. Identifying patient beliefs regarding the cause of their seizures is of particular importance with PNES, as these beliefs may be part of the process resulting in seizures. For example, a pathway of mild TBI moderated by posttraumatic stress disorder (PTSD) has been associated with the development of PNES in Veterans [9].
⁎ Corresponding author at: Portland VAMC Epilepsy Center of Excellence, 3710 SW US Veterans Hospital Road (P3ECOE), Portland, OR 97239, United States. Tel.: +1 503 494 5682; fax: +1 503 494 6658. E-mail address: [email protected] (M. Salinsky).
http://dx.doi.org/10.1016/j.yebeh.2016.02.003 1525-5050/Published by Elsevier Inc.
Studies investigating the link between TBI and PNES have generally relied on patient history as reflected in chart documentation [5–8]. However, chart documentation of TBI and the relationship of TBI to seizures can be inconsistent. Specific questions regarding TBI and seizures may not have been asked, may have been asked in a variety of ways, and may not have been clearly documented. Assigning a ‘principal’ cause for seizures can be difficult when multiple risk factors are documented. Also, it can be difficult to separate patient beliefs from the opinion of the provider. A review of primary TBI documentation is an alternative, but the information is often not available. This is particularly the case for Veterans where military TBI documentation cannot be routinely accessed. To obtain a more consistent measure of patient beliefs regarding TBI as the cause of their seizures, we designed a patient-based seizure etiology questionnaire (PSEQ). The PSEQ was developed for use in patientbased PNES research studies where beliefs regarding TBI as a cause of seizures could be a factor influencing seizure development [9]. The current study was performed to validate PSEQ content against chart documentation in Veterans undergoing epilepsy monitoring unit (EMU) evaluation for seizures. We also measured agreement between two independent expert reviewers.
142
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144
2. Methods
2.4. Statistical analysis
2.1. Patients
Scores from the PSEQ and the chart reviews were entered into a SAS dataset for analysis. The planned primary analysis was the extent of agreement between the PSEQ scores (YES; NO) and the expert chart reviews (YES; NO). Agreement was measured in two ways: (1) comparison of PSEQ scores to each expert chart review (N = 180 independent chart reviews for 90 patients) and (2) comparison of PSEQ scores to chart reviews for which reviewer responses were combined (N = 90). For the combined comparison, only those patients for whom both reviewers agreed on a YES response were coded as YES. Otherwise, the combined reviewer responses were coded as NO. Agreement between the PSEQ and chart review scores, and between the two reviewers, was assessed by Kappa statistic [10,11]. Fisher's exact test was used to compare the frequency of YES responses in the two methods.
Patients were evaluated in the EMUs of the Portland Oregon, Madison Wisconsin, and San Francisco California Veterans Affairs Medical Centers. Thirty Veterans were evaluated at each site (N = 90). All were participating in a multicenter study of seizures in Veterans. Evaluations included the PSEQ. The first 30 patients completing the PSEQ at each site were selected for study. Patient selection was not contingent upon the EMU discharge diagnosis or other clinical characteristics. The EMU discharge diagnoses were classified as: (1) epileptic seizures only; (2) psychogenic nonepileptic seizures only; (3) mixed epileptic and psychogenic seizures; (4) other nonepileptic events; and (5) inconclusive [8]. 2.2. PSEQ The PSEQ was completed by patients independently after brief instruction. The questionnaire asked patients to rate how strongly they believed each of 11 possible seizure etiologies was the original cause of their seizures. A 5-point Likert scale for each item included (0) ‘No’, (1) ‘unlikely’, (2) ‘possibly’, (3) ‘probably’, and (4) ‘definitely’ (Appendix A). Although the questionnaire covered a broad range of etiologies, it was designed to evaluate TBI as the proposed cause of seizures. Traumatic brain injury was covered in two items, one for ‘military TBI’ and one for ‘civilian TBI’. The overall PSEQ was scored as ‘YES’ (TBI was the primary cause for the seizures) vs. ‘NO’ using criteria developed from a previous pilot study. ‘YES’ required a score of ‘probably’ or ‘definitely’ for either the ‘civilian TBI’ or ‘military TBI’ questions, with no other etiology score higher than the highest TBI score. If the PSEQ did not meet criteria for ‘YES’, it was scored ‘NO’. We divided ‘YES’ scores into those generated by the military TBI question, those generated by the civilian TBI question, and those where the military and civilian questions tied. 2.3. Chart reviews At each site, two providers specializing in epilepsy care independently reviewed the EMU admission chart notes and additional outpatient notes as necessary. A reviewer response form asked two questions: whether chart notes indicated that the patient believed TBI to be the primary cause of the seizures and whether the attending provider's formulation indicated that TBI was the primary cause of the seizures. There were four possible responses to each question: (1) YES (TBI was the probable cause for the seizures); (2) NO (no cause was specified); (3) NO (another cause was specified); and (4) NO (TBI was among the possibilities but not sufficient for assigning probable cause). For statistical analyses, the three ‘NO’ categories were combined. A YES response required specifying whether the TBI was civilian, military, or both. Reviewers did not have access to the PSEQ results and were asked not to discuss their impressions with the other reviewer at their site. There were two independent reviews of 30 charts at each of the three sites (180 independent chart reviews). To assure consistent evaluations, each reviewer underwent training prior to reviewing patient charts. Reviewers read several brief case scenarios with explanations of the response form completion. Each reviewer was then required to pass an examination, completing response forms for 10 additional fictional cases (with ≥ 90% correct scoring). Scoring results from the chart reviews were simplified to YES (TBI was thought to be the primary cause of seizures) or NO for each of the 2 response questions. To simplify analysis, a YES response to either of the two questions was considered an overall YES response for that chart review.
2.5. Protocol approval, patient consent This study was approved by the VA Central Institutional Review Board. All patients provided informed consent. 3. Results 3.1. EMU diagnoses The EMU discharge diagnoses for the 90 patients are provided in Fig. 1. The distribution of diagnoses was nearly identical to that of a 2013–14 national survey of VA Epilepsy Center EMU discharge diagnoses (N = 705) [12]. 3.2. PSEQ results Individual item responses for the PSEQ are presented in Table 1. The PSEQ scored 36/90 (40%) YES responses for a TBI seizure etiology and 54 (60%) NO responses (Fig. 2). Twenty-one YES scores were for a military TBI, 10 were for a civilian TBI, and 5 were for both. Five of 26 patients (19%) with an EMU discharge diagnosis of epileptic seizures scored YES on the PSEQ; and 7/18 patients (39%) with an EMU diagnosis of PNES scored YES (ns; Fisher's exact test). 3.3. Chart review results There were 2 independent chart reviews per patient (180 reviews for 90 patients). Reviewers scored 51 charts (28%) as YES (probable TBI seizure etiology) and the remainder as NO. Fifty of the YES scores were based on the patient's report as reflected in the chart notes (33 reviews) or both the patient's report and the formulation of the admitting clinician (17 reviews). In only one case was the YES response based
Epileptic 29%
PNES Mixed
42%
Other NES Inconclusive 20% 8% 1% Fig. 1. EMU discharge diagnoses for the 90 patients. PNES — psychogenic nonepileptic seizures. Other NES — nonepileptic seizures of physiologic origin.
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144 Table 1 Individual item response summary for the PSEQ (n = 90 patients). Each number is the percentage of patients providing the response.
Table 2 Agreement between the PSEQ scores and chart review scores. Each score assessed TBI as the primary seizure etiology.
PSEQ item TBI military TBI civilian Stress Neurological Medication Family Poison Alcohol Premature Surgery Infection
143
PSEQ score No
Unlikely
Possibly
Probably
Definitely
43.3 50.0 24.4 53.3 58.9 73.3 55.6 74.4 86.6 91.1 81.1
8.9 17.8 25.6 10.0 16.7 15.6 14.4 14.4 6.7 2.2 8.9
18.9 15.6 28.9 24.4 17.8 6.7 22.2 7.8 4.4 6.7 10.0
11.1 11.1 16.7 6.7 6.7 2.2 6.7 3.3 1.1 0.0 0.0
17.8 5.6 4.4 5.6 0.0 2.2 1.1 0.0 1.1 0.0 0.0
Chart review score
NO
YES
Total
NO Row percent Column percent YES Row percent Column percent Total
102 79.1 94.4 6 11.8 5.6 108
27 20.9 37.5 45 88.2 62.3 72
129
51
180
Table 3 Agreement between the combined chart review scores and PSEQ scores. PSEQ score
solely on the attending clinician's formulation. The most common reason for a NO response was that no seizure etiology was specified in the chart notes (55% of all NO responses). The 2 reviewers agreed on the YES vs. NO score for 75 of 90 charts (83%; 18 YES responses, 57 NO responses). The Kappa coefficient was 0.59 (moderate–substantial agreement). The chart reviewers disagreed on chart scoring for nine patients categorized as having a TBI seizure etiology on the PSEQ and six patients categorized as not having a TBI seizure etiology on the PSEQ. 3.4. Agreement between PSEQ and chart reviews (primary analysis) 3.4.1. PSEQ vs. individual chart reviews (N = 180) YES scores were more frequent on the PSEQ as opposed to chart reviews (40% vs. 28%; p b 0.001). The overall agreement between the PSEQ scoring (YES:NO) and chart review scoring (YES:NO) is provided in Table 2. The table includes all 180 independent reviews for the 90 patients. The PSEQ agreed with the chart review scoring for 147/180 (81.7%) reviews, including 45/51 YES responses (sensitivity 88.2%) and 102 of 129 NO responses (specificity 79.1%). The false positive rate was 37.5% and the false negative rate 5.6%. The Kappa coefficient for agreement was 0.60. 3.4.2. PSEQ vs. combined reviewers scoring (N = 90) For this analysis, we considered a chart review score as YES (probable TBI seizure etiology) only if both reviewers scored the patient as YES. Otherwise, the patient was scored as NO. Results for agreement with the PSEQ are shown in Table 3. The PSEQ agreed with the combined reviewer's scoring 75% of the time (72/90). Agreement was 100% (sensitivity) for the combined reviewer YES responses and 60% (specificity) for NO responses. The false positive rate was 50% and the false negative rate 0%. Kappa coefficient was 0.55. 4. Conclusions The PSEQ provides a direct measure of a patient's beliefs regarding TBI as the primary cause of their seizures. The results demonstrate
90 patients
36 TBI 21 Military TBI
10 Civilian TBI
54 NO TBI
5 Both
Fig. 2. PSEQ scoring for all patients.
Chart review score (reviewers combined)
NO
YES
Total
NO Row percent Column percent YES Row percent Column percent Total
54 75.0 100.0 0 0.0 0.0 54
18 25.0 50.0 18 100.0 50.0 36
72
18
90
that the PSEQ will generally agree with expert chart review in assigning a probable TBI etiology for a patient's seizures. Agreement was found for 82% of 180 chart reviews with a sensitivity of 88%. The PSEQ sensitivity increased to 100% when two independent chart reviewers agreed on a TBI seizure etiology. The overall agreement rate was similar to that found between the two expert reviewers at each site (83%). Our study design has limitations. We chose to compare the PSEQ with expert review of chart documentation since chart review has been used in studies of the relationship between TBI and PNES [6,7,9]. However, chart documentation is not an ideal comparator. Chart documentation of TBI and its relationship to seizures primarily reflects history provided by the patient, as noted by our reviewers. However, patient interviews are generally not structured. Questions about TBI may or may not have been asked and may have been framed in a variety of ways. Patient responses may be misinterpreted by the interviewer prior to documentation or may not be documented at all, leading to the impression that the patient did not implicate TBI as the cause of their seizures. It is due to these concerns that we developed the PSEQ. A structured interview covering each of the PSEQ items would have provided a more reliable standard for validation of the PSEQ. The PSEQ may be particularly useful for research studies in which patient attribution of a TBI seizure etiology is of interest. Civilian studies have shown that mild TBI is commonly cited as the cause for seizures later proven to be PNES [6,7]. In Veterans, the well-known association between TBI and the development of epilepsy may facilitate the development of PNES following mild TBI, particularly in those suffering from PTSD [9,13]. In the present study, Veterans diagnosed with PNES were twice as likely to indicate that TBI was the cause of their seizures as compared with Veterans with epileptic seizures, results similar to those of a previous retrospective study [9]. The PSEQ may potentially be useful in clinical settings as well, providing a standardized method of determining patient beliefs regarding the cause of their seizures. Conflict of interest None of the authors report any direct or indirect conflicts of interest regarding this research.
144
M. Salinsky et al. / Epilepsy & Behavior 57 (2016) 141–144
Acknowledgment The authors thank S. Chen FNP, R. Kotloski MD PhD, and C. Evrard FNP for their participation in this project. This work was supported by grant # 5101CX000721-04 from the Department of Veterans Affairs.
Appendix A. Patient Seizure Etiology Questionnaire (PSEQ)
References [1] Raymont V, Salizar AM, Lipsky R, Goldman D, Tasick G, Grafman J. Correlates of posttraumatic epilepsy 35 years after combat head injury. Neurology 2010;75: 224–9. [2] Annegers JF. The epidemiology of epilepsy. In: Wylie E, editor. The treatment of epilepsy: principles and practice. Baltimore, Md: Williams and Wilkins; 1996. p. 166–72. [3] Pugh MJ, Orman JA, Jaramillo CA, Salinsky MC, Eapen BC, Towne AR, et al. The prevalence of epilepsy and association with traumatic brain injury in Veterans of the Afghanistan and Iraq wars. J Head Trauma Rehabil 2015;30:29–37. [4] Mahler B, Carlsson S, Andersson T, Adelow C, Ahlbom A, Tomson T. Unprovoked seizures after traumatic brain injury: a population-based case–control study. Epilepsia 2015;56:1438–44. [5] LaFrance Jr WC, Deluca M, Machan JT, Fava JL. Traumatic brain injury and psychogenic nonepileptic seizures yield worse outcomes. Epilepsia 2013;54:718–25. [6] Westbrook LE, Devinsky O, Geocadin R. Nonepileptic seizures after head injury. Epilepsia 1998;39:978–82. [7] Barry E, Krumholz A, Bergey GK, Chatha H, Alemayehu S, Grattan G. Nonepileptic posttraumatic seizures. Epilepsia 1998;39:427–31. [8] Salinsky MC, Spencer D, Ferguson F, Boudreau E. Psychogenic seizures in U.S. Veterans. Neurology 2011;77:945–50. [9] Salinsky MC, Storzbach D, Goy E, Evrard C. Traumatic brain injury and psychogenic seizures in Veterans. J Head Trauma Rehabil 2015;30:e65–70. [10] Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of Kappa and weighted Kappa. Psychol Bull 1969;72:323–7. [11] Landis JR, Kochen S. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [12] Epilepsy Centers of Excellence Annual Report 2014; 2014. http://www.epilepsy.va. gov/About_the_ECoEs.asp. [13] Salinsky MC, Evrard C, Storzbach D, Pugh MJ. Psychiatric comorbidity in Veterans with psychogenic seizures. Epilepsy Behav 2012;25:345–9.