- Email: [email protected]
Examination of volunteer bias in research involving patients diagnosed with psychogenic nonepileptic seizures
Epilepsy & Behavior 15 (2009) 524–526
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Brief Communication
Examination of volunteer bias in research involving patients diagnosed with psychogenic nonepileptic seizures Alex M. Taylor a,b, Deborah A. Cahn-Weiner b,*, Paul A. Garcia b a b
Alliant International University, San Francisco, CA, USA Department of Neurology, University of California, San Francisco, CA, USA
a r t i c l e
i n f o
Article history: Received 20 March 2009 Revised 8 May 2009 Accepted 7 June 2009 Available online 16 July 2009 Keywords: Psychogenic nonepileptic seizures Epileptic seizures Volunteer bias Methodology External validity Clinical trials
a b s t r a c t There is some evidence to suggest that the same personality and emotional features observed in patients diagnosed with psychogenic nonepileptic seizures (PNES) may be related to nonparticipation in studies aimed at determining the most effective interventions for treating this patient population. This study aimed to document the frequency with which patients referred to an epilepsy monitoring unit (EMU) for video/EEG telemetry who are diagnosed with PNES or epileptic seizures (ES) participate in survey research. Of the 139 patients approached, 16 patients declined and 23 produced invalid or incomplete surveys. There was no significant difference in participation rates between patients diagnosed with PNES and those diagnosed with ES. These findings argue against a volunteer bias in research involving PNES and suggest that the results of other studies with these patients generalize to the population as a whole. Ó 2009 Elsevier Inc. All rights reserved.
1. Introduction The personality characteristics and mood features associated with psychogenic nonepileptic seizures (PNES) have been related to lower quality of life (QOL) when compared with patients diagnosed with ES [1–4]. Presumably, a better understanding of the personality and mood features associated with PNES would inform clinicians of the most targeted efficacious treatments that might help reduce the symptoms of this disorder as well as underlying psychopathologies. Although progress in the diagnosis and treatment of PNES has been made recently [5], the same personality and mood features associated with this disorder are potentially related to problems determining appropriate treatments. There is a growing awareness that controlled trials are needed to determine the best interventions for treating this challenging patient population [6]. Before the results of these studies can be put into practice, however, the external validity of these findings must be established. One threat to external validity that has not been addressed within this patient population is volunteer bias (LaFrance, personal communication, 12 April 2006). Volunteer bias is considered a threat to validity and occurs when those who participate in a study differ from those who do not [7]. Epidemiological studies examining mental illness, health care utilization, and disability have identified a number of factors that con* Corresponding author. Address: UCSF Epilepsy Center, 400 Parnassus Avenue, San Francisco, CA 94143-0138, USA. Fax: +1 415 3534494. E-mail address: [email protected] (D.A. Cahn-Weiner). 1525-5050/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.yebeh.2009.06.008
tribute to participation or nonparticipation in research. For example, socioeconomic status [8], marital status [9], and presence of psychiatric disorders [10] have been shown to be related to nonparticipation in survey research. Greater rates of nonparticipation have also been associated with younger age and male gender [11,12]. In psychological research, studies have demonstrated that personality variables, including agreeableness and openness to experience, significantly correlate with willingness to participate in research [13]. There is also some evidence from previous work with patients diagnosed with PNES that volunteer bias may exist. In one study of 53 patients diagnosed with PNES who were contacted after receiving their diagnosis and asked to participate in a telephone survey, only 23 agreed to participate [14]. Other studies have also suggested low participation rates for patients with PNES [15,16], but there is a paucity of research that systematically examines this methodological issue [17]. In the process of gathering data for a separate study examining personality and emotional predictors of QOL involving patients admitted to our epilepsy monitoring unit (EMU) for video/EEG monitoring, we were concerned that patients diagnosed with PNES would decline participation at a greater rate than patients diagnosed with ES [14–17]. We undertook this study to determine the potential contribution of volunteer bias in patients admitted for video/EEG monitoring and who are approached for participation in research. The specific aim was to document and compare the frequency with which patients diagnosed with PNES or ES participate in personality and QOL research.
525
A.M. Taylor et al. / Epilepsy & Behavior 15 (2009) 524–526
2. Methods This study was approved by the University of California, San Francisco, (UCSF) Committee on Human Research. Patients who were consecutively admitted to the EMU from January 2006 to January 2007 were considered for participation in a study examining personality and emotional predictors of QOL. For the QOL study, patients who were under 18 years of age, too cognitively impaired to complete research protocols independently, or unable to read at a sixth grade level were excluded and, therefore, not approached. Patients who received diagnostic confirmation prior to being admitted to the EMU were also excluded (e.g., patients with a confirmed diagnosis who were admitted for presurgical workup). All patients who met eligibility criteria were asked to participate in the QOL study. Measures in the QOL study included the Personality Assessment Inventory (PAI), Profile of Mood States (POMS), and Quality of Life in Epilepsy-89 (QOLIE-89). Patients who consented completed their assessments on the day of admission and prior to receiving diagnostic confirmation from their neurologist. Diagnosis of PNES versus ES was based on video/EEG findings. A definitive diagnosis of PNES was made if the seizures were characterized by unresponsiveness or convulsions while the EEG background remained normal. Normal recordings during seizures consisting of subtle symptoms such as limb numbness were considered nondiagnostic. Those patients who received neither a PNES nor an ES diagnosis (e.g., migraine) and those who were diagnosed with both PNES and ES were excluded from the final data analyses. The present study identified those patients who met inclusion criteria for the QOL study and who were approached for participation. Approval from the UCSF Committee on Human Research was obtained to review the medical records of patients who declined to participate in the QOL study for the purpose of determining their diagnosis, age, gender, and education. No potentially identifying information was recorded. On the basis of the patients’ participation status and diagnoses, contingency tables were generated for v2 analysis. The level for statistical significance was set at P < 0.05. 3. Results One hundred-thirty nine patients admitted to the EMU for video/EEG monitoring met inclusion criteria and were approached for participation in the QOL study. Patients with no confirmed diagnosis of PNES or ES, patients with both PNES and ES, and patients with neither disorder (e.g., migraine) were not included in any further analysis (n = 16). The demographic characteristics of patients with a confirmed diagnosis of PNES or ES are summarized in Table 1. There were no differences in gender or years of education between subjects who declined and those who participated, but those who declined tended to be older [t(122) = 2.11, P < 0.05]. Participation rates for patients diagnosed with PNES or ES are listed in Table 2. Four of 40 (10%) patients diagnosed with PNES re-
Table 1 Demographic characteristics of patients with a confirmed diagnosis of PNES (n = 40) or ES (n = 83). Consent a
Age Education Gender Male Female a b
P < 0.05. Mean (SD).
fused study participation and 8 of 83 (10%) patients diagnosed with ES refused participation. The overall participation rate did not differ between diagnostic groups [v2(1) = 0.004, P = 0.950]. Additionally, volunteers diagnosed with PNES who agreed to participate in the QOL study were no more likely than volunteers diagnosed with ES to turn in invalid or incomplete surveys [6/36 (17%) vs 17/75 (23%), v2(1) = 0.533, P = 0.465]. 4. Discussion Methodological problems such as volunteer bias can compromise the interpretation and limit the generalizability of clinical research findings. Because previous research has suggested that patients diagnosed with PNES may decline participation in research [14–17], we were concerned that these patients may have been refusing to participate in our study examining personality and emotional predictors of QOL at a higher rate than patients diagnosed with ES. Such a finding could confound diagnostic comparisons between these groups. Furthermore, a low rate of participation in patients with PNES would limit the generalizability of intervention studies in this population. The results of this study must be considered in the context of its limitations. Our methodology involved gaining access, retrospectively, to the medical records of the same patients who declined to participate in our prospective research examining the personality and emotional predictors of QOL in patients diagnosed with PNES. As is standard for studies in which patient consent is not required, information obtained was limited strictly to that which could not potentially identify patients including age, gender, and years of education. Participation rates may have been affected by variables not considered including medications, seizure duration, age at onset, and ethnicity. Moreover, because of the methodology of the concurrent QOL study, many patients admitted to the EMU were not approached for participation, which may have also impacted the heterogeneity of the current sample. Finally, we did not include subjects with the dual diagnoses of PNES and ES or subjects who ultimately obtained other diagnoses (e.g., migraine); future studies are needed to determine if response rates in these patient groups are similar to those presented in the current study. In contrast to previous studies demonstrating that rate of participation increases with older samples, the results of the current study found that older subjects were more likely to decline, regardless of diagnosis. This finding was not explored further due to methodological constraints, but may be explored in future feasibility studies. Previous research suggests that patients who decline participation are less agreeable and less open to experience [13]. Examining these aspects of personality in patients diagnosed with PNES would likely provide useful information regarding willingness to participate in research. Nonetheless, in our study, participation rates do not appear to be moderated by diagnosis of PNES or ES. Certainly, a larger and more diverse sample size would strengthen our findings. In sum, our results argue against a volunteer bias in research involving patients admitted to the EMU for video/EEG monitoring and suggest that the personality and emotional factors associated with patients diagnosed with PNES do not impact participation rates. Given the limitations of our sample and the methodological concerns, further research in this area is warranted.
Decline b
37.8 (12.2) 11.9 (5.7)
45.7 (11.9) 14.2 (2.6)
41 (36.9%) 70 (63.1%)
5 (41.7%) 7 (58.3%)
Table 2 Participation rates by diagnosis.
Declined Consented
PNES
ES
4 (25%) 36 (29%)
8 (50%) 75 (61%)
526
A.M. Taylor et al. / Epilepsy & Behavior 15 (2009) 524–526
References [1] Breier JI, Fuchs KL, Brookshire BL, et al. Quality of life perception in patients with intractable epilepsy or pseudoseizures. Arch Neurol 1998;55:660–5. [2] Loring DW, Meador KJ, King DW, Hermann BP. Relationship between quality of life variables and personality factors in patients with epilepsy and nonepileptic seizures. In: Gates JR, Rowan AJ, editors. Non-epileptic seizures. Woburn, MA: Butterworth–Heinemann; 2000. p. 159–81. [3] Szaflarski JP, Hughes C, Szaflarski M, et al. Quality of life in psychogenic nonepileptic seizures. Epilepsia 2003;44:236–42. [4] Testa SM, Schefft BK, Szaflarski JP, Hwa-Shain Y, Privitera MD. Mood, personality, and health-related quality of life in epileptic and psychogenic seizure disorders. Epilepsia 2007;48:973–82. [5] Lafrance Jr WC, Miller IW, Ryan CE, et al. Cognitive behavioral therapy for psychogenic nonepileptic seizures. Epilepsy Behav 2009;14:591–6. [6] Lafrance Jr WC, Barry JJ. Update on treatments of psychological nonepileptic seizures. Epilepsy Behav 2005;7:364–74. [7] Sackett DL. Bias in analytic research. J Chron Dis 1979;32:51–63. [8] Goldberg M, Chastang JF, Leclerc A, et al. Socioeconomic, demographic, occupational, and health factors associated with participation in a long-term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population. Am J Epidemiol 2001;154:373–84. [9] Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol 1999;28:1134–40.
[10] Sheikh K, Mattingly S. Investigating non-response bias in mail surveys. J Epidemiol Community Health 1981;35:293–6. [11] McConnell P, Bebbinton P, McClelland K, Houghton S. Prevalence of psychiatric disorder and need for psychiatric care in Northern Ireland: population study in the district of Derry. Br J Psychiatry 2002;181:214–9. [12] Ayusu-Mateos J, Vasquez-Barquero J, Dowrick C, Dunn G, Wilkinson G, the ODIN group. Depressive disorders in Europe: prevalence figures from the ODIN study. Br J Psychiatry 2001;179:308–16. [13] Marcus B, Schutz A. Who are the people reluctant to participate in research? Personality correlates of four different types of nonresponse as inferred from self- and observer ratings. J Pers 2005;73:959–84. [14] Wilder C, Marquez AV, Farias ST, et al. Long-term follow-up study of patients with PNES. Epilepsia 2004;45(Suppl. 7):349. [15] Aboukasm A, Mahr G, Gahry BR, Thomas A, Barkley GL. Retrospective analysis of the effects of psychotherapeutic interventions on outcomes of psychogenic nonepileptic seizures. Epilepsia 1998;39:470–3. [16] Reuber M, Pukrop R, Bauer J, Helmstaedter C, Tessendorf N, Elger CE. Outcome in psychogenic nonepileptic seizures: 1 to 10-year follow-up in 164 patients. Ann Neurol 2003;53:305–11. [17] Lafrance Jr WC, Blum AS, Miller IW, Ryan CE, Keitner GI. Methodological issues in conducting treatment trials for psychological nonepileptic seizures. J Neuropsychiatry Clin Neurosci 2007;19:391–8.
Contents lists available at ScienceDirect
Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh
Brief Communication
Examination of volunteer bias in research involving patients diagnosed with psychogenic nonepileptic seizures Alex M. Taylor a,b, Deborah A. Cahn-Weiner b,*, Paul A. Garcia b a b
Alliant International University, San Francisco, CA, USA Department of Neurology, University of California, San Francisco, CA, USA
a r t i c l e
i n f o
Article history: Received 20 March 2009 Revised 8 May 2009 Accepted 7 June 2009 Available online 16 July 2009 Keywords: Psychogenic nonepileptic seizures Epileptic seizures Volunteer bias Methodology External validity Clinical trials
a b s t r a c t There is some evidence to suggest that the same personality and emotional features observed in patients diagnosed with psychogenic nonepileptic seizures (PNES) may be related to nonparticipation in studies aimed at determining the most effective interventions for treating this patient population. This study aimed to document the frequency with which patients referred to an epilepsy monitoring unit (EMU) for video/EEG telemetry who are diagnosed with PNES or epileptic seizures (ES) participate in survey research. Of the 139 patients approached, 16 patients declined and 23 produced invalid or incomplete surveys. There was no significant difference in participation rates between patients diagnosed with PNES and those diagnosed with ES. These findings argue against a volunteer bias in research involving PNES and suggest that the results of other studies with these patients generalize to the population as a whole. Ó 2009 Elsevier Inc. All rights reserved.
1. Introduction The personality characteristics and mood features associated with psychogenic nonepileptic seizures (PNES) have been related to lower quality of life (QOL) when compared with patients diagnosed with ES [1–4]. Presumably, a better understanding of the personality and mood features associated with PNES would inform clinicians of the most targeted efficacious treatments that might help reduce the symptoms of this disorder as well as underlying psychopathologies. Although progress in the diagnosis and treatment of PNES has been made recently [5], the same personality and mood features associated with this disorder are potentially related to problems determining appropriate treatments. There is a growing awareness that controlled trials are needed to determine the best interventions for treating this challenging patient population [6]. Before the results of these studies can be put into practice, however, the external validity of these findings must be established. One threat to external validity that has not been addressed within this patient population is volunteer bias (LaFrance, personal communication, 12 April 2006). Volunteer bias is considered a threat to validity and occurs when those who participate in a study differ from those who do not [7]. Epidemiological studies examining mental illness, health care utilization, and disability have identified a number of factors that con* Corresponding author. Address: UCSF Epilepsy Center, 400 Parnassus Avenue, San Francisco, CA 94143-0138, USA. Fax: +1 415 3534494. E-mail address: [email protected] (D.A. Cahn-Weiner). 1525-5050/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.yebeh.2009.06.008
tribute to participation or nonparticipation in research. For example, socioeconomic status [8], marital status [9], and presence of psychiatric disorders [10] have been shown to be related to nonparticipation in survey research. Greater rates of nonparticipation have also been associated with younger age and male gender [11,12]. In psychological research, studies have demonstrated that personality variables, including agreeableness and openness to experience, significantly correlate with willingness to participate in research [13]. There is also some evidence from previous work with patients diagnosed with PNES that volunteer bias may exist. In one study of 53 patients diagnosed with PNES who were contacted after receiving their diagnosis and asked to participate in a telephone survey, only 23 agreed to participate [14]. Other studies have also suggested low participation rates for patients with PNES [15,16], but there is a paucity of research that systematically examines this methodological issue [17]. In the process of gathering data for a separate study examining personality and emotional predictors of QOL involving patients admitted to our epilepsy monitoring unit (EMU) for video/EEG monitoring, we were concerned that patients diagnosed with PNES would decline participation at a greater rate than patients diagnosed with ES [14–17]. We undertook this study to determine the potential contribution of volunteer bias in patients admitted for video/EEG monitoring and who are approached for participation in research. The specific aim was to document and compare the frequency with which patients diagnosed with PNES or ES participate in personality and QOL research.
525
A.M. Taylor et al. / Epilepsy & Behavior 15 (2009) 524–526
2. Methods This study was approved by the University of California, San Francisco, (UCSF) Committee on Human Research. Patients who were consecutively admitted to the EMU from January 2006 to January 2007 were considered for participation in a study examining personality and emotional predictors of QOL. For the QOL study, patients who were under 18 years of age, too cognitively impaired to complete research protocols independently, or unable to read at a sixth grade level were excluded and, therefore, not approached. Patients who received diagnostic confirmation prior to being admitted to the EMU were also excluded (e.g., patients with a confirmed diagnosis who were admitted for presurgical workup). All patients who met eligibility criteria were asked to participate in the QOL study. Measures in the QOL study included the Personality Assessment Inventory (PAI), Profile of Mood States (POMS), and Quality of Life in Epilepsy-89 (QOLIE-89). Patients who consented completed their assessments on the day of admission and prior to receiving diagnostic confirmation from their neurologist. Diagnosis of PNES versus ES was based on video/EEG findings. A definitive diagnosis of PNES was made if the seizures were characterized by unresponsiveness or convulsions while the EEG background remained normal. Normal recordings during seizures consisting of subtle symptoms such as limb numbness were considered nondiagnostic. Those patients who received neither a PNES nor an ES diagnosis (e.g., migraine) and those who were diagnosed with both PNES and ES were excluded from the final data analyses. The present study identified those patients who met inclusion criteria for the QOL study and who were approached for participation. Approval from the UCSF Committee on Human Research was obtained to review the medical records of patients who declined to participate in the QOL study for the purpose of determining their diagnosis, age, gender, and education. No potentially identifying information was recorded. On the basis of the patients’ participation status and diagnoses, contingency tables were generated for v2 analysis. The level for statistical significance was set at P < 0.05. 3. Results One hundred-thirty nine patients admitted to the EMU for video/EEG monitoring met inclusion criteria and were approached for participation in the QOL study. Patients with no confirmed diagnosis of PNES or ES, patients with both PNES and ES, and patients with neither disorder (e.g., migraine) were not included in any further analysis (n = 16). The demographic characteristics of patients with a confirmed diagnosis of PNES or ES are summarized in Table 1. There were no differences in gender or years of education between subjects who declined and those who participated, but those who declined tended to be older [t(122) = 2.11, P < 0.05]. Participation rates for patients diagnosed with PNES or ES are listed in Table 2. Four of 40 (10%) patients diagnosed with PNES re-
Table 1 Demographic characteristics of patients with a confirmed diagnosis of PNES (n = 40) or ES (n = 83). Consent a
Age Education Gender Male Female a b
P < 0.05. Mean (SD).
fused study participation and 8 of 83 (10%) patients diagnosed with ES refused participation. The overall participation rate did not differ between diagnostic groups [v2(1) = 0.004, P = 0.950]. Additionally, volunteers diagnosed with PNES who agreed to participate in the QOL study were no more likely than volunteers diagnosed with ES to turn in invalid or incomplete surveys [6/36 (17%) vs 17/75 (23%), v2(1) = 0.533, P = 0.465]. 4. Discussion Methodological problems such as volunteer bias can compromise the interpretation and limit the generalizability of clinical research findings. Because previous research has suggested that patients diagnosed with PNES may decline participation in research [14–17], we were concerned that these patients may have been refusing to participate in our study examining personality and emotional predictors of QOL at a higher rate than patients diagnosed with ES. Such a finding could confound diagnostic comparisons between these groups. Furthermore, a low rate of participation in patients with PNES would limit the generalizability of intervention studies in this population. The results of this study must be considered in the context of its limitations. Our methodology involved gaining access, retrospectively, to the medical records of the same patients who declined to participate in our prospective research examining the personality and emotional predictors of QOL in patients diagnosed with PNES. As is standard for studies in which patient consent is not required, information obtained was limited strictly to that which could not potentially identify patients including age, gender, and years of education. Participation rates may have been affected by variables not considered including medications, seizure duration, age at onset, and ethnicity. Moreover, because of the methodology of the concurrent QOL study, many patients admitted to the EMU were not approached for participation, which may have also impacted the heterogeneity of the current sample. Finally, we did not include subjects with the dual diagnoses of PNES and ES or subjects who ultimately obtained other diagnoses (e.g., migraine); future studies are needed to determine if response rates in these patient groups are similar to those presented in the current study. In contrast to previous studies demonstrating that rate of participation increases with older samples, the results of the current study found that older subjects were more likely to decline, regardless of diagnosis. This finding was not explored further due to methodological constraints, but may be explored in future feasibility studies. Previous research suggests that patients who decline participation are less agreeable and less open to experience [13]. Examining these aspects of personality in patients diagnosed with PNES would likely provide useful information regarding willingness to participate in research. Nonetheless, in our study, participation rates do not appear to be moderated by diagnosis of PNES or ES. Certainly, a larger and more diverse sample size would strengthen our findings. In sum, our results argue against a volunteer bias in research involving patients admitted to the EMU for video/EEG monitoring and suggest that the personality and emotional factors associated with patients diagnosed with PNES do not impact participation rates. Given the limitations of our sample and the methodological concerns, further research in this area is warranted.
Decline b
37.8 (12.2) 11.9 (5.7)
45.7 (11.9) 14.2 (2.6)
41 (36.9%) 70 (63.1%)
5 (41.7%) 7 (58.3%)
Table 2 Participation rates by diagnosis.
Declined Consented
PNES
ES
4 (25%) 36 (29%)
8 (50%) 75 (61%)
526
A.M. Taylor et al. / Epilepsy & Behavior 15 (2009) 524–526
References [1] Breier JI, Fuchs KL, Brookshire BL, et al. Quality of life perception in patients with intractable epilepsy or pseudoseizures. Arch Neurol 1998;55:660–5. [2] Loring DW, Meador KJ, King DW, Hermann BP. Relationship between quality of life variables and personality factors in patients with epilepsy and nonepileptic seizures. In: Gates JR, Rowan AJ, editors. Non-epileptic seizures. Woburn, MA: Butterworth–Heinemann; 2000. p. 159–81. [3] Szaflarski JP, Hughes C, Szaflarski M, et al. Quality of life in psychogenic nonepileptic seizures. Epilepsia 2003;44:236–42. [4] Testa SM, Schefft BK, Szaflarski JP, Hwa-Shain Y, Privitera MD. Mood, personality, and health-related quality of life in epileptic and psychogenic seizure disorders. Epilepsia 2007;48:973–82. [5] Lafrance Jr WC, Miller IW, Ryan CE, et al. Cognitive behavioral therapy for psychogenic nonepileptic seizures. Epilepsy Behav 2009;14:591–6. [6] Lafrance Jr WC, Barry JJ. Update on treatments of psychological nonepileptic seizures. Epilepsy Behav 2005;7:364–74. [7] Sackett DL. Bias in analytic research. J Chron Dis 1979;32:51–63. [8] Goldberg M, Chastang JF, Leclerc A, et al. Socioeconomic, demographic, occupational, and health factors associated with participation in a long-term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population. Am J Epidemiol 2001;154:373–84. [9] Reijneveld SA, Stronks K. The impact of response bias on estimates of health care utilization in a metropolitan area: the use of administrative data. Int J Epidemiol 1999;28:1134–40.
[10] Sheikh K, Mattingly S. Investigating non-response bias in mail surveys. J Epidemiol Community Health 1981;35:293–6. [11] McConnell P, Bebbinton P, McClelland K, Houghton S. Prevalence of psychiatric disorder and need for psychiatric care in Northern Ireland: population study in the district of Derry. Br J Psychiatry 2002;181:214–9. [12] Ayusu-Mateos J, Vasquez-Barquero J, Dowrick C, Dunn G, Wilkinson G, the ODIN group. Depressive disorders in Europe: prevalence figures from the ODIN study. Br J Psychiatry 2001;179:308–16. [13] Marcus B, Schutz A. Who are the people reluctant to participate in research? Personality correlates of four different types of nonresponse as inferred from self- and observer ratings. J Pers 2005;73:959–84. [14] Wilder C, Marquez AV, Farias ST, et al. Long-term follow-up study of patients with PNES. Epilepsia 2004;45(Suppl. 7):349. [15] Aboukasm A, Mahr G, Gahry BR, Thomas A, Barkley GL. Retrospective analysis of the effects of psychotherapeutic interventions on outcomes of psychogenic nonepileptic seizures. Epilepsia 1998;39:470–3. [16] Reuber M, Pukrop R, Bauer J, Helmstaedter C, Tessendorf N, Elger CE. Outcome in psychogenic nonepileptic seizures: 1 to 10-year follow-up in 164 patients. Ann Neurol 2003;53:305–11. [17] Lafrance Jr WC, Blum AS, Miller IW, Ryan CE, Keitner GI. Methodological issues in conducting treatment trials for psychological nonepileptic seizures. J Neuropsychiatry Clin Neurosci 2007;19:391–8.