Comparing the convergent validity and clinical utility of the Behavior Assessment System for Children-Parent Rating Scales and Child Behavior Checklist in children with epilepsy

Epilepsy & Behavior 13 (2008) 237–242

Contents lists available at ScienceDirect

Epilepsy & Behavior journal homepage: www.elsevier.com/locate/yebeh

Comparing the convergent validity and clinical utility of the Behavior Assessment System for Children-Parent Rating Scales and Child Behavior Checklist in children with epilepsy H. Allison Bender a,*, Dominick Auciello b, Chris E. Morrison a, William S. MacAllister a, Charles M. Zaroff a,c a

Comprehensive Epilepsy Center, Department of Neurology, New York University Medical Center, 403 East 34th Street, 4th Floor, New York, NY 10016, USA New York University Child Study Center, Department of Child & Adolescent Psychiatry, 145 East 32nd Street, 5th Floor, New York, NY 10016, USA c Northeast Regional Epilepsy Group, 104 East 40th Street, Suites 607/608, New York, NY 10016, USA b

a r t i c l e

i n f o

Article history: Received 28 January 2008 Revised 8 March 2008 Accepted 18 March 2008 Available online 29 April 2008 Keywords: Epilepsy Psychopathology Pediatric Child behavior checklist Behavior Assessment System for ChildrenParent Rating Scales

a b s t r a c t The convergent validity and clinical utility of two parent-report child behavior rating scales, the Behavior Assessment System for Children-Parent Rating Scales (BASC-PRS) and Child Behavior Checklist/Ages 6–18 (CBCL), in children with epilepsy were examined. Analogous broadband and narrowband behavior rating scales were evaluated in 60 subjects aged 6–17 years (mean = 11.0, SD = 3.4) with Full Scale IQ >70. Correlations for each similarly labeled scale were statistically significant (P < 0.002) and greater with broadband (r = 0.71–0.79) than with narrowband (r = 0.41–0.78) scales. The BASC captured significantly less composite internalizing symptoms (P < 0.002), but more unusual thought processes (P < 0.0002) and attention problems versus a CBCL DSM-oriented attention-deficit/hyperactivity disorder scale (P < 0.002). Variation in the correlations between measures may stem from underlying differences between rationally-and empirically-derived approaches to test construction. Both the CBCL and BASC have diagnostic and clinical utility in assessing behavior problems in pediatric epilepsy. Ó 2008 Elsevier Inc. All rights reserved.

1. Introduction Epilepsy is the most prevalent pediatric neurological disorder, affecting approximately 8 in 1000 children and adolescents [1,2]. Neuropsychological, psychiatric, and social-behavioral skills impairments are important comorbidities of chronic epilepsy [3– 5]. In the classic Isle of Wight study, Rutter reported ‘‘behavior problems” in 28.6% of children with uncomplicated epilepsies [6]. More recently, the prevalence rate of psychological and maladaptive behavioral symptoms in children with epilepsy has been estimated at approximately 21–60%, representing an increased risk of P three to six times that of the general population or greater [7]. Thus, frequent monitoring of a child’s psychological status should be an ongoing and integral part of routine care. Yet, despite the known risk factors, formal evaluation of behavioral functioning may not be a regular part of a child’s care. A study by Ott and colleagues (2003) suggests a considerable unmet psychiatric need in children with epilepsy; 61% of study participants met criteria for a DSM-IV psychiatric diagnosis, although only 33% received mental health services [7]. Although numerous studies have examined the frequency and presentation of psychiatric comorbidity in pediatric epilepsy * Corresponding author. E-mail address: [email protected] (H. Allison Bender). 1525-5050/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.yebeh.2008.03.007

(e.g., depression, Attention-Deficit/Hyperactivity Disorder), empirical evaluation of the emotional and behavioral assessment methods is limited in this population. As such, the neuropsychological and psychiatry communities at-large have been largely unable to agree on a ‘‘gold standard” for the evaluation of psychiatric comorbidity in children with chronic neurological disorders. Each of the existing approaches to psychiatric assessment has considerable advantages and drawbacks. Although structured diagnostic interviews provide greater opportunity for symptom exploration and clinical judgment, these categorical measures yield dichotomous diagnoses; either the child has the disorder or he or she does not [8]. In contrast, a dimensional approach to assessment (e.g., self- and parent-report behavior rating scales), when viewed in isolation, allows for the measurement of symptom severity and subsyndromal symptomatology, yet ignores the experiences and diagnostic impressions of the clinician. Though theoretically ideal, the direct assessment of behavioral and psychological disturbance in children is problematic for several reasons. Apart from the obstacle of chronological age and concomitant cognitive developmental level, the ability to obtain reliable and valid self-report assessments may be further compromised by a child’s lack of insight. Specifically, the potential for underlying neuropathology places children with epilepsy at risk for having limited awareness and understanding of their own psychological state and behavior. Clinically, much of this

238

H. Allison Bender et al. / Epilepsy & Behavior 13 (2008) 237–242

concern can be obviated by a combined approach, using both structured or semi-structured interviews and behavior rating scales with parent or caregiver raters. Yet, lengthy clinical assessments are not always feasible in some settings, such as outpatient clinics. Parent-report behavioral rating scales are commonly administered in lieu of comprehensive diagnostic interviews, due to their relative brevity and cost-efficiency. While no single parent-report behavior rating scale provides an adequate substitute for direct observation and clinical judgment, symptom checklists can yield valuable adjunctive information for diagnostic and treatment planning. The Child Behavior Checklist/Ages 6–18 (CBCL) [9] is one of the most widely used assessment measures of child and adolescent psychopathology. The CBCL is an empirically derived rating scale which was constructed through a series of quantitative analyses to determine the overlap of behavioral characteristics as a means of deriving specific dimensions of psychopathology. The CBCL has shown excellent reliability and validity in both clinical and non-clinical populations [9– 12]. Though the broadband scales measuring internalizing and externalizing behavior have general clinical utility [13], the CBCL is less able to distinguish between diagnoses subsumed by these larger emotional categories [14]. The relatively weaker diagnostic efficiency of the narrowband CBCL syndrome scales has been attributed largely to a discrepancy between item content and the professionally accepted conceptualization of psychopathology in childhood [15]. Additional concerns include difficulty in distinguishing between several of the narrowband scales which may, in part, be due to the inclusion of the same items on multiple, scales [13]. In contrast to the CBCL, the Behavior Assessment System for Children-Parent Report Scales (BASC-PRS) and Behavior Assessment System for Children-2-Parent Report Scales (BASC-2-PRS) (BASC-PRS when referring to both [16,17] are conceptually derived instruments, in which item inclusion was primarily determined by rational, rather than statistical means. The BASC, like the CBCL, is a self-, parent-, and teacher-report rating system with which children are evaluated on behaviors within the domains of psychological, behavioral, and adaptive functioning. According to the BASC manual [17], item content focuses on clinically or theoretically relevant material useful in arriving at DSM diagnoses. The differences in item content and conceptualization of childhood disorders between the BASC and the CBCL suggest that these measures assess similar, but not entirely overlapping constructs. Limited study of the convergent validity of the BASC-PRS and CBCL has demonstrated modest to high correlations between these scales in healthy controls [16] and in children at-risk for conduct disorder [18]. To our knowledge, no published study has systematically compared the psychometric properties of these parent-report behavior rating scales in a population with a known neurological disorder. Thorough exploration of the respective utility of these instruments is a logical first step in maximizing the clinical significance of the information obtained, while also minimizing redundancy. The primary goal of this study was to compare the convergent validity of these two commonly-administered parent-report behavior rating scales in a population with pediatric epilepsy. That is, we aimed to examine the extent to which conceptually similar broadband (e.g., internalizing and externalizing behaviors) and narrowband (e.g., CBCL Somatic Complaints Syndrome Scale and BASC Somatization Scale) syndrome scales of the BASC and CBCL assess similar behaviors in this population [18–20]. Given the relatively high correlations between scales presented in previous investigations [16,18], we predict a similar degree of agreement in a sample of children with epilepsy.

2. Methods 2.1. Participants A total of 60 children and adolescents (6–17 years of age) receiving a neuropsychological evaluation at a tertiary referral center (New York University Comprehensive Epilepsy Center, NYUCEC) from 2004 to 2006 were examined. A diagnosis of epilepsy, as defined by the International Classification of Epilepsies, was required for inclusion. A Full Scale IQ P70 on the Wechsler Abbreviated Scale of Intelligence (WASI) [21] or Wechsler Intelligence Scale for Children Fourth Edition (WISC-IV) [22] was also required. An exclusionary criterion based on overall intellectual functioning was applied due to the increased risk of behavioral disturbance in children with global developmental delay [23]. We excluded all screened participants with a documented history of vision or hearing impairment and/or a history of intracranial surgical intervention for seizure disorder (e.g., resection). One child was excluded for an invalid response profile on the BASC. Five children currently receiving psychopharmacological intervention for an existing psychiatric condition were not included in any of the analyses; exclusion of such children was based on the goal of examining a cohort treatment-naïve with respect to psychotropic medications. Thirty-five males and 25 females were selected to participate in this study. Participant characteristics for the total sample are summarized in Table 1. More than half of the studied sample were receiving monotherapy at the time of evaluation (N = 28). Six children were receiving polytherapy and 18 were unmedicated. The primary seizure type observed within the sample was generalized epilepsy (N = 32), followed by partial (N = 15), and mixed (N = 3). Seizure type was not established in 10% of the children. 2.2. Measures 2.2.1. Intellectual assessment The WASI [21] or the WISC-IV [22] was administered to each child individually. In some cases, a Full Scale IQ was derived from the administration of two subtests of the WASI (Vocabulary and Matrix Reasoning). Prior validation studies have indicated that the WASI two subtest IQ shows strong correlations with longer measures of intelligence; a 0.81 correlation was observed between the WASI two subtest Full Scale IQ and the Wechsler Intelligence Scale for Children-III Full Scale IQ [22]. 2.2.2. Parent-report behavior rating scales The CBCL/Ages 6–18 [24] is a parent-report behavior rating scale appropriate for 6- to 18-year-old children. Parents respond to 118 items by rating their child on a 3-point scale (e.g., 0 = ‘‘not true,” 1 = ‘‘somewhat or sometimes true,” and 2 = ‘‘very true”). In several cases, the same item contributes to more than one scale. The CBCL yields an overall symptom index (Total Problems), two broadband dimensions (Internalizing and Externalizing), and eight narrowband syndrome scales (Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Social Problems, Thought Problems, Attention Problems, Rule-Breaking Behavior, and Aggressive Behavior). Computer software available from the Achenbach System of Empirically Based Assessment was used to score the checklists. In a method similar to Dorenbaum and colleagues [25], normative comparisons were based on the standardization sample provided by the CBCL scoring program. Scores for the CBCL consist of raw scores and converted T scores (M = 50, SD = 10). On the CBCL syndrome scales, below-average T scores are truncated at 50; scores were not truncated for the Internalizing, Externalizing, and Total Problem scales. For the broadband scales, T scores from 60 to 63 represent Borderline Clinical symptoms and T scores P64 represent Clinical range symptoms. For the syndrome scales, T scores from 65 to 69 represent Borderline Clinical symptoms, whereas T scores P70 represent Clinical range symptoms. The BASC [17] is a parent-report adaptive functioning and behavior rating scale that has developmentally-appropriate versions for preschool (2–5 years), child (6– 11 years), and adolescent (12–21 years) age groups. Parents respond to 130 items by rating their child on a 4-point scale of behavior frequency (ranging from ‘‘never” to ‘‘almost always”). Individual items assess aspects of childhood behavior and personality across home, community, and school settings. There are also scales assessing the validity of individual’s responses.

Table 1 Sample characteristics

Age IQ Length of education (years) Age at seizure onset (years) Duration of epilepsy (years)

Mean

SD

11.0 90.1 6.71 6.55 5.39

3.4 14.0 3.02 4.12 4.11

239

H. Allison Bender et al. / Epilepsy & Behavior 13 (2008) 237–242 Broadband composite scores on the BASC include the Internalizing and Externalizing Symptoms, as well as an overall Behavioral Symptom Index. Narrowband scales include Aggression, Anxiety, Attention Problems, Atypicality, Conduct Problems, Depression, Hyperactivity, Withdrawal, and Somatization. Like the CBCL, the BASC may be used in conjunction with self-report and teacher ratings for a multimodal approach. However, unlike the CBCL, the BASC was developed so that each item contributes to only one narrowband scale, thereby allowing each scale to remain a distinct measure of that specific construct [16]. Scores for the BASC consist of raw scores and converted T scores (M = 50, SD = 10). T scores falling between 40 and 60 are considered to be within normal limits. T scores of 60–69 (1–2 SD above the mean) on the clinical scales are considered At Risk, whereas T scores of P70 fall in the Clinical range. The Anxiety scale (both child and adolescent) has lower cutoff scores for the Clinical range; scores of 60–64 are considered At Risk, and scores P65 are within the Clinical range. All BASC standardized scores were generated using the nonclinical normative sample to provide information on rates of symptomatology compared to the overall general population. The original BASC (BASC-PRS) [16] was administered to nine participants who were evaluated prior to release of the BASC-2-PRS. The original and revised versions of the BASC are very similar in their item scope, content, and scoring, and have strong correlations between their resultant broad- and narrowband scales [17]. As noted in the BASC manual, the PRS has excellent internal consistency, test–retest reliability, and content and construct validity [16]. 2.3. Procedures The present study was approved by the Institutional Review Board of New York University School of Medicine. Parents of the 52 participants completed the behavior rating scales during a single inpatient admission or outpatient visit at NYUCEC. Eight parents were contacted via telephone and were asked to complete the BASC and CBCL independently and remit them to NYUCEC for analysis. The questionnaires were administered in random order. 2.4. Analyses Descriptive statistics include mean and standard deviation scores on the BASC and CBCL. The percentage of the sample obtaining scores in the clinically significant range, denoted Clinical, was computed for each scale. The BASC term at risk was adopted for those scores falling in the At Risk range on the BASC and for scores falling in the Borderline Clinical range on the CBCL. To estimate the degree of agreement among analogous broad- and narrowband scale T scores, Pearson correlation coefficients were calculated. Given the truncated T scores on the narrowband syndrome scales of the CBCL, nonparametric correlations were also calculated (Spearman rank correlation coefficient), as this analysis does not assume that the relationship between variables is linear. v2 analyses were calculated based on the frequency of elevated scores on the BASC, to determine statistically significant differences in the sensitivities of each questionnaire for each analogous scale. To correct for multiple comparisons, a Bonferroni correction was applied and significant levels were accordingly set for 0.002.

3. Results Table 2 lists the CBCL and BASC scale means. All but one of the parents’ scores on the validity scales were within the ‘‘acceptable” range of the BASC; one parent generated responses resulting in an elevated ‘‘F” index, a measure of unusually extreme maladaptive behavior. Given the elevation on this scale, these data were excluded from further study. Table 3 shows the frequency of elevated scores on each scale, reflecting children falling within and outside of normal limits (e.g., total frequency of children falling within the Borderline and Clinically Significant ranges). On the CBCL, none of the mean scores fell in the At Risk or Clinical ranges. However, children with epilepsy were at 2–3.5 times greater risk for developing Internalizing and Externalizing Problems (55.9 and 30.5%, respectively) within the Clinically Significant range, when compared with the normative sample (16.0%). On the CBCL, children with epilepsy received the highest mean syndrome scale score on the Attention Problems scale (M = 63.7) and the lowest mean score on the Rule-Breaking Behaviors scale (M = 54.6). Elevated risks were also observed on the three syndrome scales comprising the Internalizing Problems Composite: Withdrawn/Depressed (36.7%), Anxious/Depressed (31.7%), and Somatic Complaints (23.3%). Within the subscales of the Externalizing Problems domain, elevations were evident in 21.7% of children on the Aggressive scale and 13.3% of children on the Rule-Breaking Behaviors scale. The overall mean score for the BASC Behavioral Symptoms Index did not fall within the At Risk or Clinical range. However, elevations on the broadband scales were observed in approximately 28% of the sample on the Internalizing Problems scale and in approximately 20% on the Externalizing Problems scale. The highest mean score occurred on the Attention Problems scale (M = 58.2, SD = 12.6), whereas the lowest score occurred on the Conduct Problems scale (M = 48.9, SD = 11.8). The overall mean symptom indices on the two scales were similar (BASC Behavioral Symptoms Index mean = 56.7; CBCL Total Problems mean = 58.4). Mean scores on the BASC Attention Problems scale and the CBCL DSM-IV Oriented AD/HD Problems scales, and the BASC Atypicality and the CBCL Thought Problems scales were nearly identical. However, mean scale scores on the CBCL were, in general, uniformly higher

Table 2 BASC/CBCL mean scores BASC

Mean

SD

CBCL

Mean

SD

Behavioral Symptoms Index Internalizing Problems Externalizing Problems Anxiety Anxiety Depression Depression Depression Somatization Somatization Aggression Aggression Aggression Aggression Conduct Problems Conduct Problems Conduct Problems Conduct Problems Atypicality Attention Problems Attention Problems Hyperactivity Hyperactivity

56.7 53.7 51.2 52.2 52.2 55.7 55.7 55.7 49.6 49.6 50.6 50.6 50.6 50.6 48.9 48.9 48.9 48.9 57.1 58.2 58.2 54.8 54.8

14.7 12.1 13.6 12.6 12.6 15.4 15.4 15.4 9.7 9.7 12.5 12.5 12.5 12.5 11.8 11.8 11.8 11.8 14.6 12.6 12.6 15.2 15.2

Total Problems Internalizing Problems Externalizing Problems DSM-IV Anxiety Problems Anxious/Depressed DSM-IV Affective Problems Withdrawn/Depressed Anxious/Depressed Somatic Complaints DSM-IV Somatic Problems Rule-Breaking Behaviors Aggressive Behavior DSM-IV Oppositional Defiant Problems DSM-IV Conduct Problems Rule-Breaking Behaviors Aggressive Behavior DSM-IV Oppositional Defiant Problems DSM-IV Conduct Problems Thought Problems Attention Problems DSM-IV AD/HD Problems DSM-IV AD/HD Problems Attention Problems

58.4 59.6 53.8 58.9 58.8 61.9 60.5 58.8 59.2 58.3 54.6 58.2 58.3 56.3 54.6 58.2 58.3 56.3 57.5 63.7 58.1 58.1 63.7

10.4 10.5 11.3 7.3 8.1 8.5 9.6 8.1 8.0 8.0 6.4 9.8 10.1 8.8 6.4 9.8 10.1 8.8 7.6 11.2 8.4 8.4 11.2

240

H. Allison Bender et al. / Epilepsy & Behavior 13 (2008) 237–242

Table 3 Percentage of elevated scores on the BASC and CBCL (compared with normative data)

BASC Behavioral Symptoms Index (N = 19) CBCL Total Problems (N = 27) BASC Internalizing Problems (N = 16) CBCL Internalizing Problems (N = 33) BASC Externalizing Problems (N = 12) CBCL Externalizing Problems (N = 18) BASC Anxiety (N = 16) CBCL DSM-IV Oriented Anxiety Problems (N = 16) CBCL Anxious/Depressed (N = 19) BASC Depression (N = 17) CBCL DSM-IV Oriented Affective Problems (N = 25) CBCL Withdrawn/Depressed (N = 22) CBCL Anxious/Depressed (N = 19) BASC Atypicality (N = 22) CBCL Thought Problems (N = 9) BASC Somatization (N = 11) CBCL Somatic Complaints (N = 14) CBCL DSM-IV Oriented Somatic Problems (N = 14) BASC Attention Problems (N = 28) CBCL Attention Problems (N = 21) CBCL DSM-IV Oriented AD/HD Problems (N = 14) BASC Hyperactivity (N = 16) CBCL DSM-IV Oriented AD/HD Problems (N = 14) CBCL Attention Problems (N = 21) BASC Aggression (N = 10) CBCL Rule-Breaking Behaviors (N = 8) CBCL Aggressive Behavior (N = 13) CBCL DSM-IV Oriented Oppositional Defiant Problems (N = 15) CBCL DSM-IV Oriented Conduct Problems (N = 12) BASC Conduct Problems (N = 10) CBCL Rule-Breaking Behaviors (N = 8) CBCL Aggressive Behavior (N = 13) CBCL DSM-IV Oriented Oppositional Defiant Problems (N = 15) CBCL DSM-IV Oriented Conduct Problems (N = 12)

Table 4 BASC/CBCL scale correlations

Clinical

At Risk

BASC

CBCL

r

10.2% 6.8% 19.0% 15.3% 6.8% 5.1% 13.6% 15.5% 21.7% 15.5% 20.7% 21.7% 21.7% 10.0% 8.3% 16.7% 10.0% 8.9% 25.0% 10.0% 13.8% 8.3% 13.8% 10.0% 8.3% 10.0% 5.0% 12.1%

22.0% 39.0% 8.6% 40.7% 13.6% 25.4% 13.6% 12.1% 10.0% 13.8% 22.4% 15.0% 10.0% 26.7% 6.7% 1.7% 13.3% 16.1% 21.7% 25.0% 10.3% 18.3% 10.3% 25.0% 8.3% 3.3% 16.7% 13.8%

Behavioral Symptoms Index Internalizing Problems Externalizing Problems Anxiety Anxiety Depression Depression Depression Somatization Somatization Aggression Aggression Aggression

Total Problems

0.75

Internalizing Problems Externalizing Problems DSM-IV Oriented Anxiety Problems Anxious/Depressed DSM-IV Oriented Affective Problems Withdrawn/Depressed Anxious/Depressed Somatic Complaints DSM-IV Oriented Somatic Problems Rule-Breaking Behaviors Aggressive Behavior DSM-IV Oriented Oppositional Defiant Problems DSM-IV Oriented Conduct Problems Rule-Breaking Behaviors Aggressive Behavior DSM-IV Oriented Oppositional Defiant Problems DSM-IV Oriented Conduct Problems Thought Problems Attention Problems DSM-IV Oriented AD/HD Problems DSM-IV Oriented AD/HD Problems Attention Problems

0.71 0.79 0.58 0.65 0.6 0.58 0.53 0.43 0.64 0.64 0.78 0.65

10.3% 6.8% 10.0% 5.0% 12.1%

10.3% 10.2% 3.3% 16.7% 13.8%

10.3%

10.3%

Aggression Conduct Problems Conduct Problems Conduct Problems Conduct Problems Atypicality Attention Problems Attention Problems Hyperactivity Hyperactivity

0.65 0.74 0.65 0.59 0.68 0.41 0.7 0.58 0.73 0.69

Note. All correlations were significant at the 0.002 level.

between the BASC and the CBCL are listed in Table 4. Nonparametric correlational analyses yielded similar ranges for all narrowband comparisons. 4. Discussion

than theoretically similar scales on the BASC, with a mean difference of 6.0. Additionally, greater variability was observed on the BASC, as the majority of SDs were P10, whereas on the CBCL, only 7 of 24 SDs were >10. Interestingly, whereas mean broadband scale scores were more comparable across measures than the narrowband scale scores, the opposite pattern was generally observed when percentages of elevated scores were examined. The broadband scales of the CBCL captured more elevations than analogous BASC broadband scales, although the difference reached statistical significance only between the scales measuring internalizing symptoms (v2 = 23.7, P < 0.002). In contrast, most of the analogous narrowband scales captured elevated scores within 10% of one another. The notable exception was the difference between the scales measuring unusual thought processes, on which nearly 37% of the BASC sample had elevated scores, compared to only 15% of the CBCL sample, a difference that reached statistical significance (v2 = 12.129, P < 0.002). The BASC also identified a higher percentage of children with attention problems (regardless of which CBCL attention-oriented scale was used) and the difference between the BASC Attention Problems scale and the CBCL DSM-IV Oriented AD/HD Problems scale was statistically significant (v2 = 11.828, P < 0.002). Correlations were high (r = 0.71–0.79) for the broadband scales of these instruments. Comparison of the conceptually analogous narrowband scales yielded greater variability (range = 0.41–0.78), but all correlations were significant at the 0.002 level. Correlations were generally higher for scales measuring externalizing behaviors (range = 0.58–0.78) than for scales measuring internalizing behaviors (range = 0.43–0.65). Complete parametric correlations

To our knowledge, the present study is the first to compare the convergent validity of the empirically-derived CBCL and the more conceptually-derived BASC for use with children with epilepsy. Given its strong psychometric properties, several investigations have commented on the usefulness of the CBCL as an assessment tool in the psychological and social assessment of children with epilepsy [5,25–31]. On the other hand, the applicability of the BASC in this population has apparently not been examined. Our results reveal significant agreement between mean scores on broadband scales of the BASC and CBCL, consistent with correlations reported in an investigation of children with attention difficulties [20], as well as in the BASC-2-PRS and CBCL manuals [17,24]. There was also significant concordance between mean scores on several of the analogous narrowband syndrome scales, although more variability was observed. The highest correlations were generally obtained on scales assessing aspects of disruptive, externalizing behaviors (e.g., attention problems, aggression, and rule-breaking behavior). In contrast, comparatively less robustness was observed on scales assessing internalizing disorders, such as depression, anxiety, and somatization. It is not entirely surprising that the depression and anxiety-related scales of the two measures were not as highly correlated as analogous externalizing narrowband scaled scores. The two behavior rating systems differ in their conceptualization and evaluation of affective disorders; the CBCL divides depression-related symptomatology into two separate dimensions, Withdrawn/Depressed and Anxious/Depressed, whereas the BASC does not. Compared with the BASC Depression and Anxiety scales, these specific CBCL scales captured a similar percentage of children with epilepsy exhibiting clinically-significant levels of psychological disturbance. However, as seen in Table 3, a higher frequency of children were identified as

H. Allison Bender et al. / Epilepsy & Behavior 13 (2008) 237–242

having clinical scores on the dual depression-related syndrome scales of the CBCL (CBCL Withdrawn/Depressed = 21.7% and CBCL Anxious/Depressed = 21.7%), as compared to children falling within the At Risk/Clinical range on the BASC scales assessing depressionand anxiety-related symptoms (BASC Anxiety = 13.6%, Depression = 15.5%). This difference may reflect either greater sensitivity of the CBCL relative to the BASC in evaluating subthreshold or emergent affective disorders in children with epilepsy. In contrast to the mean overall composite and broadband scale scores (Table 2), which were quite similar across measures, the overall composite score captured a lower percentage of psychopathology on the BASC, and this appeared somewhat specific to the higher percentage of internalizing problems captured on the CBCL (Table 3). However, in general, most of the narrowband scales captured similar rates of psychopathology and all correlations were significant. The high degree of agreement between CBCL and BASC in identifying children with epilepsy as having subclinical or clinical levels of psychopathology suggests that the analogous syndrome scales assess similar behaviors in this population. Notable exceptions to this trend were observed on scales measuring unusual thoughts/behaviors representing possible precursors to a thought disorder. The BASC captured a higher rate of such symptoms. Comparative analysis of the items comprising these two narrowband scales would be useful in establishing clearer comparisons of each scale’s convergent and discriminant validity. Similarly, the BASC captured a higher rate of elevated scores in the domain of attention, considerably more so than a scale measuring ADHD-related symptomatology on the CBCL, suggesting the need for further comparative study in this area [18– 20]. Behavioral ratings and checklists, although not a substitute for direct observation and limited in their adequateness for providing diagnostic decisions [32], may provide clinicians with a ‘‘snapshot” of psychological functioning when a lengthier evaluation is not feasible. The relative ease of parent-report rating scales may encourage their administration in a wide range of settings. Given the necessity of frequent, long-term monitoring of emotional and behavioral status in children with epilepsy, behavior rating scales may be useful tools in providing additional information of clinical value. Throughout the literature, broadband syndrome scales, such as the CBCL Internalizing and Externalizing Problems scales, have been reported to be highly effective at determining the presence and intensity of global psychiatric impairment [13,33]. In contrast, the narrowband scales of the empirically-derived CBCL are often criticized for inclusion of items not directly related to aspects of psychiatric symptomatology (e.g., ‘‘clumsy”) [13]. Critics also note that several items are included in more than one subscale, thereby reducing these Scales0 specificity and ability to measure distinct, non-overlapping constructs [13]. Perhaps as a result, several authors report difficulty distinguishing between diagnostic categories assessed by several of the narrowband scales of the Achenbach measures [10,14,34,35]. For example, in a study examining the convergence between the empirical-quantitative approach of the CBCL and the categorical approach of the DSM, Kasius and colleagues [12] found that the CBCL Somatic Complaints syndrome scale independently predicted the presence of DSM-III-R disorders in multiple diagnostic categories (e.g., any anxiety disorder, Major Depression, Dysthymia, and AD/HD). Although the criterion-related validity of the BASC has not been widely studied, the absence of strong DSM counterparts to narrowband syndrome scales creates the risk of ‘‘false positives” and ‘‘false negatives” when interpreting results yielded by dimensional rating scales, such as the CBCL and BASC. Despite the potential contributions, much of the research exploring psychiatric comorbid conditions of children with epilepsy is limited, in large part, by methodological constraints. This

241

study is no exception. Confounding factors such as selection bias hinder the majority of research studies [36]; children with more medically-refractory epilepsies (representing approximately 17– 20% of the entire epileptic population) are often chosen as participants in research studies [36,37]. This bias may increase the estimated/presumed incidence of psychological and behavioral disturbances in the population of children with epilepsy [38]. As participants were evaluated at a tertiary referral center, the present sample likely included more children with frequent, impairing, or treatment-resistant epilepsies than would be found in a community-based sample. As such, caution should be exercised before generalizing the present findings to a cohort with fewer psychiatric symptoms. Conversely, by excluding children with low IQs, we are presumably lowering the overall frequency and severity of psychiatric symptomatology. Numerous studies have reported that intellectual disability is associated with psychiatric illness in pediatric epilepsy [4,30,39,40]. The high rates of psychopathology captured during the current study cannot be regarded as definitive estimates of psychopathology in this population given that quantitative rating scales may not accurately capture the true level of symptomotology in chronically ill populations. CBCL scores, in particular, have reportedly been over-sensitive to impairment in this cohort [41,42]. Although commonly-administered in studies of children with medical illnesses including cystic fibrosis [43], diabetes mellitus [44], cancer [45], asthma [46], and inflammatory bowel disease [47], concerns exist regarding the usefulness of the CBCL in populations with chronic somatic complaints. At the very least, the interpretation of somatic symptom scales within larger broadband measures need to be qualified in such populations. Additionally, the endorsement of pain, and other disease-related symptoms, may increase scores on several narrowband syndrome scales, which can, in turn, ‘‘inappropriately” elevate internalizing and total symptom composites. Without relevant clinical normative groups consisting of children with epilepsy (or other chronic neurological disorders), the degree to which behavior rating scales identify ‘‘abnormal” behavior is often limited. Furthermore, caution should also be exercised when interpreting the correlations obtained between the narrowband scales of the CBCL and BASC T scores. As discussed, the CBCL narrowband syndrome scale T scores are truncated at 50; scores were not truncated for the broadband scales or the BASC T scores [16,17]. In this case, the truncation procedure reduces differentiation among low scores (e.g., T = 50 on the narrowband syndrome scales). Although Achenbach [24] recommends using raw scores for statistical analyses of the truncated narrowband CBCL scales, like others in the literature [20], the present study used T scores in an effort to control for age and gender variance of the sample. Of note, similar findings were obtained when evaluating the relationship between variables using both parametric and nonparametric analyses, suggesting that the effect of T-score truncation on our correlational analyses was minimal. Limitations notwithstanding, our data suggest the utility of both the CBCL and BASC in evaluating psychopathology in children with epilepsy. Taken together, the BASC and CBCL yield similar, yet clinically distinct constructs appropriate for a comprehensive screening of psychological and social-emotional functioning in this population. Further study is warranted to better appreciate the diagnostic utility of the CBCL and BASC in children with epilepsy, as well as to explore the different merits of the two scales. This being said, each instrument has its own unique strengths that may lead a clinician in a given setting to choose one over the other. For example, the CBCL has been used in numerous previous studies of epilepsy [5,25–31]. Accordingly, it has proven its sensitivity to psychiatric difficulties in this young population. Further, the present investigation suggests that the CBCL may be more sensitive to

242

H. Allison Bender et al. / Epilepsy & Behavior 13 (2008) 237–242

affective disturbances in this population than is the BASC, as corresponding scales were more frequently elevated on the CBCL than were similar BASC scales. However, the BASC has considerable strengths that make it a viable alternative. One such advantage is the separate scales for depressed versus anxious moods, in comparison to more general scales of affective distress seen on the CBCL (e.g., Anxious/Depressed, Withdrawn/Depressed). Thus, some interpretive clarity is offered. Another positive attribute of the BASC is its included validity indicators that allow a clinician to determine if the instrument was validly completed; the BASC provides indices to assess positive response biases (i.e., under-reporting symptomology), negative response biases (i.e., over-reporting symptomology), and inconsistent responding. Another positive attribute of the BASC is its use of different form versions for varying age ranges (i.e., ages 2–5, 6–11, and 12–21). This allows for more developmentally appropriate item content. Analyses of the role of demographic and seizure-related variables (i.e., age at seizure onset, antiepileptic drug therapy, seizure frequency) would also be especially useful in determining the sensitivity of these instruments for use with children with epilepsy. Acknowledgment This study was supported by an Epilepsy Foundation of America (EFA) Behavioral Sciences Student Fellowship. References [1] David RB. Child and adolescent neurology. St Louis, MO: Mosby–Year Book Inc.; 1998. [2] Hauser WA. The prevalence and incidence of convulsive disorders in children. Epilepsia 1994;35(Suppl. 2):S1–6. [3] Austin JK, Smith MS, Risinger MW, McNelis AM. Childhood epilepsy and asthma: comparison of quality of life. Epilepsia 1994;35:608–15. [4] Kolk A, Beilmann A, Tomberg T, Napa A, Talvik T. Neurocognitive development of children with congenital unilateral brain lesion and epilepsy. Brain Dev 2001;23:88–96. [5] Ott D, Caplan R, Guthrie D, et al. Measures of psychopathology in children with complex partial seizures and primary generalized epilepsy with absence. J Am Acad Child Adolesc Psychiatry 2001;40:907–14. [6] Graham P, Rutter M. Organic brain dysfunction and child psychiatric disorder. Br Med J 1968;3:695–700. [7] Ott D, Siddarth P, Gurbani S, et al. Behavioral disorders in pediatric epilepsy: unmet psychiatric need. Epilepsia 2003;44:591–7. [8] Cantwell DP. Classification of child and adolescent psychopathology. J Child Psychol Psychiatry 1996;37:3–12. [9] Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington: Univ. of Vermont, Department of Psychiatry; 1991. [10] Edelbrock C, Costello AJ. Convergence between statistically derived behavior problem syndromes and child psychiatric diagnoses. J Abnorm Child Psychol 1988;16:219–31. [11] Gould MS, Bird H, Jaramillo BS. Correspondence between statistically derived behavior problem syndromes and child psychiatric diagnoses in a community sample. J Abnorm Child Psychol 1993;21:287–313. [12] Kasius MC, Ferdinand RF, van den Berg H, Verhulst FC. Associations between different diagnostic approaches for child and adolescent psychopathology. J Child Psychol Psychiatry 1997;38:625–32. [13] Macmann GM, Wilkins GG, O’Malley D. The Devereux Adolescent Behavior Rating Scale: a tentative model of second-order factor structure across independent clinical samples. J Clin Psychol 1992;48:182–93. [14] Jensen PS, Salzberg AD, Richters JE, Watanabe HK. Scales, diagnoses, and child psychopathology: I. CBCL and DISC relationships. J Am Acad Child Adolesc Psychiatry 1993;32:397–406. [15] Diagnostic and statistical manual of mental disorders, fourth edition, technical revision (DSM-IV-TR). Washington, DC: American Psychiatric Assoc.; 2000. [16] Reynolds CR, Kamphaus RW. Behavioral Assessment System for Children: manual. Circle Pines, MN: American Guidance Services; 1992. [17] Reynolds CR, Kamphaus RW. Behavior Assessment System for Children: second edition manual. Circle Pines, MN: American Guidance Services; 2004. [18] Doyle A, Ostrander R, Skare S, Crosby RD, August GJ. Convergent and criterionrelated validity of the Behavior Assessment System for Children—Parent Rating Scale. J Clin Child Psychol 1997;26:276–84.

[19] Ostrander R, Weinfurt KP, Yarnold PR, August GJ. Diagnosing attention deficit disorders with the Behavioral Assessment System for Children and the Child Behavior Checklist: test and construct validity analyses using optimal discriminant classification trees. J Consult Clin Psychol 1998;66: 660–72. [20] Vaughn ML, Riccio CA, Hynd GW, Hall J. Diagnosing ADHD (predominantly inattentive and combined type subtypes): discriminant validity of the behavior assessment system for children and the Achenbach parent and teacher rating scales. J Clin Child Psychol 1997;26:349–57. [21] Wechsler D. Wechsler Abbreviated Scale of Intelligence. New York: Psychological Corp.; 1999. [22] Wechsler D. Wechsler Intelligence Scale for Children. fourth edition. New York: Psychological Corp.; 2003. [23] Caplan R, Austin JK. Behavioral aspects of epilepsy in children with mental retardation. Ment Retard Dev Disabil Res Rev 2000;6:293–9. [24] Achenbach TM, Rescorla LA. Manual for the ASEBA School-Age Forms and Profiles. Burlington: Univ. of Vermont, Research Center for Children, Youth, and Families; 2001. [25] Dorenbaum D, Cappelli M, Keene D, McGrath PJ. Use of a child behavior checklist in the psychosocial assessment of children with epilepsy. Clin Pediatr (Phila) 1985;24:634–7. [26] Fang PC, Chen YJ. Using the child behavior checklist to evaluate behavioral problems in children with epilepsy. Acta Paediatr Taiwan 2007;48:181–5. [27] Hermann BP. Psychopathology in epilepsy. Br J Psychiatry 1982;141:313–4. [28] Austin JK, Harezlak J, Dunn DW, Huster GA, Rose DF, Ambrosius WT. Behavior problems in children before first recognized seizures. Pediatrics 2001;107:115–22. [29] Dunn DW, Austin JK, Huster GA. Behaviour problems in children with newonset epilepsy. Seizure 1997;6:283–7. [30] Schoenfeld J, Seidenberg M, Woodard A, et al. Neuropsychological and behavioral status of children with complex partial seizures. Dev Med Child Neurol 1999;41:724–31. [31] Caplan R, Siddarth P, Gurbani S, Ott D, Sankar R, Shields WD. Psychopathology and pediatric complex partial seizures: seizure-related, cognitive, and linguistic variables. Epilepsia 2004;45:1273–81. [32] Myers K, Winters NC. Ten-year review of rating scales: I. Overview of scale functioning, psychometric properties, and selection. J Am Acad Child Adolesc Psychiatry 2002;41:114–22. [33] Drotar D. Shared dilemmas of research and practice in developmental and behavioral pediatrics: introduction to the special issue on methodology. J Dev Behav Pediatr 1991;12:347–9. [34] Koriath U, Gualtieri CT, Van Bourgondien ME, Quade D, Werry JS. Construct validity of clinical diagnosis in pediatric psychiatry: relationship among measures. J Am Acad Child Psychiatry 1985;24:429–36. [35] Weinstein SR, Noam GG, Grimes K, Stone K, Schwab-Stone M. Convergence of DSM-III diagnoses and self-reported symptoms in child and adolescent inpatients. J Am Acad Child Adolesc Psychiatry 1990;29:627–34. [36] Hermann BP, Whitman S. Behavioral and personality correlates of epilepsy: a review, methodological critique, and conceptual model. Psychol Bull 1984;95:451–97. [37] Eriksson KJ, Koivikko MJ. Prevalence, classification, and severity of epilepsy and epileptic syndromes in children. Epilepsia 1997;38:1275–82. [38] Hoare P, Kerley S. Psychosocial adjustment of children with chronic epilepsy and their families. Dev Med Child Neurol 1991;33:201–15. [39] Buelow JM, Austin JK, Perkins SM, Shen J, Dunn DW, Fastenau PS. Behavior and mental health problems in children with epilepsy and low IQ. Dev Med Child Neurol 2003;45:683–92. [40] Pavone P, Bianchini R, Trifiletti RR, Incorpora G, Pavone A, Parano E. Neuropsychological assessment in children with absence epilepsy. Neurology 2001;56:1047–51. [41] Harris ES, Canning RD, Kelleher KJ. A comparison of measures of adjustment, symptoms, and impairment among children with chronic medical conditions. J Am Acad Child Adolesc Psychiatry 1996;35:1025–32. [42] Holmes CS, Respess D, Greer T, Frentz J. Behavior problems in children with diabetes: disentangling possible scoring confounds on the Child Behavior Checklist. J Pediatr Psychol 1998;23:179–85. [43] Simmons RJ, Corey M, Cowen L, Keenan N, Robertson J, Levison H. Behavioral adjustment of latency age children with cystic fibrosis. Psychosom Med 1987;49:291–301. [44] Brown RT, Kaslow NJ, Sansbury L, Meacham L, Culler FL. Internalizing and externalizing symptoms and attributional style in youth with diabetes. J Am Acad Child Adolesc Psychiatry 1991;30:921–5. [45] Brown RT, Kaslow NJ, Hazzard AP, et al. Psychiatric and family functioning in children with leukemia and their parents. J Am Acad Child Adolesc Psychiatry 1992;31:495–502. [46] Reichenberg K, Broberg AG. Emotional and behavioural problems in Swedish 7- to 9-year olds with asthma. Chron Respir Dis 2004;1:183–9. [47] De Boer M, Grootenhuis M, Derkx B, Last B. Health-related quality of life and psychosocial functioning of adolescents with inflammatory bowel disease. Inflamm Bowel Dis 2005;11:400–6.