- Email: [email protected]
Scaling Structured Interview Data: A Comparison of Two Methods
Scaling Structured Interview Data: A Comparison of Two Methods JULIE CEREL, M.A., AND MARY A. FRISTAD, PH.D., A.B.P.P.
ABSTRACT Objective: Although structured interviews are currently considered essential assessment strategies for conducting research, the data they generate are typically not used for purposes beyond making categorical determinations about diagnoses. Because of the need for dimensional scales to be used in conjunction with categorical data, two dimensional scales constructed from structured interviews are presented and examined. One scale, Behavior, Anxiety, Mood, and Other (BAMO), provides an overall score by summing the percentage of symptoms endorsed for each of 20 behavior, anxiety, mood, and other disorders found in the Diagnostic Interview for Children and Adolescents-Revised (DICA-R, DSM-III-R version). Another scale, DICA-SUM, is constructed by summing all endorsed symptoms on the interview. In this study the psychometric and pragmatic characteristics of BAMO and DICA-SUM are compared. Method: Data were obtained from 570 children (331 bereaved, 110 depressed, 129 community) aged 5 to 18 years (mean ± SD = 11.3 ± 3.2) who were interviewed as part of an ongoing longitudinal childhood bereavement study from 1987 to 1996. Results: Discriminant and convergent validity with other child psychopathology measures are comparable for BAMO and DICA-SUM. However, BAMO more clearly conveys information regarding the approximate number of diagnoses endorsed. Conclusion: This study identified two methods of creating dimensional scales from structured interviews. Use of such dimensional scales might allow for improved comparison of results across studies. J. Am. Acad. Child Adolesc. Psychiatry, 2001, 40(3):341–346. Key Words: structured interviews, assessment, child psychopathology.
Structured interviews are currently considered the “gold standard” for clinical and research psychopathology assessment (Kendall et al., 1989). They provide the means for well-trained junior-level interviewers to obtain a great deal of information about a wide variety of problems an individual has experienced. Although structured interviews provide an effective means of obtaining and quantifying information about symptomatology, they frequently are not used to their fullest potential. Interviews are often used merely to assess the presence or absence of symptoms. However, psychopathology research could benefit from more accurate measurement of all symptoms endorsed, not only those symptoms that fulfill diagnostic criteria for a specific disorder. Thus the wealth of Accepted September 26, 2000. Ms. Cerel is a doctoral candidate in the Department of Psychology, and Dr. Fristad is an Associate Professor in the Departments of Psychiatry and Psychology, The Ohio State University, Columbus. Supported by NIMH grants 1 RO1 MH44135 and 1 RO1 MH45534. Reprint requests to Dr. Fristad, Division of Child/Adolescent Psychiatry, 1670 Upham Drive, Suite 460 Columbus, OH 43210-1250; e-mail: [email protected]. 0890-8567/01/4003-0341䉷2001 by the American Academy of Child and Adolescent Psychiatry.
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
information obtained on structured interviews regularly goes unused. In a recent examination of structured interviews, Hodges (1993) stressed the importance of obtaining information on the severity and presence of symptoms and of creating psychometrically reliable scales that would allow structured interviews to be used to their fullest potential. Although structured interviews are useful for making categorical diagnoses, they can also, as Hodges suggested, be used to make dimensional statements about dysfunction. A dimensional approach conveys more information about severity, is more reliable, and creates greater power for statistical analysis than does a categorical approach (Klein and Riso, 1993). As highlighted by the recent special section in this Journal, “better crosswalks are needed between the various dimensional and categorical approaches to the assessment of psychopathology” (Jensen et al., 1999, p. 119). These authors note that more refined approaches are needed if we are to understand the course of psychopathology. This includes early identification of children displaying subclinical symptoms who might later be at risk for serious impairment. Jensen and 341
CEREL AND FRISTAD
Watanabe (1999) suggest that children who score above a cutpoint on a scale yet do not meet full diagnostic criteria appear to be at high risk for subsequent psychopathology (Jensen and Watanabe, 1999). However, few attempts have been made to develop dimensional scales from structured interviews for clinical or research use. The purpose of this study is to describe a new dimensional scale created from the Diagnostic Interview for Children and Adolescents-Revised (DICA-R) (Reich and Welner, 1988; Welner et al., 1987), a commonly used structured interview that assesses DSM-III-R symptoms in children. As its name indicates, the Behavior, Anxiety, Mood, and Other scale (BAMO) provides a means of conveying information about the approximate number of behavior, anxiety, mood, and other diagnoses of childhood psychopathology. This study will examine psychometric properties of the BAMO scale and compare the new scale with an alternative dimensional strategy for scaling structured interview data. METHOD Participants Data were gathered from 570 children and their informant parent as part of an ongoing longitudinal examination of childhood bereavement. Participants included (1) 331children who had experienced the death of a parent within 2 months of study entry; (2) 110 children who had never experienced the death of a parent, were recruited from the inpatient unit and outpatient clinic in the Division of Child and Adolescent Psychiatry at a university hospital, and had a diagnosis of depression (major depressive disorder [55%], dysthymic disorder [23%], major depression/dysthymic disorder [20%], or bipolar disorder-depressed [2%]) at baseline; and (3) 129 community control children who had never experienced the death of a parent and who had not received mental health treatment in the 2 years prior to study entry. Children ranged in age from 5 to 18 years (mean ± SD = 11.3 ± 3.2). Procedure Children and their informant parent completed individual face-toface interviews at baseline and 5, 12, and 24 months later. Data presented are from the initial interviews, which were conducted between 1987 and 1996. All parents and children gave informed consent/ assent before the interview. Interviews were conducted by highly trained graduate and undergraduate students who were required to achieve interrater reliability ratings greater than 0.90 before interviewing independently. Instruments Several instruments from the full interview battery for the grief study were used in the present study. The DICA-R is a structured interview that is designed to assess the presence or absence of DSMIII-R symptoms in children (Reich and Welner, 1988; Welner et al., 1987). It accounts for symptoms across the range of childhood psychological disorders. The DICA has been demonstrated to have high
342
test-retest reliability and moderate correlations with chart diagnoses (Welner et al., 1987). Two clinical rating scales were used. The Children’s Depression Rating Scale-Revised is a 17-item rating scale that assesses current severity of depressive symptoms in children and adolescents (Poznanski et al., 1984). This scale has been shown to have good interrater reliability and test-retest reliability over a 4-week interval (Poznanski et al., 1984). The Children’s Global Assessment Scale (CGAS) was completed on each child as well (Shaffer et al., 1983). The CGAS is the childhood version of the Global Assessment Scale (Endicott et al., 1976) and is designed for completion by a clinician after the child has been interviewed. Scores range from 0 to 100, with lower scores reflecting overall severity of disturbance. In this study, CGAS scores were assigned after consensus conference review by two experienced child psychopathology experts. Children and their parents also completed self-report instruments. The Children’s Depression Inventory (CDI) is a 27-item self-report scale used to assess depressive symptoms in children and adolescents (Kovacs, 1985). The CDI also has an equivalent measure for the parent to complete about his or her child (CDI-P) (Weller and Weller, 1979). Test-retest reliability, concurrent validity, and sensitivity to severity of depressive illness have been demonstrated for the CDI (Kovacs, 1992). CDI-P scores have been shown to differentiate depressed children and psychiatric controls from community controls (Fristad et al., 1991). The Conners Revised Parent Rating Scale was also completed by parents (Conners, 1989). The Conners is a widely used, 48-item instrument originally used to help identify hyperactive children. However, it is also useful for identifying other symptoms or factor patterns such as conduct, learning, psychosomatic, and anxiety problems (Glow et al., 1982). When consent was obtained, several questionnaires were sent to one of the child’s teachers. The Child Behavior Checklist-Teacher’s Report Form provides a measure of the child’s behavior problems and adaptive functioning by obtaining information about the child’s behavior in school (Achenbach, 1991). Standard scores are obtained for internalizing and externalizing behaviors reported by the teacher. A large body of reliability and validity data has been published on this instrument. The Conners Revised Teacher Rating Scale is a 28-item self-report designed to identify problem behavior in children, including conduct, anxiety, and learning problems (Conners, 1973). The factors of the Conners have well-established reliability (Glow et al., 1982). Dimensional Scale Descriptions Both dimensional scales encompass the 20 disorders assessed on the DICA-R. These 20 disorders include six in behavior (attentiondeficit/hyperactivity disorder [ADHD], oppositional defiant disorder, conduct disorder, cigarette dependence, drug abuse, and alcohol abuse); six in anxiety (overanxious disorder, separation anxiety, avoidant disorder, phobia, obsessive-compulsive disorder, and posttraumatic stress disorder); two in mood (major depression and mania); and six in other areas (anorexia, bulimia, enuresis, encopresis, psychosis, and somatization disorder). Child and parent reports were obtained by separate interviewers, and an “either” score was created to capture information obtained by either child or parent report (BAMO-E; DICA-SUM-E). Scale 1: DICA-SUM. The simplest way to create a scale from a structured interview is merely to sum all endorsed symptoms. DICA-SUM does just this, treating all symptoms equally regardless of their “importance.” This yields a range of 0 to 205. However, this method can create unequal weighting, so that disorders represented by the most questions can become disproportionately influential. For example, on the DICA there are 15 symptom items for ADHD and 8 for overanxious disorder. By the DICA-SUM method, ADHD gets almost double the weight of overanxious disorder (Table 1).
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
SCALING STRUCTURED INTERVIEWS
TABLE 1 Clinical Vignettes Showing Differences Between DICA-SUM and BAMO Scale Properties Vignette No.
Case
DICA Symptom Endorsements
Clinical Diagnosesa
DICA-SUM (out of 205)
BAMO (out of 20)
1
15-year-old male
20/22 Substance abuse sxb
Polysubstance dependence
20
0.91
2
10-year-old male
12/15 Attention-deficit/ hyperactivity disorder sx 7/9 Oppositional defiant disorder sx Smokes cigarettes
Attention-deficit/hyperactivity disorder
20
2.08
Oppositional defiant disorder Nicotine abuse
5/7 Obsessive-compulsive sx 8/10 Separation anxiety disorder sx 6/8 Generalized anxiety disorder sx Dark phobia
Obsessive-compulsive disorder Separation anxiety disorder Generalized anxiety disorder Specific phobia
20
2.76
3
12-year-old female
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. a Clinical diagnoses = DSM-IV diagnoses as obtained by standard clinician interview. b sx = symptom. Scale 2: BAMO. To avoid the problem of unequal weighting, we created a second scale designed to treat all major disorders equally regardless of the number of symptoms contained in each disorder. Thus, for each disorder, the number of symptoms endorsed is divided by the total number of symptoms for that disorder. This yields a score between 0 (no symptoms) and 1 (all possible symptoms endorsed) for each disorder. The score obtained for each disorder is then summed. By doing this, we use BAMO to treat most disorders equally. To account for differential impact on functioning, cigarette smoking, enuresis, encopresis, and phobia are down-weighted by half in the final summation. Thus the total BAMO score can range from 0 (i.e., no endorsements) to 18 (i.e., every symptom endorsed for all 20 disorders). This weighting of questions to equalize the contribution of most disorders to the total score was thought to more accurately communicate findings across studies and to more clearly convey the extent of endorsements about a child. To illustrate this point, three vignettes are provided in Table 1 to demonstrate how DICA-SUM can remain essentially constant as the number of clinical diagnoses and BAMO scores increase. To show that values of DICA-SUM and BAMO that correspond to each “diagnosis equivalent” differ widely and overlap
substantially, actual values of DICA-SUM and BAMO—including means, standard deviations, and ranges—are provided in Table 2.
TABLE 2 Comparison of Actual Values of DICA-SUM and BAMO
TABLE 3 Descriptive Statistics for BAMO and DICA-SUM
Data Analysis Descriptive statistics were used to provide information on the distribution of scores for DICA-SUM and BAMO. Convergent validity was determined with Pearson correlation coefficients, and descriptive validity was determined with analyses of variance (ANOVAs). All tests were two-tailed. Degrees of freedom reported as nonwhole numbers resulted from the Satterthwaite correction for unequal variance (Satterthwaite, 1946). RESULTS Descriptive Statistics
Descriptive statistics for BAMO and DICA-SUM are presented in Table 3. As expected, both scales were posi-
DICA-SUM BAMO
Mean
SD
Range
n
0.0–0.49 0.5–0.99 1.0–1.99 2.0–2.99 3.0–3.99 4.0+
2.5 8.8 17.7 30.5 38.7 51.1
2.2 3.2 5.0 5.5 7.2 5.1
0–10 3–21 7–34 20–45 22–54 41–62
207 109 140 68 31 15
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents. BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder.
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
BAMO Mean SD Range DICA-SUM Mean SD Range
Child
Parent
Either
0.7 0.8 0–4.8
0.7 0.9 0–4.7
1.2 1.1 0–5.4
8.2 9.6 0–50
8.6 10.4 0–60
14.0 13.2 0–61
Note: BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents.
343
CEREL AND FRISTAD
tively skewed because the overall sample was relatively healthy.
erately correlated with child-informant and teacherinformant measures (range: r = 0.34–0.51).
Convergent Validity
Discriminant Validity
BAMO-E and DICA-SUM-E scores were significantly correlated with scores from child, parent, and teacher informants (Table 4). As expected, BAMO-E and DICASUM-E were strongly and positively correlated with most combined-informant, child-informant, and parentinformant child psychopathology measures (range: r = 0.52–0.82). BAMO-E and DICA-SUM-E scores were moderately correlated with teacher-informant measures (range: r = 0.39–0.50). Also as expected, BAMO-C and DICA-SUM-C scores were strongly and positively correlated with other combined-informant and child-informant measures (range: r = 0.59–0.71; Table 4). BAMO-C and DICASUM-C scores were moderately correlated with parentinformant measures (range: r = 0.51–0.53) and modestly correlated with teacher-informant measures (range: r = 0.26–0.37). Likewise, BAMO-P and DICA-SUM-P ratings showed strong positive correlations to other combined-informant and parent-informant measures (range: r = 0.60–0.79; Table 4). BAMO-P and DICA-SUM-P scores were mod-
ANOVAs were conducted to ascertain whether BAMO and DICA-SUM scores would differ significantly between three known groups. BAMO-E scores were significantly different between depressed, community control, and bereaved children (F2,567 = 226.04, p < .001). Post hoc pairwise comparisons indicated that depressed children scored significantly higher than bereaved children (t439 = 18.07, p < .001) and that bereaved children scored significantly higher than community controls (t458 = 6.02, p < .001). Essentially identical results were obtained when BAMO-C and BAMO-P were used to discriminate groups. An ANOVA conducted on DICA-SUM-E showed similar results (F2,567 = 230.94, p < .001), with depressed children scoring higher than bereaved children (t439 = 17.50, p < .01) and bereaved children scoring higher than community controls (t458 = 7.40, p < .001). When DICASUM-C and DICA-SUM-P were used to discriminate groups, the results were essentially identical. Because both scales were positively skewed, we reanalyzed the data after using the square-root transformation of the scales. Results were essentially identical with those
TABLE 4 Convergent Validity for Either (E), Child (C), and Parent Report (P) DICA-SUM and BAMO BAMO
Combined report CDRS-R CGAS Child report CDRS-R CDI Parent report CDI-P CRPRS Teacher report CRTRS TRF-Internalizing TRF-Externalizing TRF
DICA-SUM
E
C
P
E
C
P
0.72* 0.77*
0.66* –0.67*
0.60* –0.71*
0.77* –0.82*
0.71* –0.71*
0.62* –0.76*
0.64* 0.52*
0.64* 0.59*
0.48* 0.35*
0.69* 0.54*
0.70* 0.62*
0.51* 0.36*
0.72* 0.71*
0.51* 0.51*
0.72* 0.75*
0.74* 0.74*
0.51* 0.53*
0.73* 0.79*
0.43* 0.40* 0.40* 0.50*
0.26* 0.28* 0.29* 0.37*
0.46* 0.37* 0.39* 0.47*
0.43* 0.39* 0.39* 0.49*
0.25* 0.27* 0.27* 0.35*
0.45* 0.34* 0.38* 0.46*
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents. BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. CDRS-R = Children’s Depression Rating Scale-Revised; CGAS = Children’s Global Assessment Scale; CDI = Children’s Depression Inventory; CDI-P = Children’s Depression Inventory-Parent Version; CRPRS = Conners Revised Parent Rating Scale-Hyperactivity Index; CRPRS = Conners Revised Teacher Rating Scale-Hyperactivity Index; TRF = Teacher’s Report Form, Sum T score. * p < .0001.
344
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
SCALING STRUCTURED INTERVIEWS
obtained on the untransformed data, so we chose to report the nontransformed results, to provide more easily understood scales. DISCUSSION
In this study we created two scales from a commonly used structured interview for children. DICA-SUM sums total number of endorsements (range: 0–205). BAMO calculates a sum of the percentage of each disorder endorsed. Descriptive statistics, convergent validity, and discriminant validity were presented for each scale, on the basis of a sample of 610 children. Because the sample was relatively healthy, both scales were positively skewed. Both showed high correlations with other child and parent instruments and moderate correlations with teacher data. Both scales can successfully differentiate known groups. Despite these similarities, BAMO and DICA-SUM also have some differences. Each has unique advantages and disadvantages. DICA-SUM is easier to calculate than BAMO, which requires slightly more programming to determine a score. However, DICA-SUM scores are more difficult to interpret because the raw number conveys little meaning about clinical presentation. On the other hand, BAMO more clearly conveys clinical information by approximating the number of disorders a child is experiencing. This is done by providing a number equivalent to the number of disorders the child would be experiencing if he or she were to endorse all possible symptoms for each disorder. In addition, if similar transformations are made with other structured interviews, such as the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Epidemiologic version (Orvaschel, 1985) or the National Institute of Mental Health Diagnostic Interview Schedule for Children (DISC-IV) (Shaffer et al., 2000), BAMO scores could be compared across studies, as long as the number of disorders being compared were comparable (i.e., if BAMO-DICA and BAMO-DISC each have a potential range of 0–18, a score of 2.48 achieved on each would suggest that a similar number of disorders were being endorsed on each interview). Limitations
This study describes two methods of developing dimensional information from data obtained from structured interviews that traditionally are used for categorical purposes only. Several limitations are notable in this study. First, the scales were created with archival data from a larger J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
study that was conducted under the DSM-III-R diagnostic system. Although the structured interview used in the study is well known and commonly used (i.e., the DICA-R, DSM-III-R version), its more recent revision conforms to more contemporary diagnostic criteria. Second, we provide convergence validity data primarily for measures of depression, as the available data set emphasized depression variables. Third, the sample used in this examination consisted primarily of relatively healthy children and adolescents. If more psychiatrically impaired children were examined, issues related to the skew of the sample would be less salient. Finally, the generalizability of the psychometric validation presented in this article is restricted by limitations of the assessment instrument used. Like most structured interviews designed to assess a broad range of psychopathology in children, the DICA does not encompass all childhood disorders (e.g., pervasive developmental disorders, tics, panic disorder). However, instruments that assess these diagnostic categories can easily be adapted into scales such as those presented in this article. Clinical Implications
Structured interviews are currently considered to be essential to conducting assessment research. By creating scales from structured interviews such as those described in this article, researchers can combine categorical and dimensional criteria that will enable them to use structured interviews to their fullest potential. BAMO offers a means of describing the extent of symptoms, as it provides an approximation of the number of disorders a child is experiencing (if he or she endorses all possible symptoms of a disorder). Creation of similar scales from other structured interviews would allow for improved comparison of results across studies and would allow more extensive use of the wealth of information obtained from structured interviews. It is hoped that the use of such scales will assist clinicians to evaluate findings across research studies, thereby helping to translate research findings into clinical practice. REFERENCES Achenbach TM (1991), Child Behavior Checklist: Teacher’s Report Form. Burlington: University of Vermont Center for Children, Youth, & Families Conners C (1989), Manual for Conner’s Rating Scales. Toronto: Multi-Health Systems Conners CK (1973), Rating scales for use in drug studies with children. Psychopharmacol Bull 9:24–84 (special issue: Pharmacotherapy With Children) Endicott J, Spitzer RL, Fleiss JL, Cohen J (1976), The Global Assessment Scale (GAS). Arch Gen Psychiatry 33:766–771 Fristad MA, Weller EB, Weller RA, Teare M, Preskorn SH (1991), The parent’s version of the Children’s Depression Inventory: a reliability and validity study. Ann Clin Psychiatry 3:341–346
345
CEREL AND FRISTAD
Glow RA, Glow PH, Rump EE (1982), The stability of child behavior disorders: a one year test-retest study of Adelaide versions of Conners Teacher and Parent Rating Scales. J Abnorm Child Psychol 6:221–236 Hodges K (1993), Structured interviews for assessing children. J Child Psychol Psychiatry 34:49–68 Jensen PS, Brooks-Gunn J, Graber JA (1999), Dimensional scales and diagnostic categories: constructing crosswalks for child psychopathology assessment. J Am Acad Child Adolesc Psychiatry 38:118–120 Jensen PS, Watanabe H (1999), Sherlock Holmes and child psychopathology assessment approaches: the case of the false-positive. J Am Acad Child Adolesc Psychiatry 38:138–146 Kendall PC, Cantwell DP, Kazdin AE (1989), Depression in children and adolescents: assessment issues and recommendations. Cogn Ther Res 13:109–146 Klein DN, Riso LP (1993), Psychiatric disorders: problems of boundaries and comorbidity. In: Basic Issues in Psychopathology, Costello CG, ed. New York: Guilford Kovacs M (1985), The Children’s Depression Inventory (CDI). Psychopharmacol Bull 21:995–998 Kovacs M (1992), The Children’s Depression Inventory (CDI) Manual. Toronto: Multi-Health Systems
346
Orvaschel H (1985), Psychiatric interviews suitable for research with children and adolescents. Psychopharmacol Bull 21:737–748 Poznanski EO, Grossman JA, Buchsbaum Y, Banegas M, Freeman L, Gibbons R (1984), Preliminary studies of the reliability and validity of the Children’s Depression Rating Scale. J Am Acad Child Psychiatry 23:191–197 Reich W, Welner Z (1988), The Diagnostic Interview for Children and Adolescents-Revised-Child/Parent Forms. St Louis: Washington University Satterthwaite FW (1946), An approximate distribution of estimates of variance components. Biometrics Bull 2:110–114 Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME (2000), NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry 39:28–38 Shaffer D, Gould MS, Brasic J et al. (1983), A children’s global assessment scale (CGAS). Arch Gen Psychiatry 40:1228–1231 Weller EB, Weller RA (1979), Children’s Depression Inventory-Parent Form. Kansas City, KS: University of Kansas Medical Center Welner Z, Reich W, Herjanic B, Jung KG (1987), Reliability, validity, and parent–child agreement studies of the Diagnostic Interview for Children and Adolescents (DICA). J Am Acad Child Adolesc Psychiatry 26:649–653
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
ABSTRACT Objective: Although structured interviews are currently considered essential assessment strategies for conducting research, the data they generate are typically not used for purposes beyond making categorical determinations about diagnoses. Because of the need for dimensional scales to be used in conjunction with categorical data, two dimensional scales constructed from structured interviews are presented and examined. One scale, Behavior, Anxiety, Mood, and Other (BAMO), provides an overall score by summing the percentage of symptoms endorsed for each of 20 behavior, anxiety, mood, and other disorders found in the Diagnostic Interview for Children and Adolescents-Revised (DICA-R, DSM-III-R version). Another scale, DICA-SUM, is constructed by summing all endorsed symptoms on the interview. In this study the psychometric and pragmatic characteristics of BAMO and DICA-SUM are compared. Method: Data were obtained from 570 children (331 bereaved, 110 depressed, 129 community) aged 5 to 18 years (mean ± SD = 11.3 ± 3.2) who were interviewed as part of an ongoing longitudinal childhood bereavement study from 1987 to 1996. Results: Discriminant and convergent validity with other child psychopathology measures are comparable for BAMO and DICA-SUM. However, BAMO more clearly conveys information regarding the approximate number of diagnoses endorsed. Conclusion: This study identified two methods of creating dimensional scales from structured interviews. Use of such dimensional scales might allow for improved comparison of results across studies. J. Am. Acad. Child Adolesc. Psychiatry, 2001, 40(3):341–346. Key Words: structured interviews, assessment, child psychopathology.
Structured interviews are currently considered the “gold standard” for clinical and research psychopathology assessment (Kendall et al., 1989). They provide the means for well-trained junior-level interviewers to obtain a great deal of information about a wide variety of problems an individual has experienced. Although structured interviews provide an effective means of obtaining and quantifying information about symptomatology, they frequently are not used to their fullest potential. Interviews are often used merely to assess the presence or absence of symptoms. However, psychopathology research could benefit from more accurate measurement of all symptoms endorsed, not only those symptoms that fulfill diagnostic criteria for a specific disorder. Thus the wealth of Accepted September 26, 2000. Ms. Cerel is a doctoral candidate in the Department of Psychology, and Dr. Fristad is an Associate Professor in the Departments of Psychiatry and Psychology, The Ohio State University, Columbus. Supported by NIMH grants 1 RO1 MH44135 and 1 RO1 MH45534. Reprint requests to Dr. Fristad, Division of Child/Adolescent Psychiatry, 1670 Upham Drive, Suite 460 Columbus, OH 43210-1250; e-mail: [email protected]. 0890-8567/01/4003-0341䉷2001 by the American Academy of Child and Adolescent Psychiatry.
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
information obtained on structured interviews regularly goes unused. In a recent examination of structured interviews, Hodges (1993) stressed the importance of obtaining information on the severity and presence of symptoms and of creating psychometrically reliable scales that would allow structured interviews to be used to their fullest potential. Although structured interviews are useful for making categorical diagnoses, they can also, as Hodges suggested, be used to make dimensional statements about dysfunction. A dimensional approach conveys more information about severity, is more reliable, and creates greater power for statistical analysis than does a categorical approach (Klein and Riso, 1993). As highlighted by the recent special section in this Journal, “better crosswalks are needed between the various dimensional and categorical approaches to the assessment of psychopathology” (Jensen et al., 1999, p. 119). These authors note that more refined approaches are needed if we are to understand the course of psychopathology. This includes early identification of children displaying subclinical symptoms who might later be at risk for serious impairment. Jensen and 341
CEREL AND FRISTAD
Watanabe (1999) suggest that children who score above a cutpoint on a scale yet do not meet full diagnostic criteria appear to be at high risk for subsequent psychopathology (Jensen and Watanabe, 1999). However, few attempts have been made to develop dimensional scales from structured interviews for clinical or research use. The purpose of this study is to describe a new dimensional scale created from the Diagnostic Interview for Children and Adolescents-Revised (DICA-R) (Reich and Welner, 1988; Welner et al., 1987), a commonly used structured interview that assesses DSM-III-R symptoms in children. As its name indicates, the Behavior, Anxiety, Mood, and Other scale (BAMO) provides a means of conveying information about the approximate number of behavior, anxiety, mood, and other diagnoses of childhood psychopathology. This study will examine psychometric properties of the BAMO scale and compare the new scale with an alternative dimensional strategy for scaling structured interview data. METHOD Participants Data were gathered from 570 children and their informant parent as part of an ongoing longitudinal examination of childhood bereavement. Participants included (1) 331children who had experienced the death of a parent within 2 months of study entry; (2) 110 children who had never experienced the death of a parent, were recruited from the inpatient unit and outpatient clinic in the Division of Child and Adolescent Psychiatry at a university hospital, and had a diagnosis of depression (major depressive disorder [55%], dysthymic disorder [23%], major depression/dysthymic disorder [20%], or bipolar disorder-depressed [2%]) at baseline; and (3) 129 community control children who had never experienced the death of a parent and who had not received mental health treatment in the 2 years prior to study entry. Children ranged in age from 5 to 18 years (mean ± SD = 11.3 ± 3.2). Procedure Children and their informant parent completed individual face-toface interviews at baseline and 5, 12, and 24 months later. Data presented are from the initial interviews, which were conducted between 1987 and 1996. All parents and children gave informed consent/ assent before the interview. Interviews were conducted by highly trained graduate and undergraduate students who were required to achieve interrater reliability ratings greater than 0.90 before interviewing independently. Instruments Several instruments from the full interview battery for the grief study were used in the present study. The DICA-R is a structured interview that is designed to assess the presence or absence of DSMIII-R symptoms in children (Reich and Welner, 1988; Welner et al., 1987). It accounts for symptoms across the range of childhood psychological disorders. The DICA has been demonstrated to have high
342
test-retest reliability and moderate correlations with chart diagnoses (Welner et al., 1987). Two clinical rating scales were used. The Children’s Depression Rating Scale-Revised is a 17-item rating scale that assesses current severity of depressive symptoms in children and adolescents (Poznanski et al., 1984). This scale has been shown to have good interrater reliability and test-retest reliability over a 4-week interval (Poznanski et al., 1984). The Children’s Global Assessment Scale (CGAS) was completed on each child as well (Shaffer et al., 1983). The CGAS is the childhood version of the Global Assessment Scale (Endicott et al., 1976) and is designed for completion by a clinician after the child has been interviewed. Scores range from 0 to 100, with lower scores reflecting overall severity of disturbance. In this study, CGAS scores were assigned after consensus conference review by two experienced child psychopathology experts. Children and their parents also completed self-report instruments. The Children’s Depression Inventory (CDI) is a 27-item self-report scale used to assess depressive symptoms in children and adolescents (Kovacs, 1985). The CDI also has an equivalent measure for the parent to complete about his or her child (CDI-P) (Weller and Weller, 1979). Test-retest reliability, concurrent validity, and sensitivity to severity of depressive illness have been demonstrated for the CDI (Kovacs, 1992). CDI-P scores have been shown to differentiate depressed children and psychiatric controls from community controls (Fristad et al., 1991). The Conners Revised Parent Rating Scale was also completed by parents (Conners, 1989). The Conners is a widely used, 48-item instrument originally used to help identify hyperactive children. However, it is also useful for identifying other symptoms or factor patterns such as conduct, learning, psychosomatic, and anxiety problems (Glow et al., 1982). When consent was obtained, several questionnaires were sent to one of the child’s teachers. The Child Behavior Checklist-Teacher’s Report Form provides a measure of the child’s behavior problems and adaptive functioning by obtaining information about the child’s behavior in school (Achenbach, 1991). Standard scores are obtained for internalizing and externalizing behaviors reported by the teacher. A large body of reliability and validity data has been published on this instrument. The Conners Revised Teacher Rating Scale is a 28-item self-report designed to identify problem behavior in children, including conduct, anxiety, and learning problems (Conners, 1973). The factors of the Conners have well-established reliability (Glow et al., 1982). Dimensional Scale Descriptions Both dimensional scales encompass the 20 disorders assessed on the DICA-R. These 20 disorders include six in behavior (attentiondeficit/hyperactivity disorder [ADHD], oppositional defiant disorder, conduct disorder, cigarette dependence, drug abuse, and alcohol abuse); six in anxiety (overanxious disorder, separation anxiety, avoidant disorder, phobia, obsessive-compulsive disorder, and posttraumatic stress disorder); two in mood (major depression and mania); and six in other areas (anorexia, bulimia, enuresis, encopresis, psychosis, and somatization disorder). Child and parent reports were obtained by separate interviewers, and an “either” score was created to capture information obtained by either child or parent report (BAMO-E; DICA-SUM-E). Scale 1: DICA-SUM. The simplest way to create a scale from a structured interview is merely to sum all endorsed symptoms. DICA-SUM does just this, treating all symptoms equally regardless of their “importance.” This yields a range of 0 to 205. However, this method can create unequal weighting, so that disorders represented by the most questions can become disproportionately influential. For example, on the DICA there are 15 symptom items for ADHD and 8 for overanxious disorder. By the DICA-SUM method, ADHD gets almost double the weight of overanxious disorder (Table 1).
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
SCALING STRUCTURED INTERVIEWS
TABLE 1 Clinical Vignettes Showing Differences Between DICA-SUM and BAMO Scale Properties Vignette No.
Case
DICA Symptom Endorsements
Clinical Diagnosesa
DICA-SUM (out of 205)
BAMO (out of 20)
1
15-year-old male
20/22 Substance abuse sxb
Polysubstance dependence
20
0.91
2
10-year-old male
12/15 Attention-deficit/ hyperactivity disorder sx 7/9 Oppositional defiant disorder sx Smokes cigarettes
Attention-deficit/hyperactivity disorder
20
2.08
Oppositional defiant disorder Nicotine abuse
5/7 Obsessive-compulsive sx 8/10 Separation anxiety disorder sx 6/8 Generalized anxiety disorder sx Dark phobia
Obsessive-compulsive disorder Separation anxiety disorder Generalized anxiety disorder Specific phobia
20
2.76
3
12-year-old female
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. a Clinical diagnoses = DSM-IV diagnoses as obtained by standard clinician interview. b sx = symptom. Scale 2: BAMO. To avoid the problem of unequal weighting, we created a second scale designed to treat all major disorders equally regardless of the number of symptoms contained in each disorder. Thus, for each disorder, the number of symptoms endorsed is divided by the total number of symptoms for that disorder. This yields a score between 0 (no symptoms) and 1 (all possible symptoms endorsed) for each disorder. The score obtained for each disorder is then summed. By doing this, we use BAMO to treat most disorders equally. To account for differential impact on functioning, cigarette smoking, enuresis, encopresis, and phobia are down-weighted by half in the final summation. Thus the total BAMO score can range from 0 (i.e., no endorsements) to 18 (i.e., every symptom endorsed for all 20 disorders). This weighting of questions to equalize the contribution of most disorders to the total score was thought to more accurately communicate findings across studies and to more clearly convey the extent of endorsements about a child. To illustrate this point, three vignettes are provided in Table 1 to demonstrate how DICA-SUM can remain essentially constant as the number of clinical diagnoses and BAMO scores increase. To show that values of DICA-SUM and BAMO that correspond to each “diagnosis equivalent” differ widely and overlap
substantially, actual values of DICA-SUM and BAMO—including means, standard deviations, and ranges—are provided in Table 2.
TABLE 2 Comparison of Actual Values of DICA-SUM and BAMO
TABLE 3 Descriptive Statistics for BAMO and DICA-SUM
Data Analysis Descriptive statistics were used to provide information on the distribution of scores for DICA-SUM and BAMO. Convergent validity was determined with Pearson correlation coefficients, and descriptive validity was determined with analyses of variance (ANOVAs). All tests were two-tailed. Degrees of freedom reported as nonwhole numbers resulted from the Satterthwaite correction for unequal variance (Satterthwaite, 1946). RESULTS Descriptive Statistics
Descriptive statistics for BAMO and DICA-SUM are presented in Table 3. As expected, both scales were posi-
DICA-SUM BAMO
Mean
SD
Range
n
0.0–0.49 0.5–0.99 1.0–1.99 2.0–2.99 3.0–3.99 4.0+
2.5 8.8 17.7 30.5 38.7 51.1
2.2 3.2 5.0 5.5 7.2 5.1
0–10 3–21 7–34 20–45 22–54 41–62
207 109 140 68 31 15
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents. BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder.
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
BAMO Mean SD Range DICA-SUM Mean SD Range
Child
Parent
Either
0.7 0.8 0–4.8
0.7 0.9 0–4.7
1.2 1.1 0–5.4
8.2 9.6 0–50
8.6 10.4 0–60
14.0 13.2 0–61
Note: BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents.
343
CEREL AND FRISTAD
tively skewed because the overall sample was relatively healthy.
erately correlated with child-informant and teacherinformant measures (range: r = 0.34–0.51).
Convergent Validity
Discriminant Validity
BAMO-E and DICA-SUM-E scores were significantly correlated with scores from child, parent, and teacher informants (Table 4). As expected, BAMO-E and DICASUM-E were strongly and positively correlated with most combined-informant, child-informant, and parentinformant child psychopathology measures (range: r = 0.52–0.82). BAMO-E and DICA-SUM-E scores were moderately correlated with teacher-informant measures (range: r = 0.39–0.50). Also as expected, BAMO-C and DICA-SUM-C scores were strongly and positively correlated with other combined-informant and child-informant measures (range: r = 0.59–0.71; Table 4). BAMO-C and DICASUM-C scores were moderately correlated with parentinformant measures (range: r = 0.51–0.53) and modestly correlated with teacher-informant measures (range: r = 0.26–0.37). Likewise, BAMO-P and DICA-SUM-P ratings showed strong positive correlations to other combined-informant and parent-informant measures (range: r = 0.60–0.79; Table 4). BAMO-P and DICA-SUM-P scores were mod-
ANOVAs were conducted to ascertain whether BAMO and DICA-SUM scores would differ significantly between three known groups. BAMO-E scores were significantly different between depressed, community control, and bereaved children (F2,567 = 226.04, p < .001). Post hoc pairwise comparisons indicated that depressed children scored significantly higher than bereaved children (t439 = 18.07, p < .001) and that bereaved children scored significantly higher than community controls (t458 = 6.02, p < .001). Essentially identical results were obtained when BAMO-C and BAMO-P were used to discriminate groups. An ANOVA conducted on DICA-SUM-E showed similar results (F2,567 = 230.94, p < .001), with depressed children scoring higher than bereaved children (t439 = 17.50, p < .01) and bereaved children scoring higher than community controls (t458 = 7.40, p < .001). When DICASUM-C and DICA-SUM-P were used to discriminate groups, the results were essentially identical. Because both scales were positively skewed, we reanalyzed the data after using the square-root transformation of the scales. Results were essentially identical with those
TABLE 4 Convergent Validity for Either (E), Child (C), and Parent Report (P) DICA-SUM and BAMO BAMO
Combined report CDRS-R CGAS Child report CDRS-R CDI Parent report CDI-P CRPRS Teacher report CRTRS TRF-Internalizing TRF-Externalizing TRF
DICA-SUM
E
C
P
E
C
P
0.72* 0.77*
0.66* –0.67*
0.60* –0.71*
0.77* –0.82*
0.71* –0.71*
0.62* –0.76*
0.64* 0.52*
0.64* 0.59*
0.48* 0.35*
0.69* 0.54*
0.70* 0.62*
0.51* 0.36*
0.72* 0.71*
0.51* 0.51*
0.72* 0.75*
0.74* 0.74*
0.51* 0.53*
0.73* 0.79*
0.43* 0.40* 0.40* 0.50*
0.26* 0.28* 0.29* 0.37*
0.46* 0.37* 0.39* 0.47*
0.43* 0.39* 0.39* 0.49*
0.25* 0.27* 0.27* 0.35*
0.45* 0.34* 0.38* 0.46*
Note: DICA-SUM refers to a scale that sums all possible symptom endorsements of the Diagnostic Interview for Children and Adolescents. BAMO refers to the Behavior, Anxiety, Mood, and Other scale. Scores represent a sum of percentages of symptoms endorsed for each disorder. CDRS-R = Children’s Depression Rating Scale-Revised; CGAS = Children’s Global Assessment Scale; CDI = Children’s Depression Inventory; CDI-P = Children’s Depression Inventory-Parent Version; CRPRS = Conners Revised Parent Rating Scale-Hyperactivity Index; CRPRS = Conners Revised Teacher Rating Scale-Hyperactivity Index; TRF = Teacher’s Report Form, Sum T score. * p < .0001.
344
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
SCALING STRUCTURED INTERVIEWS
obtained on the untransformed data, so we chose to report the nontransformed results, to provide more easily understood scales. DISCUSSION
In this study we created two scales from a commonly used structured interview for children. DICA-SUM sums total number of endorsements (range: 0–205). BAMO calculates a sum of the percentage of each disorder endorsed. Descriptive statistics, convergent validity, and discriminant validity were presented for each scale, on the basis of a sample of 610 children. Because the sample was relatively healthy, both scales were positively skewed. Both showed high correlations with other child and parent instruments and moderate correlations with teacher data. Both scales can successfully differentiate known groups. Despite these similarities, BAMO and DICA-SUM also have some differences. Each has unique advantages and disadvantages. DICA-SUM is easier to calculate than BAMO, which requires slightly more programming to determine a score. However, DICA-SUM scores are more difficult to interpret because the raw number conveys little meaning about clinical presentation. On the other hand, BAMO more clearly conveys clinical information by approximating the number of disorders a child is experiencing. This is done by providing a number equivalent to the number of disorders the child would be experiencing if he or she were to endorse all possible symptoms for each disorder. In addition, if similar transformations are made with other structured interviews, such as the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Epidemiologic version (Orvaschel, 1985) or the National Institute of Mental Health Diagnostic Interview Schedule for Children (DISC-IV) (Shaffer et al., 2000), BAMO scores could be compared across studies, as long as the number of disorders being compared were comparable (i.e., if BAMO-DICA and BAMO-DISC each have a potential range of 0–18, a score of 2.48 achieved on each would suggest that a similar number of disorders were being endorsed on each interview). Limitations
This study describes two methods of developing dimensional information from data obtained from structured interviews that traditionally are used for categorical purposes only. Several limitations are notable in this study. First, the scales were created with archival data from a larger J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1
study that was conducted under the DSM-III-R diagnostic system. Although the structured interview used in the study is well known and commonly used (i.e., the DICA-R, DSM-III-R version), its more recent revision conforms to more contemporary diagnostic criteria. Second, we provide convergence validity data primarily for measures of depression, as the available data set emphasized depression variables. Third, the sample used in this examination consisted primarily of relatively healthy children and adolescents. If more psychiatrically impaired children were examined, issues related to the skew of the sample would be less salient. Finally, the generalizability of the psychometric validation presented in this article is restricted by limitations of the assessment instrument used. Like most structured interviews designed to assess a broad range of psychopathology in children, the DICA does not encompass all childhood disorders (e.g., pervasive developmental disorders, tics, panic disorder). However, instruments that assess these diagnostic categories can easily be adapted into scales such as those presented in this article. Clinical Implications
Structured interviews are currently considered to be essential to conducting assessment research. By creating scales from structured interviews such as those described in this article, researchers can combine categorical and dimensional criteria that will enable them to use structured interviews to their fullest potential. BAMO offers a means of describing the extent of symptoms, as it provides an approximation of the number of disorders a child is experiencing (if he or she endorses all possible symptoms of a disorder). Creation of similar scales from other structured interviews would allow for improved comparison of results across studies and would allow more extensive use of the wealth of information obtained from structured interviews. It is hoped that the use of such scales will assist clinicians to evaluate findings across research studies, thereby helping to translate research findings into clinical practice. REFERENCES Achenbach TM (1991), Child Behavior Checklist: Teacher’s Report Form. Burlington: University of Vermont Center for Children, Youth, & Families Conners C (1989), Manual for Conner’s Rating Scales. Toronto: Multi-Health Systems Conners CK (1973), Rating scales for use in drug studies with children. Psychopharmacol Bull 9:24–84 (special issue: Pharmacotherapy With Children) Endicott J, Spitzer RL, Fleiss JL, Cohen J (1976), The Global Assessment Scale (GAS). Arch Gen Psychiatry 33:766–771 Fristad MA, Weller EB, Weller RA, Teare M, Preskorn SH (1991), The parent’s version of the Children’s Depression Inventory: a reliability and validity study. Ann Clin Psychiatry 3:341–346
345
CEREL AND FRISTAD
Glow RA, Glow PH, Rump EE (1982), The stability of child behavior disorders: a one year test-retest study of Adelaide versions of Conners Teacher and Parent Rating Scales. J Abnorm Child Psychol 6:221–236 Hodges K (1993), Structured interviews for assessing children. J Child Psychol Psychiatry 34:49–68 Jensen PS, Brooks-Gunn J, Graber JA (1999), Dimensional scales and diagnostic categories: constructing crosswalks for child psychopathology assessment. J Am Acad Child Adolesc Psychiatry 38:118–120 Jensen PS, Watanabe H (1999), Sherlock Holmes and child psychopathology assessment approaches: the case of the false-positive. J Am Acad Child Adolesc Psychiatry 38:138–146 Kendall PC, Cantwell DP, Kazdin AE (1989), Depression in children and adolescents: assessment issues and recommendations. Cogn Ther Res 13:109–146 Klein DN, Riso LP (1993), Psychiatric disorders: problems of boundaries and comorbidity. In: Basic Issues in Psychopathology, Costello CG, ed. New York: Guilford Kovacs M (1985), The Children’s Depression Inventory (CDI). Psychopharmacol Bull 21:995–998 Kovacs M (1992), The Children’s Depression Inventory (CDI) Manual. Toronto: Multi-Health Systems
346
Orvaschel H (1985), Psychiatric interviews suitable for research with children and adolescents. Psychopharmacol Bull 21:737–748 Poznanski EO, Grossman JA, Buchsbaum Y, Banegas M, Freeman L, Gibbons R (1984), Preliminary studies of the reliability and validity of the Children’s Depression Rating Scale. J Am Acad Child Psychiatry 23:191–197 Reich W, Welner Z (1988), The Diagnostic Interview for Children and Adolescents-Revised-Child/Parent Forms. St Louis: Washington University Satterthwaite FW (1946), An approximate distribution of estimates of variance components. Biometrics Bull 2:110–114 Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME (2000), NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry 39:28–38 Shaffer D, Gould MS, Brasic J et al. (1983), A children’s global assessment scale (CGAS). Arch Gen Psychiatry 40:1228–1231 Weller EB, Weller RA (1979), Children’s Depression Inventory-Parent Form. Kansas City, KS: University of Kansas Medical Center Welner Z, Reich W, Herjanic B, Jung KG (1987), Reliability, validity, and parent–child agreement studies of the Diagnostic Interview for Children and Adolescents (DICA). J Am Acad Child Adolesc Psychiatry 26:649–653
J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 4 0 : 3 , M A RC H 2 0 0 1