- Email: [email protected]
Internal Consistency of DSM-III Diagnoses Using the Symptom Scales of the Child Assessment Schedule
Internal Consistency of DSM-III Diagnoses Using the Symptom Scales of the Child Assessment Schedule KAY HODGES , PH.D. , WILLIAM B. SAUNDERS, M .P .H. , JAVAD KASHANI, M .D . , KIM HAMLETT, PH.D. , AND ROBERT J. THOMPSON, JR., PH.D. Abstract. Internal consistency of the major diagnostic categories for children was assessed, using the symptom scale scores of the Child Assessment Schedule . Alpha coefficients were calculated for three samples : 116 nonpsychotic psychiatrically disturbed children, 63 children with cystic fibrosis, and 177 children from a community based sample. For the psychiatric sample, a high level of internal consistency was demonstrated for all the symptom scales (i.e., attention deficit, conduct, anxiety, and depression). For the nonpsychiatric samples , the attention deficit and depression scales were reliable, with lower levels of endorsement and more variability observed for the other scales. These results are supportive of the clustering of diagnostic criteria present in DSM-Ill. J . Am . Acad. Child Adolesc. Psychiatry , 1990,29,4:635-641. Key Words: interviewing, internal consistency, diagnosis. The purpose of this study was to assess the internal consistency of the major diagnostic categories for children , based on the DSM-III (American Psychiatric Association, 1980). A relatively high degree of internal consistency would be anticipated for the various disorders, given the assumptions of DSM-III. For each disorder , essential features are identified and variations of the disorder are categorized into subtypes. Symptoms are identified , and these symptoms are arranged in clusters that regularly co-occur, constituting syndromes. Disorders are defined by syndromes which are differentially associated with etiological factors, biological correlates , natural history of the illness , and responsiveness to specific interventions (Carlson and Cantwell, 1980; George et al., 1989). In a technical sense, internal consistency reflects on diagnoses at the symptom level. In this study, internal consistency refers to the reliability of a group of symptoms that make up a syndrome. The most widely used technique to assess reliability is Cronbach' s coefficient alpha that ranges from 0 to 1 (Nunnally, 1978). Coefficient alpha can be thought of as the average correlation of each symptom with every other symptom in the syndrome. For example, if a syndrome is shown to be internally consistent (i.e., with an alpha above 0.70), a child with one or two symptoms is
most likely to be characterized by the other symptoms as well. In contrast, for a syndrome with low internal consistency , the endorsement of a symptom in a syndrome will not predict the endorsement of the other symptoms. Thus , if a syndrome can be shown to be highly consistent, a clinician who detects endorsement of a symptom can be confident that the other symptoms in the syndrome are likely to be endorsed . Establishing that the syndrome of symptoms identified in a disorder regularly co-occur is a logical prerequisite to studying the disorder. In fact , in the adult literature , the degree of internal consistency of the DSM-III diagnoses has been found to vary considerably across diagnoses and across diagnostic subtypes (George et al., 1989). The development of structured diagnostic interviews for children provides an opportunity to study the internal consistency of the childhood disorders. However, little attention has been given to this issue (Edelbrock and Costello, 1988; Hodges and Cools, 1990). Data on internal consistency have only been reported for the Schedule for Affective Disorders and Schizophrenia for School-Aged Children (K-SADS) (Puig-Antich and Chambers, 1978). Internal consistency was based on the relationship between the items in the scale and summary ratings made by the interviewer that reflected a synthesis of information from a variety of sources. High internal consistency was generally observed for the depression summary scales as well as for conduct disorder. However, the alpha values were markedly low for somatization (i.e., ex = 0.25) and for anxiety disorders (i.e., ex = 0.39) (Chambers et al., 1985). In the present study, the symptom scale scores of the Child Assessment Schedule (CAS) (Hodges et al. , 1982) were used. The CAS is a semistructured interview that was developed in 1978 from a traditional clinical interview for children (Hodges et al., 1981). A subsequent revision of the interview (dated 1983/85) was used in the present study . This version primarily included some additional diagnostic items and an onset and duration section . However, the content of the core items and the general format were retained . Since this study was begun , items have also been added to incorporate the criteria for DSM-III-R. This most recent version of the CAS, for DSM-III-R, and the diagnostic index
Accepted August 30, 1989. Dr. Hodges is Associate Professor, William Saunders is Research Analyst, and Dr. Thompson is Professor in the Department of Psychiatry at Duke University Medical Center. Dr. Kashani is Professor in the Department of Psychiatry at the University ofMissouri Medical Center. Dr . Hamlett was a Postdoctoral Fellow in the Division of Medical Psychology at Duke University Medical Center and is now in the Department of Pediatrics at Rainbow Babies and Children's Hospital . The authors thank Dr . Lea O'Qui nn f or her support of this project, Dr. William H . Wilson for his statistical advice, G. Kay Bishop, who served as research assistant , and Mildred Crabtree for manuscript preparation. . This research project was supported in part by National Institute of Health Grant ROI HL 3 7548 to Dr . Thompson. Request reprints to Dr. Hodges, who is now at Eastern Michigan University, 537 Mark Jefferson , Ypsilanti, MI48197. 0890-8567/90/2904-0635$02.0010© 1990 by the American Academy of Child and Adolescent Psychiatry.
635
HODGES ET AL.
for determining diagnoses can be obtained from the first author. The CAS generates diagnoses as well as symptom scores for scales containing diagnostically-related items. The CAS is thematically organized around 11 topic areas including: school, friends and activities and hobbies, family , fears, worries and anxieties, self-image, mood (especially sadness) , physical complaints, expression of anger, and reality testing symptomatology. Embedded within these content areas are diagnostically-related items that correspond to the DSM-III and DSM-III-R (American Psychiatric Association, 1987) diagnostic criteria . Internal consistency of the CAS diagnostically-related symptom scales was determined for three samples of children, as follows: psychiatric patients, children with a chronic medical illness (cystic fibrosis), and children from a community-based sample. These samples reflect a continuum of risk for psychiatric disorder. Children with chronic medical illness have a 10% to 15% increased risk of psychological/behavioral problems (Pless and Roghmann, 1971; Pless et al., 1972). Furthermore, in recent studies using parental report , children with cystic fibrosis were found to have higher rates of psychiatric illness (Steinhausen et al., 1983) and more symptoms suggesting oppositional ity and general neurotic disturbance (Breslau, 1985), compared to controls. Internal consistency was also determined for the parent version of the CAS that was available for the psychiatric and community-based samples . Thus, the authors were also able to determine the generalizability of the findings to a different informant, the parents, and to nonpsychiatric samples of children . Method Subjects Psychiatrically disturbed children. This group consisted of 116 subjects, 87 of whom were inpatients at Duke University Medical Center and 29 of whom were outpatients attending a psychiatric clinic at the University of Missouri Medical School. The outpatient sample has been previously described in Hodges et al. (1987) . The entire sample included 76 boys, with a mean age of 10 years (SD = 1.97) , and 40 girls, also with a mean age of 10 years (SD = 2.27) . The age range was 6 to 16 years, with 96% of the subjects being 13-years-old or younger . Seventy-two percent of the sample were Caucasian and 28% were black . The socioeconomic status (SES) of subjects was based on the Hollingshead (1975, unpublished manuscript) Four Factor Index of Social Status and was as follows: 21% Class I; 21% Class II; 23.5 % Class III; 18.5% Class IV; and 16% Class V. For both the inpatient and outpatient samples, the subjects were consecutive referrals that met the criteria for inclusion in the studies . The inclusionary criteria were: (1) agreement to participate in the study, (2) not psychotic , mentally retarded, or diagnosed as organic brain syndrome, and (3)' had a parental figure who was currently living with the child and was able to provide a history about the child's symptoms. For the outpatient sample, all patients who met the above criteria were invited to participate in the study and
636
all but one were willing to do so. For the inpatient sample, the CAS was part of the formal evaluation procedures on the unit. However, for 14 of the inpatient children, no interview was attempted although their parents were administered the parent version of the CAS. The reasons included: being untestable due to severe noncompliance (i.e., refused to participate with expected diagnostic or therapeutic activities) (N = 3) , severe stuttering and vocalizations (N = 1), transfer from the unit shortly after admission (N = 2), and unavailability of the child interviewer while on leave (N = 8), For both samples, the requirements of the institutional review boards were met. Parents ojpsychiatrically disturbed children. Ninety-two parents were administered the parent version of the CAS , in which they reported on their children's symptomatology. There were 62 parents of the inpatient children at Duke and 30 parents of the outpatients at the University of Missouri. One parent was included in the outpatient sample, but her child was not included in the sample because the child had a significant hearing loss that interfered with his ability to communicate with the examiner. For the inpatient sample, parent interviews were not accomplished for the first 30 children who were administered the CAS. Also there were an additional nine parents who were so uncooperative or unavailable that they did not participate in the routine evaluation procedures on the unit. The children of the entire parent sample included 59 males , with a mean age of 10 years (SD = 2.25), and 33 females, with a mean age of 11 years (SD = 2.19). Chronically medically ill children . Sixty-three children with cystic fibrosis were interviewed as part of a larger , ongoing project on coping skills of children with cystic fibrosis. They were recruited and interviewed during routine visits to the Cystic Fibrosis Clinic at Duke University Medical Center. Two of the participants were seen during brief hospitalizations for routine assessment of pulmonary functioning. The sample consisted of 34 boys with a mean age of 10 years (SD = 3.07) and 29 girls with a mean age of 12 years (SD = 3.17) . The ages ranged from 6 to 17 years, with 73% of the sample being 13-years-old or younger. All subjects were Caucasian, as would be anticipated given that cystic fibrosis is an autosomal recessive genetic disorder almost exclusively seen in Caucasians. The SES of the group consisted of: none in Class I; 10% in Class II; 22% in Class III; 37% in Class IV; and 31% in Class V. The exclusionary criteria were: (1) the child was mentally retarded or (2) the parent was not willing to participate as an informant. Only the children were administered the CAS. Community-based sample ojchildren. The subject sample is described elsewhere (Kashani et aI., 1989). This group consisted of 177 children and adolescents who represented a systemic sampling of a midwestern public school system, stratified by age and sex. An equal number of males and females were interviewed for three age groups : 8-, 12-, and 17-year-olds. For the purposes of this study, the 17-yearolds who were post-high school were deleted since they were not asked questions about school and peers. There were 89 boys, with a mean age of 11 years (SD = 3.39), and 88 girls, with a mean age of 11 years (SD = 3.32). J. Am. Acad. Child Adolesc .Psychiatry, 29:4, July 1990
INTERNAL CONSISTENCY
Eighty-seven percent of the subjects were Caucasian . The SES of the subjects was as follows: 22% Class I; 36% Class II; 25% Class III; 16% Class IV; and 1% Class V. Parents of the community based sample of children. At least one parent for each of the children in the sample was interviewed as part of the requirements for participation in the study. The informant was usually the mother. The characteristics of the children of these parents is the same as described above. Child Assessment Schedule There is considerable evidence of the reliability and validity of the CAS diagnostically-related symptom scales . Psychometric studies of the CAS have included assessment of interrater, test-retest, and interinformant reliability as well as contrast group validity and construct validity (Hodges, 1987; Hodges and Cools , 1990). The CAS consists of three sections. First , the child is asked questions about 11 topic areas (i.e., school, friends , etc.). Following these inquiries, the child is asked about onset and duration for positive symptoms. In a separate section, the interviewer records observations about the child for 53 targeted behaviors . Each CAS item is scored as "yes," "no, " " ambiguous," or " nonscorable." Response items are phrased such that an affirmative response always indicates the presence of symptomatology . Guidelines for scoring are detailed in the training manual for the CAS (Hodges, 1983, unpublished manuscript) . Using these guidelines as a reference, the child's literal "yes" or "no" responses do not necessarily reflect the recorded scores. The interviewer scores the response item based on his/her understanding of the child's intended communication (e.g., whether the child is trying to truly describe ruminations or obsessions). Scores for the diagnostically-related scales were computer generated. The computer program was based on a diagnostic index that matches the CAS items to specific diagnostic criteria (Hodges, 1985 unpublished manuscript). Since this study was begun before DSM-III-R items were added to the CAS, data relevant to DSM-III diagnostic criteria are presented. The parent form of the CAS (P-CAS) parallels the child interview . The same inquiries are made; parents are asked about their child. There are fewer items on the P-CAS because the third section , consisting of observational judgments about the child, is omitted. The P-CAS also yields the same diagnoses and scores as the CAS. Procedures At all three sites, the interviewers were trained with the CAS "Guidelines to Aid in Establishing Interrater Reliability" (Hodges, 1983, unpublished manuscript). Psychiatric sample . The interview was administered separately to parents and children and by different interviewers who were blind to each others ' rating. The interviewers for the children were as follows: (1) for the inpatients , an experienced female child psychologist (i.e. , the senior author) and (2) for the outpatients, two male medical students. The l .Am .Acad. Child Adolesc . Psychiatry,29:4,July 1990
interviewers for the parents were: (1) for the inpatients, two experienced , female psychiatric social workers , and (2) for the outpatients, two female undergraduate students . Lay examiners were used in the outpatient study because it was preferred to have interviewers who were blind to all sources of information about the child and also did not have a clear preference for a particular interview (Hodges et al., 1987). All of the interviewers for both the outpatient and inpatient study were trained by the senior author. In the outpatient study, training demonstrations were conducted via a one-way mirror. The criterion was an average item-byitem agreement of 0 .85% or better. For the inpatient study, videotaped interviews were used for training and the criterion was five consecutive interviews with a kappa of 0.75 or greater. The inpatients were typically administered the interview within the first 2 weeks of admission to the hospital as part of the routine clinical evaluation. The outpatients were interviewed during either the first or second visit to the clinic in a counterbalanced design. Chronically medically ill sample. The interview was administered only to the children. The interviewer was a female, clinical child psychologist. The interviewer was trained by the senior author, with a kappa of 0 .75 or greater for five consecutive interviews used as a criterion . There was periodic consultation to guard against interviewer drift. Community based sample. The interview took place in the subject's home and involved both the child and parent . Families were contacted initially by telephone, at which time the purposes and procedures of the project were delineated and participation payments ($50 .00 per child) were offered . The interviews were conducted by doctoral students in psychology or child and family development, all of whom had previous experience working with children . Training in administration was done until the following criteria were met: (1) achieving an item-by-item criterion of a Kappa of ~ 0.80 during the practice interview role playing; and (2) achieving an interrater reliability of K = 0.85 . All interviewers were periodically monitored to prevent interviewer drift. Analyses Alpha coefficient was calculated for each diagnosticallyrelated scale, using the Kuder-Richardson formula. For the purposes of analysis , endorsement of a symptom corresponded to a "yes" response and nonendorsement included all other response options. Alpha ranges from 0 to 1, with an alpha value of 0.70 or greater typically regarded as acceptable for traditional multiitem psychological question naires (Nunnally , 1978). For the purposes of this study, alpha values of 0.70 or above are considered high, and values between 0.60 and 0 .70 are considered moderate. The size of the reliability coefficient is a function of both the average correlation among items as well as the number of items in the scale. Thus, relatively lower coefficients would be anticipated for scales with fewer items or for mildly symptomatic samples (i.e., in which only a few items are endorsed). 637
HODGES ET AL. TABLE
1. Internal Consistency of the Diagnostically-Related Scales of the Child Version of the Child Assessment Schedule (CAS)
Alpha Coefficients Community Based Sample (N = 177)
No. of Items in CAS Scale
Psychiatric Patients (N = 116)
Attention deficit disorder ADD w/hyperactivity ADD w/out hyperactivity
14 11
0.81 0.83
0.74 0.75
0.77 0.78
Conduct disorder symptoms Aggressive subtype Nonaggressive subtype
17 10 7
0.80 0.73 0.64
0.60"
0.69 b 0.61" 0.65
Depression symptoms Major depressive episode Dysthymia
43 31 36
0.87 0.83 0.86
0.87 0.80 0.87
0.75 0.68 0.75
Anxiety symptoms Separation anxiety Overanxious
19 12 7
0.80 0.71 0.65
0.62 0.56 0.56
0.74 0.68 0.58
DSM-III-Based Scales
a b
Medically Chronically III (N = 63)
Excluded because fewer than five items were endorsed by any subject. Half or more of the items in the scale were not endorsed by any subject in the sample.
Results
Child Version Psychiatric patients. The results for all three samples for the child version of the CAS are presented in Table 1. High levels of internal consistency were observed for all the major diagnostic categories, all having alpha values of 0.80 or greater. Additionally, the subtypes for attention deficit disorder (ADD) (with and without hyperactivity) as well as for depression (major depressive episode and dysthymia) were also characterized by good internal consistency. For conduct disorder, the aggressive subtype had a high reliability at 0.73, with nonaggressive being somewhat lower (alpha = 0.64). The same pattern was observed in regard to the anxiety disorders, with separation anxiety characterized by a high alpha (0.71), with a lower value resulting for overanxious disorder (alpha = 0.65). All of the items on the diagnostic scales were endorsed by at least one subject, with the exception of two items from the observational judgment section (i.e., difficulty separating from parent and slow speech with long latencies). Medically chronically ill. High internal consistency was observed for ADD and depression as well as for the subtypes within these diagnostic categories (alpha > 0.70). Moderate levels of reliability were observed for the major categories of conduct disorder symptoms and anxiety symptoms. Only 41 % of the conduct disorder items were endorsed by any of the subjects. The results for subtypes of conduct disorder could not be calculated because there were fewer than five items on both of these scales. The somewhat lower value observed for conduct disorder probably reflects the lower number of items in the scale. Lower levels of internal consistency were also observed for anxiety symptoms, despite the fact that there was no reduction in the number of items in the scale compared to the psychiatric group. The alpha values observed for sep638
aration anxiety and overanxious disorder (0.56 for both scales) were notably lower compared to any of the other scales. Community based sample. The alpha values for each of the major diagnostic categories were acceptable (0.69 to 0.77). High reliability was observed for both subtypes for ADD. The lowest value was observed for conduct disorder symptoms (0.69). These results are similar to those observed for the chronically medically ill. There were only seven items endorsed, with moderate correlations observed for both aggressive and nonaggressive subtypes that consisted of five and six items, respectively. When all anxiety symptoms were combined, the results were considerably better (0.74) than for each of the subtypes. The same pattern was observed as seen in the psychiatric group, with the alpha value being higher for separation anxiety than for overanxious disorder.
Parent Version Psychiatric sample. Table 2 contains the results for the parent version of the CAS. High internal consistency was observed for all the major diagnostic categories (alpha of 0.77 or greater). In addition, both subtypes of ADD (with or without hyperactivity) as well as both subtypes of depression were characterized by very good internal consistency. For conduct disorder and anxiety, all the subtypes were characterized by at least moderate levels of reliability. Somewhat higher internal consistency was observed for nonaggressive conduct disorder, compared to aggressive. There was no notable difference between separation anxiety and overanxious disorder. Community based sample. High internal consistency was observed for ADD, with both subtypes having an alpha value of 0.78. The scales for anxiety symptoms and depressive symptoms were characterized by moderate levels of reliaJ.Am. Acad. Child Adolesc. Psychiatry, 29:4, July 1990
INTERNAL CONSISTENCY T ABLE
2. Internal Consistency of the Diagnostically-Related Scales of the Parent Version of the Child Assessment Schedule (P-CAS) Alpha Coeffic ients Number of Items in P-CAS Scale
Psychiatric Sample (N = 92)
Community Based Sample (N = 177)
Attention deficit disorder ADD w/hyperactivity ADD w/out hyperactivity
12 II
0.85 0.85
0.78 0.78
Conduct disorder symptoms Aggressive subtype Nonaggressive subtype
17 10 7
0.77 0.63 0.71
0.4 0" 0.40
Depression symptoms Major depressive episode Dysthymia
42 30 36
0.89 0.85 0.88
0.66 0.55 0.67
Anxiety symptoms Separation anxiety Overanxious
19 12 7
0.78 0.67 0.69
0.66 0.52 0.63
DSM-lll-Based Scales
a b
No items were endorsed by any subject. Half or more of the items in the scale were not endorsed by any subject in the sample.
bility. In contrast to the other scales, there was far less endorsement of conduct disorder symptoms. Two-thirds of the items were not endorsed by any parent. Thus , the conduct disorder scale consisted of only six items and had low internal consistenc y. Discussion
This set of findings yields several implications regarding diagnostic nomenclature for children . ADD appears to consist of related symptoms (i.e., with alpha coefficients ranging from 0.74 to 0. 83), whether assessed in highly symptomatic or in largely asymptomatic children, or whether assessed via child self-report or parent report. The high reliability for the DSM-IIl subtype of ADD with hyperactivity is supportive of DSM-III-R, given that this subtype most approximates attention deficit hyperactivity disorder in DSM-IIl-R. The high internal consistency observed for ADD without hyperactivity would suggest that perhaps this diagnostic category, which was present in DSM-III , should not have been eliminated in DSM-III-R . Research from factor analytic studies would also lend support to this viewpoint; separate factors have been identified for the dimensions of inattention/di sorganization and motor hyperactivity/impulsivity (Lahey et aI. , 1988). This has led some researchers to advocate for differentially using the DSM-III -R categories of attention deficit hyperactivity disorder and undifferentiat ed ADD, with the latter employed for cases in which the feature of hyperactivity is absent (Lahey et aI., 1988). The conduct disorder scale was found to be reliable for the psychiatric samples based on the parent or child report. For the two nonpsychiatric samples, the low endorsement rate reduced the coefficients of internal consistency in these samples. These results are very supportive of the diagnostic criteria; high reliability was observed for the psychiatric sample, and low prevalence of symptoms was observed for the nonpsychiatric samples. J .Am.Acad. Child Adolesc. Psy chiatry , 29:4, July 1990
The depression symptom scale as well as the scales for both subtypes of depression were characterized by very high internal consistency for all child samples and for the parents of the psychiatrically disturbed children. These results lend support to the decision to use adult criteria for depression in diagnosing children , as do the empirical finding s regarding the phenomenology of depression in children (Ryan et aI., 1987; Mitchell et aI., 1988). However , while it may be the case that the essential features of depression may be similar for children and adults, many empirical questions remain to be addressed. For example, there may be agespecific associated features of depression that are different for children (e.g., see Kovacs and Paulauskas, 1984). Larger cross-sectional as well as longitudinal studies are needed. Also, while these depressive symptoms clustered together into a meaningful and internally consistent syndrome for children, the question still remains as to whether the disorder of depression in children is similar to the adult disorders in terms of etiology, course of illness, biological correlates , and response to treatment (Carlson and Garber, 1986). Similar phenomenolog y does not imply similar underlying mechanisms. The results relevant to the anxiety disorders suggest that more empirical research related to nosology may be warranted. The scales for anxiety symptoms (combining both subtypes) and for separation anxiety disorder had adequate reliability for the psychiatrically disturbed children. There was a tendency for the separation anxiety scale to be characterized by higher internal consistenc y than the overanxious scale . However, this may be partially secondary to the fact that there were almost half as many items in the overanxious scale than in the separation anxiety scale . The more moderate reliability values observed for anxiety in the cystic fibrosis sample are somewhat surprising given that children with chronic illness in general and more specifically, children with cystic fibrosis, are characterized by more internalizing symptoms, including anxiety (Breslau , 639
HODGES ET AL.
1985; Thompson, 1985). Perhaps the anxiety symptoms clusters delineated in the DSM -III and (DSM -III-R) disorders do not capture well the phenomeno logy of anxiety present in nonpsychiatric, anxious children. Empirical studies comparing anxiety-disordered psychiatric children to nonpsychiatric samples, who are at risk for heightened anxiety (e.g., chronically ill, psychosomatic or traumatized/stressed children), may help differentiate meaningful patterns of representation of anxiety in children . Chronically ill children may have heightened anxiety symptoms because of their dependence on their parents to do their daily medical care. Terminally ill children have heightened anxiety symptoms because issues of loss are more imminent for them. It is probably important to consider the role of trait- versus statedependent (i.e., situational) symptoms in assessing anxiety. The results of this study compare favorably with the only other data available on interviews for children. As with the K-SADS, alpha values for depression and conduct disorder were high. In contrast, for anxiety, low internal consistency was observed with the K-SADS . In the present study, high reliability was observed for the anxiety symptom scale for the psychiatric and community based sample, based on child report, and for the psychiatric group , based on parent report. Thus, the findings in this study lend support to the syndromes encompassed in the childhood disorders in DSM -III and DSM-III-R. Additional studies to determine whether these findings are generalizable to other samples are warranted, given that the subjects in the present study may not necessarily be representative of the relevant populations (i.e., of psychiatrically ill children, chronically ill children , etc.) . For example, there was a larger representation of higher SES families among the subjects obtained primarily at Duke University Medical Center, compared to the community sample. This, in part, may be secondary to Duke being a private, teaching hospital. Also the cystic fibrosis sample was restricted to Caucasians and to persons willing to participate in a treatment study. Additionally, there were method differences across the sites, such as level of training of interviewers (i.e ., lay versus mental health professionals). However, the empirical data to date have not demonstrated significant differences between lay and clinician interviewers provided that both administer the same diagnostic interview (Costello et al. , 1984). In any case, this degree of variability across the samples, which was in part intentional (i.e., risk for psychiatric disorder), does not compromise the findings relevant to internal consistency. The stability of the relationships among the items (i.e ., internal consistency) would not be assumed to be determined by these exogenous factors, as reflected by the findings. These results provide additional evidence of the psychometric integrity of the CAS , in as much as the expression of the criteria via the CAS items reflects positively on the instrument. Continuing to explore the psychometric properties of diagnostically-related symptom scales is a critical prerequisite to using these scales. Likewise, the operationalization of diagnostic criteria for childhood disorders deserves critical scrutiny. 640
References Americ an Psychiatric Association (1987), Diagnostic and Statistical Manual of Mental Disorders, Third Edition-Revised. Washington, DC: American Psychiatric Assoc iation. - - (1980) , Diagnostic and Statistical Manual of Mental Disorders, Third Edition. Washington, DC : American Psychiatric Associ ation. Breslau, N. (1985), Psychiatric disorder in children with physical disabilities . l. Am. Acad. Child Psychiatry, 24:87-94 . Carlson, G. & Garber , J . (1986), Developmental issues in the classification of depression in children . In: Depression in Young People: Developmental and Clinical Perspective. ed . M. Rutt er, C . Izard , & P. Read. New York: Guilford Press, pp . 399--434. - - Cantwell , D. (1980) , A survey of depressive symptoms , syndrome and disorder in a child psychiatric popu lation. l . Child Psychol. Psychiatry, 21:19-25. Chambers , W. J., Puig-Anti ch, J., Hirsch , M., Paez, P., Ambrosini, P. J., Tabrizi, M . A . & Davies, M . (198 5) , The assessment of affective disorde rs in children and adolesce nts by semis tructured interview : test-retest reliability of the K-SADS . Arch. Gen . Psychiatry, 42:696--702. Costello, A. J . , Edelbrock, L. S. , Dulcan , J . K. , Kalas, R. & Klaric , S. H. (1984), Report on the NIMH Diagnostic Interview Schedule for Children (DIS-C). Washington, DC: National Institut e of Mental Health . Edelbrock , C . & Costello , A. J . (1988) , Structured psychiatric interviews for children. In: Assessment and Diagnosis in Child Psychopathology , ed. M. Rutter, A. H. Tuma & I. Lann . New York: Guilford Press, pp. 87-112 . George , L., Blazer, D. , Woo dbury , M . & Manton, K. (1989), Internal consistency of DSM-III diagnoses. In: The Validity of Psychiatric Diagnosis, ed. L. N. Robins, & J. E. Barrett. New York: Raven Press, pp. 99-125 . Hodges, K. (1987), Assessing children with a cli nical research interview : the child assessment schedule. In: Advances in Behavioral Assessment ofChildren and Families, Vol. 3. ed. R. J. Prinz. Green wich , CT: JAI Press, pp . 203-233, - - Cools, J . N. (1990), Structured diagnostic interviews. In: Through the Eyes of the Child: Obtaining Self-repo rt fro m Children and Adolescents. ed. A. M . LaGreca. Boston , MA: Allyn and Bacon , pp. 109-149. - - McKnew, D. , Burbach , D. J . & Roebuck, L. (1987) , Diagnostic concorda nce between the Child Asse ssment Schedul e (CA S) and the Schedule for Affective Disorders and Schizophrenia for SchoolAge Children (K-SADS ) in an outp atient sample usin g lay interviewers. l. Am. Acad. Child Adolesc. Psychiatry, 26:654-661. - - Kline , J. , Stem , L. , Cytryn , L. & McKnew , D. (1982), The development of a child assess ment schedule for research and cli nical use . J , Abnorm. Child Psychol. , 10:173- 189. - - Kline, J. , Fitch , P., McKnew, D . & Cytry n , L. (1981) , The Child Assess ment Schedule: a diagnostic interview for researc h and clinical use . Catalog of Selected Documents in Psychology, 11:56. Kashani , J ., Orvaschel , H. , Rosenberg, T . & Reid , J. (1989) , Psychopathology among a community sample of children and adolescents: a developmental perspective . J , Am . Acad . Child Adolesc . Psychiatry, 28:70 1- 706 . Kovacs, M . & Paulauskas, S. L. (1984), Developmental stage and the expression of depress ive disorders in children: an empirical analysis. In: Childhood Depression: New Directions for Child Development, D. Cicchetti & K. Schneider-Rosen . San Fran cisco: JosseyBass, pp. 59--80 . Lahey , B. B. , Pelham , W. E. , Schaughency , E. A . et al. (1988), Dimensions and types of attention deficit disorder. J , Am . Acad. Child Adolesc. Psychiatry. 27:330-335 . Mitchell, J. , McCauley , E . , Burke , P. & Moss, S. 1. (1988) , Phenomenology of depression in children and adolescents . J , Am . Acad. Child Adolesc. Psychiatry. 27:12-20. Nunnally , J . (1978) , Psychometric Theory , 2nd Ed . New York: McGraw-Hili Book Comp any . Pless, I. B. & Roghmann , K. J. (1971), Chronic illness and its consequences: observations based on three epidemiologic surveys. J, Pediatr., 79:351- 359.
l.Am .Acad . Child Adolesc . Psychiatry, 29:4, Jul y 1990
INTERNAL CONSISTENCY
- - - - Haggerty, R. (1972), Chronic illness, family functioning, and psychosocial adjustment: a model for the allocation of mental health services. Int. J. Epidemiol., 1:271-277. Puig-Antich, J & Chambers, W. (1978), The Schedule for Affective Disorders and Schizopheniafor School-Age Children (Kiddie-SADS). New York: New York State Psychiatric Institute. Ryan, N. D., Puig-Antich, J., Ambrosini, P. et al. (1987). The clinical picture of major depression in children and adolescents. Arch. Gen.
J.Am.Acad. Child Adolesc.Psychiatry, 29:4, July 1990
Psychiatry, 44:854-861. Steinhausen, H., Schindler, H. & Stephan, H. J. (1983), Comparative psychiatric studies on children and adolescents suffering from cystic fibrosis and bronchial asthma. Child Psychiatry Hum. Dev. 14:117130. Thompson, R. J., Jr. (1985), Delineation of children's behavior problems: a basis for assessment and intervention. J. Dev. Behav. Pediatr. 6:37-50.
641
most likely to be characterized by the other symptoms as well. In contrast, for a syndrome with low internal consistency , the endorsement of a symptom in a syndrome will not predict the endorsement of the other symptoms. Thus , if a syndrome can be shown to be highly consistent, a clinician who detects endorsement of a symptom can be confident that the other symptoms in the syndrome are likely to be endorsed . Establishing that the syndrome of symptoms identified in a disorder regularly co-occur is a logical prerequisite to studying the disorder. In fact , in the adult literature , the degree of internal consistency of the DSM-III diagnoses has been found to vary considerably across diagnoses and across diagnostic subtypes (George et al., 1989). The development of structured diagnostic interviews for children provides an opportunity to study the internal consistency of the childhood disorders. However, little attention has been given to this issue (Edelbrock and Costello, 1988; Hodges and Cools, 1990). Data on internal consistency have only been reported for the Schedule for Affective Disorders and Schizophrenia for School-Aged Children (K-SADS) (Puig-Antich and Chambers, 1978). Internal consistency was based on the relationship between the items in the scale and summary ratings made by the interviewer that reflected a synthesis of information from a variety of sources. High internal consistency was generally observed for the depression summary scales as well as for conduct disorder. However, the alpha values were markedly low for somatization (i.e., ex = 0.25) and for anxiety disorders (i.e., ex = 0.39) (Chambers et al., 1985). In the present study, the symptom scale scores of the Child Assessment Schedule (CAS) (Hodges et al. , 1982) were used. The CAS is a semistructured interview that was developed in 1978 from a traditional clinical interview for children (Hodges et al., 1981). A subsequent revision of the interview (dated 1983/85) was used in the present study . This version primarily included some additional diagnostic items and an onset and duration section . However, the content of the core items and the general format were retained . Since this study was begun , items have also been added to incorporate the criteria for DSM-III-R. This most recent version of the CAS, for DSM-III-R, and the diagnostic index
Accepted August 30, 1989. Dr. Hodges is Associate Professor, William Saunders is Research Analyst, and Dr. Thompson is Professor in the Department of Psychiatry at Duke University Medical Center. Dr. Kashani is Professor in the Department of Psychiatry at the University ofMissouri Medical Center. Dr . Hamlett was a Postdoctoral Fellow in the Division of Medical Psychology at Duke University Medical Center and is now in the Department of Pediatrics at Rainbow Babies and Children's Hospital . The authors thank Dr . Lea O'Qui nn f or her support of this project, Dr. William H . Wilson for his statistical advice, G. Kay Bishop, who served as research assistant , and Mildred Crabtree for manuscript preparation. . This research project was supported in part by National Institute of Health Grant ROI HL 3 7548 to Dr . Thompson. Request reprints to Dr. Hodges, who is now at Eastern Michigan University, 537 Mark Jefferson , Ypsilanti, MI48197. 0890-8567/90/2904-0635$02.0010© 1990 by the American Academy of Child and Adolescent Psychiatry.
635
HODGES ET AL.
for determining diagnoses can be obtained from the first author. The CAS generates diagnoses as well as symptom scores for scales containing diagnostically-related items. The CAS is thematically organized around 11 topic areas including: school, friends and activities and hobbies, family , fears, worries and anxieties, self-image, mood (especially sadness) , physical complaints, expression of anger, and reality testing symptomatology. Embedded within these content areas are diagnostically-related items that correspond to the DSM-III and DSM-III-R (American Psychiatric Association, 1987) diagnostic criteria . Internal consistency of the CAS diagnostically-related symptom scales was determined for three samples of children, as follows: psychiatric patients, children with a chronic medical illness (cystic fibrosis), and children from a community-based sample. These samples reflect a continuum of risk for psychiatric disorder. Children with chronic medical illness have a 10% to 15% increased risk of psychological/behavioral problems (Pless and Roghmann, 1971; Pless et al., 1972). Furthermore, in recent studies using parental report , children with cystic fibrosis were found to have higher rates of psychiatric illness (Steinhausen et al., 1983) and more symptoms suggesting oppositional ity and general neurotic disturbance (Breslau, 1985), compared to controls. Internal consistency was also determined for the parent version of the CAS that was available for the psychiatric and community-based samples . Thus, the authors were also able to determine the generalizability of the findings to a different informant, the parents, and to nonpsychiatric samples of children . Method Subjects Psychiatrically disturbed children. This group consisted of 116 subjects, 87 of whom were inpatients at Duke University Medical Center and 29 of whom were outpatients attending a psychiatric clinic at the University of Missouri Medical School. The outpatient sample has been previously described in Hodges et al. (1987) . The entire sample included 76 boys, with a mean age of 10 years (SD = 1.97) , and 40 girls, also with a mean age of 10 years (SD = 2.27) . The age range was 6 to 16 years, with 96% of the subjects being 13-years-old or younger . Seventy-two percent of the sample were Caucasian and 28% were black . The socioeconomic status (SES) of subjects was based on the Hollingshead (1975, unpublished manuscript) Four Factor Index of Social Status and was as follows: 21% Class I; 21% Class II; 23.5 % Class III; 18.5% Class IV; and 16% Class V. For both the inpatient and outpatient samples, the subjects were consecutive referrals that met the criteria for inclusion in the studies . The inclusionary criteria were: (1) agreement to participate in the study, (2) not psychotic , mentally retarded, or diagnosed as organic brain syndrome, and (3)' had a parental figure who was currently living with the child and was able to provide a history about the child's symptoms. For the outpatient sample, all patients who met the above criteria were invited to participate in the study and
636
all but one were willing to do so. For the inpatient sample, the CAS was part of the formal evaluation procedures on the unit. However, for 14 of the inpatient children, no interview was attempted although their parents were administered the parent version of the CAS. The reasons included: being untestable due to severe noncompliance (i.e., refused to participate with expected diagnostic or therapeutic activities) (N = 3) , severe stuttering and vocalizations (N = 1), transfer from the unit shortly after admission (N = 2), and unavailability of the child interviewer while on leave (N = 8), For both samples, the requirements of the institutional review boards were met. Parents ojpsychiatrically disturbed children. Ninety-two parents were administered the parent version of the CAS , in which they reported on their children's symptomatology. There were 62 parents of the inpatient children at Duke and 30 parents of the outpatients at the University of Missouri. One parent was included in the outpatient sample, but her child was not included in the sample because the child had a significant hearing loss that interfered with his ability to communicate with the examiner. For the inpatient sample, parent interviews were not accomplished for the first 30 children who were administered the CAS. Also there were an additional nine parents who were so uncooperative or unavailable that they did not participate in the routine evaluation procedures on the unit. The children of the entire parent sample included 59 males , with a mean age of 10 years (SD = 2.25), and 33 females, with a mean age of 11 years (SD = 2.19). Chronically medically ill children . Sixty-three children with cystic fibrosis were interviewed as part of a larger , ongoing project on coping skills of children with cystic fibrosis. They were recruited and interviewed during routine visits to the Cystic Fibrosis Clinic at Duke University Medical Center. Two of the participants were seen during brief hospitalizations for routine assessment of pulmonary functioning. The sample consisted of 34 boys with a mean age of 10 years (SD = 3.07) and 29 girls with a mean age of 12 years (SD = 3.17) . The ages ranged from 6 to 17 years, with 73% of the sample being 13-years-old or younger. All subjects were Caucasian, as would be anticipated given that cystic fibrosis is an autosomal recessive genetic disorder almost exclusively seen in Caucasians. The SES of the group consisted of: none in Class I; 10% in Class II; 22% in Class III; 37% in Class IV; and 31% in Class V. The exclusionary criteria were: (1) the child was mentally retarded or (2) the parent was not willing to participate as an informant. Only the children were administered the CAS. Community-based sample ojchildren. The subject sample is described elsewhere (Kashani et aI., 1989). This group consisted of 177 children and adolescents who represented a systemic sampling of a midwestern public school system, stratified by age and sex. An equal number of males and females were interviewed for three age groups : 8-, 12-, and 17-year-olds. For the purposes of this study, the 17-yearolds who were post-high school were deleted since they were not asked questions about school and peers. There were 89 boys, with a mean age of 11 years (SD = 3.39), and 88 girls, with a mean age of 11 years (SD = 3.32). J. Am. Acad. Child Adolesc .Psychiatry, 29:4, July 1990
INTERNAL CONSISTENCY
Eighty-seven percent of the subjects were Caucasian . The SES of the subjects was as follows: 22% Class I; 36% Class II; 25% Class III; 16% Class IV; and 1% Class V. Parents of the community based sample of children. At least one parent for each of the children in the sample was interviewed as part of the requirements for participation in the study. The informant was usually the mother. The characteristics of the children of these parents is the same as described above. Child Assessment Schedule There is considerable evidence of the reliability and validity of the CAS diagnostically-related symptom scales . Psychometric studies of the CAS have included assessment of interrater, test-retest, and interinformant reliability as well as contrast group validity and construct validity (Hodges, 1987; Hodges and Cools , 1990). The CAS consists of three sections. First , the child is asked questions about 11 topic areas (i.e., school, friends , etc.). Following these inquiries, the child is asked about onset and duration for positive symptoms. In a separate section, the interviewer records observations about the child for 53 targeted behaviors . Each CAS item is scored as "yes," "no, " " ambiguous," or " nonscorable." Response items are phrased such that an affirmative response always indicates the presence of symptomatology . Guidelines for scoring are detailed in the training manual for the CAS (Hodges, 1983, unpublished manuscript) . Using these guidelines as a reference, the child's literal "yes" or "no" responses do not necessarily reflect the recorded scores. The interviewer scores the response item based on his/her understanding of the child's intended communication (e.g., whether the child is trying to truly describe ruminations or obsessions). Scores for the diagnostically-related scales were computer generated. The computer program was based on a diagnostic index that matches the CAS items to specific diagnostic criteria (Hodges, 1985 unpublished manuscript). Since this study was begun before DSM-III-R items were added to the CAS, data relevant to DSM-III diagnostic criteria are presented. The parent form of the CAS (P-CAS) parallels the child interview . The same inquiries are made; parents are asked about their child. There are fewer items on the P-CAS because the third section , consisting of observational judgments about the child, is omitted. The P-CAS also yields the same diagnoses and scores as the CAS. Procedures At all three sites, the interviewers were trained with the CAS "Guidelines to Aid in Establishing Interrater Reliability" (Hodges, 1983, unpublished manuscript). Psychiatric sample . The interview was administered separately to parents and children and by different interviewers who were blind to each others ' rating. The interviewers for the children were as follows: (1) for the inpatients , an experienced female child psychologist (i.e. , the senior author) and (2) for the outpatients, two male medical students. The l .Am .Acad. Child Adolesc . Psychiatry,29:4,July 1990
interviewers for the parents were: (1) for the inpatients, two experienced , female psychiatric social workers , and (2) for the outpatients, two female undergraduate students . Lay examiners were used in the outpatient study because it was preferred to have interviewers who were blind to all sources of information about the child and also did not have a clear preference for a particular interview (Hodges et al., 1987). All of the interviewers for both the outpatient and inpatient study were trained by the senior author. In the outpatient study, training demonstrations were conducted via a one-way mirror. The criterion was an average item-byitem agreement of 0 .85% or better. For the inpatient study, videotaped interviews were used for training and the criterion was five consecutive interviews with a kappa of 0.75 or greater. The inpatients were typically administered the interview within the first 2 weeks of admission to the hospital as part of the routine clinical evaluation. The outpatients were interviewed during either the first or second visit to the clinic in a counterbalanced design. Chronically medically ill sample. The interview was administered only to the children. The interviewer was a female, clinical child psychologist. The interviewer was trained by the senior author, with a kappa of 0 .75 or greater for five consecutive interviews used as a criterion . There was periodic consultation to guard against interviewer drift. Community based sample. The interview took place in the subject's home and involved both the child and parent . Families were contacted initially by telephone, at which time the purposes and procedures of the project were delineated and participation payments ($50 .00 per child) were offered . The interviews were conducted by doctoral students in psychology or child and family development, all of whom had previous experience working with children . Training in administration was done until the following criteria were met: (1) achieving an item-by-item criterion of a Kappa of ~ 0.80 during the practice interview role playing; and (2) achieving an interrater reliability of K = 0.85 . All interviewers were periodically monitored to prevent interviewer drift. Analyses Alpha coefficient was calculated for each diagnosticallyrelated scale, using the Kuder-Richardson formula. For the purposes of analysis , endorsement of a symptom corresponded to a "yes" response and nonendorsement included all other response options. Alpha ranges from 0 to 1, with an alpha value of 0.70 or greater typically regarded as acceptable for traditional multiitem psychological question naires (Nunnally , 1978). For the purposes of this study, alpha values of 0.70 or above are considered high, and values between 0.60 and 0 .70 are considered moderate. The size of the reliability coefficient is a function of both the average correlation among items as well as the number of items in the scale. Thus, relatively lower coefficients would be anticipated for scales with fewer items or for mildly symptomatic samples (i.e., in which only a few items are endorsed). 637
HODGES ET AL. TABLE
1. Internal Consistency of the Diagnostically-Related Scales of the Child Version of the Child Assessment Schedule (CAS)
Alpha Coefficients Community Based Sample (N = 177)
No. of Items in CAS Scale
Psychiatric Patients (N = 116)
Attention deficit disorder ADD w/hyperactivity ADD w/out hyperactivity
14 11
0.81 0.83
0.74 0.75
0.77 0.78
Conduct disorder symptoms Aggressive subtype Nonaggressive subtype
17 10 7
0.80 0.73 0.64
0.60"
0.69 b 0.61" 0.65
Depression symptoms Major depressive episode Dysthymia
43 31 36
0.87 0.83 0.86
0.87 0.80 0.87
0.75 0.68 0.75
Anxiety symptoms Separation anxiety Overanxious
19 12 7
0.80 0.71 0.65
0.62 0.56 0.56
0.74 0.68 0.58
DSM-III-Based Scales
a b
Medically Chronically III (N = 63)
Excluded because fewer than five items were endorsed by any subject. Half or more of the items in the scale were not endorsed by any subject in the sample.
Results
Child Version Psychiatric patients. The results for all three samples for the child version of the CAS are presented in Table 1. High levels of internal consistency were observed for all the major diagnostic categories, all having alpha values of 0.80 or greater. Additionally, the subtypes for attention deficit disorder (ADD) (with and without hyperactivity) as well as for depression (major depressive episode and dysthymia) were also characterized by good internal consistency. For conduct disorder, the aggressive subtype had a high reliability at 0.73, with nonaggressive being somewhat lower (alpha = 0.64). The same pattern was observed in regard to the anxiety disorders, with separation anxiety characterized by a high alpha (0.71), with a lower value resulting for overanxious disorder (alpha = 0.65). All of the items on the diagnostic scales were endorsed by at least one subject, with the exception of two items from the observational judgment section (i.e., difficulty separating from parent and slow speech with long latencies). Medically chronically ill. High internal consistency was observed for ADD and depression as well as for the subtypes within these diagnostic categories (alpha > 0.70). Moderate levels of reliability were observed for the major categories of conduct disorder symptoms and anxiety symptoms. Only 41 % of the conduct disorder items were endorsed by any of the subjects. The results for subtypes of conduct disorder could not be calculated because there were fewer than five items on both of these scales. The somewhat lower value observed for conduct disorder probably reflects the lower number of items in the scale. Lower levels of internal consistency were also observed for anxiety symptoms, despite the fact that there was no reduction in the number of items in the scale compared to the psychiatric group. The alpha values observed for sep638
aration anxiety and overanxious disorder (0.56 for both scales) were notably lower compared to any of the other scales. Community based sample. The alpha values for each of the major diagnostic categories were acceptable (0.69 to 0.77). High reliability was observed for both subtypes for ADD. The lowest value was observed for conduct disorder symptoms (0.69). These results are similar to those observed for the chronically medically ill. There were only seven items endorsed, with moderate correlations observed for both aggressive and nonaggressive subtypes that consisted of five and six items, respectively. When all anxiety symptoms were combined, the results were considerably better (0.74) than for each of the subtypes. The same pattern was observed as seen in the psychiatric group, with the alpha value being higher for separation anxiety than for overanxious disorder.
Parent Version Psychiatric sample. Table 2 contains the results for the parent version of the CAS. High internal consistency was observed for all the major diagnostic categories (alpha of 0.77 or greater). In addition, both subtypes of ADD (with or without hyperactivity) as well as both subtypes of depression were characterized by very good internal consistency. For conduct disorder and anxiety, all the subtypes were characterized by at least moderate levels of reliability. Somewhat higher internal consistency was observed for nonaggressive conduct disorder, compared to aggressive. There was no notable difference between separation anxiety and overanxious disorder. Community based sample. High internal consistency was observed for ADD, with both subtypes having an alpha value of 0.78. The scales for anxiety symptoms and depressive symptoms were characterized by moderate levels of reliaJ.Am. Acad. Child Adolesc. Psychiatry, 29:4, July 1990
INTERNAL CONSISTENCY T ABLE
2. Internal Consistency of the Diagnostically-Related Scales of the Parent Version of the Child Assessment Schedule (P-CAS) Alpha Coeffic ients Number of Items in P-CAS Scale
Psychiatric Sample (N = 92)
Community Based Sample (N = 177)
Attention deficit disorder ADD w/hyperactivity ADD w/out hyperactivity
12 II
0.85 0.85
0.78 0.78
Conduct disorder symptoms Aggressive subtype Nonaggressive subtype
17 10 7
0.77 0.63 0.71
0.4 0" 0.40
Depression symptoms Major depressive episode Dysthymia
42 30 36
0.89 0.85 0.88
0.66 0.55 0.67
Anxiety symptoms Separation anxiety Overanxious
19 12 7
0.78 0.67 0.69
0.66 0.52 0.63
DSM-lll-Based Scales
a b
No items were endorsed by any subject. Half or more of the items in the scale were not endorsed by any subject in the sample.
bility. In contrast to the other scales, there was far less endorsement of conduct disorder symptoms. Two-thirds of the items were not endorsed by any parent. Thus , the conduct disorder scale consisted of only six items and had low internal consistenc y. Discussion
This set of findings yields several implications regarding diagnostic nomenclature for children . ADD appears to consist of related symptoms (i.e., with alpha coefficients ranging from 0.74 to 0. 83), whether assessed in highly symptomatic or in largely asymptomatic children, or whether assessed via child self-report or parent report. The high reliability for the DSM-IIl subtype of ADD with hyperactivity is supportive of DSM-III-R, given that this subtype most approximates attention deficit hyperactivity disorder in DSM-IIl-R. The high internal consistency observed for ADD without hyperactivity would suggest that perhaps this diagnostic category, which was present in DSM-III , should not have been eliminated in DSM-III-R . Research from factor analytic studies would also lend support to this viewpoint; separate factors have been identified for the dimensions of inattention/di sorganization and motor hyperactivity/impulsivity (Lahey et aI. , 1988). This has led some researchers to advocate for differentially using the DSM-III -R categories of attention deficit hyperactivity disorder and undifferentiat ed ADD, with the latter employed for cases in which the feature of hyperactivity is absent (Lahey et aI., 1988). The conduct disorder scale was found to be reliable for the psychiatric samples based on the parent or child report. For the two nonpsychiatric samples, the low endorsement rate reduced the coefficients of internal consistency in these samples. These results are very supportive of the diagnostic criteria; high reliability was observed for the psychiatric sample, and low prevalence of symptoms was observed for the nonpsychiatric samples. J .Am.Acad. Child Adolesc. Psy chiatry , 29:4, July 1990
The depression symptom scale as well as the scales for both subtypes of depression were characterized by very high internal consistency for all child samples and for the parents of the psychiatrically disturbed children. These results lend support to the decision to use adult criteria for depression in diagnosing children , as do the empirical finding s regarding the phenomenology of depression in children (Ryan et aI., 1987; Mitchell et aI., 1988). However , while it may be the case that the essential features of depression may be similar for children and adults, many empirical questions remain to be addressed. For example, there may be agespecific associated features of depression that are different for children (e.g., see Kovacs and Paulauskas, 1984). Larger cross-sectional as well as longitudinal studies are needed. Also, while these depressive symptoms clustered together into a meaningful and internally consistent syndrome for children, the question still remains as to whether the disorder of depression in children is similar to the adult disorders in terms of etiology, course of illness, biological correlates , and response to treatment (Carlson and Garber, 1986). Similar phenomenolog y does not imply similar underlying mechanisms. The results relevant to the anxiety disorders suggest that more empirical research related to nosology may be warranted. The scales for anxiety symptoms (combining both subtypes) and for separation anxiety disorder had adequate reliability for the psychiatrically disturbed children. There was a tendency for the separation anxiety scale to be characterized by higher internal consistenc y than the overanxious scale . However, this may be partially secondary to the fact that there were almost half as many items in the overanxious scale than in the separation anxiety scale . The more moderate reliability values observed for anxiety in the cystic fibrosis sample are somewhat surprising given that children with chronic illness in general and more specifically, children with cystic fibrosis, are characterized by more internalizing symptoms, including anxiety (Breslau , 639
HODGES ET AL.
1985; Thompson, 1985). Perhaps the anxiety symptoms clusters delineated in the DSM -III and (DSM -III-R) disorders do not capture well the phenomeno logy of anxiety present in nonpsychiatric, anxious children. Empirical studies comparing anxiety-disordered psychiatric children to nonpsychiatric samples, who are at risk for heightened anxiety (e.g., chronically ill, psychosomatic or traumatized/stressed children), may help differentiate meaningful patterns of representation of anxiety in children . Chronically ill children may have heightened anxiety symptoms because of their dependence on their parents to do their daily medical care. Terminally ill children have heightened anxiety symptoms because issues of loss are more imminent for them. It is probably important to consider the role of trait- versus statedependent (i.e., situational) symptoms in assessing anxiety. The results of this study compare favorably with the only other data available on interviews for children. As with the K-SADS, alpha values for depression and conduct disorder were high. In contrast, for anxiety, low internal consistency was observed with the K-SADS . In the present study, high reliability was observed for the anxiety symptom scale for the psychiatric and community based sample, based on child report, and for the psychiatric group , based on parent report. Thus, the findings in this study lend support to the syndromes encompassed in the childhood disorders in DSM -III and DSM-III-R. Additional studies to determine whether these findings are generalizable to other samples are warranted, given that the subjects in the present study may not necessarily be representative of the relevant populations (i.e., of psychiatrically ill children, chronically ill children , etc.) . For example, there was a larger representation of higher SES families among the subjects obtained primarily at Duke University Medical Center, compared to the community sample. This, in part, may be secondary to Duke being a private, teaching hospital. Also the cystic fibrosis sample was restricted to Caucasians and to persons willing to participate in a treatment study. Additionally, there were method differences across the sites, such as level of training of interviewers (i.e ., lay versus mental health professionals). However, the empirical data to date have not demonstrated significant differences between lay and clinician interviewers provided that both administer the same diagnostic interview (Costello et al. , 1984). In any case, this degree of variability across the samples, which was in part intentional (i.e., risk for psychiatric disorder), does not compromise the findings relevant to internal consistency. The stability of the relationships among the items (i.e ., internal consistency) would not be assumed to be determined by these exogenous factors, as reflected by the findings. These results provide additional evidence of the psychometric integrity of the CAS , in as much as the expression of the criteria via the CAS items reflects positively on the instrument. Continuing to explore the psychometric properties of diagnostically-related symptom scales is a critical prerequisite to using these scales. Likewise, the operationalization of diagnostic criteria for childhood disorders deserves critical scrutiny. 640
References Americ an Psychiatric Association (1987), Diagnostic and Statistical Manual of Mental Disorders, Third Edition-Revised. Washington, DC: American Psychiatric Assoc iation. - - (1980) , Diagnostic and Statistical Manual of Mental Disorders, Third Edition. Washington, DC : American Psychiatric Associ ation. Breslau, N. (1985), Psychiatric disorder in children with physical disabilities . l. Am. Acad. Child Psychiatry, 24:87-94 . Carlson, G. & Garber , J . (1986), Developmental issues in the classification of depression in children . In: Depression in Young People: Developmental and Clinical Perspective. ed . M. Rutt er, C . Izard , & P. Read. New York: Guilford Press, pp . 399--434. - - Cantwell , D. (1980) , A survey of depressive symptoms , syndrome and disorder in a child psychiatric popu lation. l . Child Psychol. Psychiatry, 21:19-25. Chambers , W. J., Puig-Anti ch, J., Hirsch , M., Paez, P., Ambrosini, P. J., Tabrizi, M . A . & Davies, M . (198 5) , The assessment of affective disorde rs in children and adolesce nts by semis tructured interview : test-retest reliability of the K-SADS . Arch. Gen . Psychiatry, 42:696--702. Costello, A. J . , Edelbrock, L. S. , Dulcan , J . K. , Kalas, R. & Klaric , S. H. (1984), Report on the NIMH Diagnostic Interview Schedule for Children (DIS-C). Washington, DC: National Institut e of Mental Health . Edelbrock , C . & Costello , A. J . (1988) , Structured psychiatric interviews for children. In: Assessment and Diagnosis in Child Psychopathology , ed. M. Rutter, A. H. Tuma & I. Lann . New York: Guilford Press, pp. 87-112 . George , L., Blazer, D. , Woo dbury , M . & Manton, K. (1989), Internal consistency of DSM-III diagnoses. In: The Validity of Psychiatric Diagnosis, ed. L. N. Robins, & J. E. Barrett. New York: Raven Press, pp. 99-125 . Hodges, K. (1987), Assessing children with a cli nical research interview : the child assessment schedule. In: Advances in Behavioral Assessment ofChildren and Families, Vol. 3. ed. R. J. Prinz. Green wich , CT: JAI Press, pp . 203-233, - - Cools, J . N. (1990), Structured diagnostic interviews. In: Through the Eyes of the Child: Obtaining Self-repo rt fro m Children and Adolescents. ed. A. M . LaGreca. Boston , MA: Allyn and Bacon , pp. 109-149. - - McKnew, D. , Burbach , D. J . & Roebuck, L. (1987) , Diagnostic concorda nce between the Child Asse ssment Schedul e (CA S) and the Schedule for Affective Disorders and Schizophrenia for SchoolAge Children (K-SADS ) in an outp atient sample usin g lay interviewers. l. Am. Acad. Child Adolesc. Psychiatry, 26:654-661. - - Kline , J. , Stem , L. , Cytryn , L. & McKnew , D. (1982), The development of a child assess ment schedule for research and cli nical use . J , Abnorm. Child Psychol. , 10:173- 189. - - Kline, J. , Fitch , P., McKnew, D . & Cytry n , L. (1981) , The Child Assess ment Schedule: a diagnostic interview for researc h and clinical use . Catalog of Selected Documents in Psychology, 11:56. Kashani , J ., Orvaschel , H. , Rosenberg, T . & Reid , J. (1989) , Psychopathology among a community sample of children and adolescents: a developmental perspective . J , Am . Acad . Child Adolesc . Psychiatry, 28:70 1- 706 . Kovacs, M . & Paulauskas, S. L. (1984), Developmental stage and the expression of depress ive disorders in children: an empirical analysis. In: Childhood Depression: New Directions for Child Development, D. Cicchetti & K. Schneider-Rosen . San Fran cisco: JosseyBass, pp. 59--80 . Lahey , B. B. , Pelham , W. E. , Schaughency , E. A . et al. (1988), Dimensions and types of attention deficit disorder. J , Am . Acad. Child Adolesc. Psychiatry. 27:330-335 . Mitchell, J. , McCauley , E . , Burke , P. & Moss, S. 1. (1988) , Phenomenology of depression in children and adolescents . J , Am . Acad. Child Adolesc. Psychiatry. 27:12-20. Nunnally , J . (1978) , Psychometric Theory , 2nd Ed . New York: McGraw-Hili Book Comp any . Pless, I. B. & Roghmann , K. J. (1971), Chronic illness and its consequences: observations based on three epidemiologic surveys. J, Pediatr., 79:351- 359.
l.Am .Acad . Child Adolesc . Psychiatry, 29:4, Jul y 1990
INTERNAL CONSISTENCY
- - - - Haggerty, R. (1972), Chronic illness, family functioning, and psychosocial adjustment: a model for the allocation of mental health services. Int. J. Epidemiol., 1:271-277. Puig-Antich, J & Chambers, W. (1978), The Schedule for Affective Disorders and Schizopheniafor School-Age Children (Kiddie-SADS). New York: New York State Psychiatric Institute. Ryan, N. D., Puig-Antich, J., Ambrosini, P. et al. (1987). The clinical picture of major depression in children and adolescents. Arch. Gen.
J.Am.Acad. Child Adolesc.Psychiatry, 29:4, July 1990
Psychiatry, 44:854-861. Steinhausen, H., Schindler, H. & Stephan, H. J. (1983), Comparative psychiatric studies on children and adolescents suffering from cystic fibrosis and bronchial asthma. Child Psychiatry Hum. Dev. 14:117130. Thompson, R. J., Jr. (1985), Delineation of children's behavior problems: a basis for assessment and intervention. J. Dev. Behav. Pediatr. 6:37-50.
641