The Feasibility of Conducting Structured Diagnostic Interviews with Preadolescents: A Community Field Trial of the DISC

The Feasibility of Conducting Structured Diagnostic Interviews with Preadolescents: A Community Field Trial of the DISC GWENDOLYN E. P. ZAHNER, PH.D. Abstract. A field trial of the Diagnostic Interview Schedule for Children, Version R, was conducted with a sample of 144 preadolescents identified from a 1986-1987 New Haven, Connecticut, cross-sectional children's mental health survey. This report examines procedural and methodological issues pertaining to the feasibility of completing structured diagnostic interviews with young children in the community. Four topics relating to community acceptance and potential response problems are addressed: I) participation and completion rates; 2) quality of interview response; 3) parent and child reactions to the interview; and 4) evaluation of response bias. J. Am. Acad. Child Adolesc. Psychiatry, 1991, 30, 4:659-668. Key Words: structured interviews, children's mental health, epidemiology. The development of the Diagnostic Interview Schedule for Children (DISC) has been sponsored by the National Institute of Mental Health since 1981 to provide a structured instrument for use by lay interviewers in community surveys of children in the tradition of the adult Epidemiological Catchment Area program (Eaton et al., 1984). The process of preparing a structured child diagnostic interview with coverage of criteria for major psychiatric disorders is considerably more complex than for an adult instrument. The diagnostic criteria for children have limited empirical bases and have changed substantially between editions of the American Psychiatric Association's Diagnostic and Statistical Manual. Questions assessing the presence of individual criteria must be framed in language comprehensible to a child. Also, the input of multiple informants (parents, children, and, in some age groups, teachers) is required. Consequently, the DISC has evolved over multiple phases of expert review, revision, and field trials. A preliminary draft written by consultants (C. K. Conners, Ph.D., B. Herjanic, M.D., and J. PUig-Antich, M.D.) was revised and tested in an inpatient sample by Costello et al. at the University of Pittsburgh (Costello, A. J. et al., 1984, Costello, E. J. et al., 1985). A panel of experts reviewed results of the Accepted February II, 1991. Dr. Zahner is an Assistant Professor in the Child Study Center and Department of Epidemiology and Public Health, Yale University School of Medicine. This article presents findings from a report prepared for the Epidemiology and Psychopathology Research Branch, Division ofClinical Research, National Institute of Mental Health. Support was provided by NIMH Contract 86M00985630ID, NIMH Small Grant R03MH4I678. and NIMH Research Scientist Development Award K01MHOO745 -02. The author wishes to acknowledge the other investigators in the DISC community field trial, Drs. James F. Leckman and Donald J. Cohen, and to thank Dr. David Shaffer for providing a draft of the DISC for use in this study. In addition. the assistance of Ms. JoAnn Vitarelli and Ms. Adria DiBenedetto during fieldwork is gratefully acknowledged. The support of the Connecticut Department ofChildren and Youth Services and the New Haven public and diocesan schools is also deeply appreciated. Reprint requests to Dr. Zahner, Child Study Center. Yale School of Medicine, P.O. Box 3333, New Haven, CT 06510-8009. 0890-8567/9113OO4-0659$03.OO/0© 1991 by the American Academy of Child and Adolescent Psychiatry. l.Am.Acad. Child Adolesc. Psychiatry. 30:4, luly 1991

Pittsburgh field trial and a concurrent Albany, New York, community survey conducted by Cohen et al. (1987a,b) and concluded that the DISC was not yet ready for large-scale research. The instrument underwent further revisions under the direction of Shaffer at Columbia University (Shaffer, 1985, Shaffer et al., 1988) in a field trial involving adolescent psychiatric patients. A substantial number of reports and articles by the instrument developers and other researchers have examined reliability, validity, and parentchild agreement, essential methodological concerns in the development of a diagnostic interview (Breslau, 1987; Costello, A. J. et al., 1984; Costello, E. J. et al., 1985; Cohen et al., 1987a,b; Edelbrock et al., 1985, 1986; Shaffer, 1985; Shaffer et al., 1988; Weinstein et al., 1989). A number of additional procedural and methodological issues must also be considered in using the DISC in community surveys, particularly because this instrument was initially developed and tested in patient samples. Unlike research in patient populations where subjects come to the investigators (or their sponsoring institutions) for assistance with emotional and behavioral problems, community survey research requires that investigators seek out their subjects and ask them for help in a scientific endeavor, although offering few or no direct benefits in exchange. This shift in investigator-subject relationship in patient versus community populations can impact on the research effort in a number of ways. There are practical problems in obtaining subject participation. There are ethical concerns to be addressed in the detection of persons with untreated psychiatric conditions in the community. There are also potential measurement problems, stemming from differences in symptom recognition or willingness to report psychopathology in clinic populations where instruments are developed and in the community populations where they are applied. Little formal attention has been given to these issues in the DISC. Structured diagnostic interviews of childhood psychiatric disorders present some new methodological challenges in evaluating the role of response bias in community surveys. Although survey researchers have long recognized the problems associated with asking questions about socially undesirable behaviors, most previous research has addressed response bias in adult populations (e.g., Nunnally, 1978). 659

ZAHNER

There is little empirical information about refusal rates, psychological impact, and potential response bias associated with structured psychiatric interviewing of young children in the community. There are a number of areas in which response bias may be different with DISC interviews than with traditional adult psychodiagnostic assessments. Children's concepts of "socially acceptable" versus "unacceptable" feelings and behaviors may differ from those of adults. It is conceivable that young children have not internalized adult mores of "normal" versus "abnormal" behaviors, and that they are, in fact, less guarded than adults in sharing information about a range of behaviors and feelings. In the parent version of the DISC, the nature of response bias is also likely to be different from other adult psychiatric instruments. Little is known about how adults alter their reporting styles when the questions in an interview pertain not to themselves but to a child in their care. Here, veridical reporting is dependent not upon self-disclosure but upon parental willingness to portray his or her child in an imperfect light. Structured interviewing with very young children in a community study presents a number of additional concerns related to the age of the informant. Preadolescent children are limited in their ability to report on complex constructs and have difficulty maintaining attention on structured tasks for long periods of time. Some recent findings from empirical research employing fully structured child interviews, such as the DISC and the Diagnostic Interview for Children and Adolescents (Weiner et aI., 1987) have led to growing skepticism about interviewing young children. Agreement between parent and child has been found to be so poor that some investigators have questioned the validity ofthe child's report (Herjanic and Reich, 1982). Edelbrock et al. (1985) observed that test-retest reliability in the Pittsburgh version of the DISC was significantly lower in children aged 6 to 10 than in older children and adolescents, frequently falling below methodologically acceptable limits. In order to evaluate the feasibility of conducting parent and child psychodiagnostic interviews in epidemiological surveys of preadolescents, this report will present findings from a community field trial of the DISC, assessing community acceptance and potential response bias. Four topics will be addressed: 1) participation and completion rates in the DISC community field trial; 2) quality of interview responses and specific response problems encountered with the DISC; 3) parent and child reactions to the interview; and 4) evaluation of response bias using two experimental scales. Method Subjects Field trial subjects were selected from a cross-sectional children's mental health survey of a stratified random sample of 822 children aged 6 to II residing in New Haven, Connecticut, that was conducted in 1986 and 1987 under contract to the Connecticut Department of Children and Youth Services. In this cross-sectional survey, parents and teachers reported on the child's current emotional and behavioral problems, using an omnibus symptom inventory, 660

the Child Behavior Checklist (CBCL) (Achenbach and Edelbrock, 1983, 1986). Subjects invited to participate in the field trial consisted of all children with total scores scoring in the published clinical range (t score> 63) of the parent or teacher symptom checklists ("CBCL cases") and a one in six control random sample of "noncases" whose total scores fell below the clinical range. Ninety-seven CBCL case subjects and 47 noncase subjects were enrolled in the community field trial. Complete sets of parent and child interviews were obtained for 138 subjects (95 CBCL cases, and 43 noncases). Child interviews were not obtained for five subjects (two CBCL cases, three noncases), and one parent interview of a noncase was not available for analysis. Similar to the community from which they were sampled, the field trial subjects were predominantly minority (53% black and I0% Latino) and consisted of approximately equal numbers of 6- to 8-year-olds (N = 69) and 9- to 11-yearolds (N = 70) and slightly more girls (N = 74) than boys (N = 65). Measures Psychodiagnostic measures. The version of the DISC used in the New Haven trial was a revision of the Costello et al. (1984) instrument known as the DISC-R, written by Shaffer and colleagues at Columbia University (Shaffer et aI., 1988). Similar to its predecessor, this instrument is highly structured and is suitable for administration by a lay interviewer. The items provide coverage of most DSMIII-R criteria for major psychiatric disorders occurring in childhood. Wording in the parent and child interviews is identical for most items; however, some questions contained in the parent version do not appear in the child instrument. In the New Haven field trial, most items assessing the concept of time (e.g., age of onset or duration of symptoms) were omitted from the child protocols because pretesting indicated that these questions were especially difficult for children to answer. Interview quality

The quality of child interview responses was assessed by interviewers in three ways. 1. At the end of the session, global interview quality was assessed on a five-point scale with the anchor point of one representing an ideal interview and a rating of five indicating a worthless interview. 2. While administering the DISC-R, interviewers were asked to record each item the child seemed unable to understand or appeared uncomfortable answering. Diagnoses posing the greatest difficulty were noted on a checklist at the completion of the interview. 3. Specific types of assessment problems encountered during the interview were recorded on a checklist covering such common measurement concerns as wording clarity, yea or nay saying, attempts to portray self as perfect, guarded responses, poor comprehension, shortening the interview, or pleasing the interviewer. Subject Reactions to the DISC-R

Four evaluation questions were designed to elicit reacl.Am.Acad. Child Ado/esc. Psychiatry, 30:4.Ju/y 1991

DISC COMMUNITY FIELD TRIAL

tions to the interview from different perspectives. First, subjects were asked if they found the interview enjoyable. Second, they were asked if they would tell a friend to participate in the interview for a study. A third question asked whether they found some parts of the interview too personal. In a last set of precoded responses, interviewers noted types of questions or concerns raised during the session about the interview or the study as a whole. Response Bias

Response bias was assessed by two experimental social desirability scales developed in parallel forms for parents and children. The DISC-R Evaluation Form was devised to measure social threat or undesirability associated with different content areas of the interview, using a technique suggested by Sudman and Bradburn (1982). Parents and children reported how other parents (or, in the child form, other boys and girls) would feel answering questions reflecting II different topics covered by the DISC-R. Parents were asked to rate whether other parents would feel' 'upset" or "ok"; children were asked to indicate whether other boys and girls would feel "bad" or "ok" answering questions on the DISC-R. To assure privacy, the forms were completed without assistance, unless help was needed, and subjects were given an envelope in which they could seal their completed questionnaires. The Columbia DISC-R also contained 10 items modelled after a classical "lie" or defensive response style scale, denoted here by the prefix "S" for social desirability. "S" items assessed extremely socially positive behaviors unlikely to be found in most children (e.g., In the past year, have you always told the truth? Are you always nice to everybody?). An attempt was made to balance the number of "S" items for which affirmative (N = 6) versus negative responses (N = 4) represented socially desirable behaviors and to intersperse items across multiple diagnoses.

Community Interviewing Procedures

Interviewers were required to have a Master's level training in a social science and experience assessing young children. Most interviewers were either elementary school teachers or school social workers, and, although not clinically trained, they were highly skilled in working with young children. Training procedures consisted of 3 days of seminars and practice interviews. Interviewing took place in subjects' homes or, if the parents preferred, at the Yale Child Study Center. At the start of the session, written, informed consent was obtained from the mother (or primary caregiver); assent forms were read to and signed by the child. A field supervisor edited all study materials and provided feedback to interviewers regarding errors and ambiguous information. Data Analysis

Data were fully verified and range-checked upon entry. Because subjects were selected from a larger survey with two sampling rates, data were weighted to provide an estimate of the proportions of response problems .that would be found in a general population sample. A weighting factor of 6 was applied to responses by children screened as noncases on the CBCL in the first-stage survey, and CBCL cases received a weight of one. Tests of significance were undertaken with unweighted data, using logistic or linear regression models in which sampling design (i.e., CBCL case/noncase sampling status) was controlled as a covariate. The CBCL case/noncase covariate term also served as an indicator of the relationship between total symptom levels and feasibility of DISC-R administration. In these models, tests for age effects were made to determine whether response problems were limited to younger children in the sample (i.e., children ages 6 through 8). Gender differences were examined to assess whether higher symptom levels typically reported for preadolescent boys were a function of reporting biases. Results

Procedures for Contacting Subjects

Before interviewers were assigned to households to conduct DISC-R interviews, an introductory letter was sent to the parent (or primary caregiver) who had completed the first stage needs assessment questionnaire. A follow-up telephone call was made by either the principal investigator or field supervisor, who described the types of questions parents and children would be answering, procedures for protecting confidentiality, and the time commitment required for the project (2 Yz hours from parents and I Yz hours from children). In instances where families had unlisted or disconnected telephones (about one-fifth of this inner-city sample), an attempt was made to contact relatives or friends whose names were provided in first-stage screening questionnaires. If the families still could not be reached, experienced interviewers were sent to the families' homes with written instructions on how to describe the nature and purposes of the study before obtaining consent to conduct the interview. l.Am.Acad. Child Adolesc. Psychiatry, 30:4,luly 1991

Participation Rates

Future investigators interested in surveying elementary school-aged community cohorts with the DISC will need information on rates and types of nonparticipation in order to project sample sizes and to design strategies to minimize nonresponse. Table I summarizes the outcome of initial telephone and home visit contacts with families in New Haven, Connecticut, for the DISC-R field trial. The rate of nonparticipation is substantial but not in excess of what one might expect when a poor, disadvantaged population is asked to participate in a lengthy (2 Yz hour) interview. Slightly more than one-third (36%) of subjects who were contacted for the study declined to participate. Nonparticipation rates for parents of symptomatic youngsters on the first-stage screening instruments (CBCL cases) were somewhat lower than for parents of noncase subjects (34% and 40%, respectively). Of the 82 nonparticipants, close to half (N = 36) were parents who refused to be in the study without offering an explanation, even when one was requested. The 661

ZAHNER TABLE

1. Subject Recruitment in New Haven DISC-R Field Trial (Unweighted Frequencies) First-Stage CBCL Status Screen Positives (Community "Cases")

Total Sample

N

% Families sampled for DISC-R study Families without valid telephone/address information Families with valid telephone or address information Families contacted for study Subjects determined to be ineligible after initial contact (non-English speaking, hearing impaired, child deceased) Eligible families Eligible families contacted for study Eligible families participating in DISC-R project Eligible families not participating in DISC-R project Reasons for nonparticipation (expressed as % of eligible families) Refused, no reason given Mother's work schedule too heavy Mother not interested in providing further information for children's services study Husband refused for mother-child interviews to take place Subject matter too personal Repeated failure to be at home at time of scheduled meeting Subject not at home for ten or more phone contacts Total

(N

= 319) 21.3

247 247)

78.7

21 226 226) 144

9.2 90.8

36.3

=

N

%

207)

(N

=

112)

44

25.0

163 163)

75.0

7.1 92.9

65.5

15 148 148) 97

60.3

47

82

34.5

51

39.7

31

15.9 5.8

36 13

14.2 5.4

21 8

19.2 6.4

15 5

4.9

11

4.7

7

5.1

4

1.8 0.4

4 1

0.7 0.7

3.8 0.0

3 0

3.5

8

5.4

8

0.0

0

4.0 36.3

9 82

3.4 34.5

5 51

5.1 39.7

4 31

77.4 (N

8.5 91.5 (N 63.7

=

=

second most frequent form of nonparticipation was the lack of enough free time for the study, a response given by 6% of eligible households and 16% of the nonparticipants. This explanation was given primarily by working mothers holding more than one job. An additional 11 parents (6% of the eligible subjects contacted; 13% of all nonparticipants) refused to enroll in the study because they felt they had already contributed enough to the research effort by completing the first-stage needs assessment questionnaire. Nonparticipation also took the form of indirect refusal-repeated failure to be at home for scheduled interview appointments (10% of nonparticipants). The subject matter of the interview was the least frequent explanation offered for nonparticipation, although this factor no doubt played a role in other decisions not to participate. Only one mother (l % of nonparticipants) reported that the interview was too personal. One factor to consider in performing community surveys of children is that because the interview is a family concern the refusal can come from more than one member of the household. In this study, four mothers (5% of nonparticipants) reported that although they would like to participate, their husbands would not permit the interview to take place. It should also be noted that among the 144 participating households, incomplete sets of parent-child DISC-R's were obtained for five child subjects because the

662

(N

72

22.6

N

%

Screen Negatives ("Noncases")

(N

(N

=

=

28 84

(N

=

84)

6 78

(N

=

78)

parent judged that the interview would be too difficult for the child, given the current family circumstances or child's emotional state (N = 3), or because the child refused to participate (N = 2). In addition to initial participation rates, one basic concern about administering lengthy structured diagnostic interviews to young children is whether they have sufficiently long attention spans to complete the task. Interviewers were instructed to terminate sessions if children appeared unable to provide meaningful responses or became very upset by the interview. Only two community interviews terminated early. These were two of the youngest children in the field trial, both screened as cases by the CBCL. Quality of DISC-R Interview Information Provided by Children Table 2 summarizes interviewer ratings on the global quality of interview information for children participating in the field trial. Ratings for parent interviews are provided for comparisons between child and adult interview quality. Overall, these interviewer ratings indicate that interview information obtained from children was of high quality. The highest two ratings (l and 2) were given for 83% of the 6to 8-year-olds and 85% of the 9- to ll-year-olds. As expected, the quality of child interviews was not as high as J.Am.Acad. Child Adolesc.Psychiatry, 30:4, July 1991

DISC COMMUNITY FIELD TRIAL TABLE

2. Interviewer Ratings on Overall Quality of DISC-R IntervieW" Children (%)

Rating (I) Ideal (2) Above average (3) Average (4) Below average (5) Worthless

Parents(%)

6-8 Years (N = 69)

9-11 Years (N = 70)

Total Sample (N = 139)

(N = 143)

27.1 56.0 12.0 3.0 1.8

51.9 33.5 13.5 1.1 0.0

40.2 44.2 12.8 2.0 0.9

72.3 24.9 2.0 0.8 0.0

°Percentages are estimated from weighted data incorporating CBCL case/noncase sampling rates. Samples sizes are given for unweighted group Ns.

TABLE

3. Interviewer Ratings of Problematic Areas of DISC-Ra Children (%)

Content area Worries/fears Rules at home Peer social relations School behavior Feeling sad/moodiness Strange thoughts/behaviors Eating/weight problem Enuresis/encopresis Trouble with law Drinking Drug abuse

6-8 Years (N = 69)

9-11 Years (N = 70)

Total Sample (N = 139)

Parents (%) (N = 143)

4.8 7.8 7.8 6.6 18.7 6.6 1.2 0.0 1.2 0.0 0.0

7.0 0.5 1.6 2.7 7.0 5.4 2.7 0.5 4.9 0.0 0.0

6.0 4.0 4.6 4.6 12.5 6.0 2.0 0.3 3.1 0.0 0.0

3.1 2.8 2.3 4.0 5.4 0.3 1.1 0.0 0.0 0.0 0.0

aPercentages are estimated from weighted data incorporating CBCL case/noncase sampling rates. Samples sizes are given for unweighted group Ns.

for parents, almost all of whom (97%) received the ideal or very good interview ratings. Significantly poorer ratings were observed for young children than older children (p < 0.05) and for CBCL screen cases than noncases (p < 0.003). No sex differences were observed in the quality of interview ratings. Table 3 indicates specific areas of the DISC-R in which interviewers encountered response problems most frequently. For both parents and children, rates of problems recorded for individual content areas of the DISC-R were generally low, ranging from 0% to 5% in parents and 0% to 12% in children. In most instances, rates of problems for children were only slightly higher than for parents (within 3%). Items dealing with depressive symptomatology were the

most problematic in child interviews; somewhat more 6- to 8-year-old children (19%) were reported to have problems in this section than youngsters aged 9 to 11 (7%), although this difference is not statistically significant. No significant differences in problems of reporting depressive symptomatology were observed between boys and girls or between symptomatic and asymptomatic youngsters screened on the first-stage parent and teacher checklists. Compared to depressive symptoms, a somewhat lower percentage of children were reported to have problems reJ.Am.Acad. Child Adolesc. Psychiatry, 30:4, July 1991

sponding to questions about worries and fears (6%). Problems in this section of the DISC-R were found exclusively among children scoring as cases in the first-stage parent and teacher screening questionnaires (p < 0.000). Two other problem areas of the DISC-R in which CBCL screen cases showed elevated trends compared with noncases were school behavior problems (p < 0.10) and eating/weight problems (p < 0.07). The data presented in Table 3 suggest that problems in reporting behavior or social problems at home, school, or with peers were concentrated among younger children in the sample. In an examination of gender differences, more boys (10%) were observed to have problems responding to questions about social relations with peers than were girls

«

1%). These differences failed to reach statistical signif-

icance, however. Table 4 summarizes the proportion of children and parents displaying the different forms of survey response problems, as recorded by interviewers at the completion of the session. The frequencies of different types of difficulties experienced by children were generally low, ranging from 3% to 14%. Problems reported for more than 10% of the weighted sample included the following: guarded response; unclear item wording; child's low intelligence or mental disturbance interfering with ability to respond; and nay saying. There were no statistically significant age or

663

ZAHNER

TABLE 4. Interviewer Ratings of Specific Problems Encountered with the DISC-Ra Children (%)

Type of problem Unclear wording Respondent's intelligence/mental state Yea saying Nay saying Tried to please interviewer Tried to shorten interviewer Tried to portray self as perfect Response guarded Uncomfortable with personal nature of DISC-R questions Other problem

6-8 Years (N = 69)

9-11 Years (N = 70)

Total Sample (N = 139)

Parents (%) (N = 143)

12.0 9.0 10.2 10.2 7.8 4.2 12.0 13.3

10.8 11.4 6.5 9.7 8.1 1.6 14.6

11.4 10.3 8.3 10.1 8.0 2.6 6.6 14.0

5.4 3.7 0.6 0.6 0.6 0.0 2.8 2.3

6.0 10.2

5.4 9.2

5.7 9.7

0.6 11.0

I.I

apercentages are estimated from weighted data incorporating CBCL case/noncase sampling rates. Samples sizes are given for unweighted group Ns. TABLE 5. Subject Evaluations of the DISC-Ra Children (%)

Rating Found interview enjoyable Would tell friend to participate Found some parts too personal

6-8 Years (n = 69)

9-11 Years (N = 70)

Total Sample (N = 139)

Parents (%) (N = 143)

81.5 88.7 15.6

81.9 94.8 7.9

81.7 91.8 11.6

90.8 95.4 4.3

apercentages are estimated from weighted data incorporating CBCL case/noncase sampling rates. Samples sizes are given for unweighted group Ns.

sex effects observed in children's response problems. Three problems occurred at significantly higher rates in children scoring above the clinical threshold on the screening instruments than in screen negatives: item wording (observed for 23% of CBCL cases versus 7% of noncases); yea saying (18% versus 5%); and discomfort with personal nature of questions (15% versus 2%). Further logistic analyses were undertaken to examine types of response difficulties with the three most problematic areas of the DISC-R identified in the preceding section: depression, anxiety, and psychosis. Different patterns of difficulties emerged for these three diagnostic areas. Children noted to have problems with the depression items were also observed to show some discomfort with the personal nature of the DISC-R items (p < 0.05). Unclear item wording (p < 0.01) and guarded response (p < 0.05) were listed as the most frequent problems in protocols where difficulty in the psychosis section was encountered. Among children experiencing problems with anxiety items (all of whom were screened cases), problems were found with the child's intelligence/mental state (p < 0.05), guarded response (p < 0.05), and attempts to please the interviewer (p < 0.05).

Parent and Child Evaluation of the DISC-R Interview Community reaction to the DISC-R appears to have been positive. Table 5 displays the responses given by children and parents to DISC-R evaluation questions, weighted to represent general population rates. At a minimum, 90% of 664

parents gave a favorable assessment of the DISC-R on these items. Children were somewhat less likely than parents to give positive evaluations of the DISC-R, although the majority of children (at least four out of five children in the study) found the experience to be a favorable one. No age or sex differences were observed in children's evaluation of the DISC-R; however, there was some evidence suggesting that symptomatic youngsters reacted differently to the DISC-R than did asymptomatic children. Of children whose scores fell below the clinical threshold on the CBCL (noncases), almost all (98%) reported that they would tell a friend to participate in the study. This rate is comparable with the percentage of all parents in the study (96%) who responded positively to this question. By comparison, only three-quarters of children scoring above the clinical threshold on the CBCL (cases) reported that they would tell a friend to participate in the study. The difference in proportions of child CBCL cases and noncases in responding affirmatively to this question was statistically significant (p < 0.001). Subjects' requests or questions raised during the interview session were assessed as another form of evaluating the impact of the DISC-R on respondents. Parents made more requests for information than did children. Close to onehalf of parents (45%) asked for additional information about the study at some point during the interview. Among both parent and child subjects, these requests were made more often among the children who were screened negative by l.Am.Acad. Child Adolesc. Psychiatry, 30:4, luly 1991

DISC COMMUNITY FIELD TRIAL TABLE

6. Summary of Linear Regression Analyses Testing the Association between Social Desirability Scales and

Other Measures of Response Bias in Child Interviews" DISC-R Evaluation Form Total Score beta CBCL screening status (Case = I; Noncase = 0) 2.01 Child's age (6-8 yrs = I; 9-11 yrs = 0) Child's sex (boy = I; girl = 0) Child's verbal report of DISC-R acceptability (yes = I; no = 0) a. Found interview enjoyable b. Would tell friend to participate 1.02 c. Found parts of interview too personal Interviewer ratings of specific types of response problems observed in child interview (for b-j: yes = I; no = 0) a. Overall quality of information (I = ideal; 5 = worthless) b. Child's psychological state/low IQ interfered with ability to respond accurately c. Yea saying occurred d. Nay saying occurred e. Child tried to respond according to interviewer's expectations f. Child responded in a manner that would shorten interview g. Child tried to portray self as perfect h. Child's response guarded, reason unknown i. Child uncomfortable with personal content of DISCR questions 1.36 j. Other response problem

DISC-R "S" Items Total Score

p value

beta

p value

p < 0.0001 NS NS

0.66

P < 0.05

NS NS NS NS NS

NS NS p < 0.10

NS

0.41

p < 0.05

NS NS NS

0.92 1.25 1.23

p < 0.05 p < 0.05 p < 0.01 NS

NS NS NS NS

p < 0.10 NS

1.77

NS p < 0.01 NS NS NS

"Unweighted data, with covariate adjustment for sampling design.

the CBCL than among CBCL cases. No sex or age differences were observed in the rate with which these questions were asked. A modest proportion of parents also made requests for information about treatment (15%) or asked for assurances that the child was psychologically healthy (10%); however, these types of requests were never made by children in the present study. Assessment of Response Bias with the DISC-R Evaluation Form and DISC-R "S" Items The interview protocol contained two experimental measures of response bias: the DISC-R evaluation form, a measure of social threat associated with different content areas of the DISC-R, and DISC-R "S" items modelled in the tradition of a classical lie or defensive response scale. Table 6 summarizes the results of linear regression analyses exploring relationships between total scores on these social desirability measures and other field trial measures of DISC-R acceptance and response problems. These analyses have been restricted to the child's report because parent data contained little variation in the independent variables selected for these analyses (i.e., few problems with the DISC-R were reported by the parents themselves or were observed by interviewers among participating parents). The results of these analyses suggest that total scores on the DISC-R Evaluation Form and DISC-R "S" items tap different aspects of response bias. The variable showing the strongest association with the J. Am. Acad. Child Adolesc. Psychiatry. 30:4•July 1991

DISC-R Evaluation Form was CBCL screening status. Significantly more negative ratings were given by the CBCL case subjects (an average of 5.9 negative ratings out of a possible 11) versus noncases (reporting an average of 3.9 negative ratings). In addition, two other measures of social desirability showed statistical trends suggesting an association with the DISC-R Evaluation Form: the child's report that some parts of the interview were too personal (p < 0.10) and the interviewer ratings that the child appeared uncomfortable with the personal nature of the DISC-R questions (p < 0.06). A distinctly different pattern of associations was observed in relation to the total score for "S" items on the DISC-R. No differences were observed between children scoring as cases compared with noncases on the CBCL. However, the "S" scale was significantly related to both poor overall quality of the interview and comprehension problems stemming from low intelligence or psychological disturbance. In addition, several distinct response styles were associated with the "S" scale's total score: yea saying, nay saying, and attempts to portray self in a favorable light. In addition to information provided by total scores, individual items on the DISC-R Evaluation Form can shed light on the social desirability of different component diagnoses of the DISC-R. Each item was written to correspond to a broad diagnostic area covered by the DISC-R, and the frequency with which negative ratings are given to each item can be interpreted as a partial measure of its social

665

ZAHNER TABLE

7. Child and Parent Attitudes of Social Desirability of Different Topics Covered on the DISC-R as Measured by Responses to Items on the DISC-R Evaluation Form" Children (%) (N = 69)

9-11 Years (N = 70)

(N = 139)

44.2 12.0

20.8 19.1

32.2 15.6

3.1 8.1

13.3 15.1 48.5

11.6 11.4 43.4

12.4 13.2 45.9

2.8 9.2 7.8

43.4 20.0

33.5 30.6

38.4 25.4

15.1 4.7

77.1 63.1 57.8 62.0

77.5 52.6 64.2 64.2

77.3 57.7 61.1 63.1

21.2 44.1 37.7 40.5

6-8 Years

Total

Parent (%) (N

=

143)

% of subjects reporting who said that other children/parents

would feel bad or upset answering DISC-R questions about a. Things that worry or scare the child b. How the child obeys rules at home c. How the child gets along with friends or other children at school d. How the child behaves in school e. If the child feels sad f. If the child hears, sees, or smells things that other children don't g. How the child eats or how much the child weighs h. If the child wets the bed/has bowel movements outside toilet i. If the child has had trouble with the police j. If the child drinks alcohol k. If the child abuses drugs

"Percentages are estimated from weighted data incorporating CBCL case/noncase sampling rates. Samples sizes are given for unweighted group Ns.

undesirability. Patterns of responses given by parents and children to individual items on the DISC-R Evaluation Form are displayed in Table 7. Consistent with other measures of interview acceptability, children tended to rate the DISC-R somewhat less positively than did their parents. As one would predict from general social conventions, areas of the DISC-R receiving the most negative ratings by both parents and children were areas of the DISC-R pertaining to illegal behaviors (trouble with the police, drug, or alcohol abuse). Between 38% and 44% of parents indicated that other parents would be "upset" answering questions about different types of illicit activities; over one-half (58%) to close to two-thirds (63%) of children gave negative ratings on these items. Questions concerning enuresis and encopresis received the highest proportion of negative ratings by children (77%), but this was not viewed as a socially unacceptable area of questioning by most parents (21 %). Questions dealing with inner thoughts and emotions (feeling sad, worries and fears, psychosis) also received high levels of negative ratings from children but, again, did not emerge as areas of concern for most parents. The areas of greatest acceptability from the perspectives of both children and parents were questions pertaining to behavioral adjustment at home, school, or with peers. Further analysis of children's responses to individual items on the DISC-R revealed no age or sex differences. However, children screened as cases on the CBCL gave more negative ratings than did noncases in many areas, including: worries or fears (all negative ratings were made by cases); school behavior; feeling sad; seeing, hearing, and smelling things others don't; trouble with the police; drinking; and drug use. Conclusion In this report, the feasibility of conducting community interviews with the DISC in preadolescent cohorts was ex-

666

amined from a number of practical perspectives that may influence completion rates and quality of information obtained in epidemiological studies. Participation rates, subject assessment of the interview, children's comprehension and response problems, and response bias related to the social undersirability of the interview's content were examined. When this spectrum of information is drawn on, a central question that provided much of the impetus for this field trial can be addressed: are there potential risks associated with interviewing young children with the Diagnostic Interview Schedule for Children in a community setting? This community field trial provided no evidence of harm to young children from participation in a diagnostic interview. No significant adverse effects were observed by interviewers or were reported by children or their parents. Furthermore, a large majority of subjects reported that the experience was enjoyable. Any conclusions that a structured diagnostic interview is suitable for use with young children, however, are made with provisos that appropriate steps are taken to prepare families for the interview and to handle problems that may arise during the interviewing session. The findings of this field trial suggest that children without emotional or behavioral problems are particularly likely to find the interview enjoyable. Thus, there appears to be little basis for the belief that an average child in the community will be distressed by the extended questioning about emotional and behavioral symptoms in the DISC-R. These findings should not be taken to suggest that these children consider a structured diagnostic interview to be "fun." It is noted, rather, that the extended attention by an adult interviewer appears to be appreciated by the children. The majority of symptomatic children enrolled in this study (CBCL cases) also found the interview to be an agreeable experience but were more likely to give negative evaluations than were asymptomatic youngsters (noncases). Unl.Am. Acad. Child Adolesc. Psychiatry, 30:4, July 1991

DISC COMMUNITY FIELD TRIAL

like asymptomatic child subjects who were administered the diagnostic interview, these children were asked to recall personal difficulties they have experienced during the past year, which may have contributed to their discomfort during the interview. The types of negative reactions observed by our interviewers were minimal and transient, such as fidgeting or making facial contortions while answering a question. Another important finding emerging from the assessment of community acceptance of this structured diagnostic interview is that younger children (Le., those between ages 6 to 8) did not seem to find the experience any less enjoyable than their older counterparts in the sample (youngsters 9 to 11). Only one age effect was observed in analyses of a number of community acceptance scales. Younger children had higher total scores on the "S" items measuring defensive responses on the DISC-R. It may be that this association does not reflect social undesirability of the DISC-R item content for young subjects but, rather, negative reactions resulting from comprehension problems. Parents in the community also reacted positively to the interview but often asked for additional information about the DISC-R and our research project. These findings point to the importance of preparing interviewers to answer questions about the study, which are likely to be asked in about one-half of community households, even when parents have had a previous opportunity to discuss the study with senior project staff. A list of community services should be included in the fieldwork packet for distribution to all subjects requesting such information. Written guidelines for responding to other commonly asked questions regarding, for example, procedures for protecting confidentiality, access to study results, or the impact of the study on the child's school program should be provided for interviewers. The findings for age, sex, and symptom levels associated with response problems were similar to those observed with the child's reported enjoyment of the interview. Although age or sex effects with survey comprehension were infrequently noted, high levels of symptomatology, as measured by parent and teacher behavior problem checklists, were consistently associated with DISC-R response problems. There are two implications of the observed association between symptomatology and difficulties responding to the DISC-R in the present study: one positive and one negative. It is encouraging that children who have not experienced the types of symptoms covered by the DISC-R do not appear to encounter inordinate difficulties with the interview-a concern that is frequently raised about community assessment with an instrument focusing exclusively on psychopathology. On the other hand, it is unfortunate, but perhaps not unexpected, that symptomatic youngsters will have difficulty reporting detailed information about their psychopathology. The areas of the DISC-R that were most problematic, items concerning depression, anxiety, and psychosis, are the types of inner experiences that adult informants are least able to corroborate. Response problems were detected in these areas both in interviewer ratings and in analyses of social desirability. The question can be raised whether the l.Am.Acad. Child Adolesc.Psychiatry, 30:4, July 1991

problems observed in asking children questions about inner states simply reflect developmental limitations in comprehension and not special social attitudes about these behaviors. Children may feel that other boys and girls will not want to be asked questions about emotions and sensations simply because they will have difficulty understanding them. Clearly, explication of the nature of the response problems in child reports of inner feelings and sensations warrants continuing attention in future studies employing structured diagnostic interviews in preadolescent cohorts. Although the development of the DISC was stimulated by an interest in conducting a national program of multisite community surveys similar to the adult Epidemiologic Catchment Area study, the availability of this structured diagnostic interview promises to have benefits for clinical as well as community-based research. Use of a structured diagnostic interview, such as the DISC, as a core instrument in a clinical assessment battery can provide a link between community and clinical research. Many of the findings reported here regarding acceptance and response rates are not applicable to clinical studies, but previous field trials suggest that these are not major obstacles in patient cohorts (Costello et aI., 1984). The increased difficulties with the DISC-R experienced by symptomatic children in this community study, however, suggests that it may be important to supplement child reports with observational data or other supplemental symptom measures in patient populations. This study has addressed a number of issues concerning the feasibility of conducting structured diagnostic interviews with preadolescent children in community settings. Reliability and validity, psychometric concerns that have been the focus of most other research efforts engaged in the development of the DISC, have not been considered here. Although the instrument examined in this report, the DISC-R, has undergone several revisions since this community field trial was completed, the issues in diagnostic interviewing raised in this report are applicable to most contemporary structured diagnostic interviews and can be considered in planning community surveys, irrespective of the instrument selected for study. References Achenbach, T. M. & Edelbrock, C. S. (1983), Manualfor the Child Behavior Checklist and the Revised Child Behavior Profile. Burlington, VT: University of Vermont Department of Psychiatry. - - - - (1986), Manua/for the Teacher Report Form and Teacher Version ofthe Child Profile. Burlington, VT: University of Vermont Department of Psychiatry. Breslau, N. (1987), Inquiring about the bizarre: false positives in Diagnostic Interview Schedule for Children (DISC) ascertainment

of obsessions, compulsions, and psychotic symptoms. J. Am. Acad. Child Ado/esc. Psychiatry, 26:645-648. Cohen, P., O'Connor, P., Lewis, S., Velez, C. N. & Malachowski, B. (l987a), Comparison of DISC and K-SADS-P interviews of an epidemiological sample of children. J. Am. Acad. Child Adolesc. Psychiatry, 26:662-667. - - Velez, C. N., Kohn, M., Schwab-Stone, M., & Johnson, J. (l987b), Child psychiatric diagnosis by computer algorithm: theoretical issues and empirical tests. J. Am. Acad. Child Ado/esc. Psychiatry, 26:631-638. Costello, A. J., Edelbrock, C., Dulcan, M. K., Kalas, R., & Klaric, S. H. (1984), Development and testing of the NIMH Diagnostic

667

ZAHNER Interview Schedule for Children in a clinical population: final report (Contract No. RFP-DB-81-0027). Rockville, MD: Center of Epidemiological Studies, National Institute of Mental Health. Costello, E. J., Edelbrock, C. S. & Costello, A. J. (1985), Validity of the NIMH Diagnostic Interview Schedule for Children: a comparison between psychiatric and pediatric referrals. J. Abnorm. Child Psychol., 13:579-595. Eaton, W. W., Holzer, C. E., Von Korff, M. et a!. (1984), The design of the Epidemiologic Catchment Areas surveys. Arch. Gen. Psychiatry, 41:942-948. Edelbrock, C. ,Costello, A. J., Dulcan, M. K., Kalas, R. & Conover, N. C. (1985), Age differences in the reliability of the psychiatric interview of the child. Child Dev. 56:265-275. - - - - - - Conover, N. C. & Kalas, R. (1986), Parent-child agreement on child psychiatric symptoms assessed via structured interview. J. Child Psychol. Psychiatry, 27:181-190. Herjanic, B. & Reich, W. (1982), Development of a structured psychiatric interview for children: agreement between child and parent on individual symptoms. J. Abnorm. Child Psychol., 10:307-324. Nunnally, J. C. (1978), Psychometric Theory. New York: McGrawHill. Shaffer, D. (1985), Critical Review of the DISC. Rockville, MD: Division of Epidemiology, National Institute of Mental Health.

668

- - Schwab-Stone, M., Fisher, P., Davies, M., Piacentini, 1. & Gioia, P. (1988), A Revised Version of the Diagnostic Interview Schedule for Children (DISC-R): Results of a Field Trial and Proposals for a New Instrument (DISC-2). Rockville, MD: Epidemiology and Psychopathology Research Branch, Division of Clinical Research, National Institute of Mental Health. Sudman, S. & Bradburn, N. M. (1982), Asking Questions: A Practical Guide to Questionnaire Design. Washington, DC: Jossey-Bass Publishers. ' Weinstein, S. R., Stone, K., Noam, G. G., Grimes, K. & SchwabStone, M. (1989), Comparison of DISC with clinicians' DSM-Ill diagnoses in psychiatric inpatients. J. Am. Acad. Child Adolesc. Psychiatry, 28:53-60. Weiner, Z., Reich, W., Herjanic, B. et a!. (1987), Reliability, validity, and parent-child agreement studies of the Diagnostic Interview for Children and Adolescents (DICA). J. Am. Acad. Child Ado/esc. Psychiatry, 26:649-653. Zahner, G. E. P., Leckman, J. F., Benedict, T. L., Riddle, M., Woolston, J. & Cohen, D. J. (1988) Report on the Suitability of the NIMH Diagnostic Interview Schedule for Children for Use in Children Ages Six to E/even. Rockville, MD: Epidemiology and Psychopathology Research Branch, Division of Clinical Research, National Institute of Mental Health.

J.Am.Acad. Child Ado/esc. Psychiatry, 30:4, July 1991