Child Psychopathology Rating Scales and Interrater Agreement: I. Parents' Gender and Psychiatric Symptoms

Child Psychopathology Rating Scales and Interrater Agreement: I. Parents' Gender and Psychiatric Symptoms PETER S. JENSEN, M.D., MAJ., M.e., JOHN TRAYLOR, M.D., COL., M.e., STEPHEN N. XENAKIS, M.D., COL., M.e., AND HARRY DAVIS, M.S.

Abstract. Although various parent-report and child self-report rating scales appear reliable and valid with child psychiatric inpatients, significant problems have been encountered with their use in outpatient and community samples. To assess possible gender and psychologic factors affecting correspondence of parents' and children's reports of children's symptoms, 100 children (45 girls, 55 boys) and their parents completed behavior problems scales. Results indicated that mothers rated sons and daughters significantly higher in symptoms than did fathers. Although interparent reliabilities were moderate for ratings of sons and daughters, parent-parent and parent-child agreement about reports of children's symptoms were significantly related to the parents' own levels of psychiatric symptoms, the gender of the child, and the type of parental symptoms. J. Am. Acad. Child Adolesc. Psychiatry, 1988, 27, 4:442-450. Key Words: rating scales, interrater agreement, parent-reports, child-reports.

Along with recent efforts to define more clearly the diagnostic criteria for childhood psychiatric disorders has come a proliferation of children's symptom and behavior problem scales. Many of these scales are designed for completion by parents and children. Although some are proving to be useful diagnostic, screening, and research instruments, a number of methodologic and conceptual problems associated with their use are coming to light. Most notably, in studies using children's depressive symptoms scales, researchers have found disappointingly low correlations between parent-completed and child-completed ratings (e.g., Kazdin et al., 1983b; Leon et al., 1980; Weissman et al., 1980). Although parents and children may be reporting different but relevant aspects of the same variable, the low correlations might reflect low convergent validities between the scales (Kazdin et al., 1982). In the case of depressive symptom scales, low validities may be indicative of problems such as children's tendency to underreport symptoms (Weissman et al., 1980), parents' tendency to underreport their children's depression (Kashani et al., 1985), or they may reflect continuing conceptual difficulties in defining childhood depression. Similar difficulties are encountered in use of child behavior problem checklists: only low correlations are found between different raters across different settings, e.g., teachers, parents, and clinicians (Garrison and Earls, 1982; Miller, 1964), suggesting that such scales rate not only the child but also the Accepted October 28. 1987. From the Department of Psychiatry and Neurology. Eisenhower Army Medical Center. Fort Gordon. Georgia. The opinions or assertions contained herein are the private views ofthe authors and are not to be construed as official or as reflecting the views ofthe Department ofthe Army or the Department ofDefense. The authors wish to thank Alan Bostrom. Ph.D. and Ginny Gildengorin. Ph.D. for their statistical advice. and Sandra Ferrell for her careful preparation ofthe manuscript. Reprint requests to Dr. Jensen. P.O. Box 342. Dwight David Eisenhower Army Medical Center. Fort Gordon. GA 30905-5650. 0890-8567/88/2704-0442$02.oo/0© 1988 by the American Academy of Child and Adolescent Psychiatry.

environment to which the child reacts. Likewise, although the moderate correlations between mothers' and fathers' ratings (Achenbach, 1978; Achenbach and Edelbrock, 1979; Garrison and Earls, 1982; Kazdin et al., 1983b; Weissmann et al., 1984) suggest reliable measurements of attributes in the child, these findings may also reflect characteristics intrinsic to the parents, such as a consistently skewed perception of the child. Despite stronger correlations between mothers' and fathers' ratings (interparent reliability) than between parents' and children's ratings, undue confidence in interparent reliability estimates may not be well founded. With rare exception (Hulbert et al., 1986; Kazdin et al., 1983a, Mash and Johnston, 1983), the few studies with reported interparent reliabilities have gathered those data only incidental to validating behavior checklists (Achenbach and Edelbrock, 1978). These reliability estimates (usually based on the Pearson correlation coefficient) do not take into account the error variance between parents (Shrout and Fleiss, 1979); thus, estimates may be spuriously high. It is also important to emphasize that the reliability coefficient is not an acceptable index of agreement, since high reliabilities can coexist with significant disagreement between raters. This fact becomes particularly relevant when raters' assessments are similarly rank-ordered but consistently different in actual magnitude. An additional problem is that with very few exceptions, most studies with reported interparent reliabilities are based on clinically-derived samples; obviously, one might expect that a certain measure of agreement already exists between those parents who agree to bring a child in for clinical assessment. The assumption that either parent can be used as the reporter in studies of child psychopathology in communitybased samples does not appear to rest on any firm empirical data. Kashani et al, (1983) noted systematic differences between referred and nonreferred samples in a study of childhood depression. Furthermore, most studies of child psychopathology in community-based samples (where parents must give permission for testing the child in a school setting, etc.) rarely report the numbers of parents and children refusing to 442

RATING SCALES AND INTERRATER AGREEMENT

participate in the study. Beck et aI., (1984) demonstrated increased problematic peer relationships in those children whose parents refuse to allow the child to participate in a study of children's social development. A possible determinant of differences in parents' and children's perceptions of children's behavior problems is the gender of both parent and child. Conceivably, gender-based identifications between parent and child may influence parents' perceptions of and responses to the child (Rothbart and Maccoby, 1966). Also, the actual degree and quality of children's problems may differ according to the child's age and sex. Achenbach (1978) and Achenbach and Edelbrock (1979, 1983), using factor analytic techniques, devised parent-completed rating scales of children's behavior problems and demonstrated differing behavior problem clusters for boys and girls, as well as for different age groups. They did not devise separate standardized scoring for fathers' and mothers' ratings of children. Although they found moderate interparent reliabilities (average Pearson correlation 0.66) and only several significant differences between mothers' and fathers' ratings, interparent reliabilities were calculated using the Pearson coefficient; thus, actual reliabilities and interparent agreement were probably somewhat lower for reasons outlined above (Shrout and Fleiss, 1979). Furthermore, because of small sample sizes in their earlier reports (Achenbach, 1978; Achenbach and Edelbrock, 1979), the statistical power of tests comparing parents' ratings was very low, making it unlikely that the investigators would have found significant results if there were any (Achenbach, 1978). In their reference sample of20 6- to II-year-old girls, mothers reported more problems than fathers on all 12 of the behavior problem subscales (fJ = 12, p < 0.001, Fisher's sign test), but this finding was not noted in their analysis (Achenbach and Edelbrock, 1979). In later reports, Achenbach and Edelbrock (1983) discounted the magnitude and relative importance of mother-father differences; however, it should be noted that their conclusions are based on data from clinically referred children; thus, possible mother-father differences when screening for child psychiatric disorders in a nonreferred population may be nonetheless quite significant. Furthermore, in the later report (Achenbach and Edelbrock, 1983), mothers reported higher levels of child symptoms than did fathers on all II of the behavior problem scales (fJ = II, P < 0.00 I). This is comparable with findings for other scales that demonstrated significantly higher ratings of child symptoms by mother reporters than fathers (Hulbert et aI., 1986). For these reasons, Hulbert et aI. (1986) advised the need for father-generated scale-to-criterion studies. In addition to the questions raised about the possible systematic differences in mothers' and fathers' ratings of their children's psychopathology, concerns have been expressed that discrete, identifiable factors may affect interparent agreement about children's symptoms. Several groups of researchers have noted that maternal ratings of child behavior problems are more closely tied to marital discord than trained home observers' ratings of the child's home behaviors (Christensen et aI., 1983; Forehand et aI., 1986a,b). In a nonclinical sample, Emery and O'Leary (1984) found that marital discord was tied to the mothers' ratings of child behavior problems

443

but not to teachers' ratings. The authors hypothesized the presence of a bias in the maternal perceptions of children's problems to explain this finding. Evidence exists to support the hypothesis that parents' own psychiatric symptoms may influence their perceptions of their children (e.g., they project onto the children what they experience in themselves). Friedlander et aI. (1986) found significant correlations between the mother's level of depression and her perceptions of depression in her child. The strength and direction of this relationship depended upon both the sex of the child and whether the child was being seen in a psychiatric or pediatric clinic. The Friedlander et aI. evidence indicated that increasing depression in mothers was related to higher maternal ratings of son's aggression and delinquency, disproportionate to girls. Similarly, in a community study of monozygotic and dizygotic twins, Graham and Stevenson (1985) found that mothers' reports of adolescent psychopathology were more closely tied to mothers' ratings of their own symptoms than to fathers' or teachers' reports of the children's symptoms. Also, the authors found evidence for systematic bias in fathers' reports of daughters' symptoms. In a study of child psychiatry outpatients, Gould (1983) reported significant correlations between teachers' and "healthy" caretakers' ratings of the children's behavioral problems, but no significant correlations between teachers' and "sick" caretakers' ratings. In a series of related studies (Brody and Forehand, 1986; Forehand et aI., I986b; Greist et aI., 1979), researchers have demonstrated that maternal distress accounts for more of the variance in mothers' perceptions of children's problems than home observers' "objective" ratings of the children's behavior. These studies are fairly persuasive, but they do not address the effects of home observers on the children's behaviors, the relatively short behavior sampling time frame used by observers (compared with mothers' 24-hour observations), nor the likelihood that parents' ratings are more sensitive (although perhaps less specific) to the child's behaviors and moods than ratings of observers who do not know the child and must use specifictime sampling procedures (Burrows and Kelly, 1983). Other evidence indicates that in clinical samples, the behavior of a "targeted" (e.g.,labeled as "deviant" by the parents, or physically abused) child may not differ from that of a "nontargeted" sibling; however, parents' ratings of the targeted child's behavior are associated with parental psychopathology, while ratings of the nontargeted sibling are not (Arnold et aI., 1975; Estroff et aI., 1984; Rickard et al., 1981). The above-cited evidence suggests that correspondence of interparent and parent-child perceptions of children's symptoms and behavioral problems may be influenced by gender of the child and parent, family factors (e.g., marital problems), and the parent's psychologic state; however, a better understanding of the extent and nature of these associations is needed. The purpose of this paper is to examine the effects of parent and child gender and parental psychiatric symptoms on the reliability and agreement of children's and parents' reports of children's symptoms and behavior problems. To avoid the problems noted in the above-cited literature, the authors attempted from the outset of this study to: secure an adequate sample size (in order to avoid a type II error); obtain

444

JENSEN ET AL.

as complete participation as possible; use a nonclinical sample; use multiple raters to assessthe child's symptoms (mother, father, teacher, and child); assess both mothers' and fathers' psychiatric symptoms (not merely depressive symptoms); and analyze the results separately for sex of parent and child. Method

Subjects One hundred and twenty-four intact families with children attending the same elementary school were invited by letter and a follow-up phone call (by one of the authors) to participate in a study on the effects of life stresses on children. Of this group, eight families were ineligible for participation because oftemporary father absence, while six families refused to participate. Two families that agreed to participate could not be reached at home, despite several attempts. Of the participating 108 families, 100 mothers, 95 fathers, 90 children, and 94 teachers completed their respective questionnaires. All fathers were officers or senior enlisted personnel on active duty with the U. S. Army, thus assuring moderately uniform socioeconomic status. The sample of children on which at least partial data were available was composed of 59 boys and 49 girls, ages 6 to II. Mean age of boys and girls was 8.9 and 8.7 years, respectively.

Instruments The Child Behavior Checklist (CBCL) was used to document the parents' reports of their children's depressive symptoms and behavioral problems (Achenbach, 1978; Achenbach and Edelbrock, 1979). The CBCL was selected for this because it allows compilation of a total behavior problem score; two "broad band" scores (internalizing and externalizing symptoms); and 12 subscale scores ("narrow band"), e.g., depression, aggression, and hyperactivity. Separate standardized norms and scoring procedures have been devised for boys and girls and different age groups; accordingly, boys' and girls' scores were tabulated separately using the standardized forms for ages 6 to II. The total CBCL score and the two CBCL broad band subscales were used to assess parents' perceptions of their children's total behavior problems and different subtypes of symptoms. (These scores were selected for analysis from the total number of CBCL scales and subscales, because much of the previous research on interparent and parentchild ratings correspondence has focused on ratings of overall behavior problems, as well as the internalizing and externalizing symptom dimensions.) The Teachers Report Form (TRF) of the CBCL (Achenbach and Edelbrock, 1983) was used to measure children's behavioral problems in the school setting, according to each child's main teacher. Like the parent-report version of the CBCL, the TRF allows the computation of narrow band scales, as well as the broad band (internalizing and externalizing) and total scale scores. To measure parents' psychologic states, a version of the Hopkins Symptom Checklist (HSCL) was used. The HSCL was modified by deleting the paranoia and psychotic symptom subscales to better accommodate a nonclinical community sample (Derogatis et al., 1974). To assess symptoms in the children, each child completed the Child Depression Inven-

tory (COl) and the Children's Manifest Anxiety Scale (CMAS). The COl is a 27-item multiple choice instrument that documents the child's symptoms of depression from his or her perspective (Kovacs and Beck, 1977). The CMAS is a 37-item true/false questionnaire (Reynolds and Richmond, 1978).Gathering these reports of psychiatric symptoms from mother, father, teacher, and child provided a means of crosschecking the reliabilities of ratings between various combinations of the family members.

Procedures A research worker delivered the parents' questionnaires personally to each house and gathered basic demographic information about the child and family from the parent at home during the visit. Parents were asked to complete the HSCL and CBCL at home, independently of each other, and then return these materials by mail in a stamped, addressed envelope provided with the questionnaire. The children completed the COl and CMAS at school in a testing room in the presence of one of the authors or a trained research assistant. Following the recommendations of Shrout and Heiss (1979), intraclass correlations were used to assess interparent reliability on the total CBCL and CBCL broad band subscale scores; t tests were also calculated to test for significant differences in the magnitude between parents' CBCL ratings. Also calculated were the means of the absolute differences of parents' ratings of boys' and girls' behavior problems. Separating the children and parents by gender, the authors calculated parent-parent, parent-child, and parent-teacher correlations of the COl, CMAS, HSCL, and CBCL. To determine how parents' reports on the CBCL may be affected by the parents' own levels of psychiatric symptoms, Pearson correlations between parents' CBCL difference scores (each mother's score minus the corresponding father's score, after Z score transformation) and the parents' HSCL scores were calculated as well as the correlation between parent-teacher CBCL difference scores (each parent's CBCL score minus the corresponding teacher's CBCL score, after Z score transformation) and the parents' HSCL scores. Z scores were computed for the COl and CMAS as well, to allow determination of the correlations between parent-child difference scores and parents' HSCL scores. Results Parents' mean ratings differed significantly for sons but not for daughters in the internalizing broad band subscale and total CBCL scores (Table I); nonetheless, interparent reliabilities were similar for parents' ratings of sons and daughters. Means of the absolute differences between each mother's and father's ratings of their child's CBCL scores (not shown) were sizable, ranging from 0.56 to 0.79 S.D. for both sons and daughters, even though the t tests in Table I did not show this effect for daughters. Correlations between parents' self-ratings of psychiatric symptoms (HSCL) and their ratings of the children's symptoms (CBCL total and broad band scales) are shown in Table 2. Correlations between parents' ratings of the children's symptoms (CBCL total and subscale scores)and the children's self-ratings of depression (COl) and anxiety (CMAS) are shown in Tables 3 and 4.

445

RATING SCALES AND INTERRATER AGREEMENT TABLE I. Comparisons ofParents' Total and Broad Band CBCL Scores Internalizing Scores Rated By TValue Mother Father

Externalizing Scores

Interclass Correlation

Rated By

T Value Mother Father

Total CBCL Scores

Interclass Correlation

Rated By TValues

Interclass Correlation

Mother Father

Boys

12.7 (8.4)"

9.8 (7.5)

-2.60·

0.49

15.8 (9.0)

13.9 (8.3)

-1.55

0.53

33.6 (17.8)

27.9 (17.0)

-2.05·

0.49

Girls

10.0 (7.1)

8.0 (6.1)

-1.82

0.41

13.8 (8.9)

12.8 (10.2)

-0.83

0.67

26.5 (14.7)

23.1 (14.7)

-1.72

0.59

"Values in parentheses are S.D. • p<0.05. TABLE 2. Correlations ofParents' HSCL and CBCL Scores Parents' SelfReports HSCL (fathers)"

HSCL (fathers)

HSCL (mothers)

HSCL (mothers)

x

x

x

x

TABLE 3. Correlations ofCDl and CBCL

Parents' Reports of Child Symptoms

Pearson

(Boys) Total CBCL score Internalizing subscale Externalizing subscale (Girls) Total CBCL score Internalizing subscale Externalizing subscale (Boys) Total CBCL score Internalizing subscale Externalizing subscale (Girls) Total CBCL score Internalizing subscale Externalizing subscale

0.41·· 0.31· 0.44··

51 51 51

COl

0.51·· 0.54·· 0.59··

44 44 44

COl (boys)

0.56·· 0.49·· 0.56··

47 47 47

COl (girls)

0.38· 0.36· 0.41·

46 46 46

COl (girls)

r

N

" Rated persons in parentheses. • p < 0.05; •• p < 0.01.

Table 5 shows the correlations between parents' and teachers' CBCL reports, separated for boys and girls. Tables 6 and 7 present the correlations of mothers' psychiatric symptoms (HSCL) with the mother-father and mother-teacher difference scores (after scores were converted to z scores, the differences between mothers' CBCL z score and each of the other two reporting sources' z scores were calculated by substraction). Similarly, Tables 8 and 9 depict the correlations between fathers' psychiatric symptoms (HSCL) with father-mother and father-teacher difference scores. The correlations between parents' HSCL scores and parent-child difference scores (e.g., mother CBCL internalizing z score minus the child COl z score) are shown in Tables 10 (mother) and II (father). Discussion

Results suggest that fathers and mothers differ significantly in their ratings of their son's behavioral problems. These findings parallel the reports by Achenbach and Edelbrock (1983) about clinical samples. Although significant differences between fathers' and mothers' ratings for girls were not found in this study, mothers did tend to rate daughters higher (as in Achenbach and Edelbrock's findings). Thus, interparent reliability appears to be related to the gender of the rating parent and the rated child. The fact that parents show fairly good reliabilities when rating sons (despite significant differences in

x (boys)"

x

x

x

(Fathers) Total CBCL score Internalizing subscale Externalizing subscale (Mothers) Total CBCL score Internalizing subscale Externalizing subscale (Fathers) Total CBCL score Internalizing subscale Externalizing subscale (Mothers) Total CBCL score Internalizing subscale Externalizing subscale

Pearson r

N

0.23 0.21 0.18

45 45 45

0.45·· 0.43·· 0.42·

47 47 47

0.14 0.19 0.11

38 38 38

0.17 0.11 0.21

39 39 39

" Rating person in parentheses. • p < 0.05; •• p < 0.001. TABLE 4. Correlation CMAS and CDCL

CMAS (boys)"

CMAS (boys)

CMAS (girls)

CMAS (girls)

x

x

(Fathers) Total CBCL score Internalizing subscale Externalizing subscale (Mothers) Total CBCL score Internalizing subscale Externalizing subscale (Fathers) Total CBCL score Internalizing subscale Externalizing subscale (Mothers) Total CBCL score Internalizing subscale Externalizing subscale

Pearson r

N

0.30· 0.36·· 0.17

46 46 46

0.35·· 0.37··· 0.26·

48 48 48

-0.12 -0.02 -0.12

38 38 38

0.09 0.18 0.02

39 39 39

Rating person in parentheses. • p < 0.05; •• p < 0.01; ••• p < 0.005.

a

the actual magnitude of their CBCL ratings) may indicate a higher threshold in fathers than in mothers in perceiving sons' behavioral problems. Alternatively, mothers may be unduly sensitive and overreport sons' problems. The generally higher

446

JENSEN ET AL.

TABLE 5. Correlations ofParents 'and Teachers CBCL Ratings Father

Mother Teacher Int. Girls

Boys

Int. Ext. Total

Ext.

Total

Int.

-0.081

Ext.

0.619**

0.504** 0.328**

Int. Ext. Total

Total

0.053 0.362*

0.273*

0.012 0.308*

0.251* 0.275*

0.140

* p < 0.05; ** p < 0.001. TABLE 6. Correlations ofMother-Father and Mother- Teacher CBCL Differences (Sons) with Mothers' HSCL Symptoms CBCL DitTerences ( T Scores)

Somatic

ObsessiveCompulsive

0.376***

0.324**

0.257*

Interpersonal Sensitivity

Phobia

Depressed

Anxious

0.098

0.007

0.375***

0.235*

0.232*

0.174

0.049

0.203

0.387***

0.302*

0.144

0.040

0.186

0.392***

0.297*

0.247*

0.425t

0.298*

0.375***

Hostile

Other

Total

0.193

0.001

0.321**

0.173

0.119

-0.112

0.219

0.308*

0.194

0.118

-0.042

0.284*

0.197

0.336**

0.429t

0.313*

0.122

0.388***

0.421t

0.221

0.375***

0.335**

0.403***

0.030

0.455t

0.327*

0.165

0.382***

0.323*

0.342**

0.050

0.420t

Mother-father (N = 51)

CBCL internalizing ditTerences CBCL externalizing ditTerences CBCL total score differences Mother-teacher (N = 49) CBCL internalizing ditTerences CBCL externalizing ditTerences CBCL total score differences

* p < 0.05; ** p < 0.01; *** P < 0.005; t

p < 0.001.

TABLE 7. Correlations ofMother-Father and Mother- Teacher CBCL Differences (Daughters) with Mothers' HSCL Symptoms CBCL DitTerences (TScores)

Mother-father CBCL internalizing ditTerences CBCL externalizing ditTerences CBCL total score ditTerences Mother-teacher CBCL internalizing ditTerences CBCLexternalizing ditTerences CBCLtotal score ditTerences

Somatic

ObsessiveCompulsive

Interpersonal Sensitivity

Phobia

Depressed

Anxious

Hostile

Other

Total

-0.069

0.153

0.047

-0.037

0.176

-0.033

-0.007

0.068

0.109

0.211

0.163

-0.106

0.086

0.185

0.055

-0.094

0.144

0.143

0.083

0.191

0.090

-0.006

0.234

0.040

-0.111

0.114

0.172

-0.045

0.121

0.195

-0.331*

0.206

-0.073

0.145

0.357*

0.045

0.063

0.158

0.355*

-0.099

0.079

0.032

0.214

0.052

0.070

-0.052

0.140

0.354*

-0.247

0.105

-0.030

0.17 4

0.124

0.027

* p<0.05. correlations found between parents and teachers for externalizing than internalizing symptoms (three out of four were significantly higher by z test) suggest that agreement is enhanced between raters when they are reporting on external, observable behaviors that violate social role expectations. For example, aggression in a girl may be less tolerated and more widely reported than aggression in a boy, based on gender sterotyping and adult expectations of children's appropriate sex role behaviors. The current findings of moderate correlations between the

parents' own symptoms and their reports of the child's problems reflect the findings of multiple previous studies. AIthough these correlations may suggest a distortion or bias in parents' reports of their children based on their own symptoms, an equally plausible explanation is that parents' and children's symptoms are in fact related (through either genetic or environmental factors). Although the correlations between parents' and children's reports reached as high as 0.46 (the correlation between mothers' total CBCL scores and boys' COl scores), these findings at best reflect only 21 % of the

447

RATING SCALES AND INTERRATER AGREEMENT TABLE 8. Correlations ofFather-Mother and Father-Teacher CBCL Differences (Sons) with Fathers' HSCL Symptoms CBCL Differences (TScores) Father-mother (N = 51) CBCL internalizing differences CBCL externalizing differences CBCL total score differences Father-teacher (N = 49) CBCL internalizing differences CBCL externalizing differences CBCL total score differences

Somatic

ObsessiveCompulsive

Interpersonal Sensitivity

0.047

-0.147

-0.116

0.119

-0.108

0.026

0.099

-0.149

-0.034

0.143

0.101

0.134

0.035

0.Q71

-0.011

Phobia

Depressed

Anxious

Hostile

Other

Total

-0.078

0.142

-0.159

0.031

-0.067

0.302-

0.116

0.182

-0.056

0.200

0.089

0.284-

0.007

0.182

-0.128

0.107

0.013

0.165

0.156

0.034

0.246

-0.047-

0.200

0.112

0.410---

0.347--

0.241

0.217

0.059

0.160

0.257-

0.325-

0.295-

0.136

0.167

-0.036

0.066

0.151

0.184

- p < 0.05; -- p < 0.01; *** P < 0.005. TABLE 9. Correlations ofFather-Mother and Father- Teacher CBCL Differences (Daughters) with Fathers' HSCL Symptoms CBCL Differences (T Scores) Father-mother CBCL internalizing differences CBCL externalizing differences CBCL total score differences Father-teacher CBCL internalizing differences CBCL externalizing differences CBCL total score differences

Somatic

ObsessiveCompulsive

Interpersonal Sensitivity

Phobia

Depressed

Anxious

Hostile

Other

Total

0.204

0.431***

0.307-

0.281-

0.155

0.411---

0.473***

0.227

0.450t

0.211

0.479t

0.313*

0.351*-

0.090

0.453t

0.582t

0.213

0.490t

0.229

0.399"-

0.285*

0.302*

0.124

0.304-

0.412*"

0.086

0.376*-

0.063

0.261

0.198

0.039

0.044

0.095

0.501t

0.134

0.266

-0.171

0.395*-

0.223

-0.027

0.085

0.203

0.660t

0.067

0.302-

-0.173

0.304*

0.121

-0.136

0.036

0.000

0.537t

-0.077

0.144

* p < 0.05; ** p < 0.01; *-* p < 0.005; t p < 0.001. shared variance in the description of the child's symptoms. The correlations reported here between mothers and sons are generally higher than previous reports of parent-child agreement. This may reflect the etTects of separately reporting the parent-child correlations by both the gender of the parent and gender of the child. Mother-son correlations (CBCL-CDI) were significantly higher than father-son (CBCL-CDI) correlations by z test. There were no significant ditTerences between mothers' and fathers' parent-child correlations for girls or for CBCL-CMAS correlations. These results may indicate possible ditTerential reliability and validities of these two childreport scales; they may also reflect possible sex ditTerences in the expression of observable signs and symptoms of depression for boys and girls. In addition, these results suggest increased receptivity/awareness in mothers (versus fathers) to sons' depressive symptoms. The relationships between parental symptoms and their reports of their child's symptoms suggest several possible explanations: a symptomatic parent could be more sensitive to the moods of his or her child; alternatively, symptoms in

the parent and the child may both reflect the action of a third factor (genes or environment). The direction of etTects is difficult to establish. Tables 6 and 7 suggest that mothers' own symptoms (especially depression) are systematically related to the ditTerences between their perceptions and those of husbands, the children's teachers, and the children themselves. These etTects ditTer, depending on whether the mothers are rating a son or daughter: etTects appear to be more pronounced in ratings ofsons (particularly internalizing problems). The reliability of this finding is further supported by the lack of significant ditTerences between parents' ratings of boys' externalizing symptoms, as seen in Table I. Interestingly, mothers' reports of daughters do not seem to be atTected by their own symptoms, compared with fathers' reports (Table 7). However, in the school setting, mothers' phobic symptoms lead to their underreporting what the teachers observed of the children's internalizing symptoms. In several instances, mothers' HSCL scores were related to mother-teacher ditTerence scores but not to mother-father difference scores. These scores could indicate that mothers ' HSCL symptoms atTect fathers'

448

JENSEN ET AL.

TABLE 10. Correlations ofMother-Child Symptom Score Differences with Mothers ' HSCL Symptoms Mother-Son Z Score Differences CBCL internalizing COl difference CBCL externalizing COl difference CBCLtotal score COl difference CBCL internalizing CMAS difference CBCL externalizing CMAS difference CBCL total score CMAS difference Mother-daughter z score differences CBCL internalizing COl difference CBCL externalizing COl difference CBCL total score coi difference CBCL internalizing CMAS difference CBCL externalizing CMAS difference CBCL total score CMAS difference

Somatic

ObsessiveCompulsive

Interpersonal Sensitivity

Phobia

Depressed

Anxious

Hostile

Other

Total

0.286-

0.305-

0.316-

0.008

0.249-

0.334-

0.284-

0.188

0.253-

0.149

0.359--

0.411---

0.133

0.283-

0.260-

0.256-

0.057

0.283-

0.299-

0.389---

0.395-"

0.059

0.296-

0.314-

0.276-

0.128

0.301-

0.294-

0.308-

0.190

0.157

0.367---

0.353--

0.188

0.159

0.327-

0.155

0.332--

0.265-

0.258-

0.369---

0.262-

0.168

0.033

0.333--

0.295-

0.372---

0.253-

0.205

0.400---

0.322-

0.181

0.098

0.362--

0.239

0.133

0.180

0.048

0.353-

0.181

0.229

-0.012

0.100

-0.105

0.083

0.155

0.197

0.190

0.220

0.123

-0.167

0.023

-0.138

0.157

0.244

0.127

0.304-

0.266

0.204

-0.089

0.100

0.078

0.322-

0.311-

0.144

0.387--

0.350-

0.328-

-0.019

0.238

0.241

0.262

0.265

0.274-

0.190

0.362-

0.203

-0.150

0.157

0.195

0.332-

0.355-

0.215

0.307-

0.414--

0.284-

-0.084

0.228

- p < 0.05; -- p < 0.01;"- P < 0.005.

TABLE II. Correlations ofFather-Child Symptom Score Differences with Fathers' HSCL Symptoms Father-Son Z Score Differences CBCL internalizing COl difference CBCLexternalizing COl difference CBCLtotal score COl difference CBCL internalizing CMAS difference CBCL externalizing CMAS difference CBCL total score CMAS difference Father-daughter z score differences CBCL internalizing COl difference CBCL externalizing COl difference CBCL total score COl difference CBCL internalizing CMAS difference CBCLexternalizing CMAS difference CBCLtotal score CMAS difference

0.219

ObsessiveCompulsive 0.288-

Interpersonal Sensitivity 0.387---

0.267-

0.250-

0.261-

0.291-

Somatic

Depressed

Anxious

0.246

0.325-

0.408---

0.383---

0.347--

0.425---

0.445t

0.344--

Phobia

Hostile

Other

Total

0.224

0.229

0.359--

0.436t

0.278-

0.409---

0.425t

0.404---

0.462t

0.270-

0.359--

0.443t

0.111

-0.138

0.158

0.243

0.199

0.073

0.143

0.134

0.166

0.148

-0.156

0.140

0.317-

0.278-

0.096

0.182

0.297-

0.242

0.147

-0.135

0.201

0.327-

0.266-

0.120

0.177

0.255-

0.237

0.186

0.031

0.266

0.338-

-0.123

0.124

0.559t

0.101

0.316-

0.257

0.040

0.341--

0.391--

-0,075

0.192

0.482t

0.045

0.341--

0.226

-0.019

0.288-

0.371--

-0.094

0.055

0.450---

-0.071

0.241

0.427---

0.215

0.367--

0.451---

0.180

0.287-

0.528t

0.206

0.501t

0.484t

0.219

0.430---

0.494t

0.217

0.343--

0.458---

0.153

0.517t

0.450---

0.164

0.377--

0.469t

0.196

0.218

0.424---

0.049

0.422---

- p < 0.05; -- p <0.01; --- P < 0.005; t p < 0.001.

RATING SCALES AND INTERRATER AGREEMENT

CBCL reports. (These findings will be pursued in additional papers in this series.) The findings in Tables 6 and 7 are further supported by the results in Table 10, which demonstrate consistent differences in mothers' and children's perceptions of child symptoms as a function of mothers' own psychiatric symptoms. As already suggested, these effects appear to be greater for sons than for daughters. This is somewhat ironic, since the reported motherson correlations are higher than mother-daughter correlations. Although this may seem contradictory at first glance, higher mother-son correlations (than mother-daughter correlations) do not rule out greater maternal bias in reporting sons' than daughters' symptoms. Instead, they may merely reflect that girls' depression may be more difficult to document by external observers (perhaps mediated by social and sex role expectations, since withdrawn or depressed behavior may be more tolerable and less noticeable for girls than boys). When examining the association between fathers' own symptoms and the differences between fathers' and other observers' reports of children's symptoms, only fathers' phobic symptoms seem to account for father-mother differences about son's symptoms. Similarly, in the school setting, fathers' interpersonal sensitivity and phobic symptoms seem to explain a potential source of father-teacher differences. In contrast, for fathers' views of daughters, the effects of paternal symptoms are striking: hostile and obsessive-compulsive symptoms seem to account for a significant amount of the variance in father-mother, father-teacher, and father-child differences in their views of child symptoms. Fathers' symptoms explain as much as 45% of the variance of the difference between fathers' and teachers' reports about child externalizing symptoms. Father-child differences are approximately equally correlated with fathers' symptoms for both boys and girls, suggesting that from the child's perspective (either sex), fathers overreport child symptoms. The moderate correlations in Tables 10 and II suggest that children's self-reports (depression, anxiety) systematically differ from both parents' views of the child, based on parents' symptoms. This may reflect that children in many cases do tend to underreport their own depression and anxiety. Parents may become aware of a child's nonverbal behavior before the child has the ability to accurately describe his or her internal experiences. The fact that parents' symptoms are systematically related to the parent-child differences in perceptions of child problems may indicate that the parent is experiencing the effects of external or family-related stressors themselves as well as observing nonverbal evidence of similar reactions in the child. A second paper of this series will examine factors related to the differences between the child's report and reports of other observers; these factors will include the age of the child, social desirability response sets, family stress, prior history of family break-up, and sibling position. These factors and other child characteristics may help explain some of the variance in the differences between children's report of their symptoms and reports of other observers. Overall results indicate that parent-parent, parent-teacher, and parent-child agreements about children's behavior and emotional problems are significantly affected by the parents' own gender, the sex of the child, and the parents' psychiatric

449

symptoms. Better interrater agreement appears to exist for external, observable child behaviors that violate social role expectations. Parents' own symptoms may lead to their underor overreporting of children's problems in the home setting, compared with school settings. Fathers' own symptoms seem to affect their perception of daughters more than sons, while mothers' symptoms seem to affect more their reports of sons than daughters. The strengths and limitations of this study deserve mention, as they may reflect why significant results were found here in contrast to the findings of other studies. This was a fairly complete sampling using a nonclinical population. Potential "ceiling effects" or built-in agreement based on referral patterns characteristic of clinical samples were not obvious in this study. Therefore, this study was less subject to the possibility of a type II error. Furthermore, the present analysis separated both parent and child by gender, compared with other reports that have combined opposite-sex parents and/ or children. The method of analysis (e.g., Tables 6 to II) was unique in its use of z scores to examine the differencesbetween raters' views, followed by calculation of the correlations with parents' symptoms. Although correlated measurement error cannot be ruled out, this is highly unlikely, since multiple raters were used across multiple settings. Although parents were asked to independently complete the questionnaires, no means were available to assess actual interparent collaboration. Given the likelihood that some collaboration between the parents did take place in the home setting, actual interparent reliabilities may be somewhat lower than reported in this study. Although the multitude of correlations suggest the need for the Bonferroni procedure, the authors have preferred here to report significant findings at the p < 0.05 level, based on their assumption that the variance explained by parents' symptoms affecting their views of the child may be no greater than 15 to 20%. If this is the case, the Bonferroni procedure would have eliminated or greatly reduced the significance of these findings, reduced the power to determine significant differences, and increased the possibility of type II error. The use of a military sample may also limit generalizing these findings to other populations. In the military, the mother may be a greater mainstay to family equilibrium than the father. Relatively frequent father absence and other family stresses may result in different patterns in this sample than in other settings; however, it is equally plausible that a great deal of extraneous variables were ruled out because of the homogeneity of the sample. Furthermore, review of the literature suggests that military families do not systematically differ from other populations in terms of child psychopathology and family patterns (Jensen et al., 1986). In conclusion, reliabilities of parental ratings may be enhanced by asking both parents for information about the child, a common practice in clinical settings but less common in research. Further research examining the correlations and agreement among mothers', fathers', children's, and teachers' reports about children's symptoms should separate parents and children by gender, since relevant information may be obscured by combining sexes. Devising separate standardized scoring for mothers' and fathers' ratings of their children's

450

JENSEN ET AL.

behavioral problems may also be necessary, if gender-linked differences are verified in future studies. Community-based studies examining the prevalence of psychiatric disorders in children cannot rely on either parent as equally reliable reporters, since parents may systematically differ in their perceptions of their children, based on gender of parent and child as well as their own psychiatric symptoms. Conceivably, a composite score of both the mothers' and fathers' ratings of the child's symptoms and behavioral problems may be the best predictor of the child's state. Further study of these questions is needed to improve their reliability and validity of children symptoms and behavioral problem scales as well as to clarify the continued conceptual problems in defining childhood depression and psychopathology. References Achenbach , T. M. (1978), The child behavior profile. I. Boys aged 6II. J . Consult. Clin. Psychol., 46:478-488. - - Edelbrock, C. S. (1979), The child behavior profile: II. Boys aged 12-16 and girls aged 6-11 and 12-16. J. Consult . Clin. Psychol., 47:223-233. - - - - (1978), The classification of child psychopathology: review and analysis of empirical efforts. Psycho/. Bull., 85: 1275-130 I. - - - - (1983), Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT.: University Associates in Psychiatry. Arnold, J. E., Levine, A. G. & Patterson , G. R. (1975), Changes in sibling behavior following family intervention. J. Consult. C/in. Psychol., 43:683-688. Beck, S., Collins, L., Overholser, J. & Terry, K. (1984), A comparison of children who receive and who do not receive permission to participate in research. J. Abnorm. Child Psychol., 12:573-580. Brody, G. H. & Forehand, R. (1986), Maternal perceptions of child maladjustment as a function of the combined influence of child behavior and maternal depression. J. Consult. Clin. Psychol., 54:237-240. Burrows, R. R. & Kelley, C. R. (1983), Parental, interrater reliability as a function of situational specificity and familiarity of target child. J. Abnorm. Child Psychol., II :41-48. Christensen, A., Phillips, S., Glasgow, R. E. & Johnson, S. M. (1983), Parental characteristics and interactional dysfunction in families with child behavior problems: a preliminary investigation. J. Abnorm. Child Psychol., 11:153-166 . Derogatis, L. R., Lipman, R. S., Richels, K., Uhlenhuth, E. H. & Covi, L. (1974), The Hopkins Symptom Checklist (HSCL): a selfreport symptom inventory. Behav. Sci.. 19:1-15. Emery, R. E. & Oleary, K. D. (1984), Marital discord and child behavior problems in a non-clinic sample. J. Abnorm. Child Psychol., 12:411-420. Estroff, T. W., Herrera , C., Gaines, R., Shaffer, D., Gould , M. & Green, A. H. (1984), Maternal psychopathology and perception of child behavior in psychiatrically referred and child maltreatment families. J. Am. Acad. Child Psychiatry, 23:649-652. Forehand, R., Brody, G. & Smith , K. (1986a), Contributions of child behavior and marital dissatisfaction to maternal perceptions of child maladjustment. Behav. Res. Ther., 24:43-48. - - Lautenschlager, G. J., Faust, J. & Graziano, W. G. (1986b), Parent perceptions and parent-child interactions in clinic-referred children: a preliminary investigation of the effects of maternal depressive moods. Behav. Res. Ther., 24:73-75. Friedlander, S., Weiss, D. S. & Traylor, J. (1986), Assessing the influence of maternal depression on the validity of the Child

Behavior Checklist. J. Abnorm. Child Psychol., 14:123-133. Garrison, W. & Earls, F. J. (1982, October), The CBCL as a screening instrument in a normal population of young children. Paper presented at the Annual Meeting of the American Academy of Child Psychiatry, Washington, D. C. Gould, M. (1983, October), Multiple sources of information on children 's psychopathology. Paper presented at the Annual Meeting of the American Academy of Child and Adolescent Psychiatry, Washington, D. C. Graham, P. & Stevenson, J. (1985), A twin study of genetic influences in behavioral deviance. J. Am. Acad. Child Psychiatry, 24:33-41 . Griest, D. L., Wells, K. C. & Forehand, R. (1979), An examination of predictors of maternal perceptions of maladjustment in c1inicreferred children. J. Abnorm. Psychol., 3:277-281. Hulbert, T. A., Gdowski, C. L. & Lochar, D. (1986), Interparent agreement on the Personality Inventory for Children: are substantial correlations sufficient? J. Abnorm. Child Psychol., 14:115-122 . Jensen, P. S., Lewis, R. L. & Xenakis, S. N. (1986), The military family in review; context, risk, and prevention. J. Am. Acad. Child Psychiatry, 25:225-234. Kashani, J., McGee, R. 0 ., Clarkson, S. E. et aI. (1983), Depression in a sample of9-year-old children. Arch. Gen. Psychiatry, 40:12171223. - - Orvaschel, H., Burk, J. P. & Reid, J. C. (1985), Informant variance: the issue of parent-child disagreement. J. Am. Acad. Child Psychiatry, 24:437-441. Kazdin, A. E. & Petti, T. A. (1982), Self-report and interview measures of childhood and adolescent depression. J. Child Psychol. Psychiatry, 23:437-457. - - French, N. H. & Unis, A. S. (1983a), Child, mother, and father evaluations of depression in psychiatric inpatient children. J. Abnorm. Child Psychol., 11:167-180. - - - - - - Esveldt-Dawson, R. (1983b), Assessment of childhood depression: correspondence of child and parent ratings. J. Am. Acad. Child Psychiatry, 22:157-164 . Kovacs, M. & Beck, A. T. (1977), An empirical-clinical approach toward a definition of childhood depression. In: Depression in Childhood, ed. J. G. Schulterbrandt & A. Raskin. New York: Raven, pp. 1-25. Leon, G. R., Kendall, P. C. & Garber, J. (1980), Depression in children: parent, teacher, and child perspectives. J . Abnorm. Child Psychol., 8:221-235. Mash, E. J. & Johnston, C. (1983), Parental perceptions of child behavior problems, parenting self-esteem, and mothers' reported stress in younger and older hyperactive and normal children . J. Consult. Clin. Psychol., 51:86-99. Miller, L. C. (1964), Q-sort agreement among observers of children . Am. J. Orthopsychiatry, 34:71-75 . Reynolds, C. R. & Richmond, B. O. (1978), What I think and feel: a revised measure of children's manifest anxiety. J. Abnorm. Child Psychol., 6:271-280. Rickard, K. M., Forehand , R., Wells, K. C., Griest, D. L. & McMahon, R. J. (1981), Factors in the referral of children for behavioral treatment: a comparison of mothers ofclinic-referred deviant , clinic-referred non-deviant, and non-clinic children . Behav. Res . Ther., 19:201-205. Rothbart, M. K. & Maccoby, E. E. (1966), Parents' differential reactions to sons and daughters. J . Pers. Soc. Psychol., 4:237-241. Shrout, P. E. & Fleiss, J. L. (1979), Interclass correlations: uses in assessing rater reliability. Psychol. Bull., 86:420-428 . Weissman, M. M., Orvaschel, H. & Padian, N. (1980), Children's symptom and social functioning self-report scales: comparison of mothers' and children's reports. J . Nerv. Ment . Dis., 168:736-740. - - Lechman, J. F., Merikangas, R. R., Gammon, G. D. & Prusoff, B. A. (1984), Depression and anxiety disorders in parents and children. Arch. Gen. Psychiatry, 41:845-852.