- Email: [email protected]
Screening for Psychosocial Dysfunction in Inner-City Children: Further Validation of the Pediatric Symptom Checklist
Screening for Psychosocial Dysfunction in Inner-City Children: Further Validation of the Pediatric Symptom Checklist J. MICHAEL MURPHY, ED.D., JOAN REEDE, M.D., MICHAEL S. JELLINEK, M.D., AND SANDRA J. BISHOP, M.A. Abstract. A sample of 123 6- to 12-year-old outpatients at an inner-city pediatric clinic was screened for psychosocial dysfunction using the Pediatric Symptom Checklist (PSC), a brief parent-completed questionnaire. The prevalence of positive screening scores on the PSC was 22%, significantly higher than the rate found in lower middle to upper middle-class samples. Comparing PSC case classifications with comprehensive assessments made by clinicians, overall agreement was 92% (K = 0.82; sensitivity = 88%; specificity = 100%); a comparison with several other measures provided additional support for the validity of the PSC. The PSC's reliability over time was also acceptable. These findings provide preliminary evidence that the PSC is as valid and reliable for screening children from economically disadvantaged and minority backgrounds as it is for middle and upper middle-class populations. J. Am. Acad. Child Adolesc. Psychiatry, 1992, 31, 6: 11 05-1111. Key Words: psychosocial screening, Pediatric Symptom Checklist, poverty and children. Epidemiological studies have shown that 5 to 15% of American children have psychiatric disorders (Costello et aI., 1988; Goldberg et aI., 1984; Schwartz-Gould, et aI., 1981; Starfield et aI., 1980), with the most precise recent estimates indicating that at least 12%, or 7.5 million children (Institute of Medicine, 1989; National Institute of Mental Health, 1990) may require mental health services. Poor and minority group children represent a growing proportion of our population (Children's Defense Fund, 1985), and these children are at even greater risk. Previous research has consistently found higher rates of psychosocial dysfunction among poor children (Costello, 1986; Costello, et aI., 1986; Goldberg, et aI., 1979, 1984; Schwartz-Gould et aI., 1981; Starfield et aI., 1980), with reported rates ranging from about 12% (Schwartz-Gould et aI., 1981) to 70% (Goldberg et aI., 1979) higher than more advantaged children. The distinct impact of racial and ethnic status, independent of socioeconomic status (SES), on children's mental health has not been well studied. Roghmann and associates (1984) reported rates of psychiatric problems that were 10 to 35'% higher for black children than for white but attributed these differences to the lower SES of the blacks in their
Accepted December 2, 1991 J)rs. Murphy and Jellinek are at the Child Psychiatry Service of the Massachusetts General Hospital and the Harvard Medical School. Dr. Reede is at the Harvard Medical School and Children's Hospital. Ms. Bishop is at Yale University. In addition to the support provided by the National Institute of
sample. Costello and her colleagues (1986) reported rates of positive Child Behavior Checklist (CBCL) (Achenbach and Edelbrock, 1981) scores that were approximately 50% higher for black than for white children. However, other investigators (Goldberg et aI., 1984; Starfield et aI., 1980; Williams et aI., 1979) have not found statistically significant differences in rates of psychosocial dysfunction by race. Despite the high prevalence rates of psychosocial problems, especially in economically disadvantaged samples, research consistently has found that less than half of the children experiencing psychosocial difficulties receive the service they need (Children's Defense Fund, 1985; Costello, 1986; Institute of Medicine, 1989; National Institute of Mental Health, 1990). Poor children are particl,llarly likely to have their mental health needs go unmet (Starfield et aI., 1980; Tuma, 1989). Pediatric Screening
One of the most promising methods of identifying children in need of psychiatric services is through their pediatricians who are particularly well placed to detect psychosocial disorders inasmuch as they see large numbers of children, on a regular basis, usually with their parents (Costello et aI., 1988). Moreover, pediatricians are often the first professionals consulted by parents who are concerned about their child's behavior or emotional development (Tuma, 1989). Here too, however, previous research has shown that the rates of pediatrician recognition and referral are quite low with less than half the children with psychosocial problems
Mental Health (contract 86M043903501D), the study was also funded
being identified and only a fraction of these children actually
by the W.T. Grant Foundation (86-1068-85), the Jane Hilder Harris Trust, and the Department of Child Psychiatry at Massachusetts General Hospital. We thank the parents who completed the PSC and especially those who participated in the interview phase of the study and, along with their children, shared so much of their lives with us. The help of Kathy Gainor, the project director, and Yvonne Vest and Kyra Kulik, our research assistants, is gratefully acknowledged. Reprint requests to Dr. Murphy, Massachusetts General Hospital, Child Psychiatry Service, ACC 725, Boston, MA 02114. 0890-8567/92/3106-1105$03.00/0©1992 by the American Academy of Child and Adolescent Psychiatry.
receiving additional treatment or a referral (Costello et al., 1988; Goldberg et aI., 1979; 1984). Parent-completed screening questionnaires can provide pediatricians with a means of focusing on those children most likely to have problems (Connors, 1979). There are, of course, other approaches to screening. Pediatricians can verbally ask screening questions of either the child or the parent, but even the simplest additional questioning (if applied to all patients routinely) would lengthen the average
J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
1105
MURPHY ET AL.
pediatrician's already busy day. Pediatricians also can give questionnaires to the children themselves, and this may be especially important with adolescent patients (Jellinek and Murphy, 1990). Psychosocial screening also can take place in schools or in the community, but using such locations entails other considerations such as confidentiality, difficulty of data collection, and acceptability to parents. For school-aged children including younger adolescents, parentcompleted questionnaires administered in the pediatrician's office appear to offer the optimal combination of screening ease and efficiency as a routine, first-stage screening procedure that can be followed up by pediatrician questioning and/or referral to a mental health professional. The Pediatric Symptom Checklist (PSC) (Jellinek et a1., 1988) (Appendix) 'is one of the only questionnaires that has been validated for use in pediatric office screening (Sturner, 1991). The PSC is a 35-item questionnaire designed to be completed in the pediatrician's waiting room by parents of 6- to 12-year-old children. The PSC takes less than 5 minutes to complete and score and reflects the parent's impression ofhislher school-aged child's psychosocial functioning. Us- . ing an empirically defined cutoff point, the PSC identifies dysfunctional children likely to benefit from further psychiatric evaluation. The validity and reliability of the PSC have been demonstrated in a variety of pediatric settings (Jellinek and Murphy, 1988, 1990; Jellinek et a1., 1986, 1988; Murphy and Jellinek, 1988; Murphy et aI., 1989; Walker et aI., 1989). PSC scores have been compared with child assessments based on in-depth interviews, other validated questionnaires, and data reflecting how children are functioning in their daily lives. In one study (Jellinek et aI., 1988), the screening accuracy of the PSC was validated against comprehensive assessments made by experienced clinicians in two pediatric settings: a private practice in a predominantly white, middle to upper middle-class suburb, and an urban health maintenance organization that serves a more ethnically and socioeconomically heterogeneous group of patients. Findings indicated 79% agreement (K = 0.60) between PSC scores and clinicians' ratings of psychosocial dysfunction, with a sensitivity of 0.95, and a specificity of 0.68. In the same study, the overall agreement between the PSC and presence of a serious psychiatric diagnosis was 87% (K = 0.74; sensitivity = 0.87; specificity = 0.89). Test-retest reliability between two administrations of the PSC about 4 weeks apart was r = 0.86. Further analyses (Murphy and Jellinek, 1988) established that the PSC had similar validity and reliability for screening the relatively small number of lower middle class and minority group children in the above samples. PSC scores agreed with clinicians' ratings of psychosocial dysfunction for 79% of the lower middle-class children in the samples and 79% of the minority group children. For lower middle-class children, the reliability of the PSC over time was r = 0.87; for minority group children, it was r = 0.91. Although this investigation extended the validity and reliability of the PSC to a lower middle-class sample, it lacked children from the poorest social class, and thus prompted the present study. 1106
The Present Study The current research followed a design that replicated the methods used in the above studies of the PSC in middle and lower middle-class samples. In the current study, the validity and reliability of the PSC were assessed in an inner-city pediatric clinic in one of Boston's poorest neighborhoods to determine whether the PSC is a suitable measure for screening children from such backgrounds. Extending the validity of the PSC to children from lower-class backgrounds was considered particularly relevant, given the documented higher rates of psychosocial problems among such children as well as their unmet mental health needs. A single site was chosen because of the large commitment necessary from the clinic to allow the investigators to monitor all pediatric activity, the difficulty in finding such clincs, and the expense involved in conducting "comprehensive evaluations" on even a relatively small number of subjects. If the PSC were found to be valid in this clinic, then further work on other samples clearly would be indicated; if the PSC did not appear to be valid, then further developmental work on the PSC might be necessary. A clinic serving African-American children was selected inasmuch as they represent the largest minority group in this country and because the question of the linguistic adequacy of the PSC in Hispanic- and Asian-American groups would require additional validation. The major hypothesis tested was that case classifications based on the PSC would obtain rates of agreement with those derived from clinicians' evaluations that were statistically adequate and, we hoped, comparable with the rates obtained using the same methods in more advantaged samples. In this, as in previous studies, presence of at least one psychiatric diagnosis was used as an alternative indicator of "true" psychosocial dysfunction in evaluating the validity of the PSC.
Methods Procedure: Questionnaire Sample The procedure used in the current research was a replication of the design used in a previous study of the PSC in predominantly middle-class samples; a more complete description of the design can be found in Jellinek et a1. (1988). All visits to general pediatric sessions were monitored during a 6-month period. Because most school-aged children attended the afternoon or evening rather than morning pediatric sessions, an attempt was made to cover as many of these sessions as possible. During all of the sessions, research assistants from the study approached all parents who entered the waiting room to see a pediatrician and asked them to complete a PSC on each of the scheduled children. Participation was completely voluntary. This article focuses on children 6 to 12 years old (the age range for which the PSC has been validated), although younger and older children were also screened. Measures: Questionnaire Sample Pediatric Symptom Checklist (PSC). The PSC consists of 35 symptoms that parents rate as "often," "sometimes," or "never" present in the child. The PSC is scored by assigning J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
2, 1, or 0 points, respectively, to these ratings, and an overall score is calculated using simple addition. For 6- to 12year-old children, scores up to 27 are considered within the normal range; scores of 28 or greater suggest dysfunction and the need for further evaluation. The PSC form used in the current study also requested basic demographic data including the child's age, gender, and race; the parents' marital status and occupations; and whether the family was covered by Medicaid. The Hollinghead (1975) scale was used to code parent occupational data and to estimate family SES rank on a nine-point scale from 1 (least advantaged) to 9 (most advantaged). Cases in which occupation was left blank were checked to see whether the parent reported receiving Medicaid; if so, the family was coded as Hollingshead job status 1 (dependent on public assistance). Cases in which both insurance and occupation information were left blank were checked in the clinic's billing system. Families that the computer listed as receiving Medicaid were also coded as job status 1. Procedure: Interview Sample
After parents had completed the PSC, the research assistant told them about the interview phase of the study, asked whether they would allow their names to be included in the pool of potential interview subjects, and obtained written consent. Parents who consented were sampled in the following manner: the parents of all childre.n scoring above the PSC cutoff point were invited to participate until 14 had been interviewed. In addition, a comparison group of 10 subjects was randomly selected from those scoring below the PSC cutoff. These procedures were designed to yield an interview sample of 24 subjects, 14 PSC positive and 10 PSC negative, thus duplicating the design used in the previous study of the PSC in more advantaged samples (Jellinek et al., 1988). In this previous study, PSC cases had been oversampled in comparison with their prevalence in the general population because of our somewhat greater concern about the possibility of false positives. Interviews were conducted within 3 months after PSC screening by a doctoral student in counseling (K.G.) and a clinical psychologist (J.M.M.), both of whom have extensive experience evaluating and treating children. The interviewers were blind to children's PSC scores. To assess the reliability of the PSC over time, parents were asked to fill out a second PSC at the beginning of the interview. Measures: Interview Sample Garmezy Child Interview. The Garmezy Child Interview (Finkelman, unpublished), modified as in the previous study (Jellinek et al., 1988), was used as the basis of the clinical interview. Information obtained from the child was supplemented by information obtained from the parent. Children's Global Assessment Scale (CGAS). The interviewers rated each child on the CGAS (Schaffer et al., 1983), blindly with regard to PSC score and independently of each other. After both of the interviewers had completed CGAS ratings, they reviewed together all available clinical information on each child (excluding PSC score) and assigned a consensus CGAS rating. This consensus CGAS rating proJ. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
vided the "gold standard" against which the validity of the PSC was tested. Using the cutoff defined by Schaffer and associates, CGAS scores of 70 or below were considered indicative of a clinical range of impairment. DICA-P. The Diagnostic Interview for Children and Adolescents, parent report version (OICA-P) (Herjanic and Reich, 1982) was used to determine whether the child had any of the specific symptoms of each of the psychiatric diagnoses listed for children in DSM-III (American Psychiatric Association, 1980). As in the previous study (Jellinek et al., 1988), the diagnoses of simple phobia and mild enuresis were not considered indicative of serious psychiatric impairment and were excluded from the caluclation of the total number of diagnoses for each subject. As in the previous study, interviewers were blind to the subjects' PSC scores. Children were also coded as having any diagnosis versus having no diagnosis. Although the PSC was designed to assess psychosocial functioning (and not as a diagnostic instrument), some overlap between PSC case classification and psychiatric diagnoses was expected and has been found in previous research (Jellinek et al., 1988). Therefore, the presence of a diagnosis on the DICA-P provided an alternative standard against which the validity of the PSC was tested in the current study. Pediatrician ratings. Pediatricians, blind to children's PSC scores, rated each child on a 5-point scale (Jellinek et al., 1988) regarding the child's current psychosocial flJnctioning. Ratings of 4 or 5 (serious problem) were contrasted with ratings of 1 to 3 (adaquate functioning) for categorical analysis. Pediatricians were also asked to complete a Physician Visit Record (PVR) (Goldberg et al., 1979) for each child in the interview study. This form requests information about the presenting symptom for the current visit, as well as a number of psychosocial questions. In this article, only the data regarding reason for the current visit were analyzed. Physicians were asked to select a single response that best described the presenting symptom or main reason for the current visit from a list of seven basic categories (e.g., preventive measure such as physical exam, chronic physical condition, emotional symptoms). Children's Personal Data Inventory (CPDl). Background information was obtained using the CPDI (National Institute of Mental Health, 1973). Data regarding the parents' educations and occupations were coded using the Hollingshead (1975) scale, to provide a four-factor rating of family SES (more precise than the two-factor rating obtained from the PSC form). The CPO I also allowed an assessment of whether the child had a history of psychiatric treatment. Results Questionnaire Sample
PSC screening was conducted during 55 of the 104 pediatric sessions scheduled during the 6-month study (53% of all pediatric sessions and 70% of the afternoon and evening sessions during this time). An attempt was made to cover all afternoon and evening sessions, but the schedules of the research assistants did not permit this. 1107
MURPHY ET AL.
During the 55 monitored sessions in the current sample, 160 children aged 6 to 12 years kept their appointments. Thirty of these children were not eligible for the study for various reasons (e.g., non-English speaking, previously sampled). Of the 130 remaining children, the parents of 123 (95%) completed aPSe. The sample of 123 children included 52 (42%) boys and 71 (58%) girls; the mean age was 8.8 (SD= 1.9). All the children were from minority groups, and all but two listed their race as African-American or mixed. Sixty-three percent lived in single parent homes. It was possible to code occupational rank (Hollingshead, 1975) for 96 (78%) of the families; the mean occupational rank was 2.7 (range = 1.8; SD = 2.0). Fifty (45%) of the families were covered by Medicaid or comparable city sponsored health insurance and were below the poverty level. Thirty-six children (37.5%) were from families with SES ranks of 2-5 (working class), and 10 children (10.5%) came from middle class families (SES 6-8). Although it was not possible to compare the obtained sample with all pediatric visits during the sampling period, the currently obtained sample could be compared with a somewhat more representative sample collected at the same clinic over a 6-week period a year later in which 71 % of all visits were monitored (Murphy et al., 1992). The currently obtained sample did not differ significantly from the more inclusive sample according to gender (58% versus 54% female), Medicaid coverage (45% versus 33%), or percent with single parents (63% versus 50%). As expected, the currently obtained sample did have a somewhat but not significantly higher mean age (8.8 versus 8.3 years), given the oversampling of afternoon and evening sessions in the current study and the larger percentage of older children in these sessions. PSC scores. Twenty-seven (22%) of the 123 children had PSC scores of 28 or greater, compared with rates found in the previous study (Jellinek et aI., 1988) of 11% in the suburban private practice and 17% in the urban HMO (X2 = 5.9, df = 2, p = .05). Two-by-two X2 analyses showed that although the PSC positive rate in the current innercity sample was significantly higher than that found in the suburban private practice (X2 = 5.1, df = 1, p < .05), it was not significantly different from that found in the urban HMO. The PSC positive rate of 22% for the inner-city sample was very similar to the rate of 23.8% found for the lower SES subjects in a previous PSC study (Murphy and Jellinek, 1988). PSC scores by background variables. In the present sample, children from single-parent families were significantly more likely to score above the PSC cutoff (34%) than were children from two parent families (7%; X2 = 7.7, df = 1, p < .01). There was a trend for boys to have a higher PSC positive rate (31 %,16/52) than girls (16%,11/71; X2 = 3.2, df = 1, P < 0.1). The PSC positive rate appeared to be influenced by SES within this sample, although the results did not reach statistical significance. For children from Medicaid-supported families, the PSC positive rate was 22% (11/ 50); for working class children the rate was 25% (9/36) and 1108
TABLE
1. Relationship between PSC and Other Measures" PSC Score
CGAS consensus Functional Dysfunctional
DSM-lII diagnosis No diagnosis Has diagnosis Pediatrician rating Functional Dysfunctional Psychotherapeutic treatment Never had Rx Had Rx
<28
28+
8 (100) 2 (12)
0 (0) 14 (88)
6 (75) 4 (25)
2 (25) 12 (75)
9 (50) 1 (17)
9 (50) 5 (83)
10 (45) 0 (0)
12 (55) 2 (100)
X2
p
13.4
.001
3.6
.06
(NS)
(NS)
Note: PSC = Pediatric Symptom Checklist. a Values in parentheses are percentages.
for middle class children the PSC positive rate was 10% (1/ 10). Interview Sample
The parents of 115 (94%) of the 123 children agreed to be included in the pool of potential interview subjects. To obtain the desired sample of 14 PSC positive and 10 PSC negative children, 30 families (16 PSC positive and 14 PSC negative) had to be invited to participate (24/30; 80% completion rate). The interview sample consisted of 12 boys; 9 (75%) of whom were identified as cases by the PSC, and 12 girls, 5 (42%) of whom were so identified. All 24 children were from minority groups (23 African-American, one mixed African-American and Native American), and their mean age was 8.5 (SD = 1.8). Eighteen (75%) children lived in singleparent homes, and 11 (46%) were covered by Medicaid. Hollingshead four-factor ratings of SES ranged from 17 to 49, with a mean of 29.8 (SD = 10.3). Fifty percent (N = 12) of the families were from the two least advantaged SES groups (IV and V). Clinicians' CGAS classifications. The two interviewers' CGAS case classifications agreed for 20 of the 24 children (83%; X2 = 7.4, df= l,p < .01; K = .64). Overall agreement between clinicians' consensus CGAS classifications and PSC case/not case classifications was 92% (22/24; X2 = 13.4, df = 1, P < .001), with a K = .82 (Table 1). Fourteen of the 16 children classified as dysfunctional by the clinicians had been correctly identified by the PSC, a sensitivity of 0.88 in this sample. The PSC correctly classified all eight of the children who were judged by the clinicians to be functioning adequately, a specificity of 1.0. Psychiatric diagnoses. Sixteen children were given at J. Am. A cad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
least one diagnosis (excluding phobia and enuresis) on the DICA-P. The most prevalent diagnoses were attention deficit disorder (ADD) (N = 9) and oppositional disorder (N = 9); separation anxiety (N = 5) and overanxious disorders (N = 3) also were found. Twelve (75%) of the children with diagnoses had been classified as cases by the PSC; of the eight children with no diagnosis, only two (25%) were PSC positive (X2 = 3.6, df = 1, p = .06; Table 1). Thus, using the presence of a psychiatric diagnosis as an alternative measure of "true" dysfunction, the PSC had a sensitivity of 0.75 and a specificity of 0.75 in this sample. The four children who received a diagnosis but screened negative on the PSC had diagnoses similar to PSC positive children (i.e., ADD, oppositional disorder, overanxious disorder). The mean number of diagnoses per child was 1.3 (SD = 1.4), with a range from 0 to 6. Children who screened positive on the PSC had a significantly higher mean number of diagnoses (1.9, compared with 0.6 for PSC negative children, t = -2.31, p < .05), providing further support for the validity of the PSC. Pediatrician ratings. Six (25%) of the children were rated by their pediatricians as having serious or very serious behavioral, emotional, or developmental problems requiring referral. Five (83%) of these six had been identified as cases by the PSC, a further demonstration of the sensitivity of the instrument. Conversely, of the 14 children rated by the PSC (and the CGAS) as dysfunctional, the pediatricians identified only five (36%) as having serious problems requiring referral. Pediatricians completed the PVR item about main presenting symptom for 22 of the children. Most (N = 15; 68%) were being seen for physical examinations or for acute conditions (N = 6; 27%). One child was being seen for an accident or injury, and none was being seen primarily for emotional symptoms or chronic physical conditions. Psychiatric history. Data from the CPDI indicated that two of the children in the interview sample had received mental health treatment. Both these children scored above the PSC cutoff, a difference that was not statistically significant, but did suggest another form of construct validity for the PSC. Conversely, from the standpoint of unmet mental health needs, 12· (86%) of the 14 children with positive PSC (and CGAS) scores had never received any treatment. Reliability of the PSC. The mean length of time between the first and second administrations of the PSC was 36 days (range 6 to 69 days). Time 2 PSC case/not case classification agreed with that from Time 1 for 20 of the 24 children (83%; X2 = 9.5, df= 1,p < .01; K = .68); the Pearson correlation between scores at the two time periods was r = .84. Discussion
The present study demonstrated that the Pediatric Symptom Checklist was a valid and reliable instrument for screening for psychosocial dysfunction in the lower SES and African-American minority group children in the current sample. When compared with comprehe!:1sive assessments made by experienced clinicians, the PSC had a sensitivity of 88% and a specificity of 100% in this sample; PSC case/ not case classifications agreed with those of the mental J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
health clinicians for 92% of the children. Thus, using clinicians' consensus ratings as the "gold standard," the validity of the PSC for screening the children in this inner-city sample was comparable with that found for more advantaged samples (Jellinek et aI., 1988; Walker et aI., 1989). A comparison of PSC case/not case classifications with several other measures provided additional evidence of the validity of the PSC for screening children in the current sample. Examining psychiatric diagnoses derived from structured interviews with parents, children who screened positive on the PSC had a significantly higher mean number of diagnoses. Using presence of a psychiatric diagnosis as an alternative validity standard, the PSC had a sensitivity of 75% and a specificity of 75%, again similar to the results obtained in the middle-class samples. Comparing PSC classifications to pediatricians' ratings of child functioning, the PSC correctly identified five of the six children rated as dysfunctional by their pediatricians. Two of the children in the sample had received mental health treatment; both scored above the PSC cutoff. Although these findings lend support to the validity of the PSC for screening inner-city children, they also highlight the unmet mental health needs of these children: less than half the children who were flagged by the PSC and CGAS ratings as in need of further evaluation were so identified by their pediatricians, and less than 15% of the children who were identified as dysfunctional by the screening had ever received any psychotherapeutic treatment. Thus, the findings of the current study replicate previous studies that found that less than half the pediatric outpatients with serious psychosocial problems are identified by their pediatricians and that only a fraction of these children get the services they need. This is further documentation of a serious, continuing unmet need in pediatrics. That all the PSC cases were independently rated as cases on the CGAS strengthens the basic point that "true cases" of dysfunction were rated as functioning adequately by their pediatricians and were not being identified as needing further evaluation and follow-up. Regarding reliability, PSC case/not case classifications showed 83% agreement over time, with a Pearson correlation of 0.84 between scores at the two time periods. The reliability of the PSC in this sample was thus very similar to that found in the study of middle-class pediatric samples where time-to-time case rating agreement was 87% and the correlation between exact scores was r = .86. When compared with children seen in a private practice in a middle- to upper-middle-class suburb (Jellinek et aI., 1988), children in the current inner-city sample exhibited a significantly higher level of impairment. This finding is consistent with previous studies documenting a higher prevalence of psychosocial problems among children from economically disadvantaged backgrounds. In fact, research to date has demonstrated a clear trend for PSC-rated dysfunction to vary with socioeconomic status: rates have ranged from 11 % in a suburban middle- to upper-middle-class sample, to 17% in an urban HMO (Jellinek et aI., 1988), 24% in a lower-middle-class sample (Murphy and Jellinek, 1988), and 22% in the current inner-city sample. These findings 1109
MURPHY ET AL.
likely reflect the increased levels of stress associated with poverty. The current study has a number of limitations. Studying a small sample in a single clinic leaves many questions unanswered. The current study focused on African-American children in a single community, and we must leave to future research the question of whether the PSC (in either English or translation) is valid for children from other minority and ethnic groups in other locations. The current study also leaves open the differential effects of lower SES and minority status, inasmuch as our subjects were both AfricanAmerican and poor. It should be noted that one of our previous studies (Murphy and Jellinek 1988) did address this question and found that although both minority status and low SES did appear to influence PSC screening scores, SES played the larger and statistically significant role. The calculation of sensitivity and specificity using such a small number of cases should be regarded as preliminary, given the possibility of chance variations within the current sample. The fact that the current figures so closely resemble those found in three previous studies (Jellinek et aI, 1988; Murphy and Jellinek 1988; Walker et aI, 1989), does however give them added credibility. From a psychometric standpoint, the PSC is valid, reliable and consistent with the expectation that children from lowerSES backgrounds have higher rates of psychosocial dysfunction than do middle-class children. From a clinical perspective, it is important to note that not all poor children are dysfunctional. The majority are not, despite their many stresses. However, the approximately 20 to 30% who show significant dysfunction need to be identified, confirmed by brief pediatric assessment, and then, if indicated, referred for further evaluation. In the hectic world of inner-city pediatrics, the availability of a validated screening instrument such as the PSC provides a needed bridge for clinicians and researchers who want to identify children with psychosocial dysfunction and to improve the process of allocating limited resources for meeting their needs.
Appendix
Pediatric Symptom Checklist Please mark under the heading that best fits your child: Never 1. Complains of aches or pains 2. Spends more time alone 3. Tires easily, little energy 4. Fidgety, unable to sit still 5. Has trouble with a teacher 6. Less interested in school
1110
Sometimes
Often
7. Acts as if driven by a motor 8. Daydreams too much 9. Distracted easily 10. Is afraid of new situations 11. Feels sad, unhappy 12. Is irritable, angry 13. Feels hopeless 14. Has trouble concentrating 15. Less interest in friends 16. Fights with other children 17. Absent from school 18. School grades dropping 19. Is down on him or herself 20. Visits doctor with doctor finding nothing wrong 21. Has trouble sleeping 22. Worries a lot 23. Wants to be with you more than before 24. Feels he or she is bad 25. Takes unnecessary risks 26. Gets hurt frequently 27. Seems to be having less fun 28. Acts younger than children his or her age 29. Does not listen to rules 30. Does not show feelings 31. Does not understand other people's feelings 32. Teases others 33. Blames others for his or her troubles 34. Takes things that do not belong to him or her 35. Refuses to share Michael Jellinek, M.D., Massachusetts General Hospital
References Achenbach, T. M. & Edelbrock, C. S. (1981), Behavioral problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monogr. Soc. Res. Child Dev., 46, Serial No. 188:1-81. American Psychiatric Association (1980), Diagnostic and Statistical Manual of Mental Disorders, 3rd edition (DSM-III). Washington, DC: American Psychiatric Association. Children's Defense Fund (1985), A Children's Defense Fund Budget. Washington, DC: Children's Defense Fund. Connors, C. K. (1979), Ratings scales. In: Basic Handbook of Child J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
Psychiatry, ed. J. D. Noshpitz. New York: Basic Books. Costello, E. J. (1986), Primary care pediatrics and child psychopathology: a review of diagnostic, treatment, and referral practices. Pediatrics, 78: 1044-1051. Costello, E. J., Costello, A. J., Edelbrock, C. et al. (1988), Psychiatric disorders in pediatric primary care: Prevalence and risk factors. Arch. Gen. Psychiatry, 45: 1107-1116. Costello, E. J., Edelbrock, C., Costello, A. et al. (1986), The diagnosis and management of psychopathology in children in an organized primary healthcare setting: final report (Contract No. 278-830006). Rockville, MD: Division of Biometry and Applied Sciences, Department of Health and Human Services. Goldberg, I. D., Regier, D., McInerny, T. K. et al. (1979), The role of the pediatrician in the delivery of mental health services to children. Pediatrics, 63:898-909. Goldberg, I. D., Roghmann, K. J., McInerny, T. K. et al. (1984), Mental health problems among children seen in pediatric practice: Prevalence and management. PediatriCs, 73:278-293. Herjanic, B. & Reich, W. (1982), Development of a structured psychiatric interview for children: agreement between child and parent, J. Abnorm. Child Psycho!., 10:307-324. Hollingshead, A. B. (1975), Four Factor Index of Social Status. New Haven, CT: Yale University Sociology Department. Institute of Medicine (1989), Research on Children and Adolescents with Mental, Behavioral, and Developmental Disorders.:-Mobilizing a National Initiative. Washington, DC, National Academy Press. Jellinek, M. S., & Murphy, J. M. (1988), Screening for psychosocial disorders in pediatric practice. Am. J. Dis. Child., 112: 1153-1157. Jellinek, M. S. & Murphy, J. M. (1990), The recognition of psychosocial disorders in pediatric practice: The current status of the Pediatric Symptom Checklist. J. Dev. Behav. Pediatr., II :273-278. Jellinek, M. S., Murphy J. M., & Burns, B. (1986), Brief psychosocial screening in outpatient pediatric practice. J. Pediatr., 109:371-378. Jellinek, M. S., Murphy, J. M. & Robinson, J., et al. (1988), Pediatric Symptom Checklist screening school-age children for psychosocial dysfunction. J. Pediatr., 112:201-209. Murphy, J. M. & Jellinek,-M. S. (1988), Screening for psychosocial
J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
dysfunction in economically disadvantaged and minority group children: further validation of the Pediatric Symptom Checklist. Am. J. Orthopsychiatry, 58:450-456. Murphy, J. M., Jellinek, M. S. & Milinsky, S. (1989), The Pediatric Symptom Checklist: Validation in the real world of the middle school. J. Pediatr. Psycho I. , 14:629-639. Murphy, J. M., Arnett, H. L., Bishop, S. 1., Jellinek, M. S. & Reede, J. Y. (1992), Screening for psychosocial dysfunction in pediatric practice: a naturalistic study of the Pediatric Symptom Checklist. Clin. Pediatr. (in press). National Institute of Mental Health (1973), Children's Personal Data Inventory. In: ECDEU Assessment Battery for Psychopharmacology. Rockville, MD: National Institute of Mental Health. National Institute of Mental Health (1990), National Plan for Research on Child and Adolescent Mental Disorders. Rockville, MD: National Institute of Mental Health. Roghmann, K. J., Babigian, H. M., Goldberg, I. D. et al. (1984), The increasing number of children using mental health services: Analysis of a cumulative case register. Pediatrics, 73:781-790. Schaffer, D., Gould, M. S., Brasio, J. et al. (1983), A Children's Global Assessment Scale (CGAS). Arch. Gen. Psychiatry, 40: 1228-1231. Schwartz-Gould, M., Wunsch-Hitzig, M. A., Dohrenweld, B. (1981), Estimating the prevalence of childhood psychopathology: a critical review. J. Am. Acad. Child Psychiatry, 20:462-476. Starfield, B. H., Gross, E., Wood, M. et al (1980), Psychosocial and psychosomatic diagnoses in primary care of children. Pediatrics, 66:159-167. Stumer, R. A. (1991), Parent questionnaires: basic office equipment? J. Dev. Behav. Pediatr., 12:51-54. Tuma, J. M. (1989), Mental health services for children: the state of the art. Am. Psycho I. , 44: 188-199. Walker, W.O., Lagrone, R. G., Atkinson, S. W. (1989), Psychosocial screening in pediatric practice: identifying high risk children. J. Dev. Behav. Pediatr., 10: 134-148. Williams, S. J., Diehr, P., Drucker, W. L. (1979), Mental health services: utilization by low income enrollees in a prepaid group practice plan and an independent practice plan. Med. Care, 17: 139145.
1111
Accepted December 2, 1991 J)rs. Murphy and Jellinek are at the Child Psychiatry Service of the Massachusetts General Hospital and the Harvard Medical School. Dr. Reede is at the Harvard Medical School and Children's Hospital. Ms. Bishop is at Yale University. In addition to the support provided by the National Institute of
sample. Costello and her colleagues (1986) reported rates of positive Child Behavior Checklist (CBCL) (Achenbach and Edelbrock, 1981) scores that were approximately 50% higher for black than for white children. However, other investigators (Goldberg et aI., 1984; Starfield et aI., 1980; Williams et aI., 1979) have not found statistically significant differences in rates of psychosocial dysfunction by race. Despite the high prevalence rates of psychosocial problems, especially in economically disadvantaged samples, research consistently has found that less than half of the children experiencing psychosocial difficulties receive the service they need (Children's Defense Fund, 1985; Costello, 1986; Institute of Medicine, 1989; National Institute of Mental Health, 1990). Poor children are particl,llarly likely to have their mental health needs go unmet (Starfield et aI., 1980; Tuma, 1989). Pediatric Screening
One of the most promising methods of identifying children in need of psychiatric services is through their pediatricians who are particularly well placed to detect psychosocial disorders inasmuch as they see large numbers of children, on a regular basis, usually with their parents (Costello et aI., 1988). Moreover, pediatricians are often the first professionals consulted by parents who are concerned about their child's behavior or emotional development (Tuma, 1989). Here too, however, previous research has shown that the rates of pediatrician recognition and referral are quite low with less than half the children with psychosocial problems
Mental Health (contract 86M043903501D), the study was also funded
being identified and only a fraction of these children actually
by the W.T. Grant Foundation (86-1068-85), the Jane Hilder Harris Trust, and the Department of Child Psychiatry at Massachusetts General Hospital. We thank the parents who completed the PSC and especially those who participated in the interview phase of the study and, along with their children, shared so much of their lives with us. The help of Kathy Gainor, the project director, and Yvonne Vest and Kyra Kulik, our research assistants, is gratefully acknowledged. Reprint requests to Dr. Murphy, Massachusetts General Hospital, Child Psychiatry Service, ACC 725, Boston, MA 02114. 0890-8567/92/3106-1105$03.00/0©1992 by the American Academy of Child and Adolescent Psychiatry.
receiving additional treatment or a referral (Costello et al., 1988; Goldberg et aI., 1979; 1984). Parent-completed screening questionnaires can provide pediatricians with a means of focusing on those children most likely to have problems (Connors, 1979). There are, of course, other approaches to screening. Pediatricians can verbally ask screening questions of either the child or the parent, but even the simplest additional questioning (if applied to all patients routinely) would lengthen the average
J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
1105
MURPHY ET AL.
pediatrician's already busy day. Pediatricians also can give questionnaires to the children themselves, and this may be especially important with adolescent patients (Jellinek and Murphy, 1990). Psychosocial screening also can take place in schools or in the community, but using such locations entails other considerations such as confidentiality, difficulty of data collection, and acceptability to parents. For school-aged children including younger adolescents, parentcompleted questionnaires administered in the pediatrician's office appear to offer the optimal combination of screening ease and efficiency as a routine, first-stage screening procedure that can be followed up by pediatrician questioning and/or referral to a mental health professional. The Pediatric Symptom Checklist (PSC) (Jellinek et a1., 1988) (Appendix) 'is one of the only questionnaires that has been validated for use in pediatric office screening (Sturner, 1991). The PSC is a 35-item questionnaire designed to be completed in the pediatrician's waiting room by parents of 6- to 12-year-old children. The PSC takes less than 5 minutes to complete and score and reflects the parent's impression ofhislher school-aged child's psychosocial functioning. Us- . ing an empirically defined cutoff point, the PSC identifies dysfunctional children likely to benefit from further psychiatric evaluation. The validity and reliability of the PSC have been demonstrated in a variety of pediatric settings (Jellinek and Murphy, 1988, 1990; Jellinek et a1., 1986, 1988; Murphy and Jellinek, 1988; Murphy et aI., 1989; Walker et aI., 1989). PSC scores have been compared with child assessments based on in-depth interviews, other validated questionnaires, and data reflecting how children are functioning in their daily lives. In one study (Jellinek et aI., 1988), the screening accuracy of the PSC was validated against comprehensive assessments made by experienced clinicians in two pediatric settings: a private practice in a predominantly white, middle to upper middle-class suburb, and an urban health maintenance organization that serves a more ethnically and socioeconomically heterogeneous group of patients. Findings indicated 79% agreement (K = 0.60) between PSC scores and clinicians' ratings of psychosocial dysfunction, with a sensitivity of 0.95, and a specificity of 0.68. In the same study, the overall agreement between the PSC and presence of a serious psychiatric diagnosis was 87% (K = 0.74; sensitivity = 0.87; specificity = 0.89). Test-retest reliability between two administrations of the PSC about 4 weeks apart was r = 0.86. Further analyses (Murphy and Jellinek, 1988) established that the PSC had similar validity and reliability for screening the relatively small number of lower middle class and minority group children in the above samples. PSC scores agreed with clinicians' ratings of psychosocial dysfunction for 79% of the lower middle-class children in the samples and 79% of the minority group children. For lower middle-class children, the reliability of the PSC over time was r = 0.87; for minority group children, it was r = 0.91. Although this investigation extended the validity and reliability of the PSC to a lower middle-class sample, it lacked children from the poorest social class, and thus prompted the present study. 1106
The Present Study The current research followed a design that replicated the methods used in the above studies of the PSC in middle and lower middle-class samples. In the current study, the validity and reliability of the PSC were assessed in an inner-city pediatric clinic in one of Boston's poorest neighborhoods to determine whether the PSC is a suitable measure for screening children from such backgrounds. Extending the validity of the PSC to children from lower-class backgrounds was considered particularly relevant, given the documented higher rates of psychosocial problems among such children as well as their unmet mental health needs. A single site was chosen because of the large commitment necessary from the clinic to allow the investigators to monitor all pediatric activity, the difficulty in finding such clincs, and the expense involved in conducting "comprehensive evaluations" on even a relatively small number of subjects. If the PSC were found to be valid in this clinic, then further work on other samples clearly would be indicated; if the PSC did not appear to be valid, then further developmental work on the PSC might be necessary. A clinic serving African-American children was selected inasmuch as they represent the largest minority group in this country and because the question of the linguistic adequacy of the PSC in Hispanic- and Asian-American groups would require additional validation. The major hypothesis tested was that case classifications based on the PSC would obtain rates of agreement with those derived from clinicians' evaluations that were statistically adequate and, we hoped, comparable with the rates obtained using the same methods in more advantaged samples. In this, as in previous studies, presence of at least one psychiatric diagnosis was used as an alternative indicator of "true" psychosocial dysfunction in evaluating the validity of the PSC.
Methods Procedure: Questionnaire Sample The procedure used in the current research was a replication of the design used in a previous study of the PSC in predominantly middle-class samples; a more complete description of the design can be found in Jellinek et a1. (1988). All visits to general pediatric sessions were monitored during a 6-month period. Because most school-aged children attended the afternoon or evening rather than morning pediatric sessions, an attempt was made to cover as many of these sessions as possible. During all of the sessions, research assistants from the study approached all parents who entered the waiting room to see a pediatrician and asked them to complete a PSC on each of the scheduled children. Participation was completely voluntary. This article focuses on children 6 to 12 years old (the age range for which the PSC has been validated), although younger and older children were also screened. Measures: Questionnaire Sample Pediatric Symptom Checklist (PSC). The PSC consists of 35 symptoms that parents rate as "often," "sometimes," or "never" present in the child. The PSC is scored by assigning J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
2, 1, or 0 points, respectively, to these ratings, and an overall score is calculated using simple addition. For 6- to 12year-old children, scores up to 27 are considered within the normal range; scores of 28 or greater suggest dysfunction and the need for further evaluation. The PSC form used in the current study also requested basic demographic data including the child's age, gender, and race; the parents' marital status and occupations; and whether the family was covered by Medicaid. The Hollinghead (1975) scale was used to code parent occupational data and to estimate family SES rank on a nine-point scale from 1 (least advantaged) to 9 (most advantaged). Cases in which occupation was left blank were checked to see whether the parent reported receiving Medicaid; if so, the family was coded as Hollingshead job status 1 (dependent on public assistance). Cases in which both insurance and occupation information were left blank were checked in the clinic's billing system. Families that the computer listed as receiving Medicaid were also coded as job status 1. Procedure: Interview Sample
After parents had completed the PSC, the research assistant told them about the interview phase of the study, asked whether they would allow their names to be included in the pool of potential interview subjects, and obtained written consent. Parents who consented were sampled in the following manner: the parents of all childre.n scoring above the PSC cutoff point were invited to participate until 14 had been interviewed. In addition, a comparison group of 10 subjects was randomly selected from those scoring below the PSC cutoff. These procedures were designed to yield an interview sample of 24 subjects, 14 PSC positive and 10 PSC negative, thus duplicating the design used in the previous study of the PSC in more advantaged samples (Jellinek et al., 1988). In this previous study, PSC cases had been oversampled in comparison with their prevalence in the general population because of our somewhat greater concern about the possibility of false positives. Interviews were conducted within 3 months after PSC screening by a doctoral student in counseling (K.G.) and a clinical psychologist (J.M.M.), both of whom have extensive experience evaluating and treating children. The interviewers were blind to children's PSC scores. To assess the reliability of the PSC over time, parents were asked to fill out a second PSC at the beginning of the interview. Measures: Interview Sample Garmezy Child Interview. The Garmezy Child Interview (Finkelman, unpublished), modified as in the previous study (Jellinek et al., 1988), was used as the basis of the clinical interview. Information obtained from the child was supplemented by information obtained from the parent. Children's Global Assessment Scale (CGAS). The interviewers rated each child on the CGAS (Schaffer et al., 1983), blindly with regard to PSC score and independently of each other. After both of the interviewers had completed CGAS ratings, they reviewed together all available clinical information on each child (excluding PSC score) and assigned a consensus CGAS rating. This consensus CGAS rating proJ. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
vided the "gold standard" against which the validity of the PSC was tested. Using the cutoff defined by Schaffer and associates, CGAS scores of 70 or below were considered indicative of a clinical range of impairment. DICA-P. The Diagnostic Interview for Children and Adolescents, parent report version (OICA-P) (Herjanic and Reich, 1982) was used to determine whether the child had any of the specific symptoms of each of the psychiatric diagnoses listed for children in DSM-III (American Psychiatric Association, 1980). As in the previous study (Jellinek et al., 1988), the diagnoses of simple phobia and mild enuresis were not considered indicative of serious psychiatric impairment and were excluded from the caluclation of the total number of diagnoses for each subject. As in the previous study, interviewers were blind to the subjects' PSC scores. Children were also coded as having any diagnosis versus having no diagnosis. Although the PSC was designed to assess psychosocial functioning (and not as a diagnostic instrument), some overlap between PSC case classification and psychiatric diagnoses was expected and has been found in previous research (Jellinek et al., 1988). Therefore, the presence of a diagnosis on the DICA-P provided an alternative standard against which the validity of the PSC was tested in the current study. Pediatrician ratings. Pediatricians, blind to children's PSC scores, rated each child on a 5-point scale (Jellinek et al., 1988) regarding the child's current psychosocial flJnctioning. Ratings of 4 or 5 (serious problem) were contrasted with ratings of 1 to 3 (adaquate functioning) for categorical analysis. Pediatricians were also asked to complete a Physician Visit Record (PVR) (Goldberg et al., 1979) for each child in the interview study. This form requests information about the presenting symptom for the current visit, as well as a number of psychosocial questions. In this article, only the data regarding reason for the current visit were analyzed. Physicians were asked to select a single response that best described the presenting symptom or main reason for the current visit from a list of seven basic categories (e.g., preventive measure such as physical exam, chronic physical condition, emotional symptoms). Children's Personal Data Inventory (CPDl). Background information was obtained using the CPDI (National Institute of Mental Health, 1973). Data regarding the parents' educations and occupations were coded using the Hollingshead (1975) scale, to provide a four-factor rating of family SES (more precise than the two-factor rating obtained from the PSC form). The CPO I also allowed an assessment of whether the child had a history of psychiatric treatment. Results Questionnaire Sample
PSC screening was conducted during 55 of the 104 pediatric sessions scheduled during the 6-month study (53% of all pediatric sessions and 70% of the afternoon and evening sessions during this time). An attempt was made to cover all afternoon and evening sessions, but the schedules of the research assistants did not permit this. 1107
MURPHY ET AL.
During the 55 monitored sessions in the current sample, 160 children aged 6 to 12 years kept their appointments. Thirty of these children were not eligible for the study for various reasons (e.g., non-English speaking, previously sampled). Of the 130 remaining children, the parents of 123 (95%) completed aPSe. The sample of 123 children included 52 (42%) boys and 71 (58%) girls; the mean age was 8.8 (SD= 1.9). All the children were from minority groups, and all but two listed their race as African-American or mixed. Sixty-three percent lived in single parent homes. It was possible to code occupational rank (Hollingshead, 1975) for 96 (78%) of the families; the mean occupational rank was 2.7 (range = 1.8; SD = 2.0). Fifty (45%) of the families were covered by Medicaid or comparable city sponsored health insurance and were below the poverty level. Thirty-six children (37.5%) were from families with SES ranks of 2-5 (working class), and 10 children (10.5%) came from middle class families (SES 6-8). Although it was not possible to compare the obtained sample with all pediatric visits during the sampling period, the currently obtained sample could be compared with a somewhat more representative sample collected at the same clinic over a 6-week period a year later in which 71 % of all visits were monitored (Murphy et al., 1992). The currently obtained sample did not differ significantly from the more inclusive sample according to gender (58% versus 54% female), Medicaid coverage (45% versus 33%), or percent with single parents (63% versus 50%). As expected, the currently obtained sample did have a somewhat but not significantly higher mean age (8.8 versus 8.3 years), given the oversampling of afternoon and evening sessions in the current study and the larger percentage of older children in these sessions. PSC scores. Twenty-seven (22%) of the 123 children had PSC scores of 28 or greater, compared with rates found in the previous study (Jellinek et aI., 1988) of 11% in the suburban private practice and 17% in the urban HMO (X2 = 5.9, df = 2, p = .05). Two-by-two X2 analyses showed that although the PSC positive rate in the current innercity sample was significantly higher than that found in the suburban private practice (X2 = 5.1, df = 1, p < .05), it was not significantly different from that found in the urban HMO. The PSC positive rate of 22% for the inner-city sample was very similar to the rate of 23.8% found for the lower SES subjects in a previous PSC study (Murphy and Jellinek, 1988). PSC scores by background variables. In the present sample, children from single-parent families were significantly more likely to score above the PSC cutoff (34%) than were children from two parent families (7%; X2 = 7.7, df = 1, p < .01). There was a trend for boys to have a higher PSC positive rate (31 %,16/52) than girls (16%,11/71; X2 = 3.2, df = 1, P < 0.1). The PSC positive rate appeared to be influenced by SES within this sample, although the results did not reach statistical significance. For children from Medicaid-supported families, the PSC positive rate was 22% (11/ 50); for working class children the rate was 25% (9/36) and 1108
TABLE
1. Relationship between PSC and Other Measures" PSC Score
CGAS consensus Functional Dysfunctional
DSM-lII diagnosis No diagnosis Has diagnosis Pediatrician rating Functional Dysfunctional Psychotherapeutic treatment Never had Rx Had Rx
<28
28+
8 (100) 2 (12)
0 (0) 14 (88)
6 (75) 4 (25)
2 (25) 12 (75)
9 (50) 1 (17)
9 (50) 5 (83)
10 (45) 0 (0)
12 (55) 2 (100)
X2
p
13.4
.001
3.6
.06
(NS)
(NS)
Note: PSC = Pediatric Symptom Checklist. a Values in parentheses are percentages.
for middle class children the PSC positive rate was 10% (1/ 10). Interview Sample
The parents of 115 (94%) of the 123 children agreed to be included in the pool of potential interview subjects. To obtain the desired sample of 14 PSC positive and 10 PSC negative children, 30 families (16 PSC positive and 14 PSC negative) had to be invited to participate (24/30; 80% completion rate). The interview sample consisted of 12 boys; 9 (75%) of whom were identified as cases by the PSC, and 12 girls, 5 (42%) of whom were so identified. All 24 children were from minority groups (23 African-American, one mixed African-American and Native American), and their mean age was 8.5 (SD = 1.8). Eighteen (75%) children lived in singleparent homes, and 11 (46%) were covered by Medicaid. Hollingshead four-factor ratings of SES ranged from 17 to 49, with a mean of 29.8 (SD = 10.3). Fifty percent (N = 12) of the families were from the two least advantaged SES groups (IV and V). Clinicians' CGAS classifications. The two interviewers' CGAS case classifications agreed for 20 of the 24 children (83%; X2 = 7.4, df= l,p < .01; K = .64). Overall agreement between clinicians' consensus CGAS classifications and PSC case/not case classifications was 92% (22/24; X2 = 13.4, df = 1, P < .001), with a K = .82 (Table 1). Fourteen of the 16 children classified as dysfunctional by the clinicians had been correctly identified by the PSC, a sensitivity of 0.88 in this sample. The PSC correctly classified all eight of the children who were judged by the clinicians to be functioning adequately, a specificity of 1.0. Psychiatric diagnoses. Sixteen children were given at J. Am. A cad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
least one diagnosis (excluding phobia and enuresis) on the DICA-P. The most prevalent diagnoses were attention deficit disorder (ADD) (N = 9) and oppositional disorder (N = 9); separation anxiety (N = 5) and overanxious disorders (N = 3) also were found. Twelve (75%) of the children with diagnoses had been classified as cases by the PSC; of the eight children with no diagnosis, only two (25%) were PSC positive (X2 = 3.6, df = 1, p = .06; Table 1). Thus, using the presence of a psychiatric diagnosis as an alternative measure of "true" dysfunction, the PSC had a sensitivity of 0.75 and a specificity of 0.75 in this sample. The four children who received a diagnosis but screened negative on the PSC had diagnoses similar to PSC positive children (i.e., ADD, oppositional disorder, overanxious disorder). The mean number of diagnoses per child was 1.3 (SD = 1.4), with a range from 0 to 6. Children who screened positive on the PSC had a significantly higher mean number of diagnoses (1.9, compared with 0.6 for PSC negative children, t = -2.31, p < .05), providing further support for the validity of the PSC. Pediatrician ratings. Six (25%) of the children were rated by their pediatricians as having serious or very serious behavioral, emotional, or developmental problems requiring referral. Five (83%) of these six had been identified as cases by the PSC, a further demonstration of the sensitivity of the instrument. Conversely, of the 14 children rated by the PSC (and the CGAS) as dysfunctional, the pediatricians identified only five (36%) as having serious problems requiring referral. Pediatricians completed the PVR item about main presenting symptom for 22 of the children. Most (N = 15; 68%) were being seen for physical examinations or for acute conditions (N = 6; 27%). One child was being seen for an accident or injury, and none was being seen primarily for emotional symptoms or chronic physical conditions. Psychiatric history. Data from the CPDI indicated that two of the children in the interview sample had received mental health treatment. Both these children scored above the PSC cutoff, a difference that was not statistically significant, but did suggest another form of construct validity for the PSC. Conversely, from the standpoint of unmet mental health needs, 12· (86%) of the 14 children with positive PSC (and CGAS) scores had never received any treatment. Reliability of the PSC. The mean length of time between the first and second administrations of the PSC was 36 days (range 6 to 69 days). Time 2 PSC case/not case classification agreed with that from Time 1 for 20 of the 24 children (83%; X2 = 9.5, df= 1,p < .01; K = .68); the Pearson correlation between scores at the two time periods was r = .84. Discussion
The present study demonstrated that the Pediatric Symptom Checklist was a valid and reliable instrument for screening for psychosocial dysfunction in the lower SES and African-American minority group children in the current sample. When compared with comprehe!:1sive assessments made by experienced clinicians, the PSC had a sensitivity of 88% and a specificity of 100% in this sample; PSC case/ not case classifications agreed with those of the mental J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
health clinicians for 92% of the children. Thus, using clinicians' consensus ratings as the "gold standard," the validity of the PSC for screening the children in this inner-city sample was comparable with that found for more advantaged samples (Jellinek et aI., 1988; Walker et aI., 1989). A comparison of PSC case/not case classifications with several other measures provided additional evidence of the validity of the PSC for screening children in the current sample. Examining psychiatric diagnoses derived from structured interviews with parents, children who screened positive on the PSC had a significantly higher mean number of diagnoses. Using presence of a psychiatric diagnosis as an alternative validity standard, the PSC had a sensitivity of 75% and a specificity of 75%, again similar to the results obtained in the middle-class samples. Comparing PSC classifications to pediatricians' ratings of child functioning, the PSC correctly identified five of the six children rated as dysfunctional by their pediatricians. Two of the children in the sample had received mental health treatment; both scored above the PSC cutoff. Although these findings lend support to the validity of the PSC for screening inner-city children, they also highlight the unmet mental health needs of these children: less than half the children who were flagged by the PSC and CGAS ratings as in need of further evaluation were so identified by their pediatricians, and less than 15% of the children who were identified as dysfunctional by the screening had ever received any psychotherapeutic treatment. Thus, the findings of the current study replicate previous studies that found that less than half the pediatric outpatients with serious psychosocial problems are identified by their pediatricians and that only a fraction of these children get the services they need. This is further documentation of a serious, continuing unmet need in pediatrics. That all the PSC cases were independently rated as cases on the CGAS strengthens the basic point that "true cases" of dysfunction were rated as functioning adequately by their pediatricians and were not being identified as needing further evaluation and follow-up. Regarding reliability, PSC case/not case classifications showed 83% agreement over time, with a Pearson correlation of 0.84 between scores at the two time periods. The reliability of the PSC in this sample was thus very similar to that found in the study of middle-class pediatric samples where time-to-time case rating agreement was 87% and the correlation between exact scores was r = .86. When compared with children seen in a private practice in a middle- to upper-middle-class suburb (Jellinek et aI., 1988), children in the current inner-city sample exhibited a significantly higher level of impairment. This finding is consistent with previous studies documenting a higher prevalence of psychosocial problems among children from economically disadvantaged backgrounds. In fact, research to date has demonstrated a clear trend for PSC-rated dysfunction to vary with socioeconomic status: rates have ranged from 11 % in a suburban middle- to upper-middle-class sample, to 17% in an urban HMO (Jellinek et aI., 1988), 24% in a lower-middle-class sample (Murphy and Jellinek, 1988), and 22% in the current inner-city sample. These findings 1109
MURPHY ET AL.
likely reflect the increased levels of stress associated with poverty. The current study has a number of limitations. Studying a small sample in a single clinic leaves many questions unanswered. The current study focused on African-American children in a single community, and we must leave to future research the question of whether the PSC (in either English or translation) is valid for children from other minority and ethnic groups in other locations. The current study also leaves open the differential effects of lower SES and minority status, inasmuch as our subjects were both AfricanAmerican and poor. It should be noted that one of our previous studies (Murphy and Jellinek 1988) did address this question and found that although both minority status and low SES did appear to influence PSC screening scores, SES played the larger and statistically significant role. The calculation of sensitivity and specificity using such a small number of cases should be regarded as preliminary, given the possibility of chance variations within the current sample. The fact that the current figures so closely resemble those found in three previous studies (Jellinek et aI, 1988; Murphy and Jellinek 1988; Walker et aI, 1989), does however give them added credibility. From a psychometric standpoint, the PSC is valid, reliable and consistent with the expectation that children from lowerSES backgrounds have higher rates of psychosocial dysfunction than do middle-class children. From a clinical perspective, it is important to note that not all poor children are dysfunctional. The majority are not, despite their many stresses. However, the approximately 20 to 30% who show significant dysfunction need to be identified, confirmed by brief pediatric assessment, and then, if indicated, referred for further evaluation. In the hectic world of inner-city pediatrics, the availability of a validated screening instrument such as the PSC provides a needed bridge for clinicians and researchers who want to identify children with psychosocial dysfunction and to improve the process of allocating limited resources for meeting their needs.
Appendix
Pediatric Symptom Checklist Please mark under the heading that best fits your child: Never 1. Complains of aches or pains 2. Spends more time alone 3. Tires easily, little energy 4. Fidgety, unable to sit still 5. Has trouble with a teacher 6. Less interested in school
1110
Sometimes
Often
7. Acts as if driven by a motor 8. Daydreams too much 9. Distracted easily 10. Is afraid of new situations 11. Feels sad, unhappy 12. Is irritable, angry 13. Feels hopeless 14. Has trouble concentrating 15. Less interest in friends 16. Fights with other children 17. Absent from school 18. School grades dropping 19. Is down on him or herself 20. Visits doctor with doctor finding nothing wrong 21. Has trouble sleeping 22. Worries a lot 23. Wants to be with you more than before 24. Feels he or she is bad 25. Takes unnecessary risks 26. Gets hurt frequently 27. Seems to be having less fun 28. Acts younger than children his or her age 29. Does not listen to rules 30. Does not show feelings 31. Does not understand other people's feelings 32. Teases others 33. Blames others for his or her troubles 34. Takes things that do not belong to him or her 35. Refuses to share Michael Jellinek, M.D., Massachusetts General Hospital
References Achenbach, T. M. & Edelbrock, C. S. (1981), Behavioral problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monogr. Soc. Res. Child Dev., 46, Serial No. 188:1-81. American Psychiatric Association (1980), Diagnostic and Statistical Manual of Mental Disorders, 3rd edition (DSM-III). Washington, DC: American Psychiatric Association. Children's Defense Fund (1985), A Children's Defense Fund Budget. Washington, DC: Children's Defense Fund. Connors, C. K. (1979), Ratings scales. In: Basic Handbook of Child J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
PSYCHOSOCIAL SCREENING OF INNER-CITY CHILDREN
Psychiatry, ed. J. D. Noshpitz. New York: Basic Books. Costello, E. J. (1986), Primary care pediatrics and child psychopathology: a review of diagnostic, treatment, and referral practices. Pediatrics, 78: 1044-1051. Costello, E. J., Costello, A. J., Edelbrock, C. et al. (1988), Psychiatric disorders in pediatric primary care: Prevalence and risk factors. Arch. Gen. Psychiatry, 45: 1107-1116. Costello, E. J., Edelbrock, C., Costello, A. et al. (1986), The diagnosis and management of psychopathology in children in an organized primary healthcare setting: final report (Contract No. 278-830006). Rockville, MD: Division of Biometry and Applied Sciences, Department of Health and Human Services. Goldberg, I. D., Regier, D., McInerny, T. K. et al. (1979), The role of the pediatrician in the delivery of mental health services to children. Pediatrics, 63:898-909. Goldberg, I. D., Roghmann, K. J., McInerny, T. K. et al. (1984), Mental health problems among children seen in pediatric practice: Prevalence and management. PediatriCs, 73:278-293. Herjanic, B. & Reich, W. (1982), Development of a structured psychiatric interview for children: agreement between child and parent, J. Abnorm. Child Psycho!., 10:307-324. Hollingshead, A. B. (1975), Four Factor Index of Social Status. New Haven, CT: Yale University Sociology Department. Institute of Medicine (1989), Research on Children and Adolescents with Mental, Behavioral, and Developmental Disorders.:-Mobilizing a National Initiative. Washington, DC, National Academy Press. Jellinek, M. S., & Murphy, J. M. (1988), Screening for psychosocial disorders in pediatric practice. Am. J. Dis. Child., 112: 1153-1157. Jellinek, M. S. & Murphy, J. M. (1990), The recognition of psychosocial disorders in pediatric practice: The current status of the Pediatric Symptom Checklist. J. Dev. Behav. Pediatr., II :273-278. Jellinek, M. S., Murphy J. M., & Burns, B. (1986), Brief psychosocial screening in outpatient pediatric practice. J. Pediatr., 109:371-378. Jellinek, M. S., Murphy, J. M. & Robinson, J., et al. (1988), Pediatric Symptom Checklist screening school-age children for psychosocial dysfunction. J. Pediatr., 112:201-209. Murphy, J. M. & Jellinek,-M. S. (1988), Screening for psychosocial
J. Am. Acad. Child Adolesc. Psychiatry, 31:6, November 1992
dysfunction in economically disadvantaged and minority group children: further validation of the Pediatric Symptom Checklist. Am. J. Orthopsychiatry, 58:450-456. Murphy, J. M., Jellinek, M. S. & Milinsky, S. (1989), The Pediatric Symptom Checklist: Validation in the real world of the middle school. J. Pediatr. Psycho I. , 14:629-639. Murphy, J. M., Arnett, H. L., Bishop, S. 1., Jellinek, M. S. & Reede, J. Y. (1992), Screening for psychosocial dysfunction in pediatric practice: a naturalistic study of the Pediatric Symptom Checklist. Clin. Pediatr. (in press). National Institute of Mental Health (1973), Children's Personal Data Inventory. In: ECDEU Assessment Battery for Psychopharmacology. Rockville, MD: National Institute of Mental Health. National Institute of Mental Health (1990), National Plan for Research on Child and Adolescent Mental Disorders. Rockville, MD: National Institute of Mental Health. Roghmann, K. J., Babigian, H. M., Goldberg, I. D. et al. (1984), The increasing number of children using mental health services: Analysis of a cumulative case register. Pediatrics, 73:781-790. Schaffer, D., Gould, M. S., Brasio, J. et al. (1983), A Children's Global Assessment Scale (CGAS). Arch. Gen. Psychiatry, 40: 1228-1231. Schwartz-Gould, M., Wunsch-Hitzig, M. A., Dohrenweld, B. (1981), Estimating the prevalence of childhood psychopathology: a critical review. J. Am. Acad. Child Psychiatry, 20:462-476. Starfield, B. H., Gross, E., Wood, M. et al (1980), Psychosocial and psychosomatic diagnoses in primary care of children. Pediatrics, 66:159-167. Stumer, R. A. (1991), Parent questionnaires: basic office equipment? J. Dev. Behav. Pediatr., 12:51-54. Tuma, J. M. (1989), Mental health services for children: the state of the art. Am. Psycho I. , 44: 188-199. Walker, W.O., Lagrone, R. G., Atkinson, S. W. (1989), Psychosocial screening in pediatric practice: identifying high risk children. J. Dev. Behav. Pediatr., 10: 134-148. Williams, S. J., Diehr, P., Drucker, W. L. (1979), Mental health services: utilization by low income enrollees in a prepaid group practice plan and an independent practice plan. Med. Care, 17: 139145.
1111