- Email: [email protected]
A comparison of outcomes for children and youth in foster and residential group care across agencies
Children and Youth Services Review 85 (2018) 19–25
Contents lists available at ScienceDirect
Children and Youth Services Review journal homepage: www.elsevier.com/locate/childyouth
A comparison of outcomes for children and youth in foster and residential group care across agencies☆
T
⁎
Sharon G. Portwooda, , Suzanne A. Boyda, Ellissa Brooks Nelsona, Tamera B. Murdockb, Jessica Hamiltonb, Angela D. Millerc a
The University of North Carolina at Charlotte, USA The University of Missouri, – Kansas City, USA c George Mason University, USA b
A B S T R A C T Working collaboratively with two state associations and their member (nonprofit) agencies providing out-ofhome care to children and youth, University researchers conducted a multi-site project to examine whether there were any differences in individual child-level outcomes between children placed in residential group care and those placed in foster. The study employed a quasi-experimental repeated measures design, with data collected at a minimum of two intervals (at intake and 3-month follow-up) and at subsequent intervals of 6 and 12 months for children remaining in care. Samples for analyses were drawn from 1082 youth in either residential group care (n = 903) or foster care (n = 179), in one of 37 agency sites across two southeastern states, who participated in a broader evaluation project. The average ages of participating youth in residential and foster care were 13.97 (SD = 2.43) and 13.65 (SD = 2.73), respectively. Based on his or her score on the Children's Global Assessment Scale (CGAS) at intake, each participant was also assigned to the low functioning group (n = 526; 53.1%), the borderline group (n = 232; 23.4%), or the high functioning group (n = 232; 23.4%). Analyses confirmed that youth in foster care tended to have higher levels of general functioning at baseline than did youth placed in group care. However, the degree to which youth progressed in care on measures of general functioning and mental and behavioral health problems did not differ based on placement setting; youth in residential group care settings progressed at the same rate as youth in community-based settings, regardless of their level of functioning at intake. The only exception to this pattern was in regard to anxiety; there was an observable, but non-significant trend of youth in foster care reporting decreases in anxiety levels, while those in group care reported increased anxiety.
During the past several decades, policymakers and practitioners have debated how best to serve the needs of children placed in out-ofhome care, with an overall focus on the relative attributes of community-based versus residential (agency-based) care. Despite a body of literature emphasizing the need for research comparing youth outcomes across foster care and residential group care to inform these decisions, the body of empirical work in this area remains sparse (James, 2011). Absent empirical data from which courts and/or the child welfare system can determine his or her optimal placement based on the relative effectiveness of available settings in meeting the type of needs with which the child presents, current policy and practice favor placement in the least restrictive setting, such that community-based foster
care is favored over residential group care. Cost can also factor into placement decisions, with foster care being favored over more expensive group care (Barth, 2005). Nonetheless, as noted by Barth (2002), there are circumstances in which children and youth may be best served in a group setting (e.g., when they have run away from foster care, they present a threat to themselves or others, or they are moving from a more restrictive setting). Moreover, the fact that a substantial number of children continue to reside in residential programs warrants examination of how child-level outcomes compare across settings (James, Zhang, & Landsverk, 2012). Despite the objections to congregate care, some studies have documented positive outcomes for those residential programs that
☆ The authors would like to acknowledge The Duke Endowment for its financial support of this project, along with the Duke Endowment staff; the South Carolina Association of Children's Homes and Family Services; Benchmarks: An Alliance of Agencies Helping Children, Adults & Families; and the many agency staff and research team members who contributed their time and expertise to the evaluation project from which the data for this manuscript were drawn. ⁎ Corresponding author. E-mail address: [email protected] (S.G. Portwood).
https://doi.org/10.1016/j.childyouth.2017.11.027 Received 4 August 2017; Received in revised form 23 November 2017; Accepted 23 November 2017 Available online 24 November 2017 0190-7409/ © 2017 Elsevier Ltd. All rights reserved.
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
(16.7%) were in high management programs; and 257 (28.4%) were in residential treatment. Notably, youth in foster care also represented placement in various service levels, with 97 (54.1%) in therapeutic foster care, 76 (42.5%) in family foster care, and 6 (3.4%) in specialized foster care. Participants were enrolled in the study for the duration of a single placement (i.e., none of the participants moved across placement settings).
reflect high standards of care, including family involvement and support and supervision by caring adults (Barth, 2002). From their review and selective meta-analysis of research studies published between 1990 and 2005 examining residential care outcomes, Knorth, Harder, Zandberg, and Kendrick (2008) rejected the notion that residential care has primarily negative outcomes for youth, instead concluding that psychosocial functioning outcomes could be expected to improve, on average, for children and youth who had been in residential care for a period of time. In contrast, Lee, Bright, Svoboda, Fakunmoju, and Barth (2010) concluded that a majority of published studies favor family foster care or treatment foster care over residential group care. Advocates for moving away from, and in some cases even dismantling, residential group care have described the outcomes and long-term treatment gains of children and youth placed residential group care as “uncertain” (Preyde, Frensch, Cameron, Hazineh, & Riosa, 2011, p. 5) and mixed (Holstead, Dalton, Horne, & Lamond, 2010), noting that some studies evidenced only short-term gains, while other, long-range studies did not find high end-state functioning. For example, following an investigation of 285 adolescents in residential treatment, Lyons, Terry, Martinovich, Peterson, and Bouska (2001) found that although there was evidence of a reduction in risk behaviors and depression, along with improved the management of psychosis, residential treatment did not appear to improve functioning; additionally, outcomes varied substantially across providers. Efforts to compare outcomes across placement settings have not only been limited in number, but they have also been complicated by methodological challenges, including selection bias given suggestions that youth placed in residential group care exhibit more problems than do those placed in foster care (Barth, 2002). In fact, a growing body of evidence suggests that youth in residential care have more severe difficulties than do youth in foster care upon entry into out-of-home placement (Leloux-Opmeer, Kuiper, Swaab, & Scholte, 2016). After careful efforts to generate matched samples in order to control for this potential bias, Lee and Thompson (2008) found that residential group care may actually be more effective than therapeutic foster care for some youth, with participants in therapeutic foster care proving to be less likely to be favorably discharged, less likely to return home, and more likely to experience a later out-of-home placement than youth in a congregate care setting. Based on their examination of a statewide system of care over a five-year period, Lyons, Woltman, Martinovich, and Hancock (2009) concluded that residential care serves an important function in serving children and youth with high levels of need. Given the contradictory findings in the existing literature, along with the many unanswered questions that remain regarding the relative effectiveness of foster and residential group care, there is a clear need for additional research in this area and, particularly, for projects that address the methodological problems present in many earlier studies. In response to this need, the current study was designed to compare outcomes of youth in foster care to those of youth in residential group care across multiple nonprofit agencies. More specifically, this study addressed the question of how youths' general functioning, behavior problems (as reported by therapists and the youth themselves), and anxiety compared across the two settings over time.
1.2. Data collection The study employed a quasi-experimental repeated measures (pretest-posttest) design. Trained agency staff collected data at a minimum of two intervals, at intake and at a three-month interval following intake. For those children remaining in care, data were also collected at 6 and 12 months after intake. (Notably, the Achenbach Child Behavior Checklist (CBCL) -Youth Self Report is designed to be administered at six-month intervals and so was not administered at three-month follow-up.) All children enrolled in the designated out-ofhome care programs at participating agencies were eligible for participation, but the design was quasi-experimental since, rather than being randomly assigned, children were placed in either foster care or residential group care in accordance with prevailing practices in the community or organization. To supplement existing admission forms, agency staff completed the Youth Demographic Form and Discharge Data Form created by the researchers to capture information on participants' characteristics and services received. The child's therapist, case worker, or other dedicated staff member completed the CGAS and the CBCL at all specified time points, beginning with the collection of baseline data within 30 days of admission; the date on which baseline data were collected then served as the reference point for subsequent follow-up data collection dates. The same trained agency staff members also administered the youth measures at the designated intervals. (Although there may have been some reassignment of cases at agencies for various reasons [e.g., staff turnover], this information was not available to the researchers.) Youth completed the measures individually in a private location. The staff member read a list of reminders to youth prior to his or her completing the study measures to ensure standardized data collection procedures across agencies. Staff members also read the questions aloud to younger children or those who had difficulty reading. Given the variance in children's ages and reading ability, the time required to complete the study measures also varied considerably. Agencies provided de-identified data to the researchers for analysis. The researchers obtained approval from their University Institutional Review Board. 1.3. Instruments 1.3.1. Children's Global Assessment Scale (CGAS) The CGAS provides a simple means for a variety of raters across disciplines to quantify the global functioning of children ages 4 to 16 years on a scale of 1 to 100. Higher scores indicate higher levels of functioning, with scores over 70 designated as normal. The scale has demonstrated interrater and test-retest reliability, as well as discriminant and concurrent validity (Shaffer et al., 1983). The CGAS has been widely used in research and shown to be a good measure of children's functional competence in clinical settings (Green, Shirk, Hanze, & Wanstrath, 1994).
1. Methods 1.1. Participants The current study examined a subset of the 1114 children and youth enrolled in out-of-home placement programs across 37 agency sites in two states who participated in a broader evaluation project (Portwood, Boyd, & Murdock, 2016). Analyses were limited to those 1082 youth enrolled in the various levels of residential group care (n = 903; 83.5%) and foster care (n = 179; 16.5%). Among those youth in residential group care settings, 409 (45.3%) were in low management programs; 86 (9.5%) were in moderate management programs; 151
1.3.2. Child Behavior Checklist (CBCL) Along with the CBCL-Youth Self Report (described below), the CBCL is part of the Achenbach System of Empirically Based Assessment (ASEBA) (Achenbach, 1991). The teacher version of the 118-item CBCL was used for clinicians to rate problem behaviors for participating children and youth. As with the parent version, the CBCL includes three broad-band scales (i.e. Internalizing, Externalizing, and Total Problems) 20
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
and eight specific narrow-band scales (e.g. attention problems). The Internalizing Scale is comprised of items to assess anxiety, depression, social withdrawal, and somatic complaints; the Externalizing Scale assesses aggression and delinquent behavior; and the Total Problems scale provides an aggregation of the two. Clinicians responded to each item using a 3-point scale from “not true” to “very true.” Detailed scoring instructions are available for this instrument, which has extensively documented reliability and validity (Achenbach & Edelbrock, 1983).
Table 1 Demographic information for subsample of youth enrolled in residential group care and foster care. Demographic characteristics
Residential group care (n = 903)
Foster care (n = 179)
Gender
510 (56.5%) 393 (43.5%) 260 (28.9%) 1 (0.1%) 547 (60.7%) 21 (2.3%) 1 (0.1%) 62 (6.9%) 8 (0.9%) 1 (0.1%) 138 (15.3%) 765 (84.7%) 13.97 (SD = 2.43)
81 (45.3%) 98 (54.7%) 93 (52.0%) 1 (0.6%) 76 (42.4%) 4 (2.2%) 1 (0.6%) 4 (2.2%) 0 (0%) 0 (0%) 39 (21.8%) 140 (78.2%) 13.65 (SD = 2.73)
Race
1.3.3. Child Behavior Checklist - Youth Self Report (CBCL-YSR) Youth ages 11 to 18 years may complete a self-report version of the CBCL, the CBCL-YSR (Achenbach & Rescorla, 2001). In response to the first seven items, youth indicate their competencies in three areas (i.e., activities, social relations, and school) that can be scored as scales and summed for a total competence score. The remaining 112 items inform eight syndrome scales (anxious/depressed, withdrawn/depressed, somatic complaints, social problems, attention problems, rule-breaking behavior, and aggressive behavior), along with the three subscales reflected in the CBCL (i.e., Internalizing, Externalizing, and Total Problems). Respondents characterize each behavior item as not true (0) or very true or often true (1) of them during the past six months. The CBCL-YSR has well-established reliability and validity (Leonard, Jang, Savik, & Plumbo, 2005).
Age group
Male Female African American Asian Caucasian Hispanic Middle-Eastern Multi-Racial Native American Undetermined 8–10 11–17
Age (average)
were Native American, two (0.2%) were Asian, and two (0.2%) were Middle-Eastern. Race/ethnicity was undetermined for one (0.1%) participant. Among youth enrolled in foster care programs, more than half (54.7%; n = 98) were female and 45.3% (n = 81) were male. Slightly over half were African American (52.0%; n = 93), followed by 42.4% (n = 76) White, 2.2% (n = 4) Hispanic, 2.2% (n = 4) multiracial, 0.6% (n = 1) Asian, and 0.6% (n = 1) Middle Eastern. This sample had a higher representation of children identified as Black than the population of foster care children in the two states examined (35% and 40% Non-Hispanic Black), and a corresponding lower representation of White children in foster care than in those states (48% and 47% NonHispanic White) during the period of this study (KIDS COUNT, n.d.). Each of the series of analyses performed were further limited to those youth (1) for whom CGAS scores were available at intake (which was required to assign participants to a functioning level group) (n = 990) and (2) for whom there were valid data, defined as scores on the relevant outcome measure at intake and the appropriate follow-up interval. Table 2 provides an overview of the demographic characteristics of the study groups of low functioning youth (n = 526; 53.1%), borderline youth (n = 232; 23.4%), and high functioning youth (n = 232; 23.4%) as determined by their scores on the Children's Global Assessment Scale (CGAS) at baseline. Consistent with standard scoring instructions, youth with CGAS scores in the clinical range (60 or below) were placed in the low functioning group; those with scores between 61 and 70 were placed in the borderline group; and those with CGAS scores in the normal range (71 or above) were assigned to the high functioning group.
1.3.4. Revised Children's Manifest Anxiety Scale (RCMAS) The 37-item RCMAS assesses the level and nature of anxiety in children and adolescents ranging in age from 6 to 19 years (Reynolds & Richmond, 1985). Respondents mark “yes” or “no” in response to each scale item, with “yes” indicating that the item accurately describes his or her feelings or actions. The Total Anxiety Score includes 28 items that are summed for a total score. The Lie Scale includes nine items to test for social desirability. Three subscale scores can also be calculated: physiological symptoms, worry, and concentration problems. Overall Cronbach's alphas for Total Anxiety range from 0.79 to 0.83 and testretest reliability ranges from 0.85 to 0.95. Reliability estimates for internal consistency are moderate, and the RCMAS has demonstrated validity. 1.4. Plan for analysis Data for the CGAS and RCMAS were collected at four time points. Because of the inherent difficulty in conducting analyses across time for children in residential and foster care, a hierarchical linear modeling approach was used with these measures in order to preserve as large a sample as possible. This approach is preferable over a repeated measures analyses which requires complete data. However, this approach was not possible for the CBCL measures collected by both self-report and clinician report. These data were collected at two time points (intake and 6 months) and were analyzed using repeated measures MANOVA. As described previously, this measure is appropriately used at six month intervals. In both of these analytic approaches, variables of functioning group level and placement setting were examined as predictors of the outcomes.
Table 2 Demographic information by functioning-level group.
2. Results 2.1. Sample characteristics
Demographic characteristics
Low functioning youth (n = 526)
Borderline youth (n = 232)
High functioning youth (n = 232)
Gender
(58.6%) 218 (41.4%) 142 (27.0%) 0 324 (61.6%) 13 (2.5%) 0 42 (8.0%) 5 (1.0%) 0 102 (19.4%) 424 (80.6%) 13.70 (2.53)
122 (52.6%) 110 (47.4%) 80 (34.5%) 0 134 (57.8%) 2 (0.9%) 0 13 (5.6%) 1 (0.4%) 1 (0.4%) 25 (10.8) 207 (89.2%) 14.45 (2.41)
117 (50.4%) 115 (49.6%) 84 (36.2%) 2 (0.9%) 123 (53.0%) 9 (3.9%) 2 (0.9%) 10 (4.3%) 2 (0.9%) 0 41 (17.7%) 191 (82.3%) 13.70 (2.50)
Race
Table 1 provides a summary of the demographic characteristics for the two study groups. The average ages for children enrolled in congregate care and foster care were 13.97 years and 13.65 years, respectively. For youth enrolled in residential group care programs, 43.5% of participants (n = 393) were female and 56.5% were male (n = 510). Over half of these participants were Caucasian (60.7%; n = 547); 28.9% (n = 260) were African-American; 6.9% (n = 62) were multiracial; and 2.3% (n = 21) were Hispanic. Eight participants (0.9%)
Age Group Age (average)
21
Male Female African American Asian Caucasian Hispanic Middle-Eastern Multi-Racial Native American Undetermined 8–10 11–17
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
2.2. Sample equivalence
90
Chi-square and independent t-tests were conducted to examine differences between youth included in comparative or growth analytic measures and those not included because of lack of data or attrition. Youth with data for at least two time points were compared to those with only one time point of data and hence not included comparative or change analyses. There was a slight but statistically significant difference in age; those included in analyses (n = 675; M = 13.6, SD = 2.51) were slightly younger than those with only one data point (n = 341; M = 14.35, SD = 2.35). There were no significant differences in gender or race for those included in the analyses as compared to the full sample. Males made up 58.0% (n = 397) of the analyses sample compared to 56.5% (n = 510) of the full sample. The race of those included in the analyses were almost exactly the same as the full sample; the largest ethnic groups were 61.2% (n = 416) Caucasian and 29.1% (n = 186) African-American. Youth cases with DSS involvement was 59% in both the analysis (n = 400) and full sample (n = 610).
80
C-GAS Score
70
Foster Low
CGAS-6 time 3
CGAS-6 time 4
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
67
49.07 (7.63) 65.55 (2.93) 80.96 (5.13)
33
56.64 (14.11) 69.42 (14.38) 76.69 (10.81)
13
59.62 (10.41) 65.45 (14.07) 75.88 (11.66)
5
63.40 (20.98) 70.33 (11.54) 77.50 (13.66)
Borderline
42
High
55
Residential Low
459
Borderline
190
High
177
47.30 (8.57) 65.86 (3.29) 80.29 (5.78)
19 26
292 121 119
54.85 (11.05) 65.68 (10.26) 75.48 (11.63)
11 17
190 78 89
57.52 (10.27) 67.99 (12.11) 72.94 (13.21)
6 8
48 30 36
Borderline Low
Time 1
Time 2
Time 3
Time 4
Fig. 1. Comparison of C-GAS scores over time by functioning group.
predictor of both the intercept and the change across time. The expected differences at intake were confirmed with the significant differences of the intercepts, both borderline (β02 = 17.49) and high (β03 = 31.46) functioning groups were significantly higher than the low functioning group represented by the intercept. The more interesting finding is the significant variation in slopes between the groups. Both the borderline (β12 = − 4.61) and the high (β13 = −8.16) differed significantly from the low functioning group. As can be seen in Fig. 1, the low functioning group experienced an increase (π1 = 5.37) over time in CGAS scores, whereas the borderline group experiences a more slight increase (+ 0.76) and the net change in the high function group average a decline (− 2.79) in scores. 2.4. Internalizing and externalizing problems The next set of analyses focused on children's levels of behavioral and mental health problems as measured by the CBCL. (Table 4 shows the functional level group scores on both of the CBCL measures at each time point.) These analyses were conducted using mixed model MANOVA with two time points. The first set of analyses focused on therapists' ratings. At six-month follow-up (which was the first time point at which CBCL follow-up data were collected), there was a significant effect of time on the Internalizing Scale, λ = 0.98, F(1, 222) = 4.55, p < 0.001 with scores increasing over time (M1 = 56.08, M2 = 57.60); there was also a significant effect of functioning group. Youth in the high functioning (n = 35; M = 52.03) and borderline groups (n = 47; M = 54.69) demonstrated significantly lower levels of problem behavior than did youth in the low functioning group (n = 146; M = 59.91). However, there were no other main effects or interactions, indicating no differences between the foster and residential care groups. For the Externalizing Scale, at six months, there was again a significant main effect of time, λ = 0.95, F(1, 222) = 11.69, p < 0.001, with scores increasing over time (M1 = 61.85, M2 = 63.85). There was again a significant main effect of functioninglevel group, F(2, 222) = 7.74, p < 0.001. High functioning youth (n = 35; M = 54.05) demonstrated significantly lower levels of problem behaviors than did youth in the borderline group (n = 47; M = 60.46), who, in turn, had significantly lower levels of problem behaviors than youth in the low functioning group (n = 146; M = 64.99). Finally, therapist' ratings of youth total problems indicated a main effect of time, λ = 0.95, F(1, 222) = 11.04, p < 0.05, with ratings increasing over time (M1 = 59.13, M2 = 61.09). There was also a significant effect of functioning level group, F(2222) = 17.29, p < 0.001, with youth in the high functioning group (n = 35; M = 52.16) rated as having significantly lower levels of problem behaviors than youth in the borderline group (n = 47; M = 57.16), whose scores were significantly lower than the youth in the low functioning group (n = 146; M = 63.02). No other significant main effects or
Table 3 Means with standard deviations and n's for CGAS at each time point. CGAS-3 time 2
High
30
Hierarchical growth modeling methods were first used to create a series of models of individual change in general functioning as measured by the CGAS. First, the shape of the trajectory was examined by estimating both linear and quadratic growth parameters. These trajectory models estimate a separate intercept and a separate growth slope for each participant in the sample. The variability across participants is also determined. Predictors are added to the models to attempt to account for differences in both intercept and in slope when enough variance is present. For these analyses the initial time point at intake was coded as “0;” therefore, any variability existing in the intercepts is variability between participants at intake. Participants' CGAS scores were modeled using four time points (i.e., intake, 3 months, 6 months, and 12 months); the means for the three functional level groups at each of these time points is shown in Table 3. The linear growth coefficient (π1 = 3.13, p < 0.05) indicated that youth experienced an average increase of 3.13 across the period, with a large amount of variation (SD = 12.94) across the sample of 720 participants with at least two time points of data. In the full model with predictors, the intercept (M = 48.01, SD = 3.79) represents the average CGAS scores for a low functioning youth in residential care. Placement setting was a significant predictor of the intercept, with youth in foster care averaging slightly higher scores (β01 = 1.27) than youth in a residential setting. However, placement did not significantly impact the change over time. Functioning group was a significant
CGAS-3 time 1
50 40
2.3. General functioning
Group
60
59.56 (12.29) 69.47 (11.00) 80.14 (11.00)
Note. CGAS = Children's Global Assessment Scale.
22
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Table 4 Means with standard deviations and n's for behaviors at each time point. Group
Youth self-rating Time 1
Internalizing behavior Foster Low Borderline High Residential Low Borderline High
Time 3
Time 1
Time 3
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
51 27 41
54.90 (13.43) 52.52 (13.39) 54.46 (9.36)
7 6 6
50.29 (10.59) 48.17 (13.11) 48.67 (10.69)
60 40 47
60.10 (10.84) 55.33 (9.74) 52.79 (10.36)
9 8 7
62.78 (11.00) 60.38 (10.57) 50.57 (7.16)
320 160 119
57.91 (11.63) 54.56 (11.55) 52.70 (10.72)
109 41 28
57.42 (12.89) 51.88 (13.05) 51.89 (11.93)
392 154 135
59.47 (9.58) 55.49 (8.50) 50.90 (8.46)
139 40 31
59.44 (9.52) 53.23 (8.14) 54.16 (8.66)
59.04 (12.93) 53.30 (11.59) 52.78 (10.19)
7 6 6
55.29 (5.88) 54.00 (10.92) 48.00 (12.63)
60 40 47
64.85 (11.35) 60.40 (7.46) 55.00 (9.63)
9 8 7
65.89 (12.65) 62.88 (11.22) 52.57 (9.61)
59.53 (11.77) 56.73 (11.40) 52.39 (10.16)
109 41 28
60.13 (10.72) 62.66 (10.02) 57.07 (8.80)
392 154 135
65.20 (8.65) 59.58 (8.38) 55.33 (7.49)
139 40 31
65.58 (6.24) 63.10 (7.86) 57.90 (9.50)
51 27 41
57.10 (13.65) 51.96 (13.66) 54.00 (9.06)
7 6 6
53.71 (7.52) 50.67 (12.99) 48.33 (9.33)
60 40 47
62.80 (11.79) 57.40 (8.65) 53.23 (10.27)
9 8 7
65.22 (13.61) 62.00 (13.44) 50.14 (10.06)
320 160 119
59.30 (11.64) 56.04 (11.36) 52.82 (10.42)
109 41 28
59.97 (12.45) 57.76 (11.30) 54.57 (10.65)
392 154 135
63.05 (8.45) 56.68 (7.24) 52.39 (6.86)
139 40 31
63.09 (7.67) 58.48 (6.53) 55.52 (8.53)
Externalizing behavior Foster Low 51 Borderline 27 High 41 Residential Low 320 Borderline 160 High 119 Total problems Foster Low Borderline High Residential Low Borderline High
Clinician rating
interactions were detected. An examination of levels of internalizing behavior problems based on youth self-reports at six months showed no significant differences. Only a marginally significant main effect of setting was found, F(1, 181) = 2.97, p = 0.09, with youth in foster care (n = 19, M = 49.76) reporting significantly lower levels of internalizing behavior than did youth in residential group care (n = 168, M = 54.11). Similarly, there was a significant main effect of placement when externalizing behavior subscales were examined, F(1, 181) = 6.34, p < 0.01, with youth in foster care (n = 19, M = 52.31) again scoring significantly lower on levels of externalizing behavior problems than the congregate care group (n = 168, M = 58.04). In regard to youth self-reports of total problems, there was a significant main effect of placement at six months, F(1, 181) = 5.16, p < 0.05, with youth in foster care (n = 19, M = 51.03) scoring significantly lower on the Total Problems Scale than youth in residential care (n = 168, M = 56.56). No other significant main effects or interactions were detected.
Table 5 Means with standard deviations and n's for anxiety at each time point.
2.5. Anxiety
coefficient, neither placement nor functioning level predicted this change. Placement was borderline significant (β12 = −2.02, p = 0.06), suggesting that foster youth experience a decline over time compared to a slight upward trend of the youth in residential care.
Group
Foster Low
Anxiety time 2
Anxiety time 3
Anxiety time 4
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
52
51.29 (12.72) 51.83 (11.92) 49.09 (9.56)
8
43.75 (10.22) 51.33 (11.55) 47.36 (10.14)
7
58.86 (17.08) 52.00 (14.73) 47.00 (11.22)
0
NA
2
37.00 (2.83) 41.00 (NA)
54.02 (11.57) 51.14 (10.01) 49.48 (10.45)
91
58.23 (10.22) 50.81 (12.01) 50.33 (11.01)
122
54.44 (11.35) 54.04 (12.99) 48.49 (12.38)
22
Borderline
30
High
46
Residential Low
Given the availability of sufficient data at four time points, an analysis of the trajectories of youth anxiety levels using the RCMAS was conducted using the same linear growth modeling method used with the CGAS and described earlier. (Mean scores for the three functional level groups at each time point are shown in Table 5.) For this outcome, there was a small, but significant linear growth coefficient (π1 = 1.88, p < 0.05). This change coefficient varied somewhat (SD = 3.16) across the sample of 703 participants. Both functioning and placement predicted differences in the intercept. At intake, high functioning youth averaged lower anxiety (β03 = −4.40) than low functioning youth. Borderline functioning youth did not differ from low functioning youth. Youth in foster care experienced slightly lower levels of anxiety (β01 = −1.90). Although there was significant variance in the change
Anxiety time 1
326
Borderline
122
High
128
6 11
21 51
5 10
47 53
1
14 14
50.82 (14.05) 52.50 (12.24) 50.21 (16.03)
3. Discussion Consistent with a growing body of literature, these data indicated that youth placed in foster care tended to have higher levels of functioning at intake than did youth placed in group care settings. However, the degree to which youth progressed in care on measures of general functioning and mental and behavioral health problems did not differ based on placement. Youth in residential group care settings progressed at the same rate as youth in community-based settings, regardless of 23
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Accordingly, it is not possible to know what a child's final “outcome” was. Analyses were further constrained by the limited information available on other variables that can impact outcomes for children and youth, including children's histories and the range and intensity of services offered, which can vary substantially across providers. For example, it may be that children's scores at “intake” to a participating agency actually reflected an “outcome” of earlier services at a nonparticipating agency. Finally, the generalizability of these results may be limited by the fact that participating agencies were located within one of two southeastern states, which may not be representative of providers nationwide.
their level of functioning at intake. Accordingly, the current study supports those prior studies demonstrating positive outcomes for children and youth in residential care and suggesting that it is an appropriate and effective placement for at least some children. Importantly, these analyses demonstrated that it was those youth with low levels of functioning who improved over time, whereas those with high levels of functioning at intake experienced a decrease in CGAS scores. This pattern was not impacted by placement; regardless of whether they were placed in foster care or a residential care setting, the scores of children with low levels of functioning at intake improved, while those of children with “normal” levels of functioning at intake demonstrated a general negative trend. While this latter finding is troubling, it should be noted that not only has this trend reversed in studies of children remaining in care for longer periods of time, but even the more problematic scores of high functioning children tended to remain in the normal range across time (Portwood et al., 2016). Moreover, some deficits are perhaps not surprising given that the likelihood of those circumstances necessitating the child's removal from his or her home continuing to present challenges is, most likely, high; these challenges could also be compounded by the disruption of the child's being placed outside the home. A relatively large percentage of youth in this study (46.8%) did not present with clinical level scores. Arguably, lower intensity services provided in community settings should be sufficient to meet the needs of these children. However, as noted by James, Roesch, and Zhang (2012, p. 11), “there might be valid nonclinical reasons, which are poorly understood and unstudied, to place a child into a residential setting.” For example, as one agency staff member explained to the researchers, several of the youth in her residential facility had expressed a preference for being place in group care rather than a foster family since this presented fewer conflicts with their parents, who felt threatened by their child's involvement with another family. While, overall, these results did not favor foster over residential group care, there was one exception to this pattern in regard to children's levels of anxiety. While not statistically significant, the observed trend in RCMAS scores suggested that while youth in foster care experienced an improvement over time, the anxiety levels of youth in residential care may increase over time. Future research should examine whether this trend persists. As noted by Bailey (1999), it is widely acknowledged that children are likely to experience at least some degree of anxiety not only upon their admission to residential care, but also as they move through the process of settling into their new environment, such that providers need to be proactive in ensuring that they do not exacerbate children's vulnerability. However, to date, there is little research investigating anxiety among children in residential care settings. Two strengths of the current research compared to the majority of previous studies are that (1) it included children in varying levels of foster care (e.g., family foster care, therapeutic foster care), and (2) it examined relatively short-term outcomes (i.e., at three and six months after placement). Both of these factors may improve the generalizability of these findings. Since the passage of the Adoptions and Safe Families Act (P.L. 105-89) in 1997, child welfare agencies have renewed their efforts to limit children's lengths of stays in foster care. Among a national sample of children and youth involved with the child welfare system who entered residential care, James, Zhang, & Landsverk (2012) found that slightly over a third (34.6%) stayed less than six months, with 23.9% staying three months or less. It follows that agencies are increasingly being expected to produce positive results over shorter periods of time, such that studies such as this one, where the focus is on outcomes at time periods consistent with the overarching goal of achieving shorter lengths of stay in care, are essential. Despite its strengths, this study was also subject to several limitations. It is not known whether children were discharged to home or to other out-of-home care settings, and in some cases, children may have continued in their current setting past the conclusion of the study.
3.1. Conclusions and directions for future research In summary, the findings from this multisite study suggest that when levels of functioning at intake are taken into consideration, there are no substantial differences in the effectiveness of residential group care compared to foster care, supporting the notion that both types of placements may contribute to an effective continuum of care for children and youth. It appears that residential group care and therapeutic foster care, in particular, are both elements of an effective continuum of care. However, given the array of services and setting that are included within both residential group care and foster care, additional research is needed to determine not only which category placement is appropriate for a particular child (i.e., when is foster care or residential group care the most effective alternative for out-of-home placement), but also when a specific form of that care can best serve the presenting needs of the child (i.e., what services are needed). As further noted, the ways in which anxiety impacts children in these settings also merits future inquiry. Conflicts of interest None. References Achenbach, T. M. (1991). Manual for the youth self-report and 1991 profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., & Edelbrock, C. (1983). Manual for the child behavior checklist and revised behavior profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington: University of Vermont, Research Center for Children, Youth, and Families. Bailey, P. (1999). From family to care: Issues for the child. Irish Journal of Applied Social Studies, 2. (Retrieved October 29, 2017, from) http://arrow.dit.ie/ijass/vol2/iss1/7. Barth, R. P. (2002). Institutions vs. foster homes: The empirical base for a century of action. Chapel Hill, NC: UNC, School of Social Work, Jordan Institute for Families. Barth, R. P. (2005). Foster home care is more cost-effective than shelter care: Serious questions continue to be raised about the utility of group care in child welfare services. Child Abuse and Neglect, 29, 623–625. Green, B., Shirk, S., Hanze, D., & Wanstrath, J. (1994). The Children's global assessment scale in clinical practice: An empirical evaluation. Journal of the American Academy of Child & Adolescent Psychiatry, 33, 1158–1164. Holstead, J., Dalton, J., Horne, A., & Lamond, D. (2010). Modernizing residential treatment centers for children and youth - an informed approach to improve long-term outcomes: The Damar pilot. Child Welfare, 89(2), 115–130. James, S. (2011). What works in group care? A structured review of treatment models for group homes and residential care. Children and Youth Services Review, 33, 308–321. James, S., Roesch, S., & Zhang, J. J. (2012). Characteristics and behavioral outcomes for youth in group care and family-based care: A propensity score matching approach using national data. Journal of Emotional and Behavioral Disorders, 20, 144–156. James, S. S., Zhang, J. J., & Landsverk, J. (2012). Residential care for youth in the child welfare system: Stop-gap option or not? Residential Treatment Children & Youth, 29, 1–15. KIDS COUNT Data Center (n.d.). Children in Foster Care by Race and Hispanic Origin. (n. d.). (Retrieved October 27, 2017, from http://datacenter.kidscount.org/). Knorth, E. J., Harder, A. T., Zandberg, T., & Kendrick, A. J. (2008). Under one roof: A review and selective meta-analysis on the outcomes of residential child and youth care. Children and Youth Services Review, 30, 123–140. Lee, B. R., Bright, C. L., Svoboda, D. V., Fakunmoju, S., & Barth, R. P. (2010). Outcomes of group care for youth: A review of comparative studies. Research on Social Work Practice, 21(2), 177–189. Lee, B. R., & Thompson, R. (2008). Comparing outcomes for youth in treatment foster
24
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Treatment for Children & Youth, 26, 71–91. Portwood, S. G., Boyd, A. S., & Murdock, T. B. (2016). Implementation of a program of outcomes research in residential care settings: Outcomes for children in care. Child and Youth Care Forum, 45, 393–407. Preyde, M., Frensch, K., Cameron, G., Hazineh, L., & Riosa, P. (2011). Mental health outcomes of children and youth accessing residential programs or a home-based alternative. Social Work in Mental Health, 9(1), 1–21. http://dx.doi.org/10.1080/ 15332985.2010.494557. Reynolds, C. R., & Richmond, B. O. (1985). Revised children's manifest anxiety scales (RCMAS). Los Angeles: Western Psychological Services. Shaffer, D., Gould, M. S., Brasco, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children's global assessment scales (C-GAS). Archives of General Psychiatry, 40, 1228–1231.
care and family-style group care. Children and Youth Services Review, 30, 746–757. Leloux-Opmeer, H., Kuiper, C., Swaab, H., & Scholte, E. (2016). Characteristics of children in foster care, family-style group care, and residential care: A scoping review. Journal of Child and Family Studies, 25, 2357–2371. Leonard, B. J., Jang, Y., Savik, K., & Plumbo, M. A. (2005). Adolescents with type 1 diabetes: Family functioning and metabolic control. Journal of Family Nursing, 11(2), 102–121. Lyons, J. S., Terry, P., Martinovich, Z., Peterson, J., & Bouska, B. (2001). Outcome trajectories for adolescents in residential treatment: A statewide evaluation. Journal of Child and Family Studies, 10(3), 333–345. (1062-1024/01/0900-0333$19.q50/0) https://doi.org/10.1023/A:1012576826136. Lyons, J. S., Woltman, H., Martinovich, Z., & Hancock, B. (2009). An outcomes perspective of the role of residential treatment in the system of care. Residential
25
Contents lists available at ScienceDirect
Children and Youth Services Review journal homepage: www.elsevier.com/locate/childyouth
A comparison of outcomes for children and youth in foster and residential group care across agencies☆
T
⁎
Sharon G. Portwooda, , Suzanne A. Boyda, Ellissa Brooks Nelsona, Tamera B. Murdockb, Jessica Hamiltonb, Angela D. Millerc a
The University of North Carolina at Charlotte, USA The University of Missouri, – Kansas City, USA c George Mason University, USA b
A B S T R A C T Working collaboratively with two state associations and their member (nonprofit) agencies providing out-ofhome care to children and youth, University researchers conducted a multi-site project to examine whether there were any differences in individual child-level outcomes between children placed in residential group care and those placed in foster. The study employed a quasi-experimental repeated measures design, with data collected at a minimum of two intervals (at intake and 3-month follow-up) and at subsequent intervals of 6 and 12 months for children remaining in care. Samples for analyses were drawn from 1082 youth in either residential group care (n = 903) or foster care (n = 179), in one of 37 agency sites across two southeastern states, who participated in a broader evaluation project. The average ages of participating youth in residential and foster care were 13.97 (SD = 2.43) and 13.65 (SD = 2.73), respectively. Based on his or her score on the Children's Global Assessment Scale (CGAS) at intake, each participant was also assigned to the low functioning group (n = 526; 53.1%), the borderline group (n = 232; 23.4%), or the high functioning group (n = 232; 23.4%). Analyses confirmed that youth in foster care tended to have higher levels of general functioning at baseline than did youth placed in group care. However, the degree to which youth progressed in care on measures of general functioning and mental and behavioral health problems did not differ based on placement setting; youth in residential group care settings progressed at the same rate as youth in community-based settings, regardless of their level of functioning at intake. The only exception to this pattern was in regard to anxiety; there was an observable, but non-significant trend of youth in foster care reporting decreases in anxiety levels, while those in group care reported increased anxiety.
During the past several decades, policymakers and practitioners have debated how best to serve the needs of children placed in out-ofhome care, with an overall focus on the relative attributes of community-based versus residential (agency-based) care. Despite a body of literature emphasizing the need for research comparing youth outcomes across foster care and residential group care to inform these decisions, the body of empirical work in this area remains sparse (James, 2011). Absent empirical data from which courts and/or the child welfare system can determine his or her optimal placement based on the relative effectiveness of available settings in meeting the type of needs with which the child presents, current policy and practice favor placement in the least restrictive setting, such that community-based foster
care is favored over residential group care. Cost can also factor into placement decisions, with foster care being favored over more expensive group care (Barth, 2005). Nonetheless, as noted by Barth (2002), there are circumstances in which children and youth may be best served in a group setting (e.g., when they have run away from foster care, they present a threat to themselves or others, or they are moving from a more restrictive setting). Moreover, the fact that a substantial number of children continue to reside in residential programs warrants examination of how child-level outcomes compare across settings (James, Zhang, & Landsverk, 2012). Despite the objections to congregate care, some studies have documented positive outcomes for those residential programs that
☆ The authors would like to acknowledge The Duke Endowment for its financial support of this project, along with the Duke Endowment staff; the South Carolina Association of Children's Homes and Family Services; Benchmarks: An Alliance of Agencies Helping Children, Adults & Families; and the many agency staff and research team members who contributed their time and expertise to the evaluation project from which the data for this manuscript were drawn. ⁎ Corresponding author. E-mail address: [email protected] (S.G. Portwood).
https://doi.org/10.1016/j.childyouth.2017.11.027 Received 4 August 2017; Received in revised form 23 November 2017; Accepted 23 November 2017 Available online 24 November 2017 0190-7409/ © 2017 Elsevier Ltd. All rights reserved.
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
(16.7%) were in high management programs; and 257 (28.4%) were in residential treatment. Notably, youth in foster care also represented placement in various service levels, with 97 (54.1%) in therapeutic foster care, 76 (42.5%) in family foster care, and 6 (3.4%) in specialized foster care. Participants were enrolled in the study for the duration of a single placement (i.e., none of the participants moved across placement settings).
reflect high standards of care, including family involvement and support and supervision by caring adults (Barth, 2002). From their review and selective meta-analysis of research studies published between 1990 and 2005 examining residential care outcomes, Knorth, Harder, Zandberg, and Kendrick (2008) rejected the notion that residential care has primarily negative outcomes for youth, instead concluding that psychosocial functioning outcomes could be expected to improve, on average, for children and youth who had been in residential care for a period of time. In contrast, Lee, Bright, Svoboda, Fakunmoju, and Barth (2010) concluded that a majority of published studies favor family foster care or treatment foster care over residential group care. Advocates for moving away from, and in some cases even dismantling, residential group care have described the outcomes and long-term treatment gains of children and youth placed residential group care as “uncertain” (Preyde, Frensch, Cameron, Hazineh, & Riosa, 2011, p. 5) and mixed (Holstead, Dalton, Horne, & Lamond, 2010), noting that some studies evidenced only short-term gains, while other, long-range studies did not find high end-state functioning. For example, following an investigation of 285 adolescents in residential treatment, Lyons, Terry, Martinovich, Peterson, and Bouska (2001) found that although there was evidence of a reduction in risk behaviors and depression, along with improved the management of psychosis, residential treatment did not appear to improve functioning; additionally, outcomes varied substantially across providers. Efforts to compare outcomes across placement settings have not only been limited in number, but they have also been complicated by methodological challenges, including selection bias given suggestions that youth placed in residential group care exhibit more problems than do those placed in foster care (Barth, 2002). In fact, a growing body of evidence suggests that youth in residential care have more severe difficulties than do youth in foster care upon entry into out-of-home placement (Leloux-Opmeer, Kuiper, Swaab, & Scholte, 2016). After careful efforts to generate matched samples in order to control for this potential bias, Lee and Thompson (2008) found that residential group care may actually be more effective than therapeutic foster care for some youth, with participants in therapeutic foster care proving to be less likely to be favorably discharged, less likely to return home, and more likely to experience a later out-of-home placement than youth in a congregate care setting. Based on their examination of a statewide system of care over a five-year period, Lyons, Woltman, Martinovich, and Hancock (2009) concluded that residential care serves an important function in serving children and youth with high levels of need. Given the contradictory findings in the existing literature, along with the many unanswered questions that remain regarding the relative effectiveness of foster and residential group care, there is a clear need for additional research in this area and, particularly, for projects that address the methodological problems present in many earlier studies. In response to this need, the current study was designed to compare outcomes of youth in foster care to those of youth in residential group care across multiple nonprofit agencies. More specifically, this study addressed the question of how youths' general functioning, behavior problems (as reported by therapists and the youth themselves), and anxiety compared across the two settings over time.
1.2. Data collection The study employed a quasi-experimental repeated measures (pretest-posttest) design. Trained agency staff collected data at a minimum of two intervals, at intake and at a three-month interval following intake. For those children remaining in care, data were also collected at 6 and 12 months after intake. (Notably, the Achenbach Child Behavior Checklist (CBCL) -Youth Self Report is designed to be administered at six-month intervals and so was not administered at three-month follow-up.) All children enrolled in the designated out-ofhome care programs at participating agencies were eligible for participation, but the design was quasi-experimental since, rather than being randomly assigned, children were placed in either foster care or residential group care in accordance with prevailing practices in the community or organization. To supplement existing admission forms, agency staff completed the Youth Demographic Form and Discharge Data Form created by the researchers to capture information on participants' characteristics and services received. The child's therapist, case worker, or other dedicated staff member completed the CGAS and the CBCL at all specified time points, beginning with the collection of baseline data within 30 days of admission; the date on which baseline data were collected then served as the reference point for subsequent follow-up data collection dates. The same trained agency staff members also administered the youth measures at the designated intervals. (Although there may have been some reassignment of cases at agencies for various reasons [e.g., staff turnover], this information was not available to the researchers.) Youth completed the measures individually in a private location. The staff member read a list of reminders to youth prior to his or her completing the study measures to ensure standardized data collection procedures across agencies. Staff members also read the questions aloud to younger children or those who had difficulty reading. Given the variance in children's ages and reading ability, the time required to complete the study measures also varied considerably. Agencies provided de-identified data to the researchers for analysis. The researchers obtained approval from their University Institutional Review Board. 1.3. Instruments 1.3.1. Children's Global Assessment Scale (CGAS) The CGAS provides a simple means for a variety of raters across disciplines to quantify the global functioning of children ages 4 to 16 years on a scale of 1 to 100. Higher scores indicate higher levels of functioning, with scores over 70 designated as normal. The scale has demonstrated interrater and test-retest reliability, as well as discriminant and concurrent validity (Shaffer et al., 1983). The CGAS has been widely used in research and shown to be a good measure of children's functional competence in clinical settings (Green, Shirk, Hanze, & Wanstrath, 1994).
1. Methods 1.1. Participants The current study examined a subset of the 1114 children and youth enrolled in out-of-home placement programs across 37 agency sites in two states who participated in a broader evaluation project (Portwood, Boyd, & Murdock, 2016). Analyses were limited to those 1082 youth enrolled in the various levels of residential group care (n = 903; 83.5%) and foster care (n = 179; 16.5%). Among those youth in residential group care settings, 409 (45.3%) were in low management programs; 86 (9.5%) were in moderate management programs; 151
1.3.2. Child Behavior Checklist (CBCL) Along with the CBCL-Youth Self Report (described below), the CBCL is part of the Achenbach System of Empirically Based Assessment (ASEBA) (Achenbach, 1991). The teacher version of the 118-item CBCL was used for clinicians to rate problem behaviors for participating children and youth. As with the parent version, the CBCL includes three broad-band scales (i.e. Internalizing, Externalizing, and Total Problems) 20
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
and eight specific narrow-band scales (e.g. attention problems). The Internalizing Scale is comprised of items to assess anxiety, depression, social withdrawal, and somatic complaints; the Externalizing Scale assesses aggression and delinquent behavior; and the Total Problems scale provides an aggregation of the two. Clinicians responded to each item using a 3-point scale from “not true” to “very true.” Detailed scoring instructions are available for this instrument, which has extensively documented reliability and validity (Achenbach & Edelbrock, 1983).
Table 1 Demographic information for subsample of youth enrolled in residential group care and foster care. Demographic characteristics
Residential group care (n = 903)
Foster care (n = 179)
Gender
510 (56.5%) 393 (43.5%) 260 (28.9%) 1 (0.1%) 547 (60.7%) 21 (2.3%) 1 (0.1%) 62 (6.9%) 8 (0.9%) 1 (0.1%) 138 (15.3%) 765 (84.7%) 13.97 (SD = 2.43)
81 (45.3%) 98 (54.7%) 93 (52.0%) 1 (0.6%) 76 (42.4%) 4 (2.2%) 1 (0.6%) 4 (2.2%) 0 (0%) 0 (0%) 39 (21.8%) 140 (78.2%) 13.65 (SD = 2.73)
Race
1.3.3. Child Behavior Checklist - Youth Self Report (CBCL-YSR) Youth ages 11 to 18 years may complete a self-report version of the CBCL, the CBCL-YSR (Achenbach & Rescorla, 2001). In response to the first seven items, youth indicate their competencies in three areas (i.e., activities, social relations, and school) that can be scored as scales and summed for a total competence score. The remaining 112 items inform eight syndrome scales (anxious/depressed, withdrawn/depressed, somatic complaints, social problems, attention problems, rule-breaking behavior, and aggressive behavior), along with the three subscales reflected in the CBCL (i.e., Internalizing, Externalizing, and Total Problems). Respondents characterize each behavior item as not true (0) or very true or often true (1) of them during the past six months. The CBCL-YSR has well-established reliability and validity (Leonard, Jang, Savik, & Plumbo, 2005).
Age group
Male Female African American Asian Caucasian Hispanic Middle-Eastern Multi-Racial Native American Undetermined 8–10 11–17
Age (average)
were Native American, two (0.2%) were Asian, and two (0.2%) were Middle-Eastern. Race/ethnicity was undetermined for one (0.1%) participant. Among youth enrolled in foster care programs, more than half (54.7%; n = 98) were female and 45.3% (n = 81) were male. Slightly over half were African American (52.0%; n = 93), followed by 42.4% (n = 76) White, 2.2% (n = 4) Hispanic, 2.2% (n = 4) multiracial, 0.6% (n = 1) Asian, and 0.6% (n = 1) Middle Eastern. This sample had a higher representation of children identified as Black than the population of foster care children in the two states examined (35% and 40% Non-Hispanic Black), and a corresponding lower representation of White children in foster care than in those states (48% and 47% NonHispanic White) during the period of this study (KIDS COUNT, n.d.). Each of the series of analyses performed were further limited to those youth (1) for whom CGAS scores were available at intake (which was required to assign participants to a functioning level group) (n = 990) and (2) for whom there were valid data, defined as scores on the relevant outcome measure at intake and the appropriate follow-up interval. Table 2 provides an overview of the demographic characteristics of the study groups of low functioning youth (n = 526; 53.1%), borderline youth (n = 232; 23.4%), and high functioning youth (n = 232; 23.4%) as determined by their scores on the Children's Global Assessment Scale (CGAS) at baseline. Consistent with standard scoring instructions, youth with CGAS scores in the clinical range (60 or below) were placed in the low functioning group; those with scores between 61 and 70 were placed in the borderline group; and those with CGAS scores in the normal range (71 or above) were assigned to the high functioning group.
1.3.4. Revised Children's Manifest Anxiety Scale (RCMAS) The 37-item RCMAS assesses the level and nature of anxiety in children and adolescents ranging in age from 6 to 19 years (Reynolds & Richmond, 1985). Respondents mark “yes” or “no” in response to each scale item, with “yes” indicating that the item accurately describes his or her feelings or actions. The Total Anxiety Score includes 28 items that are summed for a total score. The Lie Scale includes nine items to test for social desirability. Three subscale scores can also be calculated: physiological symptoms, worry, and concentration problems. Overall Cronbach's alphas for Total Anxiety range from 0.79 to 0.83 and testretest reliability ranges from 0.85 to 0.95. Reliability estimates for internal consistency are moderate, and the RCMAS has demonstrated validity. 1.4. Plan for analysis Data for the CGAS and RCMAS were collected at four time points. Because of the inherent difficulty in conducting analyses across time for children in residential and foster care, a hierarchical linear modeling approach was used with these measures in order to preserve as large a sample as possible. This approach is preferable over a repeated measures analyses which requires complete data. However, this approach was not possible for the CBCL measures collected by both self-report and clinician report. These data were collected at two time points (intake and 6 months) and were analyzed using repeated measures MANOVA. As described previously, this measure is appropriately used at six month intervals. In both of these analytic approaches, variables of functioning group level and placement setting were examined as predictors of the outcomes.
Table 2 Demographic information by functioning-level group.
2. Results 2.1. Sample characteristics
Demographic characteristics
Low functioning youth (n = 526)
Borderline youth (n = 232)
High functioning youth (n = 232)
Gender
(58.6%) 218 (41.4%) 142 (27.0%) 0 324 (61.6%) 13 (2.5%) 0 42 (8.0%) 5 (1.0%) 0 102 (19.4%) 424 (80.6%) 13.70 (2.53)
122 (52.6%) 110 (47.4%) 80 (34.5%) 0 134 (57.8%) 2 (0.9%) 0 13 (5.6%) 1 (0.4%) 1 (0.4%) 25 (10.8) 207 (89.2%) 14.45 (2.41)
117 (50.4%) 115 (49.6%) 84 (36.2%) 2 (0.9%) 123 (53.0%) 9 (3.9%) 2 (0.9%) 10 (4.3%) 2 (0.9%) 0 41 (17.7%) 191 (82.3%) 13.70 (2.50)
Race
Table 1 provides a summary of the demographic characteristics for the two study groups. The average ages for children enrolled in congregate care and foster care were 13.97 years and 13.65 years, respectively. For youth enrolled in residential group care programs, 43.5% of participants (n = 393) were female and 56.5% were male (n = 510). Over half of these participants were Caucasian (60.7%; n = 547); 28.9% (n = 260) were African-American; 6.9% (n = 62) were multiracial; and 2.3% (n = 21) were Hispanic. Eight participants (0.9%)
Age Group Age (average)
21
Male Female African American Asian Caucasian Hispanic Middle-Eastern Multi-Racial Native American Undetermined 8–10 11–17
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
2.2. Sample equivalence
90
Chi-square and independent t-tests were conducted to examine differences between youth included in comparative or growth analytic measures and those not included because of lack of data or attrition. Youth with data for at least two time points were compared to those with only one time point of data and hence not included comparative or change analyses. There was a slight but statistically significant difference in age; those included in analyses (n = 675; M = 13.6, SD = 2.51) were slightly younger than those with only one data point (n = 341; M = 14.35, SD = 2.35). There were no significant differences in gender or race for those included in the analyses as compared to the full sample. Males made up 58.0% (n = 397) of the analyses sample compared to 56.5% (n = 510) of the full sample. The race of those included in the analyses were almost exactly the same as the full sample; the largest ethnic groups were 61.2% (n = 416) Caucasian and 29.1% (n = 186) African-American. Youth cases with DSS involvement was 59% in both the analysis (n = 400) and full sample (n = 610).
80
C-GAS Score
70
Foster Low
CGAS-6 time 3
CGAS-6 time 4
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
67
49.07 (7.63) 65.55 (2.93) 80.96 (5.13)
33
56.64 (14.11) 69.42 (14.38) 76.69 (10.81)
13
59.62 (10.41) 65.45 (14.07) 75.88 (11.66)
5
63.40 (20.98) 70.33 (11.54) 77.50 (13.66)
Borderline
42
High
55
Residential Low
459
Borderline
190
High
177
47.30 (8.57) 65.86 (3.29) 80.29 (5.78)
19 26
292 121 119
54.85 (11.05) 65.68 (10.26) 75.48 (11.63)
11 17
190 78 89
57.52 (10.27) 67.99 (12.11) 72.94 (13.21)
6 8
48 30 36
Borderline Low
Time 1
Time 2
Time 3
Time 4
Fig. 1. Comparison of C-GAS scores over time by functioning group.
predictor of both the intercept and the change across time. The expected differences at intake were confirmed with the significant differences of the intercepts, both borderline (β02 = 17.49) and high (β03 = 31.46) functioning groups were significantly higher than the low functioning group represented by the intercept. The more interesting finding is the significant variation in slopes between the groups. Both the borderline (β12 = − 4.61) and the high (β13 = −8.16) differed significantly from the low functioning group. As can be seen in Fig. 1, the low functioning group experienced an increase (π1 = 5.37) over time in CGAS scores, whereas the borderline group experiences a more slight increase (+ 0.76) and the net change in the high function group average a decline (− 2.79) in scores. 2.4. Internalizing and externalizing problems The next set of analyses focused on children's levels of behavioral and mental health problems as measured by the CBCL. (Table 4 shows the functional level group scores on both of the CBCL measures at each time point.) These analyses were conducted using mixed model MANOVA with two time points. The first set of analyses focused on therapists' ratings. At six-month follow-up (which was the first time point at which CBCL follow-up data were collected), there was a significant effect of time on the Internalizing Scale, λ = 0.98, F(1, 222) = 4.55, p < 0.001 with scores increasing over time (M1 = 56.08, M2 = 57.60); there was also a significant effect of functioning group. Youth in the high functioning (n = 35; M = 52.03) and borderline groups (n = 47; M = 54.69) demonstrated significantly lower levels of problem behavior than did youth in the low functioning group (n = 146; M = 59.91). However, there were no other main effects or interactions, indicating no differences between the foster and residential care groups. For the Externalizing Scale, at six months, there was again a significant main effect of time, λ = 0.95, F(1, 222) = 11.69, p < 0.001, with scores increasing over time (M1 = 61.85, M2 = 63.85). There was again a significant main effect of functioninglevel group, F(2, 222) = 7.74, p < 0.001. High functioning youth (n = 35; M = 54.05) demonstrated significantly lower levels of problem behaviors than did youth in the borderline group (n = 47; M = 60.46), who, in turn, had significantly lower levels of problem behaviors than youth in the low functioning group (n = 146; M = 64.99). Finally, therapist' ratings of youth total problems indicated a main effect of time, λ = 0.95, F(1, 222) = 11.04, p < 0.05, with ratings increasing over time (M1 = 59.13, M2 = 61.09). There was also a significant effect of functioning level group, F(2222) = 17.29, p < 0.001, with youth in the high functioning group (n = 35; M = 52.16) rated as having significantly lower levels of problem behaviors than youth in the borderline group (n = 47; M = 57.16), whose scores were significantly lower than the youth in the low functioning group (n = 146; M = 63.02). No other significant main effects or
Table 3 Means with standard deviations and n's for CGAS at each time point. CGAS-3 time 2
High
30
Hierarchical growth modeling methods were first used to create a series of models of individual change in general functioning as measured by the CGAS. First, the shape of the trajectory was examined by estimating both linear and quadratic growth parameters. These trajectory models estimate a separate intercept and a separate growth slope for each participant in the sample. The variability across participants is also determined. Predictors are added to the models to attempt to account for differences in both intercept and in slope when enough variance is present. For these analyses the initial time point at intake was coded as “0;” therefore, any variability existing in the intercepts is variability between participants at intake. Participants' CGAS scores were modeled using four time points (i.e., intake, 3 months, 6 months, and 12 months); the means for the three functional level groups at each of these time points is shown in Table 3. The linear growth coefficient (π1 = 3.13, p < 0.05) indicated that youth experienced an average increase of 3.13 across the period, with a large amount of variation (SD = 12.94) across the sample of 720 participants with at least two time points of data. In the full model with predictors, the intercept (M = 48.01, SD = 3.79) represents the average CGAS scores for a low functioning youth in residential care. Placement setting was a significant predictor of the intercept, with youth in foster care averaging slightly higher scores (β01 = 1.27) than youth in a residential setting. However, placement did not significantly impact the change over time. Functioning group was a significant
CGAS-3 time 1
50 40
2.3. General functioning
Group
60
59.56 (12.29) 69.47 (11.00) 80.14 (11.00)
Note. CGAS = Children's Global Assessment Scale.
22
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Table 4 Means with standard deviations and n's for behaviors at each time point. Group
Youth self-rating Time 1
Internalizing behavior Foster Low Borderline High Residential Low Borderline High
Time 3
Time 1
Time 3
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
51 27 41
54.90 (13.43) 52.52 (13.39) 54.46 (9.36)
7 6 6
50.29 (10.59) 48.17 (13.11) 48.67 (10.69)
60 40 47
60.10 (10.84) 55.33 (9.74) 52.79 (10.36)
9 8 7
62.78 (11.00) 60.38 (10.57) 50.57 (7.16)
320 160 119
57.91 (11.63) 54.56 (11.55) 52.70 (10.72)
109 41 28
57.42 (12.89) 51.88 (13.05) 51.89 (11.93)
392 154 135
59.47 (9.58) 55.49 (8.50) 50.90 (8.46)
139 40 31
59.44 (9.52) 53.23 (8.14) 54.16 (8.66)
59.04 (12.93) 53.30 (11.59) 52.78 (10.19)
7 6 6
55.29 (5.88) 54.00 (10.92) 48.00 (12.63)
60 40 47
64.85 (11.35) 60.40 (7.46) 55.00 (9.63)
9 8 7
65.89 (12.65) 62.88 (11.22) 52.57 (9.61)
59.53 (11.77) 56.73 (11.40) 52.39 (10.16)
109 41 28
60.13 (10.72) 62.66 (10.02) 57.07 (8.80)
392 154 135
65.20 (8.65) 59.58 (8.38) 55.33 (7.49)
139 40 31
65.58 (6.24) 63.10 (7.86) 57.90 (9.50)
51 27 41
57.10 (13.65) 51.96 (13.66) 54.00 (9.06)
7 6 6
53.71 (7.52) 50.67 (12.99) 48.33 (9.33)
60 40 47
62.80 (11.79) 57.40 (8.65) 53.23 (10.27)
9 8 7
65.22 (13.61) 62.00 (13.44) 50.14 (10.06)
320 160 119
59.30 (11.64) 56.04 (11.36) 52.82 (10.42)
109 41 28
59.97 (12.45) 57.76 (11.30) 54.57 (10.65)
392 154 135
63.05 (8.45) 56.68 (7.24) 52.39 (6.86)
139 40 31
63.09 (7.67) 58.48 (6.53) 55.52 (8.53)
Externalizing behavior Foster Low 51 Borderline 27 High 41 Residential Low 320 Borderline 160 High 119 Total problems Foster Low Borderline High Residential Low Borderline High
Clinician rating
interactions were detected. An examination of levels of internalizing behavior problems based on youth self-reports at six months showed no significant differences. Only a marginally significant main effect of setting was found, F(1, 181) = 2.97, p = 0.09, with youth in foster care (n = 19, M = 49.76) reporting significantly lower levels of internalizing behavior than did youth in residential group care (n = 168, M = 54.11). Similarly, there was a significant main effect of placement when externalizing behavior subscales were examined, F(1, 181) = 6.34, p < 0.01, with youth in foster care (n = 19, M = 52.31) again scoring significantly lower on levels of externalizing behavior problems than the congregate care group (n = 168, M = 58.04). In regard to youth self-reports of total problems, there was a significant main effect of placement at six months, F(1, 181) = 5.16, p < 0.05, with youth in foster care (n = 19, M = 51.03) scoring significantly lower on the Total Problems Scale than youth in residential care (n = 168, M = 56.56). No other significant main effects or interactions were detected.
Table 5 Means with standard deviations and n's for anxiety at each time point.
2.5. Anxiety
coefficient, neither placement nor functioning level predicted this change. Placement was borderline significant (β12 = −2.02, p = 0.06), suggesting that foster youth experience a decline over time compared to a slight upward trend of the youth in residential care.
Group
Foster Low
Anxiety time 2
Anxiety time 3
Anxiety time 4
n
M (SD)
n
M (SD)
n
M (SD)
n
M (SD)
52
51.29 (12.72) 51.83 (11.92) 49.09 (9.56)
8
43.75 (10.22) 51.33 (11.55) 47.36 (10.14)
7
58.86 (17.08) 52.00 (14.73) 47.00 (11.22)
0
NA
2
37.00 (2.83) 41.00 (NA)
54.02 (11.57) 51.14 (10.01) 49.48 (10.45)
91
58.23 (10.22) 50.81 (12.01) 50.33 (11.01)
122
54.44 (11.35) 54.04 (12.99) 48.49 (12.38)
22
Borderline
30
High
46
Residential Low
Given the availability of sufficient data at four time points, an analysis of the trajectories of youth anxiety levels using the RCMAS was conducted using the same linear growth modeling method used with the CGAS and described earlier. (Mean scores for the three functional level groups at each time point are shown in Table 5.) For this outcome, there was a small, but significant linear growth coefficient (π1 = 1.88, p < 0.05). This change coefficient varied somewhat (SD = 3.16) across the sample of 703 participants. Both functioning and placement predicted differences in the intercept. At intake, high functioning youth averaged lower anxiety (β03 = −4.40) than low functioning youth. Borderline functioning youth did not differ from low functioning youth. Youth in foster care experienced slightly lower levels of anxiety (β01 = −1.90). Although there was significant variance in the change
Anxiety time 1
326
Borderline
122
High
128
6 11
21 51
5 10
47 53
1
14 14
50.82 (14.05) 52.50 (12.24) 50.21 (16.03)
3. Discussion Consistent with a growing body of literature, these data indicated that youth placed in foster care tended to have higher levels of functioning at intake than did youth placed in group care settings. However, the degree to which youth progressed in care on measures of general functioning and mental and behavioral health problems did not differ based on placement. Youth in residential group care settings progressed at the same rate as youth in community-based settings, regardless of 23
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Accordingly, it is not possible to know what a child's final “outcome” was. Analyses were further constrained by the limited information available on other variables that can impact outcomes for children and youth, including children's histories and the range and intensity of services offered, which can vary substantially across providers. For example, it may be that children's scores at “intake” to a participating agency actually reflected an “outcome” of earlier services at a nonparticipating agency. Finally, the generalizability of these results may be limited by the fact that participating agencies were located within one of two southeastern states, which may not be representative of providers nationwide.
their level of functioning at intake. Accordingly, the current study supports those prior studies demonstrating positive outcomes for children and youth in residential care and suggesting that it is an appropriate and effective placement for at least some children. Importantly, these analyses demonstrated that it was those youth with low levels of functioning who improved over time, whereas those with high levels of functioning at intake experienced a decrease in CGAS scores. This pattern was not impacted by placement; regardless of whether they were placed in foster care or a residential care setting, the scores of children with low levels of functioning at intake improved, while those of children with “normal” levels of functioning at intake demonstrated a general negative trend. While this latter finding is troubling, it should be noted that not only has this trend reversed in studies of children remaining in care for longer periods of time, but even the more problematic scores of high functioning children tended to remain in the normal range across time (Portwood et al., 2016). Moreover, some deficits are perhaps not surprising given that the likelihood of those circumstances necessitating the child's removal from his or her home continuing to present challenges is, most likely, high; these challenges could also be compounded by the disruption of the child's being placed outside the home. A relatively large percentage of youth in this study (46.8%) did not present with clinical level scores. Arguably, lower intensity services provided in community settings should be sufficient to meet the needs of these children. However, as noted by James, Roesch, and Zhang (2012, p. 11), “there might be valid nonclinical reasons, which are poorly understood and unstudied, to place a child into a residential setting.” For example, as one agency staff member explained to the researchers, several of the youth in her residential facility had expressed a preference for being place in group care rather than a foster family since this presented fewer conflicts with their parents, who felt threatened by their child's involvement with another family. While, overall, these results did not favor foster over residential group care, there was one exception to this pattern in regard to children's levels of anxiety. While not statistically significant, the observed trend in RCMAS scores suggested that while youth in foster care experienced an improvement over time, the anxiety levels of youth in residential care may increase over time. Future research should examine whether this trend persists. As noted by Bailey (1999), it is widely acknowledged that children are likely to experience at least some degree of anxiety not only upon their admission to residential care, but also as they move through the process of settling into their new environment, such that providers need to be proactive in ensuring that they do not exacerbate children's vulnerability. However, to date, there is little research investigating anxiety among children in residential care settings. Two strengths of the current research compared to the majority of previous studies are that (1) it included children in varying levels of foster care (e.g., family foster care, therapeutic foster care), and (2) it examined relatively short-term outcomes (i.e., at three and six months after placement). Both of these factors may improve the generalizability of these findings. Since the passage of the Adoptions and Safe Families Act (P.L. 105-89) in 1997, child welfare agencies have renewed their efforts to limit children's lengths of stays in foster care. Among a national sample of children and youth involved with the child welfare system who entered residential care, James, Zhang, & Landsverk (2012) found that slightly over a third (34.6%) stayed less than six months, with 23.9% staying three months or less. It follows that agencies are increasingly being expected to produce positive results over shorter periods of time, such that studies such as this one, where the focus is on outcomes at time periods consistent with the overarching goal of achieving shorter lengths of stay in care, are essential. Despite its strengths, this study was also subject to several limitations. It is not known whether children were discharged to home or to other out-of-home care settings, and in some cases, children may have continued in their current setting past the conclusion of the study.
3.1. Conclusions and directions for future research In summary, the findings from this multisite study suggest that when levels of functioning at intake are taken into consideration, there are no substantial differences in the effectiveness of residential group care compared to foster care, supporting the notion that both types of placements may contribute to an effective continuum of care for children and youth. It appears that residential group care and therapeutic foster care, in particular, are both elements of an effective continuum of care. However, given the array of services and setting that are included within both residential group care and foster care, additional research is needed to determine not only which category placement is appropriate for a particular child (i.e., when is foster care or residential group care the most effective alternative for out-of-home placement), but also when a specific form of that care can best serve the presenting needs of the child (i.e., what services are needed). As further noted, the ways in which anxiety impacts children in these settings also merits future inquiry. Conflicts of interest None. References Achenbach, T. M. (1991). Manual for the youth self-report and 1991 profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., & Edelbrock, C. (1983). Manual for the child behavior checklist and revised behavior profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington: University of Vermont, Research Center for Children, Youth, and Families. Bailey, P. (1999). From family to care: Issues for the child. Irish Journal of Applied Social Studies, 2. (Retrieved October 29, 2017, from) http://arrow.dit.ie/ijass/vol2/iss1/7. Barth, R. P. (2002). Institutions vs. foster homes: The empirical base for a century of action. Chapel Hill, NC: UNC, School of Social Work, Jordan Institute for Families. Barth, R. P. (2005). Foster home care is more cost-effective than shelter care: Serious questions continue to be raised about the utility of group care in child welfare services. Child Abuse and Neglect, 29, 623–625. Green, B., Shirk, S., Hanze, D., & Wanstrath, J. (1994). The Children's global assessment scale in clinical practice: An empirical evaluation. Journal of the American Academy of Child & Adolescent Psychiatry, 33, 1158–1164. Holstead, J., Dalton, J., Horne, A., & Lamond, D. (2010). Modernizing residential treatment centers for children and youth - an informed approach to improve long-term outcomes: The Damar pilot. Child Welfare, 89(2), 115–130. James, S. (2011). What works in group care? A structured review of treatment models for group homes and residential care. Children and Youth Services Review, 33, 308–321. James, S., Roesch, S., & Zhang, J. J. (2012). Characteristics and behavioral outcomes for youth in group care and family-based care: A propensity score matching approach using national data. Journal of Emotional and Behavioral Disorders, 20, 144–156. James, S. S., Zhang, J. J., & Landsverk, J. (2012). Residential care for youth in the child welfare system: Stop-gap option or not? Residential Treatment Children & Youth, 29, 1–15. KIDS COUNT Data Center (n.d.). Children in Foster Care by Race and Hispanic Origin. (n. d.). (Retrieved October 27, 2017, from http://datacenter.kidscount.org/). Knorth, E. J., Harder, A. T., Zandberg, T., & Kendrick, A. J. (2008). Under one roof: A review and selective meta-analysis on the outcomes of residential child and youth care. Children and Youth Services Review, 30, 123–140. Lee, B. R., Bright, C. L., Svoboda, D. V., Fakunmoju, S., & Barth, R. P. (2010). Outcomes of group care for youth: A review of comparative studies. Research on Social Work Practice, 21(2), 177–189. Lee, B. R., & Thompson, R. (2008). Comparing outcomes for youth in treatment foster
24
Children and Youth Services Review 85 (2018) 19–25
S.G. Portwood et al.
Treatment for Children & Youth, 26, 71–91. Portwood, S. G., Boyd, A. S., & Murdock, T. B. (2016). Implementation of a program of outcomes research in residential care settings: Outcomes for children in care. Child and Youth Care Forum, 45, 393–407. Preyde, M., Frensch, K., Cameron, G., Hazineh, L., & Riosa, P. (2011). Mental health outcomes of children and youth accessing residential programs or a home-based alternative. Social Work in Mental Health, 9(1), 1–21. http://dx.doi.org/10.1080/ 15332985.2010.494557. Reynolds, C. R., & Richmond, B. O. (1985). Revised children's manifest anxiety scales (RCMAS). Los Angeles: Western Psychological Services. Shaffer, D., Gould, M. S., Brasco, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children's global assessment scales (C-GAS). Archives of General Psychiatry, 40, 1228–1231.
care and family-style group care. Children and Youth Services Review, 30, 746–757. Leloux-Opmeer, H., Kuiper, C., Swaab, H., & Scholte, E. (2016). Characteristics of children in foster care, family-style group care, and residential care: A scoping review. Journal of Child and Family Studies, 25, 2357–2371. Leonard, B. J., Jang, Y., Savik, K., & Plumbo, M. A. (2005). Adolescents with type 1 diabetes: Family functioning and metabolic control. Journal of Family Nursing, 11(2), 102–121. Lyons, J. S., Terry, P., Martinovich, Z., Peterson, J., & Bouska, B. (2001). Outcome trajectories for adolescents in residential treatment: A statewide evaluation. Journal of Child and Family Studies, 10(3), 333–345. (1062-1024/01/0900-0333$19.q50/0) https://doi.org/10.1023/A:1012576826136. Lyons, J. S., Woltman, H., Martinovich, Z., & Hancock, B. (2009). An outcomes perspective of the role of residential treatment in the system of care. Residential
25