- Email: [email protected]
The assessment of imminence of risk of placement: Lessons from a family preservation program
Chikimnand YouthServicesReview. Vol. 13. pp. 239-256.1991 Printed in the USA. All rights resewed.
019&7409r9153.00+.00 Copyright (D 1991 Peqamon Press plc
The Assessment of Imminence of Risk of Placement: Lessons from a Family Preservation Program Marianne Berry
University of Texas at Arlington Intensive family preservation programs are a form of family services aimed at preventing the placement into foster care of children deemed at imminent risk of removal. There is considerable debate, however, on how best to determine the risk of imminent placement. This article presents an analysis of the determination of imminent risk within one family preservation program, and finds that the conventional and accepted risk factors to be used in making a risk determination at intake are more accurate at predicting placement after leaving the program than in predicting who is judged to be at imminent risk at intake. Specific elements contributing to this discrepancy are discussed, with implications for risk assessment research and policy.
Intensive family preservation services are a form of family services aimed at preventing the placement into foster care of children deemed at imminent risk of removal. Many family preservation advocates emphasize the importance of limiting family preservation efforts only to those truly at imminent risk of placement (Kinney, Haapala, Booth & Leavitt, 1988; Nelson, 1988). Opening up family preservation programs to less risky populations dilutes the intensive nature of services and can elevate preservation rates in a misleading fashion (Pecora, Fraser, Haapala & Bartlome, 1987). If these programs can be evaluated with rigor and shown to be effective with a high-risk population, the argument for their inclusion in actual service policy will be compelling. Supportfor this research was provided by the American Association of University Women in the form of a doctoral fellowship. The author thanks Mark Claycomb and JoAnn Cook at Children’s Home Society of California and Rick Barth at University of California at Berkeley for their help with this study. Send requests for reprints to: Marianne Berry at University of Texas at Arlington, School of Social Work, P.O. Box 19129. Arlington, Texas, 76019. 239
240
Berry
Family preservation programs can be viewed as elements of triangle that represents services to all families at risk of separation by child placement (Norman, 1985). The tip of the triangle contains programs for those families in need of crisis intervention, who are at risk of imminent placement. The second and somewhat larger level contains services to those families who show signs of risk but are not in immediate danger. The largest segment, the foundation of the triangle, applies to programs for families in the general population who may have characteristics that are associated with potential for risk. Thus, family preservation programs are reserved for those families at the tip of the triangle.
Definitions of Risk There is considerable debate on how best to determine the risk of imminent placement. The definition of “risk” has ranged from “families who are poor and headed by a single parent and families of adolescents” (Landsman, 1985, p. 1) to “family members judged as having high potential for removal to another living situation” (Kinney, Madsen, Fleming & Haapala, 1977, pp. 670-671). In other uses, it is not defined at all (Ruger & Wooten, 1982). This ambiguity clouds the understanding of programs designed to reach “families at risk.” Many programs like Homebuilders agree that placement is imminent if the child’s protective service worker details a plan to place the child within 24 to 48 hours (Kinney, et al., 1989), but this places the judgment of risk in the hands of child protective service workers, who may judge imminence of placement in variety of ways. Other family preservation programs broaden imminent risk to include families with many risk factors, such as prior placement, a history of abuse, or a out-of-control adolescent (Pecora, et al., 1987; Showell, Hartley, & Allen, 1987). These programs may limit their population to child protective service cases, or they can use these risk factors to define other families as soon-to-be child protective service cases, and thus, another entre’ to a designation of imminent risk of placement. The program which is the subject of this evaluation first defined families at risk (in 1983) as those families in which any of the following are true: the referral came from the Department of Social Services, the family had more than one child, and the family was of minority ethnicity (Remy & Hanson, 1983). Since 1983, risk has been further specified as pertaining to families with any of the following characteristics: a child currently in placement, a child with a prior history of placement, a history of abuse or neglect, or if more than one of the following are true: multiproblem family, a multi-racial family, possible abuse, an absent parent, a parent with problems with substance abuse, psychiatric history, developmental disability or severe physical illness.
Assessment
of Imminence
of Risk
241
Once families are deemed at imminent risk of child placement, each child in the family is also assessed as to his or her degree of imminence of risk of placement: within 72 hours, within two months, or over two months away. Thus, some families may be deemed to be imminent risk, but may have only one child out of five at imminent (within 72 hours) risk of placement, and the other children might be at lesser risk levels. Prediction of Placement Removal of a child and placement into foster care is intended to prevent continued harm to the child or the potential for danger to the child in the home. Parental and child factors associated with child abuse and neglect abound in the research literature, and these factors are included in the many risk assessment tools being developed and in use in many states (Johnson & L’Esperance, 1984; Magma & Moses, 1986; Pecora, 1988). Such factors include social isolation, a parent’s history of being abused in childhood, social and economic stress, and substance abuse. An assessment of these factors within any one family can be a good predictive tool for risk of abuse. While there are a variety of factors which may predict whether a child is in danger of abuse, however, there are additional factors which determine whether a child is placed in foster care, some concerning the same intrafamilial characteristics and problems, and others concerning the availability of foster homes, reporting mechanisms, etc. Thus, there can be a wide disparity between the children identified as being at risk of abuse and those who actually become placement statistics. The factors which predict placement are not always included in risk assessment tools, which may focus on parent-child interaction, and disregard other environmental factors or systemic decision making factors. Many studies have shown that, despite the reluctance to place for reasons of poverty (Pelton, 1989), the income level of the family is often a key predictor of whether a child is removed, over and above the severity of abuse or neglect (Katz, Hampton, Newberger, Bowles & Snyder, 1986; Lindsey, 1991). Current risk assessment tools are an important step forward in prediction of outcomes for children, but these tools are far from perfect or “magical” (Wald & Woolverton, 1990), and are also not particularly clear about whether they are predicting the risk of continued maltreatment, the risk of poor developmental outcomes, or the risk of placement. Similarly, family preservation programs often cloud the distinction between whether they are reducing the risk of placement, reducing the risk of poor developmental outcomes, or reducing the risk of continued maltreatment.
242
Berry
Factors in Reducing the Risk of Placement and Reducing the Risk of Abuse The primary goal of intensive family preservation services is to prevent the need for out-of-home placement. This usually entails resolving immediate crises and teaching skills necessary to independently maintain family integrity. This requires some level of proficiency at problem-solving and communication skills, but therapists also provide such concrete services as housecleaning and transportation. By so doing, a worker models how to do these skills, and also demonstrates his or her commitment to the family (Kinney, et al., 1988). During the provision of such “hard” services, therapists can also observe clients’ skills and talk about other problems the family is experiencing. In this way, the roundthe-clock and across-the-board availability and participation of the therapist is truly a comprehensive family service. These services are thus meant to reduce a variety of risks, with prevention of placement being the primary goal. Of course, provision of these services should also reduce the risk of poor developmental outcomes for the child and continued abuse or neglect. A variety of intensive family preservation programs have been evaluated, and most report their placement prevention rate as the primary criterion of success. These programs report success rates ranging from 71% (Pecora, et al., 1987) to 93% (Reid, Kagan, & Schlosberg, 1988), with Homebuilders averaging 90% of families intact at the end of treatment (Kinney, et al., 1988). At the one-year follow-up point, Homebuilders reports that 88% of families are still intact (Behavioral Sciences Institute, 1987), and Oregon’s Intensive Family Services finds a 73% success rate (Showell, 1985). Most of these evaluations have been focused on imminent risk cases, but the determination of imminence is, again, unclear. Placement prevention is a crucial criterion in determining treatment effectiveness, but it is not the only measure of success. Only a few evaluations of preservation programs have addressed other effects on the reduction of maltreatment factors, such as child behavior problems or family functioning (Kinney, et al., 1988; Pecora, et al., 1987). In the evaluation by Nelson and colleagues (National Resource Center on Family Based Services, 1988) of eleven family-based placement prevention programs in six states, family preservation was associated with the following changes during services: improvements in behavior, material resources, family structure, family dynamics, emotional climate, perceptions of problems, community perceptions of the family, informal support network, and community involvement. Those families who had children removed had made significant declines in behavior, family dynamics, emotional climate, community perception of the family, and
Assessment
of Imminence of Risk
243
community involvement. Given the short-tetm nature of these preventive programs, this study does not answer the question of the longevity of families’ gains, particularly since the greatest gains were made in the more intangible and elusive skills. One evaluation of the Homebuilders program (Kinney, Haapala, Booth, & Leavitt, 1988) reports that the majority of families improved in the child’s school attendance, hyperactivity, delinquent acts, and peer problems as judged by the parent or a therapist at the end of treatment. However, all families were judged to have remained the same regarding handling of medical problems or physical handicaps, and the majority of families had remained the same regarding alcohol abuse or learning disabilities. Another evaluation of the Homebuilders program looked at school adjustment, delinquent behavior, home-related behavior, and cooperation with the agency (Pecora, et al., 1987). Except for cooperation, children in the program made significant positive improvements during service. Parents also made significant improvements in supervision of younger children, parenting of older children, attitudes toward preventing placement, and their knowledge of child care. In the Utah Family Preservation program (Pecora, et al., 1987), children made significant gains in school adjustment and behavior, and parents improved in parenting behavior, attitudes and knowledge. Purpose of the Study This study evaluated one such intensive family preservation program in Northern California, in order to identify the characteristics of children who are at deemed at risk of placement, to identify those who were actually placed, and to determine whether these two groups were identical. This evaluation included a detailed examination of different service components in the program and their match to families with differing levels of placement risk. Few other programs have examined the correlates of imminent risk of placement or program effectiveness. This study examines placement rates for children at differing levels of placement risk, and also evaluates program effectiveness, in terms of services provided and the resulting skill gains, for families at differing levels of placement risk. The In-home Family Care Program
The In-home Family Care program which is the subject of this evaluation offers intensive family preservation services within an ecological framework. The program is one of the many services provided by Children’s Home Society of California, and operates in San Francisco, Oakland, and Los Angeles. The program emphasizes strengthening the
244
Berry
family’s social and ecological system, and does not follow a conventional family therapy model. This program treats parents and their children as partners in the process of strengthening the family. More than many other home-based programs, they do whatever it takes to help the family acquire tangible resources, including finding housing and day care for the family, if necessary. Staff consists of master’s level social workers and family care workers who usually have a bachelor’s or master’s degree. Each family is assigned a social worker and a family care worker. Cases are meant to be open for about three months. San Francisco workers individually carry two to five cases at a time, while Oakland workers carry three cases at a time. A team meeting is held weekly to allow the social worker and his or her family care workers to discuss all current cases and their status. The social worker and other family care workers offer treatment strategies and are good sources of information about community resources. The In-home Family Care Program emphasizes working with families at imminent risk of removal of the child. Degree of risk is determined by the Department of Social Services if they are the referral source. If the referral is from another source, the intake worker determines the degree of risk after discussion with the referral source and based on the prior placement history of the family as well as other risk factors. Families may be deemed ineligible if there is too much danger to the worker, other help is available, no children are at risk, parents are unavailable, parents decline service, or all children are older than 14 years old. Families are not excluded due to mental retardation or substance abuse, as they are in some other programs (AuClaire & Schwartz, 1986; Landsman, 1985). Referrals to the program come from a variety of sources, including hospitals, the Department of Social Services, private social services, public health agencies, and self-referrals. Depending on the family’s presenting problems, family care workers may help parents to learn positive child rearing skills instead of punitive and abusive patterns. They may also help families to acquire adequate housing and to apply for appropriate financial aid, as well as learning how to budget the income they do have. Family care workers address any problems in the condition of the family’s building or apartment, and help parents to fix broken windows and plumbing as well as helping to convince landlords to fix the heat, which is often permanently on or permanently off. Families are taught health care and nutrition practices, and are directed to legal assistance, if needed. Families are thus assessed as to their resource deficits, and these deficits are replenished with resources and skills. In-home Family Care service is not a one-way street where workers only give resources which families are free to take. When a case is opened
Assessment
of Imminence
of Risk
245
for intensive family preservation services, the worker and the family members make service agreements involving tasks that the worker and the family members agree to perform. Agreements are written and specify the task, who will perform it, and where, when and how often it will be performed. Objectives of service are to be specified in behavioral terms. For example, one such agreement may stipulate that the mother will make an appointment for her child to go to the pediatrician for a well-baby check-up. This would be followed by a task agreement that the mother would take the child to the appointment and that the worker would provide transportation. Everyone signs the agreement when it is acceptable, and workers track progress on tasks. New tasks are typically set at every meeting of family and worker. Method Sample
For the sampling frame, the following criteria were imposed: sample selection was made from cases opened and closed between January 1, 1985 and December 31, 1987; cases had to involve at least one child at risk of placement, and cases had to have received intensive services. This closing date was imposed to allow at least six months to demonstrate family stability or instability (as evidenced by the child’s placement). This sampling frame produced a population of 369 families in San Francisco and 42 families in Oakland. Placement outcomes were obtained from information from caseworkers about their knowledge of child placement in their case loads and from Department of Social Services emergency shelter and foster care logs. Each family receives follow-up visits at one-month, six-months, and one-year following termination, and follow-up forms indicate whether any children in the home were removed in that period. Cases for which an outcome of out-of-home placement of the child was known were included in the final sample. All 367 cases with known outcomes were included in this sample, 327 in San Francisco and 40 in Oakland. Only 44 cases were excluded, due to unknown outcomes (n = 9), incorrect case numbers (n = lo), or incorrect coding of whether the case received intensive services (n = 25). Measures
Each case provided the following information: outcome information, client characteristics, and service characteristics. All of this data has been gathered since 1983 in substantial detail on a number of standardized forms, which are filled out by the family care worker in the process of
246
Berry
keeping a case record for each family. All codes on these forms are determined at the time of services by family care workers and supervisors, and none were coded post-hoc by researchers. Level of Placement Risk. Each child within a family was coded by the intake worker as to the imminence of risk of placement, either within 72 hours, within 2 months, over two months away, or no risk. Several factors entered into the decision about degree of risk, the most important being a prior placement history, or a history of adjudicated abuse or neglect. Other important considerations included a psychiatric history of the child or parent, developmental disability, a multi-problem family, and possible abuse or neglect. If a child was already in placement, that was noted instead of level of placement risk. For this analysis, imminent risk cases were those at risk of placement of any child within the next 72 hours, and other families were deemed “general risk.” The average of the risk scores for all the children in the family also provided a summary risk level score for the family. Case Outcomes. Generally, outcome information included such data as whether the child was placed in out-of-home care (removed from the home and not returned home within 72 hours; relative placements were counted as out-of-home placements) and the extent of progress while in treatment. Specifically, the major outcome measure was a dichotomous variable concerning whether the child was placed in out-of-home care either while receiving In-home Family Care services or since, up to July of 1988. Client Characteristics. Client characteristics covered such areas as demographics, child and parent presenting problems, risk of placement, previous or current child protective service involvement, prior placement, assessment of the physical environment, levels of client skills in various areas, the referring agencies, the number and ages of household members, parental ethnicity, and whether or not the family has the following resources: housing, phone, food, friends or relatives able to help, AFDC, Medi-Cal, and income. Several general indicators of family level of functioning are also assessed by the family care worker: the density and condition of the environment (clean, safe, orderly, uncrowded, and comfortable), the condition of the building and living unit, the level of childhood care (general, discipline, development, and health), and parental physical and mental capacity to be involved in treatment. The typical assessment period at opening and closing lasts for one to three hours. Family care workers are trained for a month when they are hired as to how to assess family conditions, and are retrained periodically when agency assessment decisions are modified. A detailed code book specifies how various situations and conditions are coded in each category.
Assessment
of Imminence of Risk
247
Also, weekly team meetings address assessment decisions and the coding of these for case recording purposes, so fairly high interworker reliability of coding is assumed (but was not tested) for this study. Service Characteristics. Service characteristics include information on worker time, collateral agencies utilized, and problems addressed. Service logs are maintained while the case is open and the following are recorded for each contact: the location of the contact, the service provided, other agencies participating in the service contact and elapsed time (in minutes) of service provided. Due to the immense amount of data in these service logs, a subsample of 116 cases were selected for inclusion of information (a full sample of data from the service logs was too large and unwieldy for computer analysis). These cases were chosen from the latter half of the study period (July 1, 1986 to December 31, 1987), and included those which were both opened and closed between those dates, and for which there were complete service log records. These 116 cases therefore have information on specific services provided, collateral agencies used, and the actual length of service (in minutes). These 116 cases were compared to the other 25 1 cases which did not have service log data to test for representativeness of the data. Two significant differences were found between the service log sample and the remaining cases: the cases with service log data had significantly older mothers (32 years old vs. 30, on average; r = 2.11, p < .05) and the average age of the children in the home was older (5.2 years vs. 4.4 years; t = 2.03, p < .OS). There was no significant difference in the placement rate between the two groups, however, with a placement rate of 17.5% among the non-service log sample and a rate of 12.9% among the service log sample. Gain Scares. As mentioned, families are rated at initial assessment and at each periodic assessment on a variety of environmental and skill areas. Each environmental area or parenting skill was composed of a number of items for which the family received a point or two. The sum of these points provided a score for that skill at that time. Gain scores were computed for case opening to closing, as well as for gains from case closing to the one-month, six-month, and one-year follow-up points. Results
A total of 52 families, or 14%, had children removed during or subsequent to receiving In-home Family Care services. This rate is comparable to those reported for the Homebuilders program (Kinney, et al., 1988) and other family preservation programs nationwide (Landsman, 1985; Pecora, et al., 1987).
248
Berry
Predicting the Designation of Imminent Risk Only 49 of the 367 families served (13%) were judged to be at imminent risk of child placement. There were differences between the two agencies in the latitude they had in serving families: about one-third of the Oakland cases were imminent risk, while only 11% of the San Francisco cases were imminent risk. This is largely a result of the funding differences between the two counties; the Oakland office was constrained by county funding to serve imminent risk (or “reasonable efforts”) cases, while the San Francisco office had more flexible funding definitions of their service population. Imminent risk families were similar to general risk families, in that the majority had two or three children, almost three-quarters were headed by a single mother, the average age of the mothers was 3 1, an even number of children were male and female, and the average age of the children was 4.8. Imminent risk families were slightly, but not significantly, more likely to be African-American. The majority of both imminent risk and general risk families had housing, food in the home, and a phone, and about three-quarters in each group received AFDC and Medi-Cal. Imminent risk families were slightly less likely than general risk families to report that they had friends or relatives available to help them. The imminent risk designation appears to associated with standard risk factors. Imminent risk families were more likely to receive service primarily for substance abuse, parental handicap, or a request for child placement. Cases of a more general risk level were significantly more likely to receive service primarily for child emotional disturbance, child handicap, child neglect, or economic/environmental conditions. There were no significant differences between imminent risk families and nonimminent risk families in the number of children in the home, ages of children in the home, mother’s age, father’s age, ethnicity, type of dwelling, or family income. The condition of the household at intake (crowded, disordered, dangerous, dirty, and/or uncomfortable) was significantly more of a serious problem for imminent risk families, as were child care practices. Imminent risk parents were significantly less likely to be physically capable of participating in treatment. A discriminant analysis was performed to determine whether the key predictors identified by the program as imminent risk factors were used by the intake worker to define a family or a child to be at imminent risk of placement. The variables in the model included family characteristics (any prior removals, number of children, income, mother’s age, and if any relatives were nearby), parents’ presenting problems (abuse, neglect, developmental disability, and/or substance abuse), and parents’ skills at intake (living conditions, discipline skills, and health care of children).
Assessment
of Imminence of Risk
249
The results of this analysis are presented in Table 1. The key correlates of a determination of imminent risk of placement were prior removals, a low number of children in the family, parents’ good health skills and poor discipline skills at intake and poor living conditions. The presenting problem of abuse, neglect, developmental disabled parent, or substance abuse were negligible in the designation of risk, as were the mother’s age, the presence of relatives, or the income level of the family. This model was fairly poor at correctly classifying those children defined as imminent risk, with only 64% of cases correctly classified based on these factors. Therefore, intake workers were judging imminent risk on factors besides those delineated by the program, or at random. Predicting the Actual Outcome of Placement Despite the differences in presenting problem and parenting capacity between imminent risk and general risk families, the placement rate for the imminent risk families was 13%, no different than that for the rest of the sample. Regardless of the increased risk factors or the crisis nature of the presenting problem, these families were just as successful at remaining together. A more detailed examination of the services provided and skills gained illuminates the services received by imminent risk families. Imminent risk cases received an average of 96 hours of service, or about 30 more hours, on average, than non-imminent risk cases. Of these 96 hours or so, an average of 40 hours were spent in the home, significantly more time than that for non-imminent cases (23 hours on average). Overall, the percentage of service time spent in the home versus that in the agency did not differ between imminent and general risk cases, but the intensity of service (average amount of time spent per day) was significantly different (4.5 hours vs. 1 hour per day; see Table 2). Imminent risk families received somewhat more time in case planning, supplemental parenting, crisis intervention, respite care, financial skills, and home health care, while general risk cases received more parent education, teaching of family care, and referrals to other services (see Table 2). All t-tests in this study were examined for unequal sample variances, due to fluctuating sample sizes, and separate variance estimates are reported when appropriate.
Berry
250
Discriminant
Table 1 Analyses of Imminent Risk and Actual Placement Outcome
Discriminant Function Coefficient Family Characteristic Any prior removals Number of children Relatives nearby Mother’s age Income
Imminent Risks 66 -.41 -.04 -.03 .03
Placementh -.07 .39 .53 -.23 -.37
Presenting Problem Developmental disability Neglect Substance abuse Abuse
.ll -.08 -.06 -.05
.24 .45 -.25 .14
Skills at Intake Health care of children Poor living conditions Discipline skills
.82 .62 -.60
.06 .20 -.19
.24 .44 64.4%
.32 .05 72.1%
Results of the Analvsis Canonical correlation Significance level Percentage correctly classified
aA positive coefficient denotes an association with imminent negative coefficient denotes an association with less risk.
risk; a
bA positive coefficient denotes an association with placement; negative coefficient denotes an association with preservation.
a
Imminent risk cases did make gains in parenting skills over the course of services, and made the most significant gains in the acquisition of household resources and encouragement of child development (see Table 3). Gains were more substantial among families with less than imminent risk. These families made highest gains in child discipline techniques, household resources, child care skills, the orderliness and condition of the development (see Table 3). Despite the fact that their placement rates were no higher, imminent risk families ended treatment at skill levels lower than the average intake levels for general risk families.
Assessment
of Imminence
251
of Risk
Table 2 Average Time Spent Providing Services (in total hours)
Risk Total Service Provideda Case planningb Assessment Parent education Supplemental parent Teaching family care Counseling Household maintenance Transportation Referral Crisis intervention Respite Food Financial Medical Housing Legal Home health care
(116) 43.3 6.8 3.3 3.0 2.2 1.7 1.5 1.4 1.0 0.9 0.7 0.6 0.5 0.4 0.2 0.2 0.1
Imminent (15) 54.9 8.2 1.0 12.0 0.6 2.8 1.1 2.0 0.4 3.4 2.7 0.5 1.4 0.3 0.4 0.2 0.6
General (101) 42.6 6.7 3.6 *** 2 2*** 2:4** 1.6 1.5 1.3 1.1* 0.7 0.5 1.4 0.5 0.4 0.2 0.2 0.1
Intensity of service 9.5 hrslwk. 31.5 hrslwk. 7.0 hrslwk. 33 35 33 Avg. % of time in the home 42 35 Avg. % of time in the agency 41 a’Ilresefigures are based on a subsample of 116 cases. bCase planning was determined to be a catch-all category of services, that has since been refined into several more specific categories of service *pc .05, **p < .Ol, ***p< .ool.
The same model of risk factors that was used in the prediction of imminence of risk was used to predict actual placement, and in this analysis, the model was much more successful at predicting actual placement (see Table 4). The key predictors of actual placement turned out to be very different from those used to determine imminence of placement at intake. In determining who was actually placed into foster care, the key factor in this sample was whether relatives were nearby. When relatives were nearby, the child was more likely to be removed (but it is unknown
252
Berry
which or how many of these children were placed with relatives). In predicting imminence of placement, a higher number of children was associated with a judgment of lower risk, but in actual placement outcome, a higher number of children was associated with placement. Also, in predicting placement outcome, the family’s income level was a key predictor, where it had been negligible in the analysis of risk. The demographic characteristics were the most important variables in the placement model, whereas the parent’s skills at discipline and health care had been more important in the analysis of risk. Also, neglect was a minor factor in the prediction of risk, but it was a key predictor of placement in the outcome analysis.
Table 3 Parents’ Skill Gains During Service
Risk Level mmment Skill Area
General
Q!!&~Q2&&
Closing 3.34 3.71** 3.85 3.70 3.59
Non-Crowdedness Orderliness Safety Cleanliness comfort
3:19 3.48 3.39 3.32
342 3.58 3.55 3.61
353 3.77 3.63 3.50
Household resources Poor condition of building Poor condition of unit
8.94 2.88a 5.58a
9.73** 2.79 4.67
9.76 2.35 4.41
10.16*** 1.87** 3.81*
Child care skills Discipline Health care Encouragement of Development
26.52 20.04 13.72
27.96 21.68 14.28
27.37 21.29 13.30
28.51** 22.63*+* 13.69*
18.60
20.04
20.42
21.06
aDifference between imminent and general risk families at intake is statistically significant at .OS level. *Difference between opening and closing (paired t-test) is statistically significant at .OS level, ** at .Ol level, *** at .OOl level.
The model was very successful at predicting outcome, with 72% of cases correctly classified, and a canonical correlation of .32 @ < .05). These same variables used in the determination of risk of placement were much more successful at predicting actual placement than in classifying which children had been deemed at imminent risk at intake.
Assessment
of Imminence
of Risk
253
Finally, the same discriminant analysis was performed on the model predicting placement outcome again, this time adding the risk level for the family as judged by the intake worker, and then again with the addition of the amount of time spent in the home (see Table 4). When the intake worker’s risk determination is added, the prediction of placement is greatly enhanced, improving the canonical correlation to .36 (p < .Ol), and increasing the correct classification of cases to 77.3%. Then, when the amount of service time in the home is added to the analysis, the model is improved further, to a canonical correlation of .7 1 @ c .002), and a correct classification of 94.6% of cases.
Table 4 Comparison of Three Models to Predict Placement Model I: Basic Risk Factors h = 208) Predicted Qutcome Placed73.1% 26.9% 34.6% 65.4%
Actual Outconx Placed Preserved Correct Classification: Canonical Correlation
72.1% .32 @ < .05)
Model II: Basic Risk Factors and Degree of Imminence (n = 203) Predicted P)aced 77.4% 23.1%
Actual Outcome Placed Preserved
Correct Classification: Canonical Correlation
Outco e PreseZed 22.6% 76.9%
77.3% .36 @ < .Ol)
Model III: Basic Risk Factors. Degree of Imminence and Time in Home (n = 56) Actual Outcome Placed Preserved
Predicted Outcome PlacedPreserved 94.0% 6.0% 0.0% 100.0%
Correct Classification: Canonical Correlation
94.6% .71 @ < .002)
Berry
254
Discussion The findings of this study must be viewed with caution, given the differing sample sizes between the various models tested. These differences in predictive ability among the different models could be due more to differences in sample variances than to true differences. The sample variances have been checked in each of the bivariate analyses and the separate variance estimates of statistical significance have been used when appropriate. Nonetheless, these predictive models are cautious explorations of the utility of risk assessment based on the conventional family characteristics currently in use. Based on the results of these discriminant analyses, the risk factors which are used to predict imminence of placement are good at predicting actual placement after leaving the program, but are not as good at predicting which families are actually classified as being at imminent risk at the beginning of treatment. However, even though intake classification was not directly related to the risk factors meant to be assessed, the intake worker’s classification of risk was a key predictor in whether any children were actually removed after treatment. There may be other unnamed factors which are considered before classification, or there may be a “gut feeling” factor which is indeed very predictive of placement. It is possible but unlikely that this designation of imminent risk was a self-fulfilling prophecy, since removals occurred, on average, five months after the case was closed, and since workers did spend more time with families who were living in worse environmental conditions. The key predictors of placement for this sample included very basic demographic characteristics: relatives nearby, a higher number of children, and low income, as well as the presenting problem of child neglect. These are rather concrete and basic problems that prove difficult to ameliorate in a short-term program. The intractability of such basic needs demands much more proactive and universal remedies than intensive family preservation services, and must be addressed in family policy, rather than simply deemed risk factors among a high-risk population. Regardless of the classification of risk, the amount of service time spent in the home with the family was the key predictor of whether children remained in the home after treatment. Service time has also been found to be related to clients’ gains in other skills, like discipline and health care of children (discussed in Berry, 1990). Family preservation programs are focused on providing in-home and family-centered services, and this analysis supports the importance of that focus. Since this program was evaluated, the intake classification has been changed and modified at these two agencies, with an even more detailed
Assessment
of Imminence
of Risk
255
analysis of risk factors going into a determination of risk at intake. Further research will be needed to assess whether this results in better targeting of families at risk of placement, or whether the “gut feeling” factor is indeed enough.
References AuClaire, P., & Schwartz, I. M. (1986). An evuluarion of rhe effectiveness of intensive home-based services as an alternative to placement for adolescents Minneapolis: University of Minnesota. Hubert H. and their families. Humphrey Institute for Public Affairs. Behavioral Sciences Institute. (1987). Summary of King, Pierce, Snohomish, and Spokane County Homebuilders service, September 1, 1986 - August 31, 1987. Federal Way, City of Publisher, Washington: Author. Berry, M. (1990). An evaluation of family preservation services: Fitting agency services to family needs. (Manuscript submitted for publication). Johnson, W., & L’Esperance, J. (1984). Predicting the recurrence of child abuse. Social Work Research and Abstracts, 20,21-3 1. Katz, M. H., Hampton, R. L., Newberger, E. H., Bowles, R. T., 8z Snyder, J. C. (1986). Returning children home: Clinical decision making in cases of child abuse and neglect. American Journal of Orthopsychiatry, 56,253-263. Kinney, J., Haapala, D., Booth, C., & Leavitt, S. (1988). The Homebuilders model. In J. K. Whittaker, J. Kinney, E. M. Tracy, & C. Booth (Eds.), Improving practice technology for work with high-risk families (pp. 37-67). Hawthorne, New York: Aldine de Gruyter. Kinney, J. M., Madsen, B., Fleming, T., & Haapala, D. A. (1977). Homebuilders: Keeping families together. Journal of Consulting and Clinical Psychology, 45667-673. Landsman, M. J. (1985). Evaluation of fourteen child placement prevention projects in Wisconsin, 1983-1985. Iowa City, Iowa: National Resource Center on Family Based Services. Lindsey, D. (1991). Factors affecting the foster care placement decision: An analysis of national survey data. American Journal of Orthopsychiatry, 61, 272-281. Magura, S., & Moses, B. S. (1986). Outcome measures for child welfare services: Theory and applications. Washington, DC: Child Welfare League of America. National Resource Center on Family Based Services. (1988). Family bused services: A national perspective on success and failure. Iowa City, Iowa: National Resource Center on Family Based Services. Nelson, D. (1988). Recognizing and realizing the potential of “family preservation.” Miami, Florida: Edna McConnell Clark Grantees Conference. Norman, A. (1985). Keeping families together: The case for family preservation. NY: Edna McConnell Clark Foundation.
256
Berry
Pecora, P. (1988). Evaluating risk assessment systems: Methodological issues and selected research findings. In P. Schene & K. Bond (Eds.), Research issues in risk assessment for child prorecrion (pp. 000-000). Denver, CO: American Association for Protecting Children. Pecora, P. J., Fraser, M. W., Haapala, D., & Bartlome, J. A. (1987). Defining family preservation services: Three intensive home-based treatment programs.
Salt Lake City: University of Utah Social Research Institute. Pelton, L.H. (1989). For reasons of poverty. New York: Praeger. Reid, W. J., Kagan, R. M., & Schlosberg, S. B. (1988). Prevention of placement: Critical factors in program success. Child Welfare, 67, 25-36. Remy, L. L., & Hanson, S. P. (1983). Evaluation of the Emergency Family Care Program: Final report. San Francisco: Children’s Home Society. Ruger, J. B., & Wooten, R. H. (1982). A developmental approach to helping families at risk. Social Casework, 63, 3-14. Showell, W. H. (1985). 1983-85 biennial report of CSD’s Intensive Family Services. Salem: State of Oregon Children’s Services Division. Showell, W., Hartley, R., & Allen, M. (1987). Outcomes of Oregon’s family Salem: State of therapy programs: A descriptive study of 999 families. Oregon Children’s Services Division. Wald, M., & Woolverton, M. (1990). Risk assessment: The emperor’s new clothes? Child Welfare, 69, 483-5 11.
019&7409r9153.00+.00 Copyright (D 1991 Peqamon Press plc
The Assessment of Imminence of Risk of Placement: Lessons from a Family Preservation Program Marianne Berry
University of Texas at Arlington Intensive family preservation programs are a form of family services aimed at preventing the placement into foster care of children deemed at imminent risk of removal. There is considerable debate, however, on how best to determine the risk of imminent placement. This article presents an analysis of the determination of imminent risk within one family preservation program, and finds that the conventional and accepted risk factors to be used in making a risk determination at intake are more accurate at predicting placement after leaving the program than in predicting who is judged to be at imminent risk at intake. Specific elements contributing to this discrepancy are discussed, with implications for risk assessment research and policy.
Intensive family preservation services are a form of family services aimed at preventing the placement into foster care of children deemed at imminent risk of removal. Many family preservation advocates emphasize the importance of limiting family preservation efforts only to those truly at imminent risk of placement (Kinney, Haapala, Booth & Leavitt, 1988; Nelson, 1988). Opening up family preservation programs to less risky populations dilutes the intensive nature of services and can elevate preservation rates in a misleading fashion (Pecora, Fraser, Haapala & Bartlome, 1987). If these programs can be evaluated with rigor and shown to be effective with a high-risk population, the argument for their inclusion in actual service policy will be compelling. Supportfor this research was provided by the American Association of University Women in the form of a doctoral fellowship. The author thanks Mark Claycomb and JoAnn Cook at Children’s Home Society of California and Rick Barth at University of California at Berkeley for their help with this study. Send requests for reprints to: Marianne Berry at University of Texas at Arlington, School of Social Work, P.O. Box 19129. Arlington, Texas, 76019. 239
240
Berry
Family preservation programs can be viewed as elements of triangle that represents services to all families at risk of separation by child placement (Norman, 1985). The tip of the triangle contains programs for those families in need of crisis intervention, who are at risk of imminent placement. The second and somewhat larger level contains services to those families who show signs of risk but are not in immediate danger. The largest segment, the foundation of the triangle, applies to programs for families in the general population who may have characteristics that are associated with potential for risk. Thus, family preservation programs are reserved for those families at the tip of the triangle.
Definitions of Risk There is considerable debate on how best to determine the risk of imminent placement. The definition of “risk” has ranged from “families who are poor and headed by a single parent and families of adolescents” (Landsman, 1985, p. 1) to “family members judged as having high potential for removal to another living situation” (Kinney, Madsen, Fleming & Haapala, 1977, pp. 670-671). In other uses, it is not defined at all (Ruger & Wooten, 1982). This ambiguity clouds the understanding of programs designed to reach “families at risk.” Many programs like Homebuilders agree that placement is imminent if the child’s protective service worker details a plan to place the child within 24 to 48 hours (Kinney, et al., 1989), but this places the judgment of risk in the hands of child protective service workers, who may judge imminence of placement in variety of ways. Other family preservation programs broaden imminent risk to include families with many risk factors, such as prior placement, a history of abuse, or a out-of-control adolescent (Pecora, et al., 1987; Showell, Hartley, & Allen, 1987). These programs may limit their population to child protective service cases, or they can use these risk factors to define other families as soon-to-be child protective service cases, and thus, another entre’ to a designation of imminent risk of placement. The program which is the subject of this evaluation first defined families at risk (in 1983) as those families in which any of the following are true: the referral came from the Department of Social Services, the family had more than one child, and the family was of minority ethnicity (Remy & Hanson, 1983). Since 1983, risk has been further specified as pertaining to families with any of the following characteristics: a child currently in placement, a child with a prior history of placement, a history of abuse or neglect, or if more than one of the following are true: multiproblem family, a multi-racial family, possible abuse, an absent parent, a parent with problems with substance abuse, psychiatric history, developmental disability or severe physical illness.
Assessment
of Imminence
of Risk
241
Once families are deemed at imminent risk of child placement, each child in the family is also assessed as to his or her degree of imminence of risk of placement: within 72 hours, within two months, or over two months away. Thus, some families may be deemed to be imminent risk, but may have only one child out of five at imminent (within 72 hours) risk of placement, and the other children might be at lesser risk levels. Prediction of Placement Removal of a child and placement into foster care is intended to prevent continued harm to the child or the potential for danger to the child in the home. Parental and child factors associated with child abuse and neglect abound in the research literature, and these factors are included in the many risk assessment tools being developed and in use in many states (Johnson & L’Esperance, 1984; Magma & Moses, 1986; Pecora, 1988). Such factors include social isolation, a parent’s history of being abused in childhood, social and economic stress, and substance abuse. An assessment of these factors within any one family can be a good predictive tool for risk of abuse. While there are a variety of factors which may predict whether a child is in danger of abuse, however, there are additional factors which determine whether a child is placed in foster care, some concerning the same intrafamilial characteristics and problems, and others concerning the availability of foster homes, reporting mechanisms, etc. Thus, there can be a wide disparity between the children identified as being at risk of abuse and those who actually become placement statistics. The factors which predict placement are not always included in risk assessment tools, which may focus on parent-child interaction, and disregard other environmental factors or systemic decision making factors. Many studies have shown that, despite the reluctance to place for reasons of poverty (Pelton, 1989), the income level of the family is often a key predictor of whether a child is removed, over and above the severity of abuse or neglect (Katz, Hampton, Newberger, Bowles & Snyder, 1986; Lindsey, 1991). Current risk assessment tools are an important step forward in prediction of outcomes for children, but these tools are far from perfect or “magical” (Wald & Woolverton, 1990), and are also not particularly clear about whether they are predicting the risk of continued maltreatment, the risk of poor developmental outcomes, or the risk of placement. Similarly, family preservation programs often cloud the distinction between whether they are reducing the risk of placement, reducing the risk of poor developmental outcomes, or reducing the risk of continued maltreatment.
242
Berry
Factors in Reducing the Risk of Placement and Reducing the Risk of Abuse The primary goal of intensive family preservation services is to prevent the need for out-of-home placement. This usually entails resolving immediate crises and teaching skills necessary to independently maintain family integrity. This requires some level of proficiency at problem-solving and communication skills, but therapists also provide such concrete services as housecleaning and transportation. By so doing, a worker models how to do these skills, and also demonstrates his or her commitment to the family (Kinney, et al., 1988). During the provision of such “hard” services, therapists can also observe clients’ skills and talk about other problems the family is experiencing. In this way, the roundthe-clock and across-the-board availability and participation of the therapist is truly a comprehensive family service. These services are thus meant to reduce a variety of risks, with prevention of placement being the primary goal. Of course, provision of these services should also reduce the risk of poor developmental outcomes for the child and continued abuse or neglect. A variety of intensive family preservation programs have been evaluated, and most report their placement prevention rate as the primary criterion of success. These programs report success rates ranging from 71% (Pecora, et al., 1987) to 93% (Reid, Kagan, & Schlosberg, 1988), with Homebuilders averaging 90% of families intact at the end of treatment (Kinney, et al., 1988). At the one-year follow-up point, Homebuilders reports that 88% of families are still intact (Behavioral Sciences Institute, 1987), and Oregon’s Intensive Family Services finds a 73% success rate (Showell, 1985). Most of these evaluations have been focused on imminent risk cases, but the determination of imminence is, again, unclear. Placement prevention is a crucial criterion in determining treatment effectiveness, but it is not the only measure of success. Only a few evaluations of preservation programs have addressed other effects on the reduction of maltreatment factors, such as child behavior problems or family functioning (Kinney, et al., 1988; Pecora, et al., 1987). In the evaluation by Nelson and colleagues (National Resource Center on Family Based Services, 1988) of eleven family-based placement prevention programs in six states, family preservation was associated with the following changes during services: improvements in behavior, material resources, family structure, family dynamics, emotional climate, perceptions of problems, community perceptions of the family, informal support network, and community involvement. Those families who had children removed had made significant declines in behavior, family dynamics, emotional climate, community perception of the family, and
Assessment
of Imminence of Risk
243
community involvement. Given the short-tetm nature of these preventive programs, this study does not answer the question of the longevity of families’ gains, particularly since the greatest gains were made in the more intangible and elusive skills. One evaluation of the Homebuilders program (Kinney, Haapala, Booth, & Leavitt, 1988) reports that the majority of families improved in the child’s school attendance, hyperactivity, delinquent acts, and peer problems as judged by the parent or a therapist at the end of treatment. However, all families were judged to have remained the same regarding handling of medical problems or physical handicaps, and the majority of families had remained the same regarding alcohol abuse or learning disabilities. Another evaluation of the Homebuilders program looked at school adjustment, delinquent behavior, home-related behavior, and cooperation with the agency (Pecora, et al., 1987). Except for cooperation, children in the program made significant positive improvements during service. Parents also made significant improvements in supervision of younger children, parenting of older children, attitudes toward preventing placement, and their knowledge of child care. In the Utah Family Preservation program (Pecora, et al., 1987), children made significant gains in school adjustment and behavior, and parents improved in parenting behavior, attitudes and knowledge. Purpose of the Study This study evaluated one such intensive family preservation program in Northern California, in order to identify the characteristics of children who are at deemed at risk of placement, to identify those who were actually placed, and to determine whether these two groups were identical. This evaluation included a detailed examination of different service components in the program and their match to families with differing levels of placement risk. Few other programs have examined the correlates of imminent risk of placement or program effectiveness. This study examines placement rates for children at differing levels of placement risk, and also evaluates program effectiveness, in terms of services provided and the resulting skill gains, for families at differing levels of placement risk. The In-home Family Care Program
The In-home Family Care program which is the subject of this evaluation offers intensive family preservation services within an ecological framework. The program is one of the many services provided by Children’s Home Society of California, and operates in San Francisco, Oakland, and Los Angeles. The program emphasizes strengthening the
244
Berry
family’s social and ecological system, and does not follow a conventional family therapy model. This program treats parents and their children as partners in the process of strengthening the family. More than many other home-based programs, they do whatever it takes to help the family acquire tangible resources, including finding housing and day care for the family, if necessary. Staff consists of master’s level social workers and family care workers who usually have a bachelor’s or master’s degree. Each family is assigned a social worker and a family care worker. Cases are meant to be open for about three months. San Francisco workers individually carry two to five cases at a time, while Oakland workers carry three cases at a time. A team meeting is held weekly to allow the social worker and his or her family care workers to discuss all current cases and their status. The social worker and other family care workers offer treatment strategies and are good sources of information about community resources. The In-home Family Care Program emphasizes working with families at imminent risk of removal of the child. Degree of risk is determined by the Department of Social Services if they are the referral source. If the referral is from another source, the intake worker determines the degree of risk after discussion with the referral source and based on the prior placement history of the family as well as other risk factors. Families may be deemed ineligible if there is too much danger to the worker, other help is available, no children are at risk, parents are unavailable, parents decline service, or all children are older than 14 years old. Families are not excluded due to mental retardation or substance abuse, as they are in some other programs (AuClaire & Schwartz, 1986; Landsman, 1985). Referrals to the program come from a variety of sources, including hospitals, the Department of Social Services, private social services, public health agencies, and self-referrals. Depending on the family’s presenting problems, family care workers may help parents to learn positive child rearing skills instead of punitive and abusive patterns. They may also help families to acquire adequate housing and to apply for appropriate financial aid, as well as learning how to budget the income they do have. Family care workers address any problems in the condition of the family’s building or apartment, and help parents to fix broken windows and plumbing as well as helping to convince landlords to fix the heat, which is often permanently on or permanently off. Families are taught health care and nutrition practices, and are directed to legal assistance, if needed. Families are thus assessed as to their resource deficits, and these deficits are replenished with resources and skills. In-home Family Care service is not a one-way street where workers only give resources which families are free to take. When a case is opened
Assessment
of Imminence
of Risk
245
for intensive family preservation services, the worker and the family members make service agreements involving tasks that the worker and the family members agree to perform. Agreements are written and specify the task, who will perform it, and where, when and how often it will be performed. Objectives of service are to be specified in behavioral terms. For example, one such agreement may stipulate that the mother will make an appointment for her child to go to the pediatrician for a well-baby check-up. This would be followed by a task agreement that the mother would take the child to the appointment and that the worker would provide transportation. Everyone signs the agreement when it is acceptable, and workers track progress on tasks. New tasks are typically set at every meeting of family and worker. Method Sample
For the sampling frame, the following criteria were imposed: sample selection was made from cases opened and closed between January 1, 1985 and December 31, 1987; cases had to involve at least one child at risk of placement, and cases had to have received intensive services. This closing date was imposed to allow at least six months to demonstrate family stability or instability (as evidenced by the child’s placement). This sampling frame produced a population of 369 families in San Francisco and 42 families in Oakland. Placement outcomes were obtained from information from caseworkers about their knowledge of child placement in their case loads and from Department of Social Services emergency shelter and foster care logs. Each family receives follow-up visits at one-month, six-months, and one-year following termination, and follow-up forms indicate whether any children in the home were removed in that period. Cases for which an outcome of out-of-home placement of the child was known were included in the final sample. All 367 cases with known outcomes were included in this sample, 327 in San Francisco and 40 in Oakland. Only 44 cases were excluded, due to unknown outcomes (n = 9), incorrect case numbers (n = lo), or incorrect coding of whether the case received intensive services (n = 25). Measures
Each case provided the following information: outcome information, client characteristics, and service characteristics. All of this data has been gathered since 1983 in substantial detail on a number of standardized forms, which are filled out by the family care worker in the process of
246
Berry
keeping a case record for each family. All codes on these forms are determined at the time of services by family care workers and supervisors, and none were coded post-hoc by researchers. Level of Placement Risk. Each child within a family was coded by the intake worker as to the imminence of risk of placement, either within 72 hours, within 2 months, over two months away, or no risk. Several factors entered into the decision about degree of risk, the most important being a prior placement history, or a history of adjudicated abuse or neglect. Other important considerations included a psychiatric history of the child or parent, developmental disability, a multi-problem family, and possible abuse or neglect. If a child was already in placement, that was noted instead of level of placement risk. For this analysis, imminent risk cases were those at risk of placement of any child within the next 72 hours, and other families were deemed “general risk.” The average of the risk scores for all the children in the family also provided a summary risk level score for the family. Case Outcomes. Generally, outcome information included such data as whether the child was placed in out-of-home care (removed from the home and not returned home within 72 hours; relative placements were counted as out-of-home placements) and the extent of progress while in treatment. Specifically, the major outcome measure was a dichotomous variable concerning whether the child was placed in out-of-home care either while receiving In-home Family Care services or since, up to July of 1988. Client Characteristics. Client characteristics covered such areas as demographics, child and parent presenting problems, risk of placement, previous or current child protective service involvement, prior placement, assessment of the physical environment, levels of client skills in various areas, the referring agencies, the number and ages of household members, parental ethnicity, and whether or not the family has the following resources: housing, phone, food, friends or relatives able to help, AFDC, Medi-Cal, and income. Several general indicators of family level of functioning are also assessed by the family care worker: the density and condition of the environment (clean, safe, orderly, uncrowded, and comfortable), the condition of the building and living unit, the level of childhood care (general, discipline, development, and health), and parental physical and mental capacity to be involved in treatment. The typical assessment period at opening and closing lasts for one to three hours. Family care workers are trained for a month when they are hired as to how to assess family conditions, and are retrained periodically when agency assessment decisions are modified. A detailed code book specifies how various situations and conditions are coded in each category.
Assessment
of Imminence of Risk
247
Also, weekly team meetings address assessment decisions and the coding of these for case recording purposes, so fairly high interworker reliability of coding is assumed (but was not tested) for this study. Service Characteristics. Service characteristics include information on worker time, collateral agencies utilized, and problems addressed. Service logs are maintained while the case is open and the following are recorded for each contact: the location of the contact, the service provided, other agencies participating in the service contact and elapsed time (in minutes) of service provided. Due to the immense amount of data in these service logs, a subsample of 116 cases were selected for inclusion of information (a full sample of data from the service logs was too large and unwieldy for computer analysis). These cases were chosen from the latter half of the study period (July 1, 1986 to December 31, 1987), and included those which were both opened and closed between those dates, and for which there were complete service log records. These 116 cases therefore have information on specific services provided, collateral agencies used, and the actual length of service (in minutes). These 116 cases were compared to the other 25 1 cases which did not have service log data to test for representativeness of the data. Two significant differences were found between the service log sample and the remaining cases: the cases with service log data had significantly older mothers (32 years old vs. 30, on average; r = 2.11, p < .05) and the average age of the children in the home was older (5.2 years vs. 4.4 years; t = 2.03, p < .OS). There was no significant difference in the placement rate between the two groups, however, with a placement rate of 17.5% among the non-service log sample and a rate of 12.9% among the service log sample. Gain Scares. As mentioned, families are rated at initial assessment and at each periodic assessment on a variety of environmental and skill areas. Each environmental area or parenting skill was composed of a number of items for which the family received a point or two. The sum of these points provided a score for that skill at that time. Gain scores were computed for case opening to closing, as well as for gains from case closing to the one-month, six-month, and one-year follow-up points. Results
A total of 52 families, or 14%, had children removed during or subsequent to receiving In-home Family Care services. This rate is comparable to those reported for the Homebuilders program (Kinney, et al., 1988) and other family preservation programs nationwide (Landsman, 1985; Pecora, et al., 1987).
248
Berry
Predicting the Designation of Imminent Risk Only 49 of the 367 families served (13%) were judged to be at imminent risk of child placement. There were differences between the two agencies in the latitude they had in serving families: about one-third of the Oakland cases were imminent risk, while only 11% of the San Francisco cases were imminent risk. This is largely a result of the funding differences between the two counties; the Oakland office was constrained by county funding to serve imminent risk (or “reasonable efforts”) cases, while the San Francisco office had more flexible funding definitions of their service population. Imminent risk families were similar to general risk families, in that the majority had two or three children, almost three-quarters were headed by a single mother, the average age of the mothers was 3 1, an even number of children were male and female, and the average age of the children was 4.8. Imminent risk families were slightly, but not significantly, more likely to be African-American. The majority of both imminent risk and general risk families had housing, food in the home, and a phone, and about three-quarters in each group received AFDC and Medi-Cal. Imminent risk families were slightly less likely than general risk families to report that they had friends or relatives available to help them. The imminent risk designation appears to associated with standard risk factors. Imminent risk families were more likely to receive service primarily for substance abuse, parental handicap, or a request for child placement. Cases of a more general risk level were significantly more likely to receive service primarily for child emotional disturbance, child handicap, child neglect, or economic/environmental conditions. There were no significant differences between imminent risk families and nonimminent risk families in the number of children in the home, ages of children in the home, mother’s age, father’s age, ethnicity, type of dwelling, or family income. The condition of the household at intake (crowded, disordered, dangerous, dirty, and/or uncomfortable) was significantly more of a serious problem for imminent risk families, as were child care practices. Imminent risk parents were significantly less likely to be physically capable of participating in treatment. A discriminant analysis was performed to determine whether the key predictors identified by the program as imminent risk factors were used by the intake worker to define a family or a child to be at imminent risk of placement. The variables in the model included family characteristics (any prior removals, number of children, income, mother’s age, and if any relatives were nearby), parents’ presenting problems (abuse, neglect, developmental disability, and/or substance abuse), and parents’ skills at intake (living conditions, discipline skills, and health care of children).
Assessment
of Imminence of Risk
249
The results of this analysis are presented in Table 1. The key correlates of a determination of imminent risk of placement were prior removals, a low number of children in the family, parents’ good health skills and poor discipline skills at intake and poor living conditions. The presenting problem of abuse, neglect, developmental disabled parent, or substance abuse were negligible in the designation of risk, as were the mother’s age, the presence of relatives, or the income level of the family. This model was fairly poor at correctly classifying those children defined as imminent risk, with only 64% of cases correctly classified based on these factors. Therefore, intake workers were judging imminent risk on factors besides those delineated by the program, or at random. Predicting the Actual Outcome of Placement Despite the differences in presenting problem and parenting capacity between imminent risk and general risk families, the placement rate for the imminent risk families was 13%, no different than that for the rest of the sample. Regardless of the increased risk factors or the crisis nature of the presenting problem, these families were just as successful at remaining together. A more detailed examination of the services provided and skills gained illuminates the services received by imminent risk families. Imminent risk cases received an average of 96 hours of service, or about 30 more hours, on average, than non-imminent risk cases. Of these 96 hours or so, an average of 40 hours were spent in the home, significantly more time than that for non-imminent cases (23 hours on average). Overall, the percentage of service time spent in the home versus that in the agency did not differ between imminent and general risk cases, but the intensity of service (average amount of time spent per day) was significantly different (4.5 hours vs. 1 hour per day; see Table 2). Imminent risk families received somewhat more time in case planning, supplemental parenting, crisis intervention, respite care, financial skills, and home health care, while general risk cases received more parent education, teaching of family care, and referrals to other services (see Table 2). All t-tests in this study were examined for unequal sample variances, due to fluctuating sample sizes, and separate variance estimates are reported when appropriate.
Berry
250
Discriminant
Table 1 Analyses of Imminent Risk and Actual Placement Outcome
Discriminant Function Coefficient Family Characteristic Any prior removals Number of children Relatives nearby Mother’s age Income
Imminent Risks 66 -.41 -.04 -.03 .03
Placementh -.07 .39 .53 -.23 -.37
Presenting Problem Developmental disability Neglect Substance abuse Abuse
.ll -.08 -.06 -.05
.24 .45 -.25 .14
Skills at Intake Health care of children Poor living conditions Discipline skills
.82 .62 -.60
.06 .20 -.19
.24 .44 64.4%
.32 .05 72.1%
Results of the Analvsis Canonical correlation Significance level Percentage correctly classified
aA positive coefficient denotes an association with imminent negative coefficient denotes an association with less risk.
risk; a
bA positive coefficient denotes an association with placement; negative coefficient denotes an association with preservation.
a
Imminent risk cases did make gains in parenting skills over the course of services, and made the most significant gains in the acquisition of household resources and encouragement of child development (see Table 3). Gains were more substantial among families with less than imminent risk. These families made highest gains in child discipline techniques, household resources, child care skills, the orderliness and condition of the development (see Table 3). Despite the fact that their placement rates were no higher, imminent risk families ended treatment at skill levels lower than the average intake levels for general risk families.
Assessment
of Imminence
251
of Risk
Table 2 Average Time Spent Providing Services (in total hours)
Risk Total Service Provideda Case planningb Assessment Parent education Supplemental parent Teaching family care Counseling Household maintenance Transportation Referral Crisis intervention Respite Food Financial Medical Housing Legal Home health care
(116) 43.3 6.8 3.3 3.0 2.2 1.7 1.5 1.4 1.0 0.9 0.7 0.6 0.5 0.4 0.2 0.2 0.1
Imminent (15) 54.9 8.2 1.0 12.0 0.6 2.8 1.1 2.0 0.4 3.4 2.7 0.5 1.4 0.3 0.4 0.2 0.6
General (101) 42.6 6.7 3.6 *** 2 2*** 2:4** 1.6 1.5 1.3 1.1* 0.7 0.5 1.4 0.5 0.4 0.2 0.2 0.1
Intensity of service 9.5 hrslwk. 31.5 hrslwk. 7.0 hrslwk. 33 35 33 Avg. % of time in the home 42 35 Avg. % of time in the agency 41 a’Ilresefigures are based on a subsample of 116 cases. bCase planning was determined to be a catch-all category of services, that has since been refined into several more specific categories of service *pc .05, **p < .Ol, ***p< .ool.
The same model of risk factors that was used in the prediction of imminence of risk was used to predict actual placement, and in this analysis, the model was much more successful at predicting actual placement (see Table 4). The key predictors of actual placement turned out to be very different from those used to determine imminence of placement at intake. In determining who was actually placed into foster care, the key factor in this sample was whether relatives were nearby. When relatives were nearby, the child was more likely to be removed (but it is unknown
252
Berry
which or how many of these children were placed with relatives). In predicting imminence of placement, a higher number of children was associated with a judgment of lower risk, but in actual placement outcome, a higher number of children was associated with placement. Also, in predicting placement outcome, the family’s income level was a key predictor, where it had been negligible in the analysis of risk. The demographic characteristics were the most important variables in the placement model, whereas the parent’s skills at discipline and health care had been more important in the analysis of risk. Also, neglect was a minor factor in the prediction of risk, but it was a key predictor of placement in the outcome analysis.
Table 3 Parents’ Skill Gains During Service
Risk Level mmment Skill Area
General
Q!!&~Q2&&
Closing 3.34 3.71** 3.85 3.70 3.59
Non-Crowdedness Orderliness Safety Cleanliness comfort
3:19 3.48 3.39 3.32
342 3.58 3.55 3.61
353 3.77 3.63 3.50
Household resources Poor condition of building Poor condition of unit
8.94 2.88a 5.58a
9.73** 2.79 4.67
9.76 2.35 4.41
10.16*** 1.87** 3.81*
Child care skills Discipline Health care Encouragement of Development
26.52 20.04 13.72
27.96 21.68 14.28
27.37 21.29 13.30
28.51** 22.63*+* 13.69*
18.60
20.04
20.42
21.06
aDifference between imminent and general risk families at intake is statistically significant at .OS level. *Difference between opening and closing (paired t-test) is statistically significant at .OS level, ** at .Ol level, *** at .OOl level.
The model was very successful at predicting outcome, with 72% of cases correctly classified, and a canonical correlation of .32 @ < .05). These same variables used in the determination of risk of placement were much more successful at predicting actual placement than in classifying which children had been deemed at imminent risk at intake.
Assessment
of Imminence
of Risk
253
Finally, the same discriminant analysis was performed on the model predicting placement outcome again, this time adding the risk level for the family as judged by the intake worker, and then again with the addition of the amount of time spent in the home (see Table 4). When the intake worker’s risk determination is added, the prediction of placement is greatly enhanced, improving the canonical correlation to .36 (p < .Ol), and increasing the correct classification of cases to 77.3%. Then, when the amount of service time in the home is added to the analysis, the model is improved further, to a canonical correlation of .7 1 @ c .002), and a correct classification of 94.6% of cases.
Table 4 Comparison of Three Models to Predict Placement Model I: Basic Risk Factors h = 208) Predicted Qutcome Placed73.1% 26.9% 34.6% 65.4%
Actual Outconx Placed Preserved Correct Classification: Canonical Correlation
72.1% .32 @ < .05)
Model II: Basic Risk Factors and Degree of Imminence (n = 203) Predicted P)aced 77.4% 23.1%
Actual Outcome Placed Preserved
Correct Classification: Canonical Correlation
Outco e PreseZed 22.6% 76.9%
77.3% .36 @ < .Ol)
Model III: Basic Risk Factors. Degree of Imminence and Time in Home (n = 56) Actual Outcome Placed Preserved
Predicted Outcome PlacedPreserved 94.0% 6.0% 0.0% 100.0%
Correct Classification: Canonical Correlation
94.6% .71 @ < .002)
Berry
254
Discussion The findings of this study must be viewed with caution, given the differing sample sizes between the various models tested. These differences in predictive ability among the different models could be due more to differences in sample variances than to true differences. The sample variances have been checked in each of the bivariate analyses and the separate variance estimates of statistical significance have been used when appropriate. Nonetheless, these predictive models are cautious explorations of the utility of risk assessment based on the conventional family characteristics currently in use. Based on the results of these discriminant analyses, the risk factors which are used to predict imminence of placement are good at predicting actual placement after leaving the program, but are not as good at predicting which families are actually classified as being at imminent risk at the beginning of treatment. However, even though intake classification was not directly related to the risk factors meant to be assessed, the intake worker’s classification of risk was a key predictor in whether any children were actually removed after treatment. There may be other unnamed factors which are considered before classification, or there may be a “gut feeling” factor which is indeed very predictive of placement. It is possible but unlikely that this designation of imminent risk was a self-fulfilling prophecy, since removals occurred, on average, five months after the case was closed, and since workers did spend more time with families who were living in worse environmental conditions. The key predictors of placement for this sample included very basic demographic characteristics: relatives nearby, a higher number of children, and low income, as well as the presenting problem of child neglect. These are rather concrete and basic problems that prove difficult to ameliorate in a short-term program. The intractability of such basic needs demands much more proactive and universal remedies than intensive family preservation services, and must be addressed in family policy, rather than simply deemed risk factors among a high-risk population. Regardless of the classification of risk, the amount of service time spent in the home with the family was the key predictor of whether children remained in the home after treatment. Service time has also been found to be related to clients’ gains in other skills, like discipline and health care of children (discussed in Berry, 1990). Family preservation programs are focused on providing in-home and family-centered services, and this analysis supports the importance of that focus. Since this program was evaluated, the intake classification has been changed and modified at these two agencies, with an even more detailed
Assessment
of Imminence
of Risk
255
analysis of risk factors going into a determination of risk at intake. Further research will be needed to assess whether this results in better targeting of families at risk of placement, or whether the “gut feeling” factor is indeed enough.
References AuClaire, P., & Schwartz, I. M. (1986). An evuluarion of rhe effectiveness of intensive home-based services as an alternative to placement for adolescents Minneapolis: University of Minnesota. Hubert H. and their families. Humphrey Institute for Public Affairs. Behavioral Sciences Institute. (1987). Summary of King, Pierce, Snohomish, and Spokane County Homebuilders service, September 1, 1986 - August 31, 1987. Federal Way, City of Publisher, Washington: Author. Berry, M. (1990). An evaluation of family preservation services: Fitting agency services to family needs. (Manuscript submitted for publication). Johnson, W., & L’Esperance, J. (1984). Predicting the recurrence of child abuse. Social Work Research and Abstracts, 20,21-3 1. Katz, M. H., Hampton, R. L., Newberger, E. H., Bowles, R. T., 8z Snyder, J. C. (1986). Returning children home: Clinical decision making in cases of child abuse and neglect. American Journal of Orthopsychiatry, 56,253-263. Kinney, J., Haapala, D., Booth, C., & Leavitt, S. (1988). The Homebuilders model. In J. K. Whittaker, J. Kinney, E. M. Tracy, & C. Booth (Eds.), Improving practice technology for work with high-risk families (pp. 37-67). Hawthorne, New York: Aldine de Gruyter. Kinney, J. M., Madsen, B., Fleming, T., & Haapala, D. A. (1977). Homebuilders: Keeping families together. Journal of Consulting and Clinical Psychology, 45667-673. Landsman, M. J. (1985). Evaluation of fourteen child placement prevention projects in Wisconsin, 1983-1985. Iowa City, Iowa: National Resource Center on Family Based Services. Lindsey, D. (1991). Factors affecting the foster care placement decision: An analysis of national survey data. American Journal of Orthopsychiatry, 61, 272-281. Magura, S., & Moses, B. S. (1986). Outcome measures for child welfare services: Theory and applications. Washington, DC: Child Welfare League of America. National Resource Center on Family Based Services. (1988). Family bused services: A national perspective on success and failure. Iowa City, Iowa: National Resource Center on Family Based Services. Nelson, D. (1988). Recognizing and realizing the potential of “family preservation.” Miami, Florida: Edna McConnell Clark Grantees Conference. Norman, A. (1985). Keeping families together: The case for family preservation. NY: Edna McConnell Clark Foundation.
256
Berry
Pecora, P. (1988). Evaluating risk assessment systems: Methodological issues and selected research findings. In P. Schene & K. Bond (Eds.), Research issues in risk assessment for child prorecrion (pp. 000-000). Denver, CO: American Association for Protecting Children. Pecora, P. J., Fraser, M. W., Haapala, D., & Bartlome, J. A. (1987). Defining family preservation services: Three intensive home-based treatment programs.
Salt Lake City: University of Utah Social Research Institute. Pelton, L.H. (1989). For reasons of poverty. New York: Praeger. Reid, W. J., Kagan, R. M., & Schlosberg, S. B. (1988). Prevention of placement: Critical factors in program success. Child Welfare, 67, 25-36. Remy, L. L., & Hanson, S. P. (1983). Evaluation of the Emergency Family Care Program: Final report. San Francisco: Children’s Home Society. Ruger, J. B., & Wooten, R. H. (1982). A developmental approach to helping families at risk. Social Casework, 63, 3-14. Showell, W. H. (1985). 1983-85 biennial report of CSD’s Intensive Family Services. Salem: State of Oregon Children’s Services Division. Showell, W., Hartley, R., & Allen, M. (1987). Outcomes of Oregon’s family Salem: State of therapy programs: A descriptive study of 999 families. Oregon Children’s Services Division. Wald, M., & Woolverton, M. (1990). Risk assessment: The emperor’s new clothes? Child Welfare, 69, 483-5 11.