Children at risk: rationales for risk assessments and interventions

Children and Youth Services Review 26 (2004) 1155 – 1173 www.elsevier.com/locate/childyouth

Children at risk: rationales for risk assessments and interventions Rujla Osmo *, Rami Benbenishty Paul Baerwald School of Social Work, Hebrew University, Mount Scopus, Jerusalem, 91905, Israel Available online 22 June 2004

Abstract In this study, we examine and compare the rationales that social workers and laypersons give for their judgments, assessments and decisions in cases of alleged child abuse and neglect. We have used Toulmin’s framework to explore the structure of the arguments/rationales of social workers and laypersons. We analyzed separately basic and complementary level arguments. We used Rosen’s work to analyze the content of the backing provided by respondents for their claims. We presented two groups of respondents—52 social workers and 50 laypersons—with a case vignette that was a referral of a child and family made by a family physician. Respondents were asked to read the case, provide their assessment of risk to the child, recommend an intervention, and give written and detailed rationales for their assessment and recommended intervention. The findings indicate that social workers and laypersons differ in the structure and in the content of their rationales. Social workers formulated significantly more complete arguments, both basic and complementary levels of arguments, than laypersons. The content of the backing for the judgments was also significantly different: Social workers used more theory, experience and policy than the laypersons who used more general knowledge and values. In the discussion, we address the training implications of the fact that social workers pay very little attention to rebuttal of arguments, and do not refer to research evidence as a basis for the arguments they make. We recommend that more research focus on arguments and rationales given by professionals in this area. D 2004 Elsevier Ltd. All rights reserved. Keywords: Children; Risk assessment; Interventions; Rationales

Social services make major efforts to identify children and families at risk, to assess the level of risk for future abuse and neglect, and to respond with the appropriate intervention. Social work practitioners in this area are entrusted with making crucial decisions with far* Corresponding author. E-mail address: [email protected] (R. Osmo). 0190-7409/$ - see front matter D 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.childyouth.2004.05.006

1156

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

reaching consequences for families and children. One of the most difficult decisions social workers face is whether or not to remove a child at risk from his/her family. Based on the information available to them the social workers need to assess to what extent the child is at risk for future abuse and neglect, and to determine whether the child’s removal from home is the best intervention. With these high stakes, professionals would prefer to be supported by a solid base of empirical knowledge or clear policy and practice guidelines. Structured risk assessment instruments can sometimes support the decision process. Several such instruments showed reasonable reliability and validity (e.g. Lyons, Doueck, & Wodarski, 1996). In many other cases, however, practitioners in this area need to rely on less structured clinical judgments. Previous studies have identified difficulties in defining the risk a child faces (Costin, Karger, & Stoesz, 1996; Merrick, 1996). One of the important sources for these difficulties is the lack of empirical evidence connecting between information available when assessments are made and future maltreatment (i.e., the ‘predictive validity’ of information cues, see Kaplan, Pelcovitz, & Labruna, 1999). Due to many conceptual and practical problems, research is also limited in providing clear evidence on the relationships between the specific intervention chosen and the outcomes for the child and family (e.g., Parker, 1998). Furthermore, Lindsey (1992) showed that reliability among practitioners who make placement decisions in child welfare is quit low and results in error in judgment. Similarly, Ruscio (1998) reviews the findings on clinical judgment in general and in child welfare in particular and demonstrates the limitations of these unaided judgments and decisions (see also Packman, 1989 for observations on difficulties in decision making in this area). The combination of the importance of the decisions regarding children at risk and the fallibility of judgments focuses attention on social work professionals who must make crucial decisions with regard to children at risk even under these unfavorable circumstances. The judgment process and decision-making strategies of social work practitioners in this field need to be studied and critically examined. We need to ascertain what type of issues they consider in their deliberations, what are the rationales they provide for their conclusions, and what is the basis for these rationales. Our interest in the workers’ rationales stems from the fact that when the rationale for a decision is articulated in detail, it is possible to judge its justification, to learn from it, and perhaps to identify mistakes that need to be rectified (Osmo & Landau, 2001). By capturing the implicit policies of social workers and their judgment processes with regard to child abuse and neglect, it may be possible to open them to professional and public discussion and thereby improve the quality of these decisions (Benbenishty, 1992). In this study, we examine the content and structure of rationales that social workers in local ‘welfare agencies’ in Israel gave for their assessments and decisions on a case of alleged child abuse and neglect and we compare between these professional workers and laypersons. As a background, we first provide basic information on the Israeli context of decision making in the area of child abuse and neglect. The backbone of the Israeli welfare system are local ‘welfare agencies’, each covering a specified neighborhood. Within these local agencies there are ‘generalist’ social workers

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1157

assigned to client families. The minimal requirement for social work professionals in Israel is to study for three years in a school of social work in one of the universities and to obtain a B.SW. These workers are responsible for interacting with the families, assessing their needs and making decisions about services. They receive ongoing personal supervision and participate in in-service training on issues relevant to families and children. As part of their family-oriented work, these workers identify children that may be in need of services or may be at risk for harm. In many ways they serve as one of the most important gatekeepers of the child welfare system. In the majority of cases, when there is no clear case of immediate danger to the child, these generalist social workers in the social welfare agency assess the risk to the child and decide whether there is a need for further intervention. If they think that the situation of the child in the family warrants it, they bring the case to an interdisciplinary ‘decision committee’ convened in the welfare agency to discuss the intervention plan. To this conference they bring their assessments and recommendations, among them whether to remove the child from home, and they take an active role in the decision making process. In the relatively few cases in which they observe (or are informed of) significant and immediate risk to the child, they report to ‘child protection’ officers. These officers are specially trained social workers who have the responsibility and authority to conduct emergency investigations, remove a child from a dangerous home, bring cases to court, etc. Even when cases are brought directly to the attention of child protection officers (e.g., by concerned neighbors), these officers consult with the relevant family workers, often instructing them to conduct home visits, assess the situation and recommend a course of action. In summary, it is important to emphasize that social workers in our study are university-trained professionals (BSWs and MSWs) who routinely make important judgments and decisions on children that may be at risk for abuse and neglect.

1. Social workers and laypersons One way to study the decision-making processes of these professionals is to compare them with others who do not have any special training and expertise in this area. Such a comparison can identify the unique characteristics of the judgment processes of professionals, not shared by the public. Comparisons between professionals and nonprofessionals and between experts and novice are common and have a long history in research on decision-making in general and in the helping professions in particular. Intuitively, it seems that trained professionals think differently and reasons in ways that are different from those who were not trained. The findings in this area, however, are quite mixed and inconclusive (see a review in Bolger & Wright, 1994). Rowe and Wright (2001) evaluated nine empirical studies that compared judgments of risk made by laypersons and experts in various areas. They conclude that these studies find little evidence to assumptions that experts reason differently and are more accurate than laypersons. In a seminal series of studies, Meehl (see a review in Dawes, Faust, & +ACYMeehl, 1989) compared the clinical judgment of experienced clinicians with those of novices and non-professionals and showed many of the limitations inherit in clinical judgment, and raised serious doubts as the superiority of judgments made by experienced

1158

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

clinicians’ over judgments made by others. In the area of child abuse, Ganzach (1994) examined risk assessments in cases of alleged abuse and neglect and compared between experienced social workers, social work trainees and business administration students. He found that experts and laypersons used different rules of integrating the available information in order to reach their judgments and recommendations. He concluded that experts and laypersons use different theories about the relationships between case characteristics and appropriate judgments. Similarly, Benbenishty, Segev, Surkis, and Elias (2002) examined whether information on physical abuse had a different impact on these three groups. The study showed that judges with no social work training tended to assess higher risk, recommend more removal of child from home and attach heavier weight to information on physical abuse than judges who received social work training. In the area of sexual abuse, Peters (2001) compared a group of professionals with experience in conducting evaluations with a group of students from a college level introductory psychology class. The author examined whether several information cues had differential effects on the judgments made by raters from the two groups. In Peter’s study, judgments made by the professionals and by the students were quite similar. In our study we follow this research tradition and compare between professional social workers and others who were not trained in this area. An underlying assumption of this study is that differences between professionals and laypersons in the structure and content of rationales are an indicator that social work training and specialized expertise have indeed influenced professional decisions, rather than values and common sense alone. Thus, differences between the two groups are seen as indications of the differences that professional training and experience make in this area. In this area of practice we also expect certain similarities between professionals and laypersons. The judgments and decisions made by social workers may reflect their expertise, based on training and experience (Benbenishty, 1992; Drury-Hudson, 1999). This ‘technical’ expertise, however, operates within a social context. Child abuse is, to a large extent, a socially constructed problem. Its definition depends on the public discourse and on societal norms with regard to issues such as ‘familialism’, ‘parents’ rights’, ‘children’s rights’, and the relative role of society and family as responsible for raising children (Fernandez, 1996). Hence, the responses of laypersons reflect the normative climate, which professionals need to consider in their judgments and decisions. Public outcries arising on adverse outcomes of a professional decision shape future decisions (Munro, 1998; Stevenson, 1998). When social workers know that they will have to justify their recommendations in public (e.g., in court or a public inquiry), they carefully consider what is acceptable to the ‘man in the street’, i.e., the layperson. Consequently, one would expect that the values included in the rationales provided by professional social workers may not be all that different from these of laypersons. Furthermore, some researchers suggest that professional judgments in the area of child abuse are actually a mere reflection of the layperson’s views. Shapira and Benbenishty (1993) raised the possibility that the judgment models of the professionals they studied may have reflected popular lore regarding children at risk rather than professional expertise. Mass (1994) argued that when social workers testify in court on what is best for children, they rely on their value system, just as laypersons do, rather than on a body of professional knowledge, as experts should.

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1159

This line of thinking is tested in this study by comparing social workers’ and laypersons’ rationales.

2. Conceptual/methodological framework for studying rationales In their epistemological study of the process of explaining behavior, Locke and Pennington (1982) distinguish between ‘looking for causes’ and ‘finding reasons.’ In the first approach, one observes from the outside and looks for empirical regularities that reveal causal relationships between behaviors. The latter approach is to ask people why they behaved in a certain way (i.e., ask for arguments or reasons for actions and decision). This distinction has a parallel in the area of decision-making. Judgment analysis and policy capturing (e.g., Cooksey, 1996) focus on statistical modeling of the relationship between information and judgments (input– output analysis). Process tracing (Klayman, 1984), especially ‘think aloud’ methods (Ericsson & Simon, 1980), deal with the internal mechanisms that lead a person from information to judgment. This approach is similar to the one utilized in the present study that focuses on the rationales or arguments that lead from given information to conclusions based on this information. By argumentation we mean the activity of making claims, challenging them, backing them with reasons, criticizing these reasons, and rebutting the criticism offered (Toulmin, Riecke, & Janik, 1984). Kuhn (1991) claims that much of our deliberations involve silently arguing with ourselves and evaluating the arguments for and against a course of action or decision. We have used Toulmin’s framework to explore the structure of the arguments/rationales of social workers and laypersons (Toulmin et al., 1984). This framework emphasizes the structure of the arguments, i.e., configuration of the components, which allows description and breakdown of an argument/rationale into its constituent components or content. We define argument here as an assertion/statement and its accompanying justification. Toulmin identifies six types of statements. The first three comprise the basic argument: (1) a claim or a conclusion, we seek to establish by our argument; (2) we provide data, evidence, or grounds (the particulars of the situation) for the foundation of the claim; (3) we warrant or justify the connection between the data and the claim by appealing to a rule of inference, which establishes that the evidence does apply in this particular case. An example of a basic argument in this area is: ‘‘this child is at risk’’ (claim); ‘‘his parents let him roam the streets unattended’’ (evidence); ‘‘our professional experience has shown that unsupervised children at this age tend to harm themselves and are involved in deadly incidents’’ (warrant). The other three types of statement make up the complementary argument: (4) we back up, justify or otherwise support a warrant by appealing to generalizations which make explicit the body of experience used to establish the trustworthiness of the warrant, e.g. empirical data, common knowledge, professional practice, or scientific theory; (5) we qualify a claim or conclusion by expressing degrees of confidence and likelihood we can attach to our claim; (6) rebuttal of a claim or conclusion is made by stating the conditions under which it does not hold or by introducing reservations showing the limits within which the claim is made. An example of a complementary argument is ‘‘empirical findings

1160

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

show that neglected children are more disposed to suffer physical and emotional injuries’’ (backing); ‘‘ I am absolutely certain that this is a risky situation’’ (qualify); ‘‘if the parents allow the child to cross the street only when accompanied by an adult, the risk would be lower’’ (rebuttal). In addition to analyzing arguments in terms of their structure, we study their content. Toulmin et al. suggest that the criteria for assessing the content validity of an argumentation are ‘field-dependent’. That is, argumentation must be assessed according to the particular norms that are adequate in the field to which the argumentation refers. Hence, appropriate backing for warrants differs from field to field. In our case, the validity is judged in terms of the characteristics of the social work profession. In this field, we find Rosen’s work very helpful. This work analyzes social workers’ use of knowledge in making and in justifying their practice decisions (Rosen, 1994; Rosen, Proctor, MorrowHowell, & Staudt, 1995). Rosen (1994) defined knowledge used by social work practitioners ‘as the contents of the rationale that workers give for practice’ (p. 566). Rosen identifies several types of knowledge used by workers in justifying their practice decisions: conceptualization/theory; policy; value/norm; imperative claim/assertion; client wish; empirical evidence; practice experience. In this study, we use an elaboration of this conceptual tool to analyze the content of the ‘backing’ component. The role of the backing component in arguments is to explicate the body of knowledge and experience used to establish the trustworthiness of the argument. Toulmin’s conceptualization helps us to identify the existence of the backing component and Rosen’s framework is used to analyze its content. Our interest is in the structure and content of the arguments used by professionals. As a backdrop and contrast, we compare rationales made by this group of professional social workers to those of laypersons. Such comparisons were not made often in the area of children at risk. Notable exceptions are the studies by Mandel, Lehman, and Yuille (1994, 1995). These researchers have studied how social work professionals in the area of child abuse and neglect make the decision whether to remove a child from home, and compared them to undergraduates and to police officers. They presented a short sketch of a home visit that ended with the decision to remove a child from home. The respondents indicated their level of agreement with the decision and provided their reasons and the features of the case that led to their response. The authors found that professionals made fewer unwarranted assumptions, and requested more information, but generated a similar number of hypotheses. Surprisingly, despite the minimal and inconclusive evidence, a similar proportion of professionals agreed with the decision to remove a child from home as of the undergraduate students. Among the police officers, the decision to remove the child on the basis of the minimal evidence in the vignette was even higher. These studies have emphasized the importance of critical examination of the ways social work professionals reason in the field of child abuse compared with others. Our study expands this research in several ways. In contrast to the studies by Mandel et al. (1994, 1995), the respondents in our study were asked to indicate their own decision, rather than their agreement with an existing decision. Previous studies strongly suggest that such an existing decision may have an anchoring effect that may bias the judgment (Tversky & Kahneman, 1974). In order to provide enough information so that respondents could reach their own decision, we presented a much more detailed and rich vignette than

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1161

the short one provided by Mandel and his associates. The complete case-vignette provided social workers with enough information to construct complete arguments to support their evaluations and recommendations. We were therefore able to apply Toulmin’s conceptual framework to the detailed arguments provided by the participants. Our general assumption in this study is that social workers’ justification of their decisions, based on theory and supported both by professional experience and empirical evidence, distinguish between professionals and laypersons. We therefore hypothesized that social workers would provide rationales that differ in structure and content from those provided by laypersons. Specifically, we hypothesized that social workers would provide more complete arguments (i.e., both basic level and complementary), and will use more theory, professional experience and empirical evidence in backing up their arguments. We focus on the differences with regard to (a) risk assessment, and (b) recommended intervention. This study is part of a larger international research project on judgments and decisions regarding child abuse and neglect (Gold, Benbenishty, & Osmo, 2001).

3. Methods In this study we used a ‘‘critical incident’’ approach to study the rationales made by the participants. This is a common strategy for studying decision-making (e.g., Drury-Hudson, 1999; Osmo & Rosen, 2002). We presented two groups of respondents—social workers and laypersons—with a case vignette, which was a referral of a child and family. Major efforts were made to ensure the authenticity and representativeness of the vignette. The case was a composite derived from authentic files. It was intended to depict an ambiguous situation that requires careful deliberation and discretion. Briefly, the family described in the vignette contains a couple and their three young children (ages 7, 4, 2). The family has few ties within the community, lacks a supportive family network and is struggling financially due to a failed business venture. The local welfare agency receives a call from the family physician in which he expresses suspicion about the origin of bruises on the body of the oldest child. This boy is described as an average child without many friends who occasionally has violent outbursts towards other classmates. In the past, the child has had a dislocated shoulder and a fractured arm. At the age of three he was burned when according to his mother he pulled a hot kettle down from the kitchen counter. In a home visit the mother denies any physical aggression toward the child, explains the bruises as outcome of skateboarding and expresses anger towards the interviewing social worker and social services in general (the complete case is available upon request from the senior author). The vignette was examined by five experienced workers and modified according to their suggestions. Respondents were asked to read the case; to make their assessment of risk to the child, recommend an intervention, and provide a rationale for their assessment and recommended intervention. 3.1. Sample The sample consisted of 102 respondents—52 social workers and 50 laypersons. A convenience sampling was used. We asked the heads of three social welfare neighborhood

1162

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

agencies in Jerusalem (with whom we had personal and professional relationships) for permission to approach their workers during a staff meeting. We asked the social workers attending the meeting to participate in anonymous study and to complete the questionnaire during a special time set during the meeting for this study. The social workers were all professionally trained and certified social workers, with at least a BA in social work. More than 90% of them completed their questionnaires. No monetary compensation was offered. A member of the research team returned to these agencies to report the findings and discuss their implications. The laypersons were randomly sampled from two neighborhoods, one in Jerusalem area and the other from city of Modiin, and from the administrative staff of a large university. The rationale for the selection of the laypersons’ sample was to reach a sample of adults without professional training in a social work related field. In the selected neighborhoods we put questionnaires in mailboxes with return envelopes. Our return rate was very low—30 questionnaires (about 30% return rate). We also personally approached professional administrative staff in our university and asked for their participation. With this group we received much better response rate (80%) and we added 20 participants. The decision to select one or another way to give out the questionnaires was based on the most efficient approach to reach this sample. Unfortunately, due to the anonymous nature of the responses we could not separate the two groups of our layperson sample in the analyses. All respondents gave their informed consent to participate in the study and were not compensated.The length of time to complete the questionnaire was between 25 and 45 min. All 52 professionals were women. Six of the 50 laypersons were men. The mean age of the professionals was 33.9 years (SD = 6.83, range 24 –52) and of the laypersons 41.2 years (SD = 12.19, range 21 –66). The family status was similar in the two groups—about 75% were married. About 43% of the laypersons had only a high school diploma and the other 57% had a university degree. In contrast, all professionals had at least a BSW (76%) or MSW (24%) degree. None of the laypersons have professional training in a social work related field. None of them reported on any personal involvement with children at risk. Among the social work professionals, 26% said they had no prior experience working with children at risk, 49% said they had some experience and 25% said they worked extensively in this area. 3.2. Measures 3.2.1. Claims Following the work by Shapira and Benbenishty (1993), two separate professional decisions, or claims, were studied: (1) The degree of risk to the child—This was measured on a Likert type scale, ranging from 1 (‘no risk at all’) to 7 (‘very high risk’). (2) Intervention recommendation—Participants were asked to select an intervention that best suits the needs of the child and the family. The list of options represented a continuum on an ‘obtrusiveness scale’ (Shapira & Benbenishty, 1993). The following were the possible intervention choices: (1) Discontinue contact with the family with

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1163

no further intervention. (2) Continue monitoring the situation indirectly through agencies already in contact with the family. (3) Direct casework with the family without removing the child from the home. (4) Provide an extended framework for the child till the evening hours and continue direct work with the family. (5) Work towards removal of the child from the home with the parents’ consent through continued casework. (6) Removal of the child from the home, even without parental consent, by court order if necessary, while continuing direct casework with the family. The risk assessments and intervention recommendations were studied elsewhere (Gold et al., 2001). Therefore, this paper will report only on the findings that pertain to the rationales provided for these two types of decisions (claims). 3.2.2. Rationales given for the risk assessment Unstructured request: We requested the respondents to give their rationale for their responses immediately after they had assessed the risk: ‘‘What led you to this assessment of risk? Please describe in detail your rationale for this assessment’’. 3.2.3. Rationales given for the recommended interventions Unstructured request: After the respondents chose an intervention, they were asked to give their rationales for the recommended intervention: ‘‘Above, you have indicated your recommended intervention. Now we would like to understand what led you to your recommendation. Imagine that you are requested to present this case and this recommendation in a case conference. Please explain in detail the rationale/reasoning that led you to make this specific recommendation’’. Structured request: We were concerned that our respondents may not articulate their rationales in sufficient detail. We therefore added structured questions based on Toulmin’s six components, to elicit each one of the components, if these actually existed in the participants’ minds. Examples of such structured requests are: What case characteristics influenced your decisions? How do you explain the relationship between the case data that you based your decision on and your choice of intervention? What would have to be different in the case description to cause you to suggest an alternative course of action? We used a structured request only with regard to the intervention recommendations, so that we do not overburden the participants. The instruments were pre-tested by five senior social workers and five laypersons, and minor adjustments were made. 3.3. Coding of the rationales Data on structure and content of the rationales were derived from content analyses of the rationales participants provided in the open-ended responses. 3.3.1. The structure of rationales We reviewed the participants’ written statements provided as rationale to their assessments of risk and recommended interventions and looked for instances of each of the six

1164

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

different components of the argument based on Toulmin’s conceptualization (Toulmin et al., 1984). We counted the number of occurrences of each of the following components: claim—response that answers the question ‘‘what is it you are claiming?’’; evidences— ‘‘what data do you have?’’; warrant—‘‘why is your conclusion warranted?’’; backing— ‘‘what is the justification for your line of reasoning?’’; qualifier—‘‘how likely is it that what you say is correct?’’; rebuttal—response that answers the question ‘‘under what conditions would your argument break down causing you to change your mind?’’. Thus, each participant was described by the number of the occurrences of each of these six components. 3.3.2. The content of the backing component This analysis was performed on the written statements that we classified as backing components in the analyses of the structure of rationales described above. We reviewed these statements and looked for instances of each of the six distinctive categories of use of knowledge based on an elaboration of Rosen’s (1994) conceptualization. The six categories used in this study were defined as follows: general knowledge—statements which relate to acquired personal knowledge articulated in general terms without specifying its origins; theory—statements which invoke or mention by name a theory or a theorist or consists of a link between two or more professional concepts; policy— any statement referring to rules, norms or directives of the service, agency, or other administrative unit; value—the statement of a value, or a categorical assertion of universal norm or truth; empirical evidence—asserting research evidence (published or not) or referring to a ‘‘study’’; practice experience—mention of a worker’s or colleagues’ practice or the collective practice experience of several colleagues. Thus, each participant was described by the number of the occurrences of each of these six components. The senior investigator trained a social worker to code arguments contained in pre-test questionnaires, based on definitions and instructions written by the investigators (available upon request). Coding of the study data commenced after reaching an inter-rater reliability of 85% between the coder and the senior author. A third coder (doctoral student in social work), who did not participate in the refinement of the coding system, also coded 8% of responses to check for possible unwritten agreements and for decay. With respect to structure of rationales, there were 84% agreement for risk assessment and 91% agreement for recommended intervention with the coder; with respect to the content of the backing component, there were 82% agreement for risk assessment and 93% for recommended intervention with the coder.

4. Results 4.1. Structure of rationales We first present our analysis of the structure of rationales given for assessment of risk and for the recommended intervention. We explore this issue from two angles—(a) whether the respondents mentioned a certain component, and (b) how many instances the

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1165

respondent used each component (for example, the number of different evidences or number of different contents of warrants the respondent provided). This was done in order to estimate the prevalence of each component in the formulation of the rationales by the two groups of respondents. 4.1.1. Assessment of risk In both groups the claim that the child is at high risk was the mode (44% of the respondents; 26 social workers and 18 laypersons). There was no significant difference between the two groups in their risk claim. Table 1 presents: (a) the percentage of social workers and laypersons who used each of the six argument components in assessing risk to the child, and (b) the means and SDs of the number of instances each component was used by each group. The components used most by the two groups were claim and evidence given for the claim. The least used was rebuttal. Social workers used three components significantly more than the laypersons: evidence (chi-square = 4.33, df = 1, p < 0.05); warrant (chisquare = 11.57, df = 1, p < 0.001); and backing (chi-square = 3.92, df = 1, p < 0.05). There was only one significant difference in the number of instances each component was used in the two groups: Social workers gave more instances of evidence to support their risk assessment than did laypersons (means—11.17 vs. 5.76, respectively, t(96) = 4.39, p < 0.001). We also examined the level of formulation of the rationales—basic level (the intersection of the components claim with evidence and warrant) and complementary level (the intersection of the components back up, rebuttal and qualify the claim). Almost all social workers (96%) formulated a basic level argument vs. only 68% of the laypersons. Only 6% of social workers and 2% of laypersons formulated a complementary level of arguments. 4.1.2. Intervention recommendations The most common recommended intervention (claim) by both groups was (4) ‘direct intervention with the family and place the child in a day placement until the evening’ (65% Table 1 Rates, means and SD of argument components for risk assessment by social workers and laypersons Social workers

Laypersons

N

%

Mean

SD

N

%

Mean

SD

52 52 49

100 100 96

11.17 2.37

7.68 1.20

50 46 34

100 92 68

5.76 2.12

3.50 2.01

Complementary level Backing 36 Rebuttal 6 Qualify 16

71 12 31

1.97 1.20

0.88 0.04

25 4 12

50 8 24

1.56 1.00

0.96 0.00

Basic level Claim Evidence Warrant

With respect to the claim component of risk assessment (1 = no risk to 7 = extreme risk), 5 high risk was the mode; with respect to component of qualifying (1 = low to 3 = high confidence), 2 was the mode.

1166

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

of the respondents, 36 social workers and 29 laypersons). There was no significant difference between the two groups. Table 2 presents the percentage of participants who used each of the components in their rationales for recommending an intervention, as well as the means and SD of the number of instances each component was used by each group. The components used most often were claim and warrant, the least used was rebuttal. Significantly more social workers used evidence (chi-square = 6.21, df = 1, p < 0.01) and warrant (chi-square = 4.05, df = 1, p < 0.05). In addition, social workers gave more instances of evidence (means—5.27 vs. 3.00, t(84) = 3.81, p < 0.001) and warrants (means—3.02 vs. 2.25, t(90) = 2.74, p < 0.01) for the selected intervention than laypersons. Most of the social workers (94%) formulated a basic level argument (the intersection of the components claim with evidence and warrant) for the selected intervention vs. 76% of the laypersons. Only 13% of social workers and none of the laypersons formulated a complete extended level of argumentation (the intersection of the components back up, rebuttal and qualify the claim). 4.2. Categories of knowledge in backing the rationale In addition to studying the structure of the rationale given by the participants, we examined one aspect of their content—types of knowledge they mentioned as backing their assessment of risk and recommended interventions. Table 3 presents the frequency distribution of the backing component used by social workers and laypersons in their rationales for risk assessment and for the recommended intervention. Because the respondents used more than one category of knowledge in each rationale, a category was counted even when it appeared in combination with other categories.

Table 2 Rate, mean and SD of argument components of the intervention recommended by social workers and laypersons Social workers

Laypersons

N

%

Mean

SD

N

%

Mean

SD

50 48 48

98 94 94

5.27 3.02

3.37 1.40

49 38 44

98 76 88

3.00 2.25

1.63 1.22

Complementary level Backing 41 Rebuttal 10 Qualify 23

79 19 44

1.68 1.50

1.01 1.10

36 5 19

72 10 38

1.81 1.0

1.09 0.00

Basic level Claim Evidence Warrant

With respect to the claim component for the recommended intervention (1 = no further intervention to 7 = removal of the child from the home for an extended period of time even without parental agreement), 4 ‘direct intervention with the family and place the child in a day placement’ was the mode; with respect to component of qualifying (1 = low to 3 = high confidence), 2 was the mode.

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1167

Table 3 Frequencies of the backing component by Rosen’s categories, type of claim and groups of respondents Categories

General knowledge Theory Experience Values Empirical knowledge Policy

Risk assessment

Recommended intervention

SW (52), %

Layperson (50), %

Total (102), %

SW (52), %

Layperson (50), %

Total (102), %

48 32 10 8 0 0

38 14 0 2 0 0

44 24 5 5 0 0

31 35 21 15 0 12

44 14 14 40 0 2

37 25 18 28 0 7

The total is less than 100% since some of the participants did not address the backing component.

4.2.1. Risk assessment The category used most often in risk assessment rationales was general knowledge; value and experience were used the least. Empirical knowledge and policy were not used at all. Laypersons did not use the experience category and have significantly lower scores than social workers on the theory category (14% vs. 32%; chi-square = 4.95, df = 1, p = 0.026). 4.2.2. Recommended interventions The category used most often in backing the recommended intervention was general knowledge, while policy was the least used. Empirical knowledge was not used by any of the participants. Laypersons did not use the policy category, as well. They used significantly more values than social workers (40% vs. 15%; chi-square = 7.75, df = 1, p = 0.005). Social workers used significantly more theory-based knowledge than laypersons (35% vs. 14%; chi square = 5.85, df = 1, p = 0.016). 4.2.3. Rebuttal of the rational We examined to what extent participants used rebuttals in their rationales regarding the assessment of risk and recommended interventions. As mentioned in Methods, the participants responded (a) to an unstructured open invitation to provide a rationale for their risk assessments and recommended intervention and (b) to a structured question designed to elicit all the components of the rationale for the intervention they recommended. Our analysis indicates that when respondents were not specifically prompted, very few used rebuttals in rationales for their risk assessments (six social workers and four laypersons), and for their recommended interventions (six social workers and five laypersons). These few rebuttals concerning the interventions were added to those in the response to the structured question specifically requesting reference to the various components of the rationale for recommending the intervention. We found that 44 social workers (85%) provided rebuttals when prompted, with a mean number of rebuttals of 1.73 (SD = 1.02). Thirty-five of these workers (80%) formulated a rebuttal recommending a more intrusive intervention than the intervention given before. Thirty-seven laypersons (74%) provided rebuttals, with a mean number of rebuttals of 1.51 (SD = 0.73) and 25 of them (68%) formulated a more intrusive

1168

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

intervention. These differences between social workers and laypersons were not significant ( p>0.05). 4.2.4. Qualifying the rationale Sixteen social workers (31%) and 12 laypersons (24%) indicated the degree of confidence they had for the evaluated risk (low, moderate or high confidence). A high degree of confidence was the mode (13%). There were no significant differences in the degree of confidence between the two groups. Twenty-two social workers (44%) and 19 laypersons (38%) indicated the degree of confidence they had for the recommended intervention. A moderate degree of confidence was the mode (21%). There were no significant differences in the degree of confidence between the two groups. In additional analyses we examined possible differences on the structure and content of the risk evaluation and preferred intervention rationales among the social workers, due to differences on previous work experience with children at risk and on level of education (MSW vs. BSW). No significant differences were found ( p>0.05). We also examined possible differences among the laypersons due to differences in education level. No significant differences were found ( p>0.05).

5. Discussion Social workers who make decisions about children at risk have extensive discretionary powers. These powers are based on the presumption that they make decisions on the basis of professional expertise. In this study, we hypothesized that judgments and decisions made by social workers in the area of child abuse and neglect would reflect their expertise (Benbenishty, 1992; Drury-Hudson, 1999) and that rationales provided by social workers would therefore differ in structure and content from those provided by laypersons. This study is, to the best of our knowledge, the first to analyze the content of the arguments and rationales provided in making decisions in the area of children at risk. Its findings therefore should be seen as tentative and exploratory. Furthermore, the limitations of this study should not be overlooked. The study used one case vignette and convenience samples in Israel. Replications of this study with a range of cases, more representative samples, and in other countries are necessary in order to examine whether our findings can be generalized to other contexts and countries. The findings indicate that social workers and laypersons do indeed differ in the structure and in the content of their rationales. Social workers formulated significantly more basic and complementary levels of arguments than laypersons, both for the risk assessments and for recommended interventions. These differences related to aspects that seem most relevant to expertise in a specific domain: social workers gave more specific evidence from the case material as grounds for their claims, connected the evidence to their claims, and made explicit the knowledge on which they relied. Thus, social workers generally appear to use information in a more complex manner—they take into account large amounts of information, and weigh and integrate the evidence in more complex ways than laypersons (Benbenishty, 1992).

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1169

However, when social workers were not specifically prompted for particular types of argument components, they used fewer components on the complementary level of the argumentation process than one would hope. The complementary level of arguments, that focuses on the backing up of warrants and on the rebuttal and qualifying of the claim, reflects skills that are based on education and training and are important requirements for professional reasoning. Professional training is expected to help social workers recognize the deeper sources from which the reasoning of their profession draws its power (Toulmin et al., 1984). An important part of sound reasoning of professionals consists of ‘‘critical thinking’’, both about the claims they make and how they arrived to these claims. Thus, we expected that professional social workers would base their rationales significantly more on this level of argumentation. The results seem to suggest that professional social workers may not be using the kind of complex reasoning we tend to attribute to them (e.g., DruryHudson, 1999; Shanteau, 1992). This finding is of special importance with regard to the absence of the rebuttal component for the claims made by social workers. As Kuhn (1991) observed—‘‘to recognize that there are alternatives that stand in contrast to one’s own is the first vital step to (a) recognizing that one’s could be wrong and, therefore, (b) seeking to evaluate to what extent it is correct’’ (p. 114). The low number of rebuttal claims provided spontaneously may indicate that social workers are vulnerable to the confirmation bias. That is, they may search and gather information that is consistent with the professional’s initial hypothesis, coupled with inattention to information that is inconsistent with, or could discredit, the hypothesis (e.g., Haverkamp, 1993; Snyder & Thomsen, 1988). This bias may have negative impact on their decision making process. For instance, social workers may formulate their initial hypothesis based on the referral information, and may not be open enough to rebuttal their hypothesis in their home studies and investigations. Asking the respondents direct and specific questions on their rationales with respect to recommended interventions, we elicited more complete responses relating to the rebuttal component—85% of social workers and 74% of laypersons provided rebuttals. One way to interpret these findings is to assume that rebuttals have been implicitly considered and were part of the respondents’ repertoire in their deliberations, but were not articulated explicitly by them unless requested specifically. On the other hand, it is possible that rebuttal is not a regular part of the thinking process, and the participants responded to the demand characteristics of this study by ‘making up’ the rebuttal arguments. Recent findings by Osmo and Rosen (2002) are also not unequivocal: in their study social workers tended to prefer a confiramtory information search stragetgy, but were cautious and critical in the ways they integrated information to reach their conclusions. This issue needs to be further explored because of its critical implications in professional judgment and decisions. In this study we investigated what social workers and laypersons consider to be the basis and backing for their judgments and decisions. General knowledge was the most often used source for backing the risk assessment rationales by both groups. In most of the other components social workers differed from the laypersons: they used more theory, experience and policy than the laypersons who used more general knowledge and values as the backing for their judgments. Even though Pratt and Grimshaw (1985) found that social workers rest almost exclusively upon use of practical common sense and life-

1170

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

experience, we observed a trend by social workers to balance the use of general knowledge with theory-oriented knowledge in their assessment. This is consistent with DruryHudson’s (1999) model of professional knowledge, and with Hardiker’s (1981) empirical findings that social workers do employ social science concepts in their practice but that they are intermixed with other forms of knowledge. Laypersons used more value statements, whereas professionals used more theory-based backing for their recommended interventions. This indicates that at least on the explicit level, social workers rely on what they consider is relevant to their expertise as professionals, rather than on their personal values. Theories, however, are not value-free, especially in this area (Fernandez, 1996). Thus, the reliance on theory does not preclude the possibility that social workers utilize in their judgments the theories that are most in line with their own personal value system. Further research is needed to examine the boundaries between the value systems of practitioners and the theory they use in their work. Neither social workers nor laypersons used evidence based empirical knowledge in the backing of their rationales. This finding is similar to results reported by Rosen et al. (Rosen, 1994; Rosen et al., 1995) regarding the use of empirical knowledge. While laypersons are not expected to be aware of the relevant empirical findings, this is a very problematic finding for professional workers. Stevenson (1998) argues that, in practice, we are a long way from utilizing research concerning children’s needs at the field level where it really counts. Munro (1998) analyzes the results of 45 public inquiries into child abuse cases in Britain, and points out the problems identified in the social workers’ knowledge base, among them the lack of research evidence as an explicit basis for assessments and decisions. One possible explanation for this finding is that professionals do use empirical findings but are not accustomed to cite these results as part of colloquial speech. Fortune and Reid (1999) observe that using research results is by far a more varied, complex, subtle and indirect process than previously thought, and is not captured solely by direct reference to a particular study. Hence, it is possible that empirical findings are internalized and assimilated in the thinking process to the extent that they are considered general knowledge. Thus, when social workers said that they used general knowledge, they may have also been referring to empirical research. The other possibility is that social workers do not use research evidence in practice. Our informal reading of the language used by social workers in our study indicates that this is the more probable explanation. The overall pattern of findings indicates that workers use a blend of practice-based knowledge and theory and do not base judgments on any specific set of relevant empirical studies. Our informal observations of professional practice in Israel support this interpretation. Few workers read research-based professional journals and more are interested in published case studies and in theory-driven reviews of the issues relevant to child abuse and neglect. One possible response to this situation is an effort to summarize existing research and present it to practitioners. A prime example is the series published by the Department of Health in the UK, ‘Messages from Research’. In this series, experts analyze and summarize a series of studies in a relevant topic (e.g., ‘caring for children away from home’, ‘Patterns and outcomes in child placement’), to help make them available to

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1171

practitioners in the field. This should be even more relevant in countries like Israel in which English, the language of most research journals, is not the mother tongue. As social work educators, we find clear implications for social work training. This study highlights the need to instruct social workers to justify their professional judgments and to open the sources for their judgments to scrutiny. Furthermore, we should train workers to pay attention to the complementary level of their arguments and especially to consider rebuttals. From a training perspective, the fact that social workers responded to a prompt and generated rebuttals suggests that supervisors may need to actively engage workers in looking for rebuttals, as part of their focus on improving critical thinking skills of the workers they supervise. The work of Gambrill et al. is a prime example of such efforts to improve critical thinking (Gambrill, 1997; Gibbs & Gambrill, 1996). This study examined Israeli social workers. Nevertheless, studies conducted in other settings (e.g., in Canada, Benbenishty, Osmo, & Gold, 2003) indicate that the concerns and needs identified in this study are probably shared by social workers in other countries. There is a clear need for future studies to examine these issues in different cultures and diverse social work settings. Finally, the study strongly indicates that Toulmin’s conceptual framework provides an important tool for analyzing rationales. We recommend that this line of inquiry be utilized further and refined. Studies of the decision-making processes in this area should further explore the components of arguments provided by experts for their decisions in real-life cases. These studies would contribute to efforts to identify how decision makers reason, and to design methods to improve the internal processes that result in decisions with such a crucial impact on children and families.

Acknowledgements The authors wish to thank the respondents who have contributed so generously from their time and to the Social Welfare Department of the Municipality of Jerusalem for their collaboration. We acknowledge gratefully the important contribution of Aaron Rosen, Nora Gold and Lauren Wolfsfeld to the design of the instruments and material used in this study.

References Benbenishty, R. (1992). An overview of methods to elicit and model expert judgment and decision making. Social Service Review, 66, 598 – 616. Benbenishty, R., Osmo, R., & Gold, N. (2003). Rationales provided for risk assessments and for recommended interventions in child protection: A comparison between Canadian and Israeli professionals. British Journal of Social Work, 33, 137 – 155. Benbenishty, R., Segev, D., Surkis, T., & Elias, T. (2002). Information-search and decision-making by professionals and nonprofessionals in cases of alleged child-abuse and maltreatment. Journal of Social Service Research, 28(3), 1 – 18. Bolger, F., & Wright, G. (1994). Assessing the quality of expert judgment—Issues and analysis. Decision Support Systems, 11(1), 1 – 24.

1172

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

Cooksey, R. (1996). Judgment analysis: Theory, methods, and applications. San Diego, CA: Academic Press. Costin, L., Karger, H., & Stoesz, H. (1996). The politics of child abuse in America. Oxford, UK: Oxford University. Dawes, R. M., Faust, D., & + ACY-Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668 – 1674. Drury-Hudson, J. (1999). Decision-making in child protection: The use of theoretical, empirical and procedural knowledge by novices and experts and implications for fieldwork placement. British Journal of Social Work, 29, 147 – 169. Ericsson, J., & Simon, H. (1980). Verbal reports as data. Psychological Review, 87, 215 – 251. Fernandez, E. (1996). Significant harm: Unraveling child protection decisions and substitute care careers of children. Arebuny: Aldeshot. Fortune, A. E., & Reid, W. (1999). Research in social work. New York: Columbia University Press. Gambrill, E. (1997). Social work practice: A critical thinker’s guide. Oxford, UK: Oxford University Press. Ganzach, Y. (1994). Theory and configurality in expert and layperson judgment. Journal of Applied Psychology, 79(3), 439 – 448. Gibbs, L., & Gambrill, E. (1996). Critical thinking for social workers: A workbook. California, USA: Pine Forge Press. Gold, N., Benbenishty, R., & Osmo, R. (2001). Risk assessments and recommended interventions in instances of suspected abuse or neglect in Canada and Israel: A comparative study. Child Abuse and Neglect, 25(5), 607 – 622. Hardiker, P. (1981). Heart or head: The function and role of knowledge in social work. Issues in Social Work Education, 1, 85 – 111. Haverkamp, B. E. (1993). Confirmatory bias in hypothesis testing for client-identified and counselor self-generated hypotheses. Journal of Counseling Psychology, 34, 303 – 315. Kaplan, S. J., Pelcovitz, D., & Labruna, V. (1999). Child and adolescent abuse and neglect research: A review of the past 10 years. Part I: Physical and emotional abuse and neglect. Journal of the American Academy of Child Adolescent Psychiatry, 38(10), 1214 – 1222. Klayman, J. (1984). Analysis of predecisional information search patterns. In Svenson O., & Vari A. (Eds.), Analysing and aiding decision processes. Amsterdam: North-Holland. Kuhn, D. (1991). The skills of argument. Cambridge, USA: Cambridge University Press. Lindsey, D. (1992). Reliability of the foster care placement decision: A review. Research on Social Work Practice, 2, 65 – 80. Locke, D., & Pennington, D. (1982). Reasons and other causes: Their role in attribution processes. Journal of Personality and Social Psychology, 42, 212 – 223. Lyons, P., Doueck, H., & Wodarski, J. (1996). Risk assessment for child protective services: A review of the empirical literature on instrument performance. Social Work Research, 20, 143 – 155. Mandel, D., Lehman, D., & Yuille, J. (1994). Should this child be removed from home? Hypothesis generation and information seeking as predictors of case decisions. Child Abuse and Neglect, 18, 1051 – 1062. Mandel, D., Lehman, D., & Yuille, J. (1995). Reasoning about the removal of a child from home: A comparison of police officers and social workers. Journal of Applied Social Psychology, 25, 906 – 921. Mass, M. (1994). The need for a paradigm shift in social work: The study of parenting. Social Work and Social Science Review, 5(4), 130 – 145. Merrick, D. (1996). Social work and child abuse. London: Routledge. Munro, E. (1998). Improving social workers’ knowledge base in child protection work. British Journal of Social Work, 28, 89 – 105. Osmo, R., & Landau, R. (2001). The need for explicit argumentation in ethical decision making in social work. Social Work Education, 4, 483 – 492. Osmo, R., & Rosen, A. (2002). Social workers’ strategies for treatment hypothesis testing. Social Work Research, 26(1), 9 – 18. Packman, J. (1989). Decisions in child care. In Kahan B. (Ed.), Child care research, policy and practice (pp. 80 – 89). London: Hodder & Stoughton. Parker, R. (1998). Reflections on the assessment of outcomes in child care. Children & Society, 12, 192 – 201.

R. Osmo, R. Benbenishty / Children and Youth Services Review 26 (2004) 1155–1173

1173

Peters, D. F. (2001). Examining child sexual evaluations: The types of information effecting expert judgment. Child Abuse & Neglect, 25(10), 149 – 178. Pratt, J., & Grimshaw, R. (1985). A study of a social work agency: The occupational routines and working practices of the educational social work service. Sociological Review, 33, 106 – 135. Rosen, A. (1994). Knowledge use in direct practice. Social Service Review, 68, 561 – 577. Rosen, A., Proctor, E., Morrow-Howell, N., & Staudt, M. (1995). Rationales for practice decisions: Variations in knowledge use by decision task and social work service. Research on Social Work Practice, 5, 501 – 523. Rowe, G., & Wright, G. (2001). Differences in expert and lay judgment of risk: Myth or reality? Risk Analysis, 21(2), 341 – 350. Ruscio, J. (1998). Information integration in child welfare cases: An introduction to statistical decision-making. Child Maltreatment, 3(2), 143 – 156. Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53, 252 – 266. Shapira, M., & Benbenishty, R. (1993). Modeling expert judgment and decision making in cases of alleged child abuse and neglect. Social Work Research & Abstracts, 29, 14 – 20. Snyder, M., & Thomsen, C. J. (1988). Interactions between therapists and clients: Hypothesis testing and behavioral confirmation. In Turk D. C., & Salovey P. (Eds.), Reasoning, inference, and judgment in clinical psychology (pp. 124 – 152). New York: Free Press. Stevenson, O. (1998). So what is to be done? Moving forward on assessment. In Stevenson O. (Ed.), Neglected children: Issues and dilemmas (pp. 100 – 105). London: Blackwell Science. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124 – 1131. Toulmin, S. E., Riecke, R., & Janik, A. (1984). An introduction to reasoning. New York: Macmillan.