Case file analyses in child protection research: Review of methodological challenges and development of a framework

Children and Youth Services Review 108 (2020) 104551

Contents lists available at ScienceDirect

Children and Youth Services Review journal homepage: www.elsevier.com/locate/childyouth

Case file analyses in child protection research: Review of methodological challenges and development of a framework

T

Susanne Witte German Youth Institute, Nockherstr. 2, Munich 81541, Germany

A R T I C LE I N FO

A B S T R A C T

Keywords: Child protection Review Case file analysis

In cases of child maltreatment, many different professional groups and institutions are involved. This involvement, in general, leads to the creation of documents, which are commonly referred to as case files, charts, or administrative data. Although their original aim is to facilitate communication, document procedures, and legitimize actions, case files are frequently used in research on various issues related to child protection. However, no methodological framework for conducting case file analysis exists to date. First, the article proposes a threestep model for the process of case file keeping, including aspects on the level of information gathering, processing, and presentation. Second, the different types of research questions investigated and the methods used in case file analyses in child protection research are shown based on the review of 219 scientific articles. Third, drawing on the first two parts, challenges of case file analysis in child protection research are summarized following different steps of the research process. Recommendations for the use of case files in child protection research are given.

1. Introduction Since the beginning of child protection research about two centuries ago, analyzing documents on cases of child maltreatment has been a method of choice. Some of the most influential studies are based on a retrospective review of information mostly found in administrative records or case files (e.g, Finkelhor, 1984). Until today, the method of case file analysis has continued to be one of the most common research methods in child protection research (Green et al., 2015). Case file analysis is a useful method to study trajectories of maltreated children in a cost-effective and non-intrusive way whereas avoiding the ethical dilemmas of a prospective research design. Thus, analyzing cases provides the unique opportunity to gain more information about the life of children in high-risk families, the challenges they face, and the support they need. Additionally, it is often used to improve services as well as evaluate policies and guidelines (Courtney, Needell, & Wulczyn, 2004; Green et al., 2015). In the last decades, the type of data available has changed as advances in technologies allow the electronic storage and analysis of large amounts of data. However, the key aspect of case files or administrative data analysis still holds: Data is extracted from documents that are used for a practical rather than a scientific purpose. Regardless of the popularity of the method, very few authors have addressed the methodological challenges of case file analysis and their implications for the

validity of results (for exceptions cf. Green et al., 2015; Huffhines et al., 2016). 2. Scope and objectives This article focuses on the method of case file analysis. It includes empirical studies that extracted, analyzed, or interpreted data that was not exclusively collected and created for research proposes. Furthermore, the data has to relate to persons, i.e. children or families, and does not only include data about policies or other available data regarding child protection, i.e. guidelines for handling cases, financial resources of institutions, or working hours of staff. In order to reflect on the variety of methods and challenges in working with data generated by practitioners in the field of child protection, a wide definition of case file analysis is used in this paper. Thus, it includes research undertaken by practitioners, academic researchers, or both as well as qualitative, quantitative, and mixed methods studies. Moreover, it also considers the advantages and disadvantages of data from structured and already computerized administrative datasets as well as data that has been extracted by the researchers themselves from unstructured and narrative descriptions. According to the definition provided above, case file analysis has many overlaps with clinical data mining (Epstein, 2002) and is sometimes argued to be one type of clinical data mining (Lalayants et al.,

E-mail address: [email protected]. https://doi.org/10.1016/j.childyouth.2019.104551 Received 17 June 2019; Received in revised form 11 October 2019; Accepted 11 October 2019 Available online 25 November 2019 0190-7409/ © 2019 Elsevier Ltd. All rights reserved.

Children and Youth Services Review 108 (2020) 104551

S. Witte

family in order to get an expert opinion or for reasons of supervision. The quality of information gathered depends on various aspects: The amount and accuracy of information provided by family members and professionals as well as the extent of resources spent on gathering information.

2013). However, the latter also includes the extraction and analyses of data not related to persons or cases and case file analysis not necessarily includes practice-based research strategies, which is a crucial aspect of clinical data mining. Case file analysis can also be seen as one type of document analysis and many considerations regarding document analysis are useful for the method of case file analysis (Prior, 2010). However, document analysis includes the analysis of other documents of humankind as well, e.g. tombstones, books, music (Prior, 2010). For the scope of this article, case file analysis also includes those research designs that use data from statewide data sets, like NCANDS. Working with those datasets is considered secondary data analysis (Vartanian, 2011). However, secondary data analysis also refers to research with data from large surveys and do not necessarily need to be based on information provided by professionals. The objective of this article is threefold: First, a three-step process model of the creation of case files is provided. This is based on a review of literature on information seeking, processing, and documentation. Where possible, it refers to the limited empirical literature regarding this topic, particularly related to child protection. Second, the article provides an overview of the research articles published between 2010 and 2016 using case file analysis. Information on the frequencies of the type of case files, the research questions, samples, and the methodological challenges are reported. Third, challenges in case file analysis during the research process are described and recommendations for scientific case file analysis are derived from the first two parts of the article.

3.1.1. Information provided Gathering information is an interactive process. The way interviews are conducted, i.e., suggestive questions (e.g., Hritz et al., 2015) or paraphrasing (e.g., Evans, Roberts, Price, & Stefek, 2010), effects the type of information gathered. Furthermore, factors like the perceived threat by the social worker influence the willingness to share information (Arbeiter & Toros, 2017a, 2017b). Family dynamics have an impact on the extent and accuracy of information obtained as well: Parental behavior has been proven to be influential when seeking information from a child (e.g., Malloy & Mugno, 2016; Melinder, Baugerud, Ovenstad, & Goodman, 2013). Also, having experienced stress or a traumatic event can impair memory retrieval processes (e.g., Berliner, Hyman, Thomas, & Fitzgerald, 2003; Billoux, Arbus, Telmon, & Voltzenlogel, 2016; Lin et al., 2017). Information provided is likely to be linked to characteristics of the case. For example, JohnsonMotoyama, Dettlaff, and Finno (2012) noted that case files which did not contain birthplace information of parents differed significantly from those with birthplace information of parents. The children without birthplace information of parents were substantially older, had a higher level of risk and harm, and parents were more likely to abuse alcohol or drugs. For foster children, information like gestational age or birth weight are in many cases not known to the child or the caretakers, thus remain unknown to the caseworker (Schneiderman et al., 2013).

3. The process of case file keeping In professional work contexts, documents are used extensively (Hertzum, 2002). According to Hertzum (2002), documents have six different roles: 1) as personal work files, 2) as reminders of things to do, 3) to share information with some yet to withhold it from others, 4) to convey meaning, 5) to generate new meaning, and 6) to mediate contacts among people. Case files are documents created within a professional context. They contain information about an individual or a group of individuals which exist in real life. They are neither the person's own accounts nor the ‘truth’ but rather the perception of the person through the lens of a professional. Thus, Taylor (2008) states that the contents of case files are not a ‘transparent window on the world' but are influenced by numerous factors. Moreover, case files – in most cases – are created over time and thus represent the documentation of a sequence of events (Huuskonen, 2014). The creation of a case file is a process, including many aspects and variables. A three-step process model is proposed regarding the creation of case files in child protection (cf. Fig. 1). The steps are 1) the gathering of information, 2) the processing of information, and 3) the presentation of information. Although those steps seem to happen chronologically, they will repeat themselves over time. Thus, the presentation of information about one incident will most likely influence the gathering of information on the second incident, i.e., the creation and consumption of case files are bound together (Prior, 2010).

3.1.2. Resources The resources spent on investigating a case vary between cases, caseworkers, and institutions. For example, the number of contacts with family members differs based on the complexity of the case and the resources allocated by the social worker. In the face of rare resources, caseworkers might limit their attention to the most severe problems or those types of maltreatment which are more prominent (Pinto & Maia, 2013). Considering these factors, it is not surprising that one of the very few studies which combined self-report data and case file data on maltreatment experiences did not find a strong agreement between the two sources of information (Pinto & Maia, 2013). Especially for emotional abuse, the agreement was very low. Caseworkers are also likely to limit their attention to specific persons. Hence, many authors report that information about fathers is missing more frequently than information about mothers (Li, Chu, Ng, & Leong, 2014; PutnamHornstein & Needell, 2011) 3.1.3. Conflicting objectives Besides the substantiation of concerns, caseworkers have other objectives in their work with families, like establishing a relationship of trust with the family as well as offering support and information. These objectives can stand in the way of seeking more information (Arbeiter & Toros, 2017a, 2017b). The conflict between different objectives is even enforced in some states and countries as providing information and offering support at an early stage is required by law. These laws were found to influence social work practice (Jergeby & Soydan, 2002), and it is very likely that they also affect the type of data gathered.

3.1. Gathering of information The gathering of information is most likely the starting point of a case file. The caseworker1 is informed about a family in which a child or all children might need help or be suffering maltreatment. At this point, the caseworker most likely begins with gathering information by contacting family members or other persons in contact with the family. The caseworker might also talk to other professionals not in contact with the

3.2. Processing of information Although closely linked to the gathering of information, it is useful to consider the processing of information separately. 3.2.1. Individual factors Information processing depends foremost on attention, emotion, and motivation of the individual, which influence the kind of

1 For the sake of clarity, the term caseworker is used instead of naming all possible professional groups in charge of the creation of a case file (e.g., social worker, physician).

2

Children and Youth Services Review 108 (2020) 104551

S. Witte

process of case Įle keeping

phenomenon

gathering

processing

presentaƟon

• amount and accuracy of informaƟon retrieved/provided • ressources spent • conŇicƟng objecƟves

• individual factors: memory capacity, aƩenƟon, knowledge • situaƟonal factors: workload, emoƟonal burden, expectaƟons about the family • use of memory aids

• aim of presentaƟon • audience of presentaƟon • habits of documentaƟon • cultural and insƟtuƟonal pracƟces

research process

case Įůe

gathering

• ensuring access to case Įles: working together with insituƟons, retrieving case Įles • ethical consideraƟons and data security: informed consent, data privacy, ensuring the safety of the child

processing

• variable selecƟon • handling of ambiguous informaƟon and inconsistencies • handling of missing informaƟon • reliability of data abstracƟon

presentaƟon

• speciĮcity of the sample • diīerent levels of ‘reality‘ • InsƟtuƟonal and cultural context

research Įndings Fig. 1. Process model of case file keeping and case file analysis.

information memorized (Kuldas, Hashim, Ismail, & Bakar, 2015). Other factors of great importance are attitudes and ‘professional lenses’ in the evaluation of information (e.g., Fleming, Biggart, & Beckett, 2015). Knowledge shapes the perception of events but also influences the amount memorized (e.g., Tourangeau, 2000). Moreover, different persons might draw different conclusions. This is the case for risk factors, and even for medical diagnoses. McKeag et al. (2013), for example, note that the diagnosis of enlargement of subarachnoid spaces depends on the radiologist's discretion. Palusci, Wirtz, and Covington (2010) found that the number of child deaths due to child maltreatment was underestimated in comparison to the reassessment of these cases by an expert panel using only the information from the case files.

during the processing of information. These are the current workload and emotional burden of the social worker. Moreover, expectations about the family can alter the processing of information. Thus, caseworkers might tend to confirm the information about the family gathered before, also known as confirmation bias (Spratt, Devaney, & Hayes, 2015). Besides, as Tourangeau (2000) points out, people add information from a general script to descriptions of specific events in order to fill in the blanks. For example, the knowledge about high-risk families, in general, will be used to make interferences about a specific case. In this sense, social workers are not only the creators of records but also users of already available information and knowledge (e.g., Huuskonen & Vakkari, 2015).

3.2.2. Situational factors All the factors mentioned above have often been considered traits of a person, yet many situational and case-specific factors come into play

3.2.3. Memory aids Aiding memory by taking notes is also often considered a valuable tool to increase memory capacity. However, taking notes during 3

Children and Youth Services Review 108 (2020) 104551

S. Witte

conversations is effortful, as caseworkers have to split attention and make decisions about what to note. Huuskonen (2014) found that social workers frequently do not note the most important risk factors but note rather unhelpful information. Note taking becomes especially influential when there is a time delay between the encounter and the documentation within the case file. However, a time delay still poses threats to the accuracy of the information (Huuskonen & Vakkari, 2015).

(Taylor, 2008). Besides an overall tendency to document information in an impersonal style, the professional training leads to different linguistic styles, i.e., the use of ICD-10 codes for physicians and legal terms for lawyers. Some social workers might be more cautious in their linguistic choices based on the belief that reality is constructed through social interaction, and the truth is a matter of perspective (Berger & Luckmann, 1991).

3.3. Presentation of information

3.3.4. Cultural and institutional practices The presentation or organization of information depends on cultural and institutional practices and thus varies over time. Different guidelines and practices of documentation influence the content of case files. Gonzalez-Izquierdo et al. (2013) found differences in their study on victimization-related hospital admissions between England and Western Australia: In England, the hospital staff was advised to only code substantiated cases of assault, whereas in Western Australia all assaults were required to have information about a possible cause. Chen et al. (2010) found in their study on a hospital-based child protection team that the recording practice became more structured and comprehensive over time. Adaptation to new guidelines takes time to influence practice. Thus, Wells and Correia (2012) found that data on structured decision-making was missing more frequently in the first year following the implementation of a structured decision-making tool. Different scopes in child protection, e.g., family service orientation or child protection orientation, also influence the way case records are kept: In Germany, case files from child and youth welfare authorities are mostly about the whole family, in the English child protection system each child has an own case file. A tremendous change in institutional practices is the use of electronic case records, which offers new possibilities (e.g., simplified procedures for sharing of information), but also new challenges (e.g., adapting to new programs, concerns of data safety). Thus, some information is now entered in electronic forms, allowing only a specific type of information to be entered. It can also limit the amount of data provided in the case file as in some programs the space for narratives is limited (Thomas, Sorenson, & Joshi, 2010). Some countries and institutions are more open to new technologies than others. While in England, the Netherlands, and the Scandinavian countries computerbased client information systems are widely utilized (Huuskonen & Vakkari, 2013), in Germany hospitals as well as youth welfare organizations use either paper case files, electronic case files, or a combination of both.

The third step in the model is the presentation of information. It depends on four factors: 1) aim of the presentation, 2) audience of presentation, and 3) habits of documentation, which are embedded in the broader context of 4) cultural and institutional practices. 3.3.1. Aim of the presentation People use documents for various reasons: to help them memorize things and necessary next steps, to reflect on cases, or to communicate with colleagues. Recording can be used as a mean to show accountability (Kagle, 1984; Mann & Williams, 2003). Information highly crucial for one context is prioritized. Thus Wu, Lu, Lin, and Feng (2015) noted in one study using a medical chart review that physical injuries were very richly described whereas information on demographic characteristics of the child and family characteristics were missing. In contrast, Lightfoot, Hill, and LaLiberte (2011) found in their work using child welfare data that detailed information on diagnoses was lacking. Recording cases is often considered a pointless task by social workers, which reduces the time for interaction with the client (O'Rourke, 2010). Hence, if the aim of the presentation is not clear or decision-making, in this case, is considered easy, the documentation in case files might be less comprehensive. In severe or complex cases the documentation is more detailed: Hall, Slembrouck, and Sarangi (2006) as well as Huuskonen and Vakkari (2015) found that entries in case files became more detailed in critical moments. The stronger focus on documentation in severe or complex cases might be because the accuracy of case files is frequently considered a sign of the quality of practice and thus is used to evaluate the actions of professionals (Taylor, 2008). Information is not only more detailed in complex cases, but also in those cases with ongoing service provision. Esposito et al. (2013), for example, found that information about the ethnic-racial background of a child was associated with a higher likelihood for ongoing service provision.

4. Review of studies using case file analysis in child protection research

3.3.2. Audience of the presentation Case files are written for different audiences, e.g., for colleagues, other professionals, superiors, or the family. Most likely, there are multiple audiences for the whole case file or parts of it. Kagle (1984) highlighted that an expanded audience for a case file could lead to a reluctance of the social worker to spent time and effort on case file keeping. O'Rourke (2010) as well as Huuskonen and Vakkari (2015) found that caseworkers often felt ashamed to enter intimate details about the family life into the case file being aware that the family members will read it.

Whereas case file analysis is widely used in child protection research, there is limited systematic information regarding the scope and methods. Thus, in the following section, the results of a structured literature search on empirical studies using case file analysis are shown. 4.1. Method PubMed and all available databases on EBSCOhost were searched using the following search term: (case file OR chart OR administrative data) AND (child protection OR child abuse OR child maltreatment OR neglect). All articles published between January 2010 and May 2016 were considered eligible. The search yielded 822 results (PubMed: 255; EBSCOhost: 567). The following steps were made to derive the final set of articles (cf. Fig. 2): First, publications were selected based on title and abstract, which lead to a reduction to 415 articles for both databases. Second, duplets were excluded. One hundred twenty-five publications had to be excluded based on this criterion. Third, full articles and publications were sought, and authors were contacted if articles could not be retrieved via library. After reading all copies of the

3.3.3. Habits of documentation Whereas the points mentioned above are mostly considered consciously by the caseworker during the documentation process, the documentation process in itself is repeated so frequently that it becomes a habit. The characteristics of these habits stem from the idiosyncrasies of the social worker, their professional training, and the institutional context (Kagle, 1984). Taylor (2008) noted in her work on the qualitative analysis of case files that the entries are often written in a matter of fact way; thus, disguising the author and emphasizing neutrality by using the passive voice and indirect discourse. However, the case files still contain assumptions, categorization, and opinions of the author 4

Children and Youth Services Review 108 (2020) 104551

S. Witte

PubMed: 255

EBSCOhost: 567

822 exclusion following review of abstract

407 415

exclusion of duplets 125 290

exclusion following review of full paper 71

219 Fig. 2. Flowchart of selection process of publications.

articles, further 71 articles were excluded, as they did not include empirical data (e.g., Cole & Lanham, 2011), did not fit the subject (e.g., Hovens et al., 2012), or were based on aggregated data from official statistics without case specific information (e.g., Gilbert, 2012). After this process, 219 articles remained for the analysis. Articles that used the same data set or had overlapping samples were not excluded from the analysis. The articles were classified in terms of the source of the case file, the sample characteristics, the research questions, and the methods used. Articles were also reviewed regarding reports about difficulties during case file analysis. Coding was done by the author as well as a student assistant. Both reviewed all articles. Differences were solved by discussion.

Table 1 Subcategories of research questions.

Service provision Service usage Service outcome Recognition of maltreatment Diagnostic procedures Decision-making, actions of professionals Variation of service usage in time Other Outcome of maltreatment Mental health Physical health Offending Mortality Social competencies Academic achievement Revictimization Characteristics of maltreatment Risk factors of maltreatment Types of maltreatment Prevalence of maltreatment

4.2. Results 4.2.1. Research questions The articles were analyzed in terms of the type of main research question investigated. Three main groups of research questions were found: service provision (51.6%; n = 113), outcome of maltreatment (29.7%; n = 65), and characteristics of maltreatment (29.2%; n = 64). In 23 articles, two main research questions were analyzed (10.5%). Within each category, subcategories were derived, which are depicted in Table 1. For reasons of clarity, only four distinct main categories of research questions will be used in the following section, i.e., ‘service provision only,' ‘outcome of maltreatment only,' ‘characteristics of maltreatment only,' and ‘two research questions.'

n

% within subcategory

% within total sample

113 43 33 14 12 10

– 38.0 29.2 12.4 10.6 8.8

51.6 19.6 15.0 6.4 5.5 4.6

5 3 65 30 23 6 3 3 2 1 64 41 30 7

4.4 2.7 – 46.2 35.4 9.2 4.6 4.6 3.1 1.5 – 64.1 46.9 10.9

2.3 1.4 29.7 13.7 10.5 2.7 1.4 1.4 0.9 0.5 29.2 18.7 13.7 3.2

Note. Percentages add up to more than 100% as some articles were considered to investigate more than one research question.

types of institutions (n = 26). 56.6% of the articles were based on information retrieved from social services (n = 124), 45.2% from a health care institution (n = 99), and 13.2% from court (n = 29). In addition to case file data, 10.0% of the articles used census data (n = 22). The majority of articles which were based on the information of more than one type of institution included at least one health care institution and one social service (n = 24; 40.7%). 23.7% facilitated data from at least two different types of institutions based in the social service sector (n = 14). The number of different types of institutions did not substantially vary regarding research questions investigated (Kruskal Wallis; p = .842). Yet, different research questions were investigated using different types of institutions as source of information (Χ2(9) = 46.22; p < .001; Table 2). Outcome of maltreatment was more likely than expected investigated using case files from health care records only. In contrast, service provision only was more likely

4.2.2. Sources of information The majority of articles were conducted using case file information retrieved in the United States of America (65.3%; n = 143) and other primarily English speaking countries (United Kingdom: 8.6%; n = 19; Canada: 6.4%; n = 14; Australia: 3.7%; n = 8; New Zealand: 1.8%; n = 4). The remaining articles were based on information collected in Asian countries (7.3%; n = 16) or other European countries (United Kingdom excluded; 5.9%; n = 13). Only one article facilitated data collected in Africa. Two articles compared data between two different countries, both included England. The majority of articles used one source of information, i.e., one specific type of institution like social services (73.2%; n = 160). 15.1% used two types of institutions (n = 33), and 11.9% used three or more 5

Children and Youth Services Review 108 (2020) 104551

S. Witte

Table 2 Research questions and data source (type of institution).

Service provision only Outcome of maltreatment only Characteristics of maltreatment only More than one type of research question Total

Health care only

Social services only

Health care and social services

Neither

n

%

n

%

n

%

n

%

18 34 9 13 74

8.2 15.5 4.1 5.9 33.8

59 13 21 6 99

26.9 5.9 9.6 2.7 45.2

13 5 4 3 25

5.9 2.3 1.8 1.4 11.4

8 3 9 1 21

3.7 1.4 4.1 0.5 9.6

intimate partner violence only, and sexual exploitation only were rarely studied in articles using case file analysis. There were substantial differences between the assessment of maltreatment in regard to research questions: Articles focusing on service provision only were more likely to have no types of maltreatment noted to be considered in the analysis (Χ2(3) = 32.78; p < .001). In articles with research questions about the characteristics of maltreatment and those with a combination of research questions, it was more likely that only one type of maltreatment was considered compared to those articles with other research questions (Χ2(3) = 13.87; p = .003).

investigated using social service records only. For 73 articles, the number of sites where data collection took place was not reported. However, 64 of these used the data from the whole region. 42.0% of the remaining articles reported only one site for data collection (n = 92). 30.1% collected data from up to ten different sites (n = 44) and 13.7% from more than ten different sites (n = 10). There were no significant differences between the use of one, two, or more sites of data collection regarding the research question investigated (Χ2(3) = 5.11; p = .164). 4.2.3. Sample characteristics In four studies, the whole population was considered for analysis, but in most cases, a specific sample was used. The sample size varied greatly between articles, ranging from three units of analysis to over four million. 60.5% of articles had a sample size smaller than 1000 units of analysis (n = 130), and only 3.7% had a sample size larger than one million (n = 8). There were no substantial differences in the sample size in regard to research questions (Kruskal-Wallis; p = .646). In 74.4% of all articles, children and/or adolescents were the unit of the analysis (n = 163). Adults were the unit of analysis in 9.6% of the articles (n = 21) and in 5.0% of the articles children, adolescents, as well as adults were included in the sample (n = 11). In 7.7% of the articles, families or parents were the units of analysis (n = 17). Only very few articles used the report, spell, or case file as the unit of analysis (2.7%; n = 6). In the majority of the articles, female and males were included in the sample (91.3%; n = 200). There were no significant differences between sample characteristics between research questions.

4.2.5. Informed consents In 20 articles, informed consent of the persons whose case files were analyzed was sought (8.7%), in one only from the social workers handling the case. In eight articles (3.7%), it was explicitly stated that informed consent was not obtained. The remaining articles did not contain any information if informed consent was sought (86.8%; n = 190). There were no significant differences between research questions whether information on informed consent was provided or not (Χ2(3) = 0.70; p = .874). 4.2.6. Methods of data collection and analysis Results from case file analysis were often based on quantitative approaches of data analysis (83.6%; n = 183). Qualitative analysis only (2.7%; n = 6) or a combination of quantitative and qualitative approaches (13.7%; n = 30) was rare. In 15.5% of the articles, the case file analysis was combined with other sources of information (n = 34, e.g., interviews with social workers). In 11.0%, different case files or sets of administrative data were linked (n = 24). In 13.7% of the articles inter-rater reliability testing was reported (though not always the results; n = 30). The most common measures to report inter-rater reliability were Cohen’s Kappa, Intraclass Correlation Coefficients (ICC), and the percentage of agreement between raters. Cohen’s Kappas reported ranged from 0.52 to 1.00, ICCs from 0.43 to 0.98, and percentage of agreement from 65% to 96%. The majority of articles reporting inter-rater reliability used about 10% of the cases to assess inter-rater reliability (range: 5–20%). Only one study reported assessing stability of inter-rater reliability over time. Besides inter-rater

4.2.4. Types of maltreatment To further elaborate on the content of the articles, the types of maltreatment considered in the data analysis were coded. No differentiation was made between articles which used types of maltreatment as predictors, covariates, or outcomes (cf. Table 3). Only in 27 articles, the maltreatment was not included in the research design. In the majority of the remaining articles, more than one type of maltreatment was considered. However, not all differentiated between types of maltreatment. When only one type of maltreatment was considered, it was most likely sexual abuse followed by physical abuse. Death of a child, Table 3 Types of maltreatment considered and research questions.

Not documented/applicable More than one type All types not differentiated Physical abuse only Sexual abuse only Sexual exploitation only Neglect only Intimate partner violence only Death Other

Service provision only

Outcome of maltreatment only

Characteristics of maltreatment only

More than one type of research question

Total

n

%

n

%

N

%

n

%

n

%

26 39 16 2 7 1 3 1 0 4

26.3 39.4 16.2 2.0 7.1 1.0 3.0 1.0 0.0 4.0

0 29 12 4 4 0 2 0 0 3

0.0 53.7 22.2 7.4 7.4 0.0 3.7 0.0 0.0 5.6

0 18 2 4 7 2 2 1 4 3

0.0 41.9 4.7 9.3 16.3 4.7 4.7 2.3 9.3 7.0

1 12 0 1 5 0 0 1 1 2

4.3 52.2 0.0 4.3 21.7 0.0 0.0 4.3 4.3 8.7

27 98 30 11 23 3 7 3 5 12

12.3 44.7 13.7 5.0 10.5 1.4 3.2 1.4 2.3 5.5

6

Children and Youth Services Review 108 (2020) 104551

S. Witte

reliability, 64 articles reported other measures to ensure the reliability and validity of results (29.2%). These included protocols to enter data, to solve difficulties, to check the validity of administrative data entries with local authorities, and to train case file abstractors. There were no significant differences whether ensuring data quality was reported between articles with different research questions (Χ2(3) = 0.81; p = .847). In 42.5% of the articles (n = 93), the analyses were based on datasets already available to the researchers due to electronic records and datasets. No information on the case file abstractors was provided in 33.8% of the articles (n = 74) although it was apparent that no previously collected dataset was used. Forty-six articles (21.0%) reported that the case files abstractors were part of the research team or the authors. Twelve of these reported that information was at least partially extracted by research assistants or students (5.5%) and three reported the involvement of other coders, e.g., social workers (1.4%). The remaining articles reported that the coding was done by an advanced practice nurse (n = 1), a former employee of the institution providing the files (n = 1), or a person familiar with social service procedures and terminology (n = 1). One study reported that they asserted that the person was no current or former employee of the institutions providing the files. The number of case file abstractors was reported in 19.6% of the articles (n = 43). It ranged between one to ten case file abstractors. In 42.9% of the articles (n = 94), the authors reported that information was missing in the case files for certain variables. There were no substantial differences between research questions (Χ2(3) = 1.86; p = .603), but articles using data from social service records only were substantially more likely to report missing data whereas articles using data from health care records were less likely to report missing data (Χ2(3) = 15.90; p = .001).

Table 4 Criteria for scientific case file analysis. Nr.

Criteria

A A1

Accessing data Is there the possibility of bias in the selection of the site which provided the case files? Was it possible to access all case files meeting the inclusion criteria? If more than one source of information was used, were there obstacles in matching the data? Ethical considerations Was informed consent obtained? If no, why? If yes, what was the dropout? How was data privacy ensured? Was there a safety plan? How was ensured that the population under study, profited from the results? Method of data abstraction Can the variables of interest be abstracted reliably from case files? Were the variables validated by practitioners entering information in case files? Was there a pilot testing of the method? How were inconsistencies in case files handled? How was missing data in case files handled? Reliability of data abstraction How were case file abstractors trained? How was inter-rater reliability established between case file abstractors? How was the data collection process supervised? Acknowledging the institutional and cultural context What does information about procedures and data entry practices obtained from practitioners imply for the results? What do documents and guidelines about the institutional processes imply for the results? Were the results discussed with practitioners and/or the population under study? What were the results? What do social and demographical of the sample as well as institutional and cultural characteristics of the institution providing the case file imply for the generalizability of the results?

A2 A3 E E1 E2 E3 E4 M M1 M2 M3 M4 M5 R R1 R2 R3 C C1 C2 C3 C4

5. Challenges and recommendations for case file analysis Based on the results of the review of case file studies as well as literature on the use of case files, challenges and recommendations for research using this method in child protection research are derived. Therefore, the research process can be described in a manner similar to the process of case file keeping (cf. Fig. 1): At first, the researchers have to gather the information, i.e. to gain access to case files and to retrieve them. During this process they – also very similar to caseworkers – have to ensure certain legal and ethical standards. The second step is the abstracting of information from case files, which can be seen as a way of processing the information. As a final step, the findings are presented following the interpretation of the results. An overview of criteria important for the scientific use of administrative data is provided in Table 4.

on already established relationships with institutions facilitates access (Green et al., 2015), but there is the possibility that it might lead to bias; institutions more willing to participate in research might differ from those who do not (cf. Table 4, A1). Second, it is essential to find out who is in charge of the decisions regarding participation in research and issues of data privacy. Otherwise, much time may be wasted (Green et al., 2015). For the data collection, it can be beneficial to have a liaison person to help with organizational issues. These persons should be authorized by the institution providing the case files and possibly be reimbursed for their efforts by the research team. Third, researchers should ensure that research does not burden social workers with too much additional work and ultimately threaten the well-being of children and adolescents (Green et al., 2015; Huffhines et al., 2016). Fourth, providing information before, during, and after the data collection is vital to enhance the commitment of the staff to the research project (Green et al., 2015). Fifth, although these recommendations help to ensure access, researchers still need to take into account that receiving approval to access data takes time and should plan their research project accordingly (3 to 14 months, Green et al., 2015; 9 months Hayes & Devaney, 2016).

5.1. Ensuring access to case files and administrative data There are some obstacles when ensuring access to case files or administrative data when no available data set can be used, i.e., there is no statewide data collection. Researchers need to get approval to collect information from institutions. Moreover, there might be some obstacles retrieving case files even when the cooperation is established.

5.1.2. Retrieving case files Even when cooperation is established, it is sometimes not possible to retrieve all case files of interest, particularly in case of historical case files. Records might be destroyed or data deleted. In some instances, case files are kept, but the archives are unstructured and do not have an index of cases. Even when most recent cases are studied, there may be obstacles when retrieving case files kept by individual workers (e.g., Tonk et al., 2012). In a study on drug exposure of maltreated children, Oral et al. (2011) reported that 47 case files from 665 children admitted due to abuse or neglect could not be found. Though electronic storage increases the likelihood to capture all cases, studies matching different data sources still show a high number of missing cases (e.g., Osterling &

5.1.1. Working together with institutions Establishing a good cooperation with a data provider is crucial for the success of a research project. Still, this is sometimes challenging as child protection agencies already have a high workload, have concerns regarding data privacy, and do not want to be ‘judged’ by a research team. In some cases, it might be possible to obtain data from many institutions, but not from all (Tursz, Crost, Gerbouin-Rerolle, & Cook, 2010; Vellut, Cook, & Tursz, 2012). This is even the case when the data collection is supported by the state (e.g., Tonk, Adorjáni, & László, 2012). Some of the following recommendations might help: First, relying 7

Children and Youth Services Review 108 (2020) 104551

S. Witte

5.2.3. Ensuring the safety of the child In line with considerations of data privacy and data protection, the researcher has to consider other ethical values as well. The foremost in child protection research is to ensure the safety of the child. Although it might be rare that a researcher comes across information during case file analysis suggesting an immediate risk not attended to already, it is not completely unlikely. For example, in a review of medical charts – also widespread in child protection research – the physical evidence suggests the presence of maltreatment, but the case was not treated as such by the staff. This leaves the researcher with a dilemma. These cases need to be considered individually, and researchers have to be aware of the legal obligations within their country. It is very useful to have a safety plan at hand for those cases (cf. Table 4, E3). These safety plans can be based on those usually used in surveys and often include supervision by a senior researcher. Being informed about the safety plan will also limit the risk that an inexperienced case file abstractor breaches confidentiality due to being emotionally overwhelmed.

Han, 2011; Wells & Correia, 2012). The possibilities of researchers are limited when information is lost or unable to find. As these cases are likely to vary in their characteristics, it is necessary to document any obstacles in retrieving case files or establishing the correct number of cases when cases are selected randomly (cf. Table 4, A2 and A3). 5.2. Ethical considerations and data security Case file analysis does not involve direct contact with the subjects and is a non-intrusive research method; still, ethical considerations, including aspects of data security, need to be taken into account. 5.2.1. Informed consents When personal data is used for research, informed consent is required by participants (cf. Table 4, E1). The results of the review of articles show that informed consent was only obtained in a minority of the studies. Most authors did not even comment on why they did not obtain informed consent from subjects. There are many obstacles in obtaining informed consent in case file analysis: First, it is unclear to whom the data in a case file belongs if it contains information about more than one person. Second, it is difficult to contact all subjects as their addresses might have changed, and obtaining their new addresses might threaten confidentiality. Third, there is the possibility that obtaining consent will lead to a selection effect and thus lead to constraints in the generalizability of the results and hence bias the derived conclusions for service provision (Hayes & Devaney, 2016). Thus, as Hayes and Devaney (2016) point out: If the results are highly important for society in general, it sometimes outweighs the rights of individual subjects. However, researchers should not decide to waive informed consent lightly and should justify their decision in terms of the importance for society. In some cases, it might still be useful to inform subjects and provide them with the possibility to opt-out of the research project. Especially, if it is decided not to inform subjects based on the importance for the society, researchers need to put effort into disseminating the results not only in the scientific community, but also to stakeholders, policymakers, and practitioners (cf. Table 4, E4). Thus, making it possible for participants to profit from the research (e.g., Brakewood & Poldrack, 2013).

5.3. Abstracting information from case files When data from cases files are abstracted or already available administrative data is used, certain aspects of the data and the process of data acquisition have to be considered. In addition, it is crucial to consider the information available when selecting variables of interest. 5.3.1. Variable selection Some variables might be more readily abstracted from case files or used for data analysis than others (cf. Table 4, M1). Some variables – like dates of reports and duration of foster care – are most reliably assessed by using case files such as dates of admission (Kohl, JonsonReid, & Drake, 2011; Jonson-Reid, Emery, Drake, & Stahlschmidt, 2010). Jonson-Reid, Drake, and Zhou (2013) suggest that it is best to focus on the information that triggers funding, service provision, or mandated forms of continued surveillance of clients. This information is more likely to be kept in the case file in a structured way. Additionally, as Lindsay et al. (2012) pointed out, case files are more likely to contain information about the presence of a problem, e.g., delinquent behavior, than the frequency of such behavior. Other variables like poverty can be assessed using indicators in case files, e.g., Medi-Cal coverage (PutnamHornstein & Needell, 2011) or food assistance (Semanchin Jones, 2015). Other variables, e.g., the cultural orientation of the clients (Johnson-Motoyama et al., 2012), are vast constructs and are unlikely to be captured reliably in case files. As pointed out in Section 3.3.1, the type of information might be different based on the original aim and setting in which the case file was created. Thus, some variables of interest might be more likely to be captured in case files from one specific institution than in others. In a French study, for example, 71% of fatal maltreatment cases identified in courts were not found in the mortality statistics (Tursz et al., 2010). Different measures can be taken in order to select reliable and valid variables. First, practices of handling cases and entering data should be discussed with practitioners working in the institutions (e.g., JonsonReid et al., 2013; cf. Table 4, M2). Second, guidelines and policies guiding data entry should be reviewed (e.g., Jonson-Reid et al., 2013; cf. Table 4, M2). Third, pilot testing of coding schemes can help to determine which information is available in case files (e.g., Cook, Steigman, & Jonikas, 2014; Osterling, Lee, & Hines, 2012; Schneiderman et al., 2013; Tursz et al., 2010 cf. Table 4, M3). All decisions of the researchers should be carefully documented, as this might prove useful during data analysis and the interpretation of results.

5.2.2. Data privacy Data privacy has more and more become an issue of great importance as the ‘electronic turn’ (e.g., Garrett, 2005) allows to collect data but also to misuse it. All collected information of people can potentially be misused (Hayes & Devaney, 2016). The same applies to data collected from case files if they contain personal information (cf. Table 4, E2). As participants are commonly not informed about data collection, this is even a more severe issue. This results in additional data security measures that have to be taken (Green et al., 2015). Data should be anonymized as soon as possible, and only persons that have signed a confidentiality agreement should have access to personal data (Hayes & Devaney, 2016). Especially, if students or groups of students assist in the research process, the senior researcher should explain data privacy regulations and their importance to them. For some research projects, it might be useful that staff from the institution providing the case files is extracting the information. However, this is only possible if the person is reliably coding the case files and is not the social worker on the cases coded or talking to the social workers handling the case. The latter leads to a bias in the data collection and is especially harmful when comparing results from more than one site with different procedures used. Another option to anonymize data is blackening personal data in case files. Yet, it is unclear whether this is sufficient in terms of data privacy (Hayes & Devaney, 2016), and it is very time- and cost-intensive.

5.3.2. Ambiguous information and inconsistencies Case files are 'inherently messy' (Lucenko, Sharkova, Huber, Jemelka, & Mancuso, 2015; p. 56). Thus researchers come across ambiguous and inconsistent information within a case, between 8

Children and Youth Services Review 108 (2020) 104551

S. Witte

and during data collection. Case file abstractors should be trained in the coding of information and guidelines for coding information should be established (e.g., Cook et al., 2014; Li, Chu, Goh, Ng, & Zeng, 2015; Lindsay et al., 2012; cf. Table 4, R1). It can be beneficial to select case file abstractors with training in the field as well as limiting the number of case file abstractors (e.g., Annerbäck, Svedin, & Gustafsson, 2010; Murphy & Fairtlough, 2015). After a training period, inter-rater reliability has to be established either by using actual case files or vignettes (cf. Table 4, R2). Sometimes, it is also advisable to have ongoing inter-rater reliability testing during data collection. However, as data from the same case files needs to be assessed more than once, more time and resources for data collection are needed. For some research questions, it is necessary that some information is coded by a reviewer blind to other characteristics of the case (e.g., Choi, Reddy, & Spaulding, 2012; Choudhary, Bradford, Dias, Moore, & Boal, 2012; Greiner, Richards, Care, & Leach, 2013; Myhre et al., 2013). As case file abstractors encounter difficulties during the coding process, they should be able to discuss these with a senior researcher (Huffhines et al., 2016; Tursz et al., 2010). Decisions made about difficult cases should be noted and made available to all case file abstractors (cf. Table 4, R3). Establishing reliability is most crucial when information is abstracted from case files, containing narratives or even handwritten information and when researchers are in charge of data collection. However, even when already structured and available datasets or statewide datasets are used, researchers need to take into account the reliability of the variables, which relies on the training and diligence of the caseworker.

institutions (Green et al., 2015; Huffhines et al., 2016), and states (Semanchin Jones, 2015). Definitions, as well as guidelines for case file keeping, also vary over time (Chen et al., 2010). During data collection, solutions have to be found on how to deal with different definitions (Green et al., 2015; cf. Table 4, M2). Some of these issues can be resolved if case file abstractors are carefully trained and there are possibilities to discuss and solve difficulties (e.g., Burke, Hellman, Scott, Weems, & Carrion, 2011; Robboy & Anderson, 2011). One possibility to tackle inconsistencies, e.g., in definitions of maltreatment is to code maltreatment based on the narratives within the case file (English & the Longscan Investigators, 1997). Still, decisions need to be made which level of certainty is required to code, e.g., maltreatment as present. There are advantages and disadvantages to either a conservative approach including only substantiated characteristics (e.g., Hussey, 2012; Osterling et al., 2012) as well as an approach including any reports of maltreatment (e.g., PutnamHornstein & Needell, 2011). 5.3.3. Missing information Case files are created for practical purposes and can contain extensive information (e.g., up to three boxes of files per case, GoodmanDelahunty, 2014). However, they do frequently not contain all information of interest for researchers (cf. Section 4.2.6). If information is missing it is difficult for the researchers to conclude if something is truly absent, e.g., a particular risk factor, or if it was not assessed by the caseworkers (e.g., de Bortoli, Ogloff, Coles, & Dolan, 2016). As noted in Section 3.1, the amount of information gathered, and thus, the amount of missing information is unlikely to be random. Hence, there is a need to establish how to deal with this in research (Green et al., 2015) as it might threaten the validity of the findings (cf. Table 4, M5). One common way to handle missing information is the exclusion of case files with missing information (e.g., Gangat, Klein, & Heptulla, 2012; Hawkes, 2011; Jud, Perrig-Chiello, & Voll, 2011; Knott & Donovan, 2010; Long, 2014; Zhang, Fuller, & Nieto, 2013). However, this can lead to sampling bias as cases with missing information might be different from those without. The amount of missing information can also differ between sites. For example, Walsh, Lippert, Edelson, and Jones (2015) found an unusually high number of missing information in one site of their data collection, but not in others. In cases in which the missing information is relevant for an inclusion criteria, e.g., the migration background of a child (JohnsonMotoyama et al., 2012) or the relationship of the perpetrator to the maltreated child (Ben-David, Jonson-Reid, Drake, & Kohl, 2015), researchers do not have a choice but to exclude those cases. Still, the number of cases excluded due to missing information should be reported. In some studies (e.g., Klatt, Cavner, & Egan, 2014; López López, del Valle, Montserrat, & Bravo, 2011), missing information was obtained from caseworkers. This can be a good option. However, it might lead to some bias due to false memory of caseworkers or differences in the willingness to share information and availability of the caseworker. In large datasets and if the variable is not required for sampling, it is possible to impute missing variables assuming they are random (e.g., Osterling et al., 2012; Ou & Reynolds, 2010). In order to ascertain that the amount of missing information does not influence the outcome variable, an aggregated variable on the amount of missing information can be created and added to the prediction model (Ou & Reynolds, 2010).

5.4. Interpreting results from case file analysis Every research method has limitations that have to be considered when interpreting the findings. However, some of the following limitations are more inherent in some research questions than in others. The same applies to case file analysis. Nevertheless, there are some issues similar to almost all studies using this method of data collection. 5.4.1. Specificity of the sample The researchers select the samples stemming from case file analysis by using inclusion and exclusion criteria. However, before this step, the population from which to draw is already preselected. This selection process is not controlled by the researcher but is a result of a complex process of policy, structures of the child protection system, and dynamics in society. Thus, the sample is influenced by factors such as service utilization (e.g., Salwen, Hymowitz, Vivian, & O'Leary, 2014) and surveillance bias (e.g., Wilson, Tumen, Ota, & Simmers, 2015). In case of a longitudinal design the generalizability is threatened by attrition due to mobility (e.g., Gifford, Eldred, Sloan, & Evans, 2016; Green et al., 2014; Kohl et al., 2011) as well as changes in administrative areas (e.g., Zhou & Chilvers, 2010). Moreover, selection effects arise as institutions willing to take part in research projects differ from those denying access to case files and cases are excluded due to missing data (cf. Section 5.3.3; cf. Table 4, A1). The limited generalizability of findings particularly warrants caution in cases in which the research question is aiming to highlight matters of incidents, characteristic, or consequences of maltreatment besides service provision (cf. Section 4.2.1). There are no simple solutions to the problem of generalizability. Case file analysis is sometimes the only feasible way to study a phenomenon as populations with specific characteristics are hard or even impossible to recruit in other ways. However, it is advisable to carefully compare the demographic characteristics of the sample with the demographics of the region as well as other research on the specific population (cf. Table 4, C4). Furthermore, effects inherent in the selection of sites and the exclusion of cases due to missing information need to be

5.3.4. Reliability Abstracting information from case files is difficult, and the reader might not always grasp the meaning of the author straight forward (Taylor, 2008). Furthermore, different individuals might not always draw the same conclusions. This poses a threat to the validity of the data collected by case file abstractors. There are some measures to enhance inter-rater reliability before 9

Children and Youth Services Review 108 (2020) 104551

S. Witte

option. However, some particularities of this specific data collection method have to be considered at every step of the research process. Failure to embed research findings in their institutional and cultural context, particularly the practice of case file keeping would otherwise threaten the validity of the results.

considered (cf. Table 4, A1 and M5). 5.4.2. Different levels of ‘reality’ As pointed out in Section 4.2.1, case file analysis is used for very different research questions and the information obtained from case files does not reflect the reality (Taylor, 2008; cf. Section 3 and Fig. 1) or the service user’s voice (Skehill, 2010) directly. In contrast, research findings from case files reflect various ‘levels of reality.' Especially in large data sets, a small bias in the use of the administrative data can lead to severe mistakes, e.g., if the lack of precision in diagnosis (Stern, 2016) or the underreporting of incidents of abuse and neglect (Fallon et al., 2010) is not taken into account. If the research is used to amend policies, this leads possibly to a distorted service provision and thus likely aggravates the situation for children in certain circumstances even more. In order to avoid false interpretation and thus adverse outcomes for practice, researchers need to understand the practice of case file keeping in the context from which they collect the data (cf. Section 3). Information about the practice of case file keeping should be sought before data collection (cf. Table 4, C1). However, some issues of case file keeping become only known to the researcher during the data collection. These should nevertheless be documented and used for the interpretation of the results (cf. Table 4, C2). In addition, it is beneficial to share and discuss preliminary results with practitioners during the initial data analysis (e.g., Ben-David et al., 2015; cf. Table 4, C3) as well as comparing results with those from other studies using different data collection methods (e.g., Lee, Jonson-Reid, & Drake, 2012).

Acknowledgement This work was supported by the Deutsche Forschungsgemeinschaft (DFG), Germany [grant numbers WA 949/12-1, KI 2170/2-1]. Declaration of Competing Interest None. References Annerbäck, E.-M., Svedin, C.-G., & Gustafsson, P. A. (2010). Characteristic features of severe child physical abuse: A multi-informant approach. Journal of Family Violence, 25(2), 165–172. https://doi.org/10.1007/s10896-009-9280-1. Arbeiter, E., & Toros, K. (2017a). Parental engagement in child protection assessment: A qualitative analysis of worker and parent perspectives. International Social Work, 121(5), 1–13. https://doi.org/10.1177/0020872817706409. Arbeiter, E., & Toros, K. (2017b). Participatory discourse: Engagement in the context of child protection assessment practices from the perspectives of child protection workers, parents and children. Children and Youth Services Review, 74, 17–27. https:// doi.org/10.1016/j.childyouth.2017.01.020. Ben-David, V., Jonson-Reid, M., Drake, B., & Kohl, P. L. (2015). The association between childhood maltreatment experiences and the onset of maltreatment perpetration in young adulthood controlling for proximal and distal risk factors. Child Abuse & Neglect, 46, 132–141. https://doi.org/10.1016/j.chiabu.2015.01.013. Berger, P. L., & Luckmann, T. (1991). The social construction of reality: A treatise in the sociology of knowledge. Harmondsworth: Penguin. Berliner, L., Hyman, I., Thomas, A., & Fitzgerald, M. (2003). Children's memory for trauma and positive experiences. Journal of Traumatic Stress, 16(3), 229–236. https:// doi.org/10.1023/A:1023787805970. Billoux, S., Arbus, C., Telmon, N., & Voltzenlogel, V. (2016). Autobiographical memory impairment in female victims of intimate partner violence. Journal of Family Violence, 31(7), 897–902. https://doi.org/10.1007/s10896-016-9838-7. de Bortoli, L., Ogloff, J., Coles, J., & Dolan, M. (2016). Preliminary evaluation of structured professional judgment to assess removal in child protection practice. Journal of Forensic Social Work, 5(1–3), 29–48. https://doi.org/10.1080/1936928X.2015. 1093572. Brakewood, B., & Poldrack, R. A. (2013). The ethics of secondary data analysis: Considering the application of Belmont principles to the sharing of neuroimaging data. NeuroImage, 82, 671–676. https://doi.org/10.1016/j.neuroimage.2013.02.040. Burke, N. J., Hellman, J. L., Scott, B. G., Weems, C. F., & Carrion, V. G. (2011). The impact of adverse childhood experiences on an urban pediatric population. Child Abuse & Neglect, 35(6), 408–413. https://doi.org/10.1016/j.chiabu.2011.02.006. Chen, W., Glasser, S., Benbenishty, R., Davidson-Arad, B., Tzur, S., & Lerner-Geva, L. (2010). The contribution of a hospital child protection team in determining suspected child abuse and neglect: Analysis of referrals of children aged 0–9. Children and Youth Services Review, 32(12), 1664–1669. https://doi.org/10.1016/j.childyouth.2010.07. 008. Choi, K. H., Reddy, L. F., & Spaulding, W. (2012). Child abuse rating system for archival information in severe mental illness. Social Psychiatry and Psychiatric Epidemiology, 47(8), 1271–1279. https://doi.org/10.1007/s00127-011-0435-z. Choudhary, A. K., Bradford, R. K., Dias, M. S., Moore, G. J., & Boal, D. K. B. (2012). Spinal subdural hemorrhage in abusive head trauma: A retrospective study. Radiology, 262(1), 216–223. https://doi.org/10.1148/radiol.11102390. Cole, S. Z., & Lanham, J. S. (2011). Failure to thrive:: an update. American Family Physician, 83(7), 829–834. Cook, J. A., Steigman, P. J., & Jonikas, J. A. (2014). Outcomes of programs serving mothers with psychiatric disabilities and their young children: A multisite case file abstraction study. Psychiatric Rehabilitation Journal, 37(3), 232–241. https://doi.org/ 10.1037/prj0000075. Courtney, M. E., Needell, B., & Wulczyn, F. (2004). Unintended consequences of the push for accountability: The case of national child welfare performance standards. Children and Youth Services Review, 26(12), 1141–1154. https://doi.org/10.1016/j.childyouth. 2004.05.005. English, D. J., & the Longscan Investigators (1997). Modified Maltreatment Classification System (MMCS). Retrieved from http:www.iprc.unc.edu/longscan/. Epstein, I. (2002). Using available clinical information in practice-based research: Mining for silver while dreaming of gold. Social Work in Health Care, 33(3–4), 15–32. https:// doi.org/10.1300/J010v33n03_03. Esposito, T., Trocmé, N., Chabot, M., Shlonsky, A., Collin-Vézina, D., & Sinha, V. (2013). Placement of children in out-of-home care in Québec, Canada: When and for whom initial out-of-home placement is most likely to occur. Children and Youth Services Review, 35(12), 2031–2039. https://doi.org/10.1016/j.childyouth.2013.10.010. Evans, A. D., Roberts, K. P., Price, H. L., & Stefek, C. P. (2010). The use of paraphrasing in

5.4.3. Institutional and cultural context Yet in order to produce that object various auteurs have had to call upon a complex system of rules, conventions, organizational strategies and conceptual schemes. (Prior, 2010, p. 31) Prior's (2010) statement for document analysis is true for case file analysis as well. Case files and the information they contain can only be understood in their institutional and cultural context. There are rules and regulations regarding the way case files are kept or even how data is entered in administrative datasets. These are subject to change over time and place. Even when there are binding rules, there is always the possibility of institutional processes, e.g. reluctance against a new tool of risk assessment, leading to differences in data entry. Over half of the articles reviewed were based on data collected in more than one site (cf. Section 4.2.2). This provides the researcher with the opportunity to consider the results within different institutional contexts. Due to insufficient sample sizes or additional information on practices at different sites, statistical procedures to account for variables on the institutional level are rarely used. However, there is a great value in considering the institutional context of the case files in a more qualitative way when interpreting the results. This can also be done conjointly with practitioners from the field (cf. Table 4, C3). Especially, when research questions are about characteristics and outcomes of maltreatment in general, it is necessary to consider not only the institutional but also the cultural context (cf. Table 4, C4). A possibility for this is to contrast the results to findings from different regions and countries. 6. Conclusion There is a large number of studies in child protection research which uses case file analysis. Their results contribute to the knowledge in child protection research and have a high utility for practice and policy (e.g., Chen et al., 2010; Jonson-Reid et al., 2013). Some research questions like service provision cannot be answered comprehensively without the use of case file data. A sufficient sample for other research questions, like female sex offenders, may only be obtained when using case files from a more extended period of time. As there is a high need for scientific evidence in child protection, not using case file analysis is not an 10

Children and Youth Services Review 108 (2020) 104551

S. Witte

Jonson-Reid, M., Emery, C. R., Drake, B., & Stahlschmidt, M. J. (2010). Understanding chronically reported families. Child Maltreatment, 15(4), 271–281. https://doi.org/ 10.1177/1077559510380738. Jonson-Reid, M., Drake, B., & Zhou, P. (2013). Neglect subtypes, race, and poverty: Individual, family, and service characteristics. Child Maltreatment, 18(1), 30–41. https://doi.org/10.1177/1077559512462452. Jud, A., Perrig-Chiello, P., & Voll, P. (2011). Less effort in worsening child protection cases?: The time-course of intensity of services. Children and Youth Services Review, 33(10), 2027–2033. https://doi.org/10.1016/j.childyouth.2011.05.032. Kagle, J. D. (1984). Restoring the clinical record. Social Work, 29(1), 46–50. https://doi. org/10.1093/sw/29.1.46. Klatt, T., Cavner, D., & Egan, V. (2014). Rationalising predictors of child sexual exploitation and sex-trading. Child Abuse & Neglect, 38(2), 252–260. https://doi.org/10. 1016/j.chiabu.2013.08.019. Knott, T., & Donovan, K. (2010). Disproportionate representation of African-American children in foster care: Secondary analysis of the National Child Abuse and Neglect Data System, 2005. Children and Youth Services Review, 32(5), 679–684. https://doi. org/10.1016/j.childyouth.2010.01.003. Kohl, P. L., Jonson-Reid, M., & Drake, B. (2011). Maternal mental illness and the safety and stability of maltreated children. Child Abuse & Neglect, 35(5), 309–318. https:// doi.org/10.1016/j.chiabu.2011.01.006. Kuldas, S., Hashim, S., Ismail, H. N., & Bakar, Z. A. (2015). Reviewing the role of cognitive load, expertise level, motivation, and unconscious processing in working memory performance. International Journal of Educational Psychology, 4(2), 142–169. https://doi.org/10.17583/ijep.2015.832. Lalayants, M., Epstein, I., Auslander, G. K., Chan, W. C. H., Fouché, C., Giles, R., & Vertigan, A. (2013). Clinical data-mining: Learning from practice in international settings. International Social Work, 56(6), 775–797. https://doi.org/10.1177/ 0020872811435370. Lee, S., Jonson-Reid, M., & Drake, B. (2012). Foster care re-entry:: Exploring the role of foster care characteristics, in-home child welfare services and cross-sector services. Children and Youth Services Review, 34(9), 1825–1833. https://doi.org/10.1016/j. childyouth.2012.05.007. Li, D., Chu, C. M., Goh, J. T. L., Ng, I. Y. H., & Zeng, G. (2015). Impact of childhood maltreatment on recidivism in youth offenders: A matched-control study. Criminal Justice and Behavior, 42(10), 990–1007. https://doi.org/10.1177/ 0093854815598598. Li, D., Chu, C. M., Ng, W. C., & Leong, W. (2014). Predictors of re-entry into the child protection system in Singapore: A cumulative ecological-transactional risk model. Child Abuse & Neglect, 38(11), 1801–1812. https://doi.org/10.1016/j.chiabu.2014. 08.017. Lightfoot, E., Hill, K., & LaLiberte, T. (2011). Prevalence of children with disabilities in the child welfare system and out of home placement: An examination of administrative records. Children and Youth Services Review, 33(11), 2069–2075. https://doi. org/10.1016/j.childyouth.2011.02.019. Lin, P.-Z., Bai, H.-Y., Sun, J.-W., Guo, W., Zhang, H.-H., & Cao, F.-L. (2017). Association between child maltreatment and prospective and retrospective memory in adolescents: The mediatory effect of neuroticism. Child Abuse & Neglect, 65, 58–67. https:// doi.org/10.1016/j.chiabu.2017.01.010. Lindsay, W. R., Carson, D., Holland, A. J., Michie, A. M., Taylor, J. L., Bambrick, M., & Steptoe, L. (2012). A comparison of sex offenders and other types of offenders referred to intellectual disability forensic services. Psychiatry, Psychology and Law, 19(4), 566–576. Long, M. (2014). Achieving successful returns from care: What makes reunification work? Australian Social Work, 67(1), 154–155. https://doi.org/10.1080/0312407X.2013. 857637. López López, M., del Valle, J. F., Montserrat, C., & Bravo, A. (2011). Factors affecting foster care breakdown in Spain. The Spanish Journal of Psychology, 14(01), 111–122. https://doi.org/10.5209/rev_SJOP.2011.v14.n1.9. Lucenko, B. A., Sharkova, I. V., Huber, A., Jemelka, R., & Mancuso, D. (2015). Childhood adversity and behavioral health outcomes for youth: An investigation using state administrative data. Child Abuse & Neglect, 47, 48–58. https://doi.org/10.1016/j. chiabu.2015.07.006. Malloy, L. C., & Mugno, A. P. (2016). Children's recantation of adult wrongdoing: An experimental investigation. Journal of Experimental Child Psychology, 145, 11–21. https://doi.org/10.1016/j.jecp.2015.12.003. Mann, R., & Williams, J. (2003). Standards in medical record keeping. Clinical Medicine, 3(4), 329–332. https://doi.org/10.7861/clinmedicine.3-4-329. McKeag, H., Christian, C. W., Rubin, D., Daymont, C., Pollock, A. N., & Wood, J. (2013). Subdural hemorrhage in pediatric patients with enlargement of the subarachnoid spaces. Journal of Neurosurgery. Pediatrics, 11(4), 438–444. https://doi.org/10.3171/ 2012.12.PEDS12289. Melinder, A., Baugerud, G. A., Ovenstad, K. S., & Goodman, G. S. (2013). Children's memories of removal: A test of attachment theory. Journal of Traumatic Stress, 26(1), 125–133. https://doi.org/10.1002/jts.21784. Murphy, E., & Fairtlough, A. (2015). The successful reunification of abused and neglected looked after children with their families: A case-file audit. British Journal of Social Work, 45(8), 2261–2280. https://doi.org/10.1093/bjsw/bcu093. Myhre, A. K., Adams, J. A., Kaufhold, M., Davis, J. L., Suresh, P., & Kuelbs, C. L. (2013). Anal findings in children with and without probable anal penetration: A retrospective study of 1115 children referred for suspected sexual abuse. Child Abuse & Neglect, 37(7), 465–474. https://doi.org/10.1016/j.chiabu.2013.03.011. Oral, R., Bayman, L., Assad, A., Wibbenmeyer, L., Buhrow, J., Austin, A., & Bayman, E. O. (2011). Illicit drug exposure in patients evaluated for alleged child abuse and neglect. Pediatric Emergency Care, 27(6), 490–495. https://doi.org/10.1097/PEC. 0b013e31821d860f.

investigative interviews. Child Abuse & Neglect, 34(8), 585–592. https://doi.org/10. 1016/j.chiabu.2010.01.008. Fallon, B., Trocme, N., Fluke, J., MacLaurin, B., Tonmyr, L., & Yuan, Y.-Y. (2010). Methodological challenges in measuring child maltreatment. Child Abuse & Neglect, 34(1), 70–79. https://doi.org/10.1016/j.chiabu.2009.08.008. Finkelhor, D. (1984). Child sexual abuse: New theory and research (7. print). New York NY u.a.: The Free Press [u.a.]. Fleming, P., Biggart, L., & Beckett, C. (2015). Effects of professional experience on child maltreatment risk assessments: A comparison of students and qualified social workers. British Journal of Social Work, 45(8), 2298–2316. https://doi.org/10.1093/ bjsw/bcu090. Gangat, M., Klein, G. W., & Heptulla, R. A. (2012). Foster care and type 1 diabetes in the Bronx: A case series. Journal of Pediatric Endocrinology & Metabolism : JPEM, 25(7–8), 775–779. https://doi.org/10.1515/jpem-2012-0035. Garrett, P. M. (2005). Social work’s ‘electronic turn’: Notes on the deployment of information and communication technologies in social work with children and families. Critical Social Policy, 25(4), 529–553. https://doi.org/10.1177/0261018305057044. Gifford, E. J., Eldred, L. M., Sloan, F. A., & Evans, K. E. (2016). Parental criminal justice involvement and children's ivolvement with child protective services: Do adult drug treatment courts prevent child maltreatment? Substance Use & Misuse, 51(2), 179–192. https://doi.org/10.3109/10826084.2015.1089906. Gilbert, N. (2012). A comparative study of child welfare systems: Abstract orientations and concrete results. Children and Youth Services Review, 34(3), 532–536. https://doi. org/10.1016/j.childyouth.2011.10.014. Gonzalez-Izquierdo, A., Ward, A., O'Donnell, M., Li, L., Roposch, A., Stanley, F., & Gilbert, R. (2013). Cross-country comparison of victimisation-related injury admission in children and adolescents in England and Western Australia. BMC Health Services Research, 13, 260. https://doi.org/10.1186/1472-6963-13-260. Goodman-Delahunty, J. (2014). Profiling parental child sex abuse. Trends & Issues in Crime & Criminal Justice. 465, 1–8. Green, B. L., Ayoub, C., Bartlett, J. D., von Ende, A., Furrer, C., Chazan-Cohen, R., & Klevens, J. (2014). The effect of early head start on child welfare system involvement: A first look at longitudinal child maltreatment outcomes. Children and Youth Services Review, 42, 127–135. https://doi.org/10.1016/j.childyouth.2014.03.044. Green, B. L., Ayoub, C., Dym Bartlett, J., Furrer, C., von Ende, A., Chazan-Cohen, R., & Nygren, P. (2015). It's not as simple as it sounds: Problems and solutions in accessing and using administrative child welfare data for evaluating the impact of early childhood interventions. Children and Youth Services Review, 57, 40–49. https://doi. org/10.1016/j.childyouth.2015.07.015. Greiner, M. V., Richards, T. J., Care, M. M., & Leach, J. L. (2013). Prevalence of subdural collections in children with macrocrania. American Journal of Neuroradiology, 34(12), 2373–2378. https://doi.org/10.3174/ajnr.A3588. Hall, C., Slembrouck, S., & Sarangi, S. (2006). Language practices in social work: Categorisation and accountability in child welfare. London: Routledge. Hawkes, C. (2011). Description of a UK study of onset of sexually harmful behaviour before the age of ten years in boys referred to a specialist assessment and treatment service. Child Abuse Review, 20(2), 82–101. https://doi.org/10.1002/car.1133. Hayes, D., & Devaney, J. (2016). Accessing social work case files for research purposes. Qualitative Social Work: Research and Practice, 3(3), 313–333. https://doi.org/10. 1177/1473325004045667. Hertzum, M. (2002). Six roles of documents in professionals’ work. In S. Bødker, M. Kyng, & K. Schmidt (Eds.). Ecscw '99: Proceedings of the Sixth European Conference on Computer Supported Cooperative Work 12-16 September 1999, Copenhagen, Denmark (pp. 41–60). Dordrecht: Kluwer Academic Publishers. https://doi.org/10.1007/0306-47316-X_3. Hovens, J. G. F. M., Giltay, E. J., Wiersma, J. E., Spinhoven, P., Penninx, B. W. J. H., & Zitman, F. G. (2012). Impact of childhood life events and trauma on the course of depressive and anxiety disorders. Acta Psychiatrica Scandinavica, 126(3), 198–207. https://doi.org/10.1111/j.1600-0447.2011.01828.x. Hritz, A. C., Royer, C. E., Helm, R. K., Burd, K. A., Ojeda, K., & Ceci, S. J. (2015). Children's suggestibility research: Things to know before interviewing a child. Anuario de Psicología Jurídica, 25(1), 3–12. https://doi.org/10.1016/j.apj.2014.09. 002. Huffhines, L., Tunno, A. M., Cho, B., Hambrick, E. P., Campos, I., Lichty, B., & Jackson, Y. (2016). Case file coding of child maltreatment: Methods, challenges, and innovations in a longitudinal project of youth in foster care. Children and Youth Services Review, 67, 254–262. https://doi.org/10.1016/j.childyouth.2016.06.019. Hussey, D. L. (2012). An in-depth analysis of domestically adopted children with special needs and their biological mothers. Journal of Social Work, 12(5), 528–544. https:// doi.org/10.1177/1468017310393548. Huuskonen, S. (2014). Recording and use of information in a client information system in child protection work. Tampere: Tampere University Press. Huuskonen, S., & Vakkari, P. (2013). “I Did It My Way”: Social workers as secondary designers of a client information system. Information Processing & Management, 49(1), 380–391. https://doi.org/10.1016/j.ipm.2012.05.003. Huuskonen, S., & Vakkari, P. (2015). Selective clients' trajectories in case files: Filtering out information in the recording process in child protection. British Journal of Social Work, 45(3), 792–808. https://doi.org/10.1093/bjsw/bct160. Jergeby, U., & Soydan, H. (2002). Assessment processes in social work practice when children are at risk: A comparative cross-national vignette study. Journal of Social Work Research and Evaluation, 3(2), 127–144. Johnson-Motoyama, M., Dettlaff, A. J., & Finno, M. (2012). Parental nativity and the decision to substantiate: Findings from a study of latino children in the second national survey of child and adolescent well-being (NSCAW II). Children and Youth Services Review, 34(11), 2229–2239. https://doi.org/10.1016/j.childyouth.2012.07. 017.

11

Children and Youth Services Review 108 (2020) 104551

S. Witte

influence of confirmation bias in developing decision supportive reasoning. Child Abuse & Neglect, 49, 76–85. https://doi.org/10.1016/j.chiabu.2015.01.015. Stern, R. (2016). Administrative and survey data: Potential and pitfalls. The Journal of Investigative Dermatology, 136(11), 2122–2124. https://doi.org/10.1016/j.jid.2016. 07.022. Taylor, C. (2008). Trafficking in facts. Qualitative Social Work: Research and Practice, 7(1), 25–42. https://doi.org/10.1177/1473325007086414. Thomas, K. A., Sorenson, S. B., & Joshi, M. (2010). Police-documented incidents of intimate partner violence among young women. Journal of Women's Health, 19(6), 1079–1087. https://doi.org/10.1089/jwh.2009.1612. Tonk, G., Adorjáni, J., & László, É. (2012). Providing services to maltreated children and their families. Some findings of Romanian case based surveillance study. Social Work Review/Revista de Asistenta Sociala, 11(2), 91–102. Tourangeau, R. (2000). Remembering what happened: Memory errors and survey reports. In A. A. Stone (Ed.). The science of self-report: Implications for research and practice (pp. 29–47). Mahwah, N.J.: Lawrence Erlbaum. Tursz, A., Crost, M., Gerbouin-Rerolle, P., & Cook, J. M. (2010). Underascertainment of child abuse fatalities in France: Retrospective analysis of judicial data to assess underreporting of infant homicides in mortality statistics. Child Abuse & Neglect, 34(7), 534–544. https://doi.org/10.1016/j.chiabu.2009.12.005. Vartanian, T. P. (2011). Secondary data analysis. Pocket guides to social work research methods. Oxford, New York: Oxford University Press. Vellut, N., Cook, J. M., & Tursz, A. (2012). Analysis of the relationship between neonaticide and denial of pregnancy using data from judicial files. Child Abuse & Neglect, 36(7-8), 553–563. https://doi.org/10.1016/j.chiabu.2012.05.003. Walsh, W. A., Lippert, T., Edelson, M. G., & Jones, L. M. (2015). Length of time to resolve criminal charges of child sexual abuse: A three-county case study. Behavioral Sciences & the Law, 33(4), 528–545. https://doi.org/10.1002/bsl.2187. Wells, M., & Correia, M. (2012). Reentry into out-of-home care: Implications of child welfare workers' assessments of risk and safety. Social Work Research, 36(3), 181–195. https://doi.org/10.1093/swr/svs011. Wilson, M. L., Tumen, S., Ota, R., & Simmers, A. G. (2015). Predictive modeling: Potential application in prevention services. American Journal of Preventive Medicine, 48(5), 509–519. https://doi.org/10.1016/j.amepre.2014.12.003. Wu, M.-F., Lu, T.-H., Lin, C.-J., & Feng, J.-Y. (2015). Risk factors and physical signs of child abuse in hospitalized children in Taiwan. Children and Youth Services Review, 58, 137–141. https://doi.org/10.1016/j.childyouth.2015.09.015. Zhang, S., Fuller, T., & Nieto, M. (2013). Didn't we just see you?: Time to recurrence among frequently encountered families in CPS. Children and Youth Services Review, 35(5), 883–889. https://doi.org/10.1016/j.childyouth.2013.02.014. Zhou, A. Z., & Chilvers, M. (2010). Infants in Australian out-of-home care. British Journal of Social Work, 40(1), 26–43. https://doi.org/10.1093/bjsw/bcn058.

O'Rourke, L. (2010). Recording in social work: Not just an administrative task. Bristol: The Policy Press. Osterling, K. L., & Han, M. (2011). Reunification outcomes among Mexican immigrant families in the child welfare system. Children and Youth Services Review, 33(9), 1658–1666. https://doi.org/10.1016/j.childyouth.2011.04.020. Osterling, K. L., Lee, P. A., & Hines, A. H. (2012). The influence of family reunification services on racial/ethnic disparities in permanency outcomes for children in the child welfare system. Journal of Public Child Welfare, 6(3), 330–354. https://doi.org/10. 1080/15548732.2012.683372. Ou, S.-R., & Reynolds, A. J. (2010). Childhood predictors of young adult male crime. Children and Youth Services Review, 32(8), 1097–1107. https://doi.org/10.1016/j. childyouth.2010.02.009. Palusci, V. J., Wirtz, S. J., & Covington, T. M. (2010). Using capture-recapture methods to better ascertain the incidence of fatal child maltreatment. Child Abuse & Neglect, 34(6), 396–402. https://doi.org/10.1016/j.chiabu.2009.11.002. Pinto, R. J., & Maia, Â. C. (2013). A comparison study between official records and selfreports of childhood adversity. Child Abuse Review, 22(5), 354–366. https://doi.org/ 10.1002/car.2232. Prior, L. (2010). Using documents in social research. Introducing qualitative methods. London, Thousand Oaks, Calif: Sage Publications. Putnam-Hornstein, E., & Needell, B. (2011). Predictors of child protective service contact between birth and age five: An examination of California's 2002 birth cohort. Children and Youth Services Review, 33(11), 2400–2407. https://doi.org/10.1016/j.childyouth. 2011.07.010. Robboy, J., & Anderson, K. G. (2011). Intergenerational child abuse and coping. Journal of Interpersonal Violence, 26(17), 3526–3541. https://doi.org/10.1177/ 0886260511403758. Salwen, J. K., Hymowitz, G. F., Vivian, D., & O'Leary, K. D. (2014). Childhood abuse, adult interpersonal abuse, and depression in individuals with extreme obesity. Child Abuse & Neglect, 38(3), 425–433. https://doi.org/10.1016/j.chiabu.2013.12.005. Schneiderman, J. U., Smith, C., Arnold-Clark, J. S., Fuentes, J., Duan, L., & Palinkas, L. A. (2013). Overweight and obesity among Hispanic children entering foster care: A preliminary examination of polyvictimization. Child Maltreatment, 18(4), 264–273. https://doi.org/10.1177/1077559513508236. Semanchin Jones, A. (2015). Implementation of differential response: A racial equity analysis. Child Abuse & Neglect, 39, 73–85. https://doi.org/10.1016/j.chiabu.2014. 04.013. Skehill, C. (2010). History of child welfare and protection social work in Northern Ireland: Finding continuity amongst discontinuity in case files from 1950 to 1968. Child Care in Practice, 16(4), 309–325. https://doi.org/10.1080/13575279.2010. 498413. Spratt, T., Devaney, J., & Hayes, D. (2015). In and out of home care decisions: The

12