Investigative interviewing with children: evaluation of the effectiveness of a training program for child protective service workers1

Child Abuse & Neglect, Vol. 23, No. 7, pp. 701–713, 1999 Copyright © 1999 Elsevier Science Ltd Printed in the USA. All rights reserved 0145-2134/99/$–see front matter

Pergamon

PII S0145-2134(99)00042-3

SPOTLIGHT ON PRACTICE

INVESTIGATIVE INTERVIEWING WITH CHILDREN: EVALUATION OF THE EFFECTIVENESS OF A TRAINING PROGRAM FOR CHILD PROTECTIVE SERVICE WORKERS KURT A. FREEMAN

AND

TRACY L. MORRIS

Department of Psychology, West Virginia University, Morgantown, WV, USA

ABSTRACT Objective: Previous research suggests that training workshops on investigative interviewing conducted with front-line workers, such as police officers or child protective service (CPS) workers, may result in improved knowledge about appropriate strategies to use. Limited research has addressed whether such training programs result in improvements in actual interviewing skills. The present investigation assessed the impact of a training workshop on CPS workers’ knowledge about conducting and behavior during investigative interviews. Method: Twelve CPS workers participated. To evaluate the impact of the training, a quasi-experimental design was used. Participants completed outcome measures prior to, immediately following, and 3 months following the training. Outcome measures included (a) a questionnaire to assess knowledge about interviewing practices, (b) simulated interviews with a confederate to assess participant behavior during an investigative interview, and (c) questionnaires to gather information related to the simulated interviews (e.g., materials requested for use during interview, abuse status of confederate). All participants completed 6 hours of training during 3 weekly 2-hour sessions. Results: Participants’ knowledge on the topic, as well as several interviewing skills during simulated interviews, improved following the training. However, the training failed to impact several key interviewing skills such as the types of questions asked or the length of the interview. Conclusions: Knowledge-based workshop training programs may not adequately prepare CPS workers to conduct appropriate investigative interviews with children who were allegedly abused. Further, knowledge about how to conduct such interviews may not be the best indicator of whether someone is prepared for this aspect of the job. © 1999 Elsevier Science Ltd Key Words—Investigative interviewing, Training, Child protective services, Child sexual abuse.

Portions of this paper represent work completed in partial fulfullment of the requirements for a Doctor of Philosophy degree by the first author. Support for this project was provided by grants to the first author from the West Virginia University Office of Academic Affairs and the Department of Psychology Alumni Fund, both of which are gratefully acknowledged. Submitted for publication September 18, 1998, final revision received January 4, 1999, accepted January 11, 1999. Requests for reprints should be sent to Kurt A. Freeman, PhD, Counseling Psychology Program, Pacific University, 511 SW 10th Ave., 4th Floor, Portland, OR, 97205. 701

702

K. A. Freeman and T. L. Morris

DUE TO CONTINUED high occurrence of childhood sexual abuse (CSA), conducting effective and appropriate investigative interviews with children remains one of the most important tasks of child protective service (CPS) workers. Unfortunately, declassification of the CPS worker position in many states has resulted in fewer workers with social work or education backgrounds (Leung, Cheung, & Kay, 1994). Thus, more people may enter the position with little background information about, and understanding of, issues relevant to interacting effectively with children in general. Deficits may be more obvious in the context of investigating abuse allegations. Such investigations require knowledge of developmental capabilities and childhood memory. Additionally, escalating caseload sizes necessitate that CPS workers provide rapid, yet accurate, assessments of risk to the child. Therefore, possessing the necessary skills to properly interview children appears important. Further, because CPS workers may be held liable for their actions (Besharov, 1984), using proper interviewing skills may have legal benefits for workers, as well. Despite the fact that CPS and other front-line workers may not receive the training necessary to complete proper interviews with children alleged to be victims of CSA (see Ceci & Bruck, 1995; Wood, Orsak, Murphy, & Cross, 1996), as well as legal cases in which use of certain interviewing skills were questioned (e.g., State v. Michaels, 1994), there remains a paucity of reports in the literature on training programs directed to meet this need. Stone, Tyler, and Mead (1984) described a program for police officers that addressed, among other topics, special considerations when interviewing children who were allegedly sexually abused. Hibbard, Serwint, and Connolly (1987) reported outcome data on an educational program regarding evaluating alleged sexual abuse victims, with the curriculum including components specific to procedures for interviewing children. Participants included various professionals involved regularly with investigations of alleged CSA, with the majority being CPS practitioners. Using a questionnaire administered pre- and posttraining, results showed positive change in knowledge of issues relevant to investigations of CSA. More recently, Sullivan and Clancy (1990) experimentally evaluated a training on, among other topics, investigative interviewing techniques for use with adolescents who were allegedly sexually abused. A total of 350 health care practitioners who may gather initial information regarding sexual abuse (e.g., nurses, medical doctors) participated. Prior to and following the training, participants completed a one-page questionnaire designed to assess knowledge of topics presented during the training, with one question specific to investigative interviewing strategies. The authors suggested that results showed significant improvements in interviewing skills because participants “were able to identify more specific actions they could take in an interview to make it easier for an adolescent to disclose that he or she had been sexually abused” (Sullivan & Clancy, 1990, p. 212). Finally, Cheung and colleagues (Cheung, 1997; Stevenson, Leung, & Cheung, 1992) reported attempts to assess the impact of in-service training programs on interviewing skills in the context of CSA with CPS workers. Prior to and following a 10-day training, Stevenson and colleagues videotaped 36 social workers conducting simulated interviews with a confederate who depicted a 10-year-old girl with a history of sexual abuse. Using a 35-item rating scale designed specifically for the study, observers rated participants’ interviewing abilities. Results showed no significant improvements in interview skills. More recently, Cheung described the impact of training on police officers, social workers, and clinical psychologists in Hong Kong. Participants were provided with a specific interview protocol for conducting video-recorded CSA investigations. Using qualitative data, he reported that training improved how participants conducted the interview protocol. Although previous evaluations of the impact of training workshops on investigative interviewing are important, they are limited for several reasons. First, several studies (i.e., Cheung, 1997; Stone et al., 1984; Sullivan & Clancy, 1990) failed to employ adequate procedures allowing for an empirical evaluation of the impact of the training. For example, Stone and colleagues simply described their program. Additionally, Sullivan and Clancy measured the impact of training on interviewing skills with one item on a questionnaire. Such an approach likely is unable to assess

Investigative interviewing with children

703

fully the level of detail needed to make sound statements about the influence of training on participant behavior. Second, of the studies using experimental methods, little attention was given to assessing whether the training produced change in the performance of participants, the exception being the Stevenson and colleagues (1992) study. Although indicators of changes in knowledge are important, they may not serve as markers of change in interviewing behavior. Additionally, in the one study to date that has attempted to measure behavior change, methodological concerns exist. Specifically, as Stevenson and colleagues discussed, using the same case scenario during both preand post-training assessment may have limited generalization and influenced how participants interacted with the confederate in the latter interview. Further, conducting the post-training assessment immediately following the training may not have allowed participants adequate time and opportunity to learn the new skills. The purpose of the present study was to assess the impact of a training workshop on CPS workers’ abilities to conduct investigative interviews effectively with children. In addition to assessing the impact of training on participants’ knowledge, an evaluation of its effect on actual performance was included due to increasing focus on competency- and outcome-based measures of training for CPS workers (e.g., Leung et al., 1994; Mitchell, White, Wright, & Pecora, 1989; Stevenson et al., 1992). METHOD Participants Twelve Caucasian employees (nine caseworkers and three supervisors; eight women and four men) of a local CPS agency participated. Mean age of participants was 33.5 years (range 22 to 65 years). Nine participants held a Bachelor of Arts or Bachelor of Science degree, one a Master of Arts, and one had at least 1 year of education beyond a Master of Arts. Participants’ mean length of employment in their current position was 21.2 months (range 1.0 – 84.0), whereas mean length of employment at the agency in any position was 52.0 months (range 1.0 –207.0). Prior to training, participants had completed an average of 1.7 training programs related to investigative interviewing (range 0 – 4), for an average of 17.8 hours of training (range 0 – 68). Informed consent was obtained from each participant. Confederates Three Caucasian women (ages 20, 20, and 30) served as confederates during the simulated interviews (see below for discussion). Each confederate learned scripted information about a different hypothetical child. Two confederates learned information about children who experienced intra-familial sexual abuse whereas the third learned information about a child coached by her mother to provide false allegations of intra-familial abuse. Confederates learned the following information about the children they portrayed: (a) personal information (e.g., age, favorite hobbies, family make-up); (b) abuse history (e.g., acts perpetrated against them, relationship with perpetrator, chronological abuse history); (c) to whom they initially disclosed the abuse; and (d) what they were told by the perpetrators about the consequences of abuse disclosure. For the confederate acting as a child providing false allegations, details regarding how she was coached to report being abused was provided in place of abuse history information. All confederates were educated about and coached to use developmentally appropriate language. Prior to initiating the study, confederates received approximately 20 hours of instruction regarding their role. Prior to the post-training assessment, confederates reviewed one video-taped simulated interview from the pre-training assessment to re-familiarize themselves with their scenario. Finally,

704

K. A. Freeman and T. L. Morris

prior to collecting follow-up data, confederates spent approximately 10 hours reviewing and practicing their scenarios, as well as again watching a previously-recorded interview. Because of the interactive nature of the interviews, confederates were instructed to follow the interviewer’s lead while providing answers based on the case scenario information. Also, confederates were instructed to deny any false allegations of abuse by the interviewers (except for the confederate presenting false allegations) and to resist suggestion in the form of misleading questions. Case scenario information was changed across different data collection points so that time-based facts (e.g., birthdays, dates of particular events) were consistent. For example, the date that one confederate’s father moved out of the house was changed from the beginning of the school year at the pre-training assessment to around Thanksgiving at the follow-up assessment. Experimental Design To evaluate whether changes in outcome measures occurred following the training, a quasiexperimental design was used (Campbell & Stanley, 1963). As such, no experimental control group was employed. Data on the various outcome measures were completed prior to, immediately following, and 3 months following the training workshop. To avoid the influence of previous exposure to information, a counter-balanced design was used so that each participant was exposed to a different case scenario for the pre-training, post-training, and follow-up assessments of performance. MEASURES Prior to receiving training, participants completed a demographic questionnaire designed to gather information about themselves, their previous training, and their work experience. To assess the impact of the training workshop, all participants completed a measure of knowledge of procedures for best practices in investigative interviewing with children and a direct measure of the application of interviewing skills. Knowledge Measure All participants completed a 27-item Knowledge Questionnaire, developed specifically for the current investigation, to assess participants’ knowledge regarding appropriate investigative interviewing techniques to be used with children. Knowledge questions were designed to assess the participants’ recall of facts (e.g., factors influencing children’s reports, effects of sexual abuse, developmentally appropriate behaviors), steps in interviewing process (e.g., importance of establishing rapport, beginning with free-recall/open-ended questions), and definitions (e.g., of leading questions). Application questions assessed participants’ ability to use material introduced in the workshop to provide solutions to inappropriate practices in vignette form. Measure of Performance All participants completed simulations whereby they interviewed the confederates described above. Participants received case information equivalent to that typically gathered prior to investigative interviews, approximately 20 to 30 minutes prior to engaging in the simulated interview. In addition to accurate information about the case scenario, one inaccurate detail about the alleged abuse was provided. Prior to the interview, participants completed a brief questionnaire on which they indicated the information they would gather from a parent before conducting the interview, what materials (e.g., anatomically-correct drawings or dolls, toys) they would use in actual interviews, and whether they would allow a parent to be present in the room during the interview.

Investigative interviewing with children

705

The information and materials requested were provided. Participants were given 30 minutes to establish rapport with the confederate, gather abuse-related information, and conclude the interview. All interviews were videotaped for later scoring. Following the interview, the confederates left the room and participants were asked to complete a brief questionnaire to assess their impressions as to the abuse status of the confederates, as well as any further steps they would complete in the hypothetical investigation.

PROCEDURES All outcome measures, as well as the workshop, were completed at a local CPS agency. The knowledge and performance measures were completed within 2 weeks prior to the training. Participants were instructed not to collaborate or use other relevant materials when completing the knowledge questionnaires. The simulated interviews were conducted with each participant individually in one of three interviewing rooms at the agency. Each room measured approximately 9’ 3 9’, had no windows, and was supplied with a round table and adult-sized chairs. The workshop was completed with participants in two groups. The training was based on research produced within the past 10 years on issues relevant to conducting investigative interviews, with the focus on findings within the past 5 years. Training topics included the potential influences of social and developmental factors on memory, with an emphasis on the effect of various question formats (e.g., open-ended versus leading); appropriate steps in the interview process (e.g., initiating the interview, using open-ended questions); information on developmental capabilities of children of different ages; and when to refer cases to other professionals (a more detailed outline of training topics is available from the first author). The 6-hour training workshop was completed in three consecutive 2-hour weekly meetings. Instructional modalities used included lecture, role-play, group discussions, and viewing of videotapes depicting information on investigative interviewing. Due to professional emergencies, two participants did not attend 1 day of training. These participants watched the training on video-tape and then met with the first author to discuss any questions or clarifications. Following the training, the performance measure again was completed within a 2-week period. All participants completed the Knowledge Questionnaire within 1 month following the training. Participants also completed a simulated interview and Knowledge Questionnaire 3 months following the training. Data Analyses Coding of the knowledge measure. Because the Knowledge Questionnaire contained many openended questions, scoring involved a tiered system whereby participants received full credit for capturing the main idea or concept in their answers and received successively lower scores if their answer failed to do so. For example, in response to a question regarding how to initiate an interview, participants received two points if they conveyed the importance of introducing the purpose of the interview and establishing rapport with the child. One point was given if they mentioned only one concept, whereas no points were given for other responses. Closed questions that required a prescribed response were scored using a “correct–incorrect” approach. To determine participants’ scores on the questionnaires, a trained research assistant served as the primary coder. Although aware of the general purpose and hypotheses of the study, she was unaware of the time period when any individual questionnaire was completed (e.g., pre-training, follow-up). Knowledge Questionnaire total scores were determined by summing the obtained score for questions 1 through 26. The total score possible was 85. Intercoder agreement was calculated for 61% of the questionnaires. Data regarding intercoder agreement were collected by the first author only for open-ended questions for which coder bias

706

K. A. Freeman and T. L. Morris

Table 1. Categories of Interviewer Behaviors on the Modified Child Abuse Interview Interaction Coding System Type of question asked: Free recall/open-ended: any question that does not suggest a particular answer, including anything that begins with “tell me” Leading: any question that implies a certain answer, or has the answer embedded in it Direct/focused: any “Who,” ”When,” “Where,” and “How” questions that are not leading in nature; also any “What” question that is directed toward a specific circumstance Yes-No: any question that could be answered with a “yes–no” response, but is not leading or multiple choice Multiple choice: any question that gives options Type of support provided: Praise not for disclosure: compliments about child’s appearance, efforts, attention, including words such as “good,” “great,” “that’s right” Praise for disclosure: provides affirming response after child provides details about alleged abuse Type of information provided: Explain purpose and procedures: explaining general procedures that should occur at the beginning and end of an interview, including introduces self, describes purpose of interview, provides “ground rules” for interview process, discusses what will happen following the interview Instruction on the use of aids: describing anatomically correct dolls or pictures; instructing child to draw picture related to abuse, including a picture of their house, their family, etc., as long as it applies to allegations Abuse-related information: talk about topic related to the alleged abuse No talking (interval during which neither interviewer nor confederate speak) Note. Table adapted from Wood, Orsak, Murphy, & Cross (1996).

may affect the score. Agreements were defined as both coders assigning the same numerical value to a given answer and was determined by summing the total number of agreements divided by the number of agreements plus disagreements multiplied by 100. Intercoder agreement was 86.0% (range 69.0 –93.1). Coding of the performance measure. Data on participant behavior during the simulated interviews were collected using a modified version of the Child Abuse Interview Interaction Coding System (CAIICS; Wood, 1990, as cited in Wood et al., 1996). The CAIICS was designed to sample both interviewer and child behavior during investigative interviews occurring in the context of allegations of CSA. For the current study, data were collected only on interviewer behavior. Data on targeted responses were collected using partial-interval recording within continuous 10-s intervals. Using the modified CAIICS, information on specific responses was gathered (see Table 1). These specific responses were classified into three global classes of interviewer behavior: (a) the type of question asked, (b) the type of support provided, and (c) the type of information provided (Wood et al., 1996). Wood and colleagues (1996) suggested that interviewers are leading when they “direct or focus the response of the child toward abuse-related disclosures, suggest that abuse has occurred, or imply expectations of disclosure of abuse” (p. 84). However, as they discussed, certain behaviors (e.g., choice questions, providing abuse-related information already identified in child’s disclosure) may be appropriate after a child discloses the abuse. To measure whether these behaviors occurred prior to or following disclosure of abuse, observers indicated the interval in which the child first disclosed the abuse. Before viewing the simulated interviews, the primary observer attained at least 90% agreement with the first author for three consecutive sessions while watching practice tapes. Although aware of the general purpose of the study, the observer was unaware as to the point at which any individual simulated interview was conducted (i.e., pre-training, post-training, follow-up). The primary observer collected data on all of the simulated interviews. Interobserver agreement data were collected by the first author on 26.5% of the simulated interviews. Effective percent occurrences, percent nonoccurrences, and total percent agreement were calculated using the methods outlined by Hartmann (1977). Mean percent occurrence, percent nonoccurrence, and total

Investigative interviewing with children

707

percent agreement for types of questions asked were 83%, 96.1%, and 89.6%; for support provided 65.6%, 98.4%, and 89.6%; for information provided 95.4%, 99.6%, and 98%; and for no talking 83.8%, 99.7%, and 96.9%, respectively. Data collected using the modified CAIICS were summarized several ways. First, the percentage of each specific class of interviewer behavior was calculated by dividing the frequency of intervals scored with that response by the total number of intervals during the interview, multiplied by 100. Second, because open-ended questions may be considered the least leading (e.g., Kuehnle, 1996; United States Department of Justice, 1996), it may be most appropriate for interviewers to rely on their use to a greater extent. Thus, the ratio of open-ended questions in relation to all questions asked was calculated by summing the frequency of intervals scored with open-ended questions and dividing by the total number of intervals in which any type of question was asked, multiplied by 100. This is referred to as the Open-ended composite score. Third, based on Wood and colleagues’ (1996) recommendation, a composite score of the percentage of inappropriate interviewer behaviors that occurred, referred to as Inappropriate Behaviors, was derived. This was calculated by summing the percent of intervals scored with behaviors identified as potentially influencing the child’s report (i.e., any leading questions; praise for disclosures; yes–no and multiple choice questions, and providing abuse-related information prior to disclosure of abuse). Fourth, a composite score referred to as Appropriate Questions was determined by summing the frequency of intervals scored with the occurrence of free recall/open-ended or direct questions. Fifth, to assess whether participants spent more time explaining the purpose and procedures of the interview at appropriate times during the interview (e.g., introducing self in the beginning, explaining what will happen next toward the end) an Explain Purpose composite score was calculated. This was determined by calculating the percent of intervals within the first and last 2 minutes of the interview in which the participant was observed to emit behavior consistent with the explain purpose and procedure code. In addition to evaluating changes in participants’ behavior during the actual simulated interviews, responses on questionnaires completed immediately prior to and following the interviews were evaluated. Specifically, changes in the following areas were assessed: (a) the types of props requested, (b) whether participants would allow the confederate’s mother in the room during the interview, and (c) whether participants were able to identify correctly the confederates’ abuse status.

RESULTS Effects of Training on Participants’ Knowledge The first purpose of the current study was to assess whether a training on investigative interviewing would result in significant increases in participants’ knowledge about the topic, as well as to determine whether obtained improvements (if any) maintained. As noted earlier, the total score possible on the Knowledge Questionnaire was 85. Eleven participants completed the pre-training questionnaire, 12 the post-training, and 11 the follow-up. Pre- and post-training total scores on the Knowledge Questionnaire, as well as post-training and follow-up scores, were compared using paired-sample t-tests procedures. Results showed a significant increase in knowledge scores from pre-training (M 5 42.8, SD 5 8.7) to post-training (M 5 50.3, SD 5 7.5), t (10) 5 3.8, p , .01. Further, no significant differences were observe between post-training (M 5 50.36, SD 5 7.80) and follow-up (M 5 49.82, SD 5 6.08) scores, suggesting that improvements noted after the training maintained.

708

K. A. Freeman and T. L. Morris Table 2. Descriptive Statistics of Participants’ Behavior During Simulated Interviews Time of Data Collection Behavior category

Pre-training

Post-training

Follow-up

Open-ended Questions Leading Questions Direct Questions Multiple Choice Questions Yes–No Questions Praise not for Disclosure Praise for Disclosure Explain Purpose Abuse Information Instructional Aid Open-ended† Appropriate Questions† Inappropriate Behaviors† Explain Purpose†

8.8 (4.83) 4.0 (3.24) 22.0 (8.77) 2.7 (1.62) 34.7 (9.95) 6.0 (4.58)a .1 (.25) 6.8 (4.61) 3.0 (3.75) 1.0 (1.22) 12.2 (6.56)a 30.7 (8.96) 44.1 (19.8) 4.7 (3.2)a

12.6 (4.1) 3.8 (4.41) 21.8 (7.31) 2.9 (2.01) 31.8 (6.43) 3.6 (2.51)b 0.0 (0.0) 9.6 (4.72) 2.7 (2.04) 1.6 (1.63)a 17.5 (5.43)b 34.4 (7.61) 37.2 (16.96) 7.7 (2.99)b

15.3 (9.21) 2.3 (2.47) 23.4 (8.5) 2.5 (2.39) 34.4 (9.97) 2.8 (2.92) .2 (.63) 9.4 (6.62) 1.8 (1.95) .3 (.94)b 19.3 (9.27) 38.7 (13.9) 31.4 (12.38) 8.7 (7.11)

Note. Numbers represent mean percent of intervals in which the behavior was scored, except for Open-ended which is the ratio of intervals scored with open-ended questions in relation to the total number of intervals scored with any question. Numbers within the parentheses are standard deviations. Behaviors marked with “†” refer to composite scores. Numbers marked with different superscripts are statistically significant at p , .05.

Effects of Training on Participants’ Behavior During Simulated Interviews The second purpose of the study was to determine whether the training workshop significantly impacted participants’ interviewing behavior, as assessed during a simulated interview with a confederate who was allegedly sexually abused. Although participants were allowed a maximum of 30 minutes (i.e., 180 10-second intervals) for each simulated interview, actual length of the interviews varied. There were no statistically significant differences in the length, as measured by the number of 10-second intervals, of the pre-training (M 5 112.9, SD 5 39.9) and post-training (M 5 124, SD 5 39.9) interviews, nor between the post-training (M 5 120.5, SD 5 37.7) and follow-up (M 5 104.1, SD 5 47.8) interviews. Descriptive statistics and the results of paired-sample t-tests for the participant behaviors coded during the simulated interviews are presented in Table 2. Eleven participants completed the simulated interview at the pre-training and follow-up data collection points, whereas 12 completed the interview post-training. Results showed that the percent of intervals in which participants praised the confederates for verbalizations not related to disclosure of abuse decreased significantly following the training. This change maintained at follow up, as evidenced by no significant difference between post-training and follow-up scores. Further, the ratio of open-ended questions in relation to all questions asked by participants increased significantly between pre- and posttraining, as evidenced by the Open-ended composite score. Maintenance of these improvements was evidenced by no significant difference between post-training and follow-up composite scores. Also, the intervals in which participants spent explaining the purpose and procedures during the first and last two minutes of the simulated interview increased following the training as compared to prior to the training, as evidenced by a significant pre-post increase in the Explain Purpose composite score. These improvements maintained, as demonstrated by the lack of significant differences between post-training and follow-up Explain Purpose composite scores. Further, there was a significant decrease in the percent of intervals during which participants explained the use of instructional aids (e.g., dolls, drawings) to confederates between post-training and follow-up. No other significant changes in participants’ behavior during the simulated interview were obtained. In addition to the changes that occurred in certain classes of participant behavior, it is important to note the low occurrence of praise provided to confederates for disclosing information about the

Investigative interviewing with children

709

Table 3. Comparison of Percent of Abuse Allegations Correctly Classified by Participants Time of Data Collection Scenario 1 2 3

Pre-training

Post-Training

Follow-up

50% (2 of 4) 75% (3 of 4) 33% (1 of 3)

100% (4 of 4) 100% (5 of 5) 100% (3 of 3)

25% (1 of 4) 100% (2 of 2) 80% (4 of 5)

allegations of CSA. Further, participants maintained a relatively low rate of asking multiple choice questions at all points of data collection. These findings are important because they are consistent with what would be appropriate during an investigative interview. As such, although no significant changes in the occurrence of these behaviors were noted, it appears that participants were skilled in these areas prior to the training. Although not statistically significant, several interesting outcomes occurred regarding the effects on participants’ responses on questionnaires completed immediately prior to and following the simulated interviews. First, four participants stated prior to training they would allow a parent in the room while they conducted an investigative interview with a child (a decision inconsistent with proper interviewing guidelines). During the post-training simulated interview, three of those four participants reported they would not allow the parent in the room while they gathered abuse-related information. Second, the types of props requested by participants for use during the simulated interviews changed following the training. Specifically, of the three participants who requested anatomicallycorrect dolls to use during the pre-training simulated interview, two did not request the dolls during the post-training simulated interview. Further, of the six participants requesting some type of prop during the post-training simulated interview, four requested anatomically-correct drawings instead of the anatomically-correct dolls. This is important given the controversy surrounding the use of anatomically-correct dolls and the recognition that significant training is needed before one should use the dolls (e.g., Koocher et al., 1995). Finally, participants’ ability to identify correctly whether a child was sexually abused (i.e., correctly say “yes” when child was abused, correctly say “no” when child was not abused) was compared across simulated interviews conducted at the three points of data collection. Results suggest that participants’ ability to correctly identify whether the child was sexually abused improved following the training (see Table 3). Percent correct increased to 100% for all three scenarios at post-training. Further, participants’ abilities to identify correctly whether the child was abused appeared to maintain for scenarios 2 and 3 (scenarios that depicted a child that actually was abused). However, percent correct decreased from 100% to 25% for scenario 1, the scenario that depicted false allegations of CSA. DISCUSSION The present study was designed to assess whether a training workshop on investigative interviewing in the context of allegations of CSA would impact significantly CPS workers’ knowledge about conducting, and behavior during, investigative interviews. To accomplish this, participants completed Knowledge Questionnaires and simulated interviews prior to, immediately following, and 3 months after a 6-hour training workshop. Results suggest training impacted participants’ knowledge about investigative interviewing. Specifically, significant improvements in Knowledge Questionnaire total scores were obtained

710

K. A. Freeman and T. L. Morris

following training, and these improvements maintained for 3 months. It is important to note that, because the same version of the questionnaire was used at all points of data collection, practice effects potentially may account for the positive changes in the total scores. However, this potential limitation notwithstanding, the procedure of using the same knowledge measure at pre- and post-training is consistent with previous research in this area. Regarding responses assessed during simulated interviews, results showed changes in several, but not all, of the participants’ behavior during the simulated interviews (e.g., decrease in praising confederate for verbalizations not related to abuse, increase in time spent explaining the purpose and procedures of the interview). Moreover, although not statistically significant, changes in participants responses to pre-interview and post-interview questionnaires (e.g., decrease in the number of participants who would allow confederate’s mother in room during interview, increase in accuracy of identifying abuse status of confederate) suggest that the training impacted participants in a positive manner. Despite these improvements, however, no other significant changes in participants’ behavior were noted. The training impacted neither the percent of intervals during which participants emitted behaviors defined as inappropriate (i.e., the Inappropriate Behaviors composite scores) nor the percent of intervals during which participants spent asking appropriate questions (i.e., the Appropriate Questions composite scores). Further, although the ratio of open-ended questions in relation to all questions asked increased, it still remained relatively low, never reaching higher than slightly over 19% of questions asked. Even though the increase from pre-training rates was statistically significant, one is left questioning whether the difference is clinically relevant. Thus, although the training impacted the amount of time spent describing the purpose of the interview, improvements in participants’ abilities to gather information from the confederates regarding the allegations of abuse were rather limited. This lack of behavioral improvement is consistent with previous research (Stevenson et al., 1992). That is, Stevenson and colleagues found improvements in several areas, such as providing appropriate empathy, but failed to find significant improvements in skills related to gathering information in an appropriate manner. Although it is possible that the training workshop failed to produce the desired changes in participants’ behavior, several other factors may have contributed to the current results obtained from the simulations. Specifically, the small number of participants may have limited the ability to find significant results. With only 12 participants, error variance due to individual differences may have reduced the ability to detect actual differences that emerged between data that were collected at different points throughout the study. Thus, although a few significant results were found, other significant findings may have emerged given a greater number of participants. Further, factors related to the simulated interview situation may not have allowed for an accurate assessment of participants’ behavior during actual investigative interviews. Specifically, although confederates were trained to use age-appropriate language and repeatedly practiced their “scenario,” the fact that they were adults may have influenced how the participants interacted with them. Also, the fact that participants only had access to the “child” as opposed to the child and a guardian or parent is significantly different than with actual investigations. Although participants were able to “access” some information from the confederates’ mother through the principle investigator, the inability to talk face-to-face with a parent or guardian in preparation for the interview may have influenced their interaction with the confederates. Other factors such as being video-taped, having an artificial time limit, and knowing that this was not a “real” case also may have changed the participants’ behavior. Despite these limitations, the present investigation improves upon existing literature in several ways. First, the current investigation provides another example of efforts to assess for changes in participants’ behavior during investigative interviews. Currently, there is only one other study in which participant behavior was assessed (Stevenson et al., 1992). Thus, additional documentation of the impact of workshop training programs on participant’ interviewing behavior clearly is

Investigative interviewing with children

711

needed. Moreover, the methods used to assess for changes in participants’ interviewing behavior following training extended the efforts of Stevenson and colleagues. Specifically, they had participants interview the same confederate prior to and following the training workshop. Conversely, participants in the present investigation interviewed different confederates portraying dissimilar scenarios at all three points of data collection. This may have reduced the likelihood of practice effects, which were speculated to be a major contributor to the lack of findings in the Stevenson and colleagues’ study. Further, the present investigation allowed for an assessment of whether changes in behavior would either maintain or emerge given opportunities for participants to use them in their daily work. Second, and related to the first improvement, using the CAIICS to measure participant behavior during simulated interviews raises the possibility of a standardized measure of interviewer skills becoming available as a means of comparing the effectiveness of different training programs. In particular, classifying different interviewer behaviors into predetermined categories produced data useful in an objective and quantitative assessment of changes in skills. As such, this instrument has the potential of becoming a valuable and useful tool for the field. Third, as compared to several previous studies, a more comprehensive measure of knowledge was utilized in the current investigation. The Knowledge Questionnaire used in the current study contained 27 items specifically designed to assess participants’ knowledge about factors that contribute to children’s reporting abilities, as well as the methods of conducting effective investigative interviews. This is in contrast to questionnaires used in previous research that were either more general in nature (see Hibbard et al., 1987) or not detailed enough to capture extensive knowledge (see Sullivan & Clancy, 1990). Further, the current questionnaire probed for factual information as well as for the application of information regarding investigative interviewing. As a result, although the questionnaire was designed specifically for the current study, and thus its psychometric properties are unknown, it potentially allowed for a more thorough assessment of participants’ knowledge about investigative interviewing with children who were allegedly sexually abused than did previously utilized measures. In addition to improving upon existing literature, the present results suggest areas of future research. Specifically, research addressing the issue of the magnitude of change in certain interviewer behaviors required to constitute “clinically relevant” improvements seems important. For example, research suggests that using open-ended questions may be the least leading approach (e.g., Poole & White, 1991; Tucker, Mertin, & Luszcz, 1990). Evaluating the optimal ratio of open-ended to direct questions that produces the most accurate report would help determine the extent to which interviewers should utilize particular forms of questioning. Unfortunately, there are no answers to questions such as this available; however, with the advent of such instruments as the CAIICS, these important questions may be able to be answered. Researchers also should begin investigating the extent to which various training procedures and lengths of training produce differing results. The current workshop was six hours in length, was completed over three weeks, and involved some role play activities. Other workshops have been both greater and shorter in length, were completed during one day or across many, and involved experiential exercises to a greater or less extent. Currently, little is known about the differential impact of different training designs and modalities on targeted skills and knowledge. One might assume that providing longer training, over a greater time period, with more opportunities to practice the skills taught would result in better outcomes. However, this remains an empirical question, which should be answered. The current investigation adds to the growing emphasis of providing training to CPS workers on investigative interviewing strategies. As Doris, Mazur, and Thomas (1995) noted, there has been an increased focus on providing relevant training to CPS workers. The authors describe the systems of several states that provide for initial and on-going training in a variety of areas (e.g., human behavior, risk and family assessment, investigation process). Many of these programs grew out of

712

K. A. Freeman and T. L. Morris

the recognized need for further training due to lack of pre-employment training, as well as increasingly diverse caseloads and responsibilities. Although the training programs described by Doris and colleagues (1995) focus on both knowledge and skills acquisition, the authors point out that only one of the reviewed state programs (i.e., Tennessee) uses a competency-based evaluation process that includes both written and performance assessments. For the remainder of the programs that utilized some form of assessment to determine if the training has prepared employees to assume job responsibilities, tests of knowledge were used as the sole determinant. The findings of the current investigation call into question sole reliance on measures of knowledge about investigative interviewing in the context of allegations of CSA. If such an approach is utilized, the result may be many CPS workers practicing in their profession without the skills necessary to meet the needs of their clientele adequately. This may be particularly problematic in the context of investigating allegations of CSA given that little or no physical evidence may be available to corroborate children’s reports (De Jong & Rose, 1991). Thus, information gathered during interviews with children who were allegedly sexually abused may have considerable importance in how caseworkers proceed (e.g., substantiating the allegations, contacting police). Because it can be particularly difficult to gather accurate information from children using legally defensible practices, it has been advised that professionals who conduct interviews with children be well-trained (Besharov, 1994 as cited in Doris et al., 1995). Current results suggest that pre-requisite knowledge about best practices in investigative interviewing with children may not be the best marker for making such a determination. Acknowledgements—The authors would like to thank Alisha Wilson, Joanna Conrad, Dana Brinson, and Aaron Topping for assistance with data collection and analysis.

REFERENCES Besharov, D. T. (1984). Liability in child welfare. Public Welfare, 42, 28 –33. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally. Ceci, S. J., & Bruck, M. (1995). Jeopardy in the courtroom: A scientific analysis of children’s testimony. Washington, DC: American Psychological Association. Cheung, K. M. (1997). Developing the interview protocol for video-recorded child sexual abuse investigations: A training experience with police officers, social workers, and clinical psychologists in Hong Kong. Child Abuse & Neglect, 21, 273–284. De Jong, A. R., & Rose, M. (1991). Legal proof of child sexual abuse in the absence of physical evidence. Pediatrics, 88, 506 –511. Doris, J., Mazur, R., & Thomas, M. (1995). Training in child protective services: A commentary on the amicus brief of Bruck and Ceci (1993/1995). Psychology, Public Policy, and Law, 1, 479 – 493. Hartmann, D. P. (1977). Considerations in the choice of intercoder reliability estimates. Journal of Applied Behavior Analysis, 10, 103–116. Hibbard, R. A., Serwint, J., & Connolly, M. (1987). Educational program on evaluation of alleged sexual abuse victims. Child Abuse & Neglect, 11, 513–519. Koocher, G., P., Goodman, G. S., White, C. S., Friedrich, W. N., Sivan, A. B., & Reynolds, C. R. (1995). Psychological science and the use of anatomically detailed dolls in child sexual-abuse assessments. Psychological Bulletin, 118, 199 –222. Kuehnle, K. (1996). Assessing allegations of child sexual abuse. Sarasota, FL: Professional Resources. Leung, P., Cheung, K. M., & Kay, M. (1994). Advancing competent social work practice: A computer-based approach to child protective service training. Computers in Human Services, 11, 317–332. Mitchell, S., White, A., Wright, W. S., & Pecora, P. J. (1989). Counseling troubled adolescents: An evaluation of a statewide training program. Journal of Sociology and Social Welfare, 16, 95–109. Poole, D. A., & White, L. T. (1991). Effects of question repetition on eyewitness testimony of children and adults. Developmental Psychology, 27, 975–986. State v. Michaels, 136 N.J. 299, 642, A.2d 1372 (N.J., 1994). Stevenson, K. M., Leung, P., & Cheung, K. M. (1992). Competency-based evaluation of interviewing skills in child sexual abuse case. Social Work Research and Abstracts, 28, 11–16.

Investigative interviewing with children

713

Stone, L. E., Tyler, R. P., & Mead, J. J. (1984). Law enforcement officers as investigators and therapists in child sexual abuse: A training model. Child Abuse & Neglect, 8, 75– 82. Sullivan, R., & Clancy, T. (1990). An experimental evaluation of interdisciplinary training with sexually abused adolescents. Health and Social Work, 15, 207–214. Tucker, A., Mertin, P., & Luszcz, M. (1990). The effect of a repeated interview on young children’s eyewitness memory. Australian and New Zealand Journal of Criminology, 23, 117–124. United States Department of Justice. (1996). Interviewing child witnesses and victims of sexual abuse. Washington, DC: Author. Wood, B., Orsak, C., Murphy, M., & Cross, H. J. (1996). Semistructured child sexual abuse interviews: Interview and child characteristics related to credibility of disclosure. Child Abuse & Neglect, 20, 81–92.

RE´SUME´ French abstract not available at time of publication.

RESUMEN Objetivo: Inestigaciones previas sugieren que el entrenamiento en talleres sobre la entrevista investigativa conducidos con empleados de primera fila, como son los oficiales de policı´ o los adscritos al servicio de proteccio´n infantil (CPS), pueden resultar en jemoramiento de el conocimiento sobre las estrategias adecuadas que deben utilizar. Se han realizado limitadas investigaciones para enfocar si estos programas de entrenamiento resultan en mejorı´a en las habilidades reales para la entrevista. La presente investigacio´n evaluo´ el impacto de un taller de entrenamiento en los conocimientos de los empleados del CPS sobre como conducir y la conducta adecuada durante las entrevistas de investigacio´n. Me´todo: Doce empleados del CPS participaron. Para evaluar el impacto del entrenamiento se utilizo´ un disen˜o cuasiexperimental. Los participantes completaron medidas de aprovechamiento antes e inmediatamente despue´s, y 3 meses despue´s del entrenamiento. Las medidas de los resultados incluı´an (a) un cuestionario para evaluar el conocimiento sobre las pra´cticas de entrevista, (b) entrevistas simuladas con un compan˜ero para evaluar la conducta participativa durante la entrevista de investigacio´n, y (c) cuestionarios para recolectar informacio´n relacionada con las entrevistas simuladas (e.g., materiales requeridos para el uso durante la entrevista, status del abuso). Todos los participantes completaron seis horas de entrenamiento durante tres sesiones de dos horas semanalmente. Resultados: Los conocimientos de los participantes sobre el to´pocio, ası´ como varias habilidades de entrevistas durante las entrevistas simuladas mejoraron despue´s del entrenamiento. Sin embargo, el entrenamiento no influyo´ en varias habilidades claves de entrevistas como el tipo de preguntas que se harı´an o la duracio´n de la entrevista. Conclusiones: Los programas de talleres de entrenamiento basados en conocimientos pueden no preparar adecuadamente a los empleados del CPS para conducir de manera apropiada las entrevistas de investigacio´n con nin˜os que se sospecha han sido abusados. Adema´s, el concimiento sobre como conducir estas entrevistas pueden no ser el mejor indicador de si una persona esta´ preparado para este aspecto del trabajo.