- Email: [email protected]
Statewide dissemination of trauma-focused cognitive-behavioral therapy (TF-CBT)
Children and Youth Services Review 35 (2013) 1023–1029
Contents lists available at SciVerse ScienceDirect
Children and Youth Services Review journal homepage: www.elsevier.com/locate/childyouth
Statewide dissemination of trauma-focused cognitive-behavioral therapy (TF-CBT)☆ Benjamin A. Sigel a,⁎, Teresa L. Kramer a, Nicola A. Conners-Burrow b, Janice K. Church c, Karen B. Worley c, Nicholas A. Mitrani c a b c
Psychiatric Research Institute, University of Arkansas for Medical Sciences, United States Department of Family and Preventative Medicine, University of Arkansas for Medical Sciences, United States Department of Pediatrics, University of Arkansas for Medical Sciences, United States
a r t i c l e
i n f o
Article history: Received 28 November 2012 Received in revised form 25 March 2013 Accepted 27 March 2013 Available online 3 April 2013 Keywords: Dissemination Implementation Trauma Evidence-based practices Training
a b s t r a c t Despite evidence linking childhood trauma to subsequent social, emotional, psychological, and cognitive problems, many children who have experienced trauma do not receive mental health treatment that has been proven to be effective. Large-scale dissemination of evidence-based practices (EBPs) is one possible solution to enhance the current negative state of mental health treatment for these children. This article describes a dissemination effort of an EBP (i.e., Trauma-Focused Cognitive-Behavioral Therapy [TF-CBT]) for childhood symptoms of post-traumatic stress disorder throughout Arkansas. The effort targeted mental health professionals within child advocacy centers and community mental health centers across the state. The article describes the process of dissemination and implementation. Lessons learned and recommendations for future dissemination efforts are highlighted. © 2013 Elsevier Ltd. All rights reserved.
1. Introduction Nationally, the percentage of youth exposed to some form of trauma is high, ranging from 8% to 53%, depending on the type of trauma and population studied (e.g., Copeland, Keeler, Angold, and Costello, 2007; Finkelhor, Ormrod, Turner, and Hamby, 2005; U.S. Department of Health and Human Services, 2011). Despite high rates of subsequent related post-traumatic stress disorder (PTSD), and other symptomatologies, many children who have experienced trauma either do not receive treatment or receive treatment that has not proven to be effective (Burns et al., 2004; Cohen, Mannarino, and Rogal, 2001; Kolko, Cohen, Mannarino, Baumann, and Knudsen, 2009; Ringeisen, Casanueva, Urato, and Stambaugh, 2009). Without adequate and appropriate treatment, trauma symptoms may linger or exacerbate over time, developing into other mental health problems such as internalizing or externalizing disorders (Hamblen, 1999; Hernandez, Lodico, and DiClemente, 1993; Hoven et al., 2005; Siegel and Williams, 2003). In addition, lack of treatment has the potential to increase secondary adversities such as health problems, home and foster home placement disruptions, school difficulties, social maladjustment,
☆ This research was supported by a grant from the 2009 Arkansas Revenue Stabilization Act. ⁎ Corresponding author at: University of Arkansas for Medical Sciences, Psychiatric Research Institute, 4301 Markham Street, Slot 755, Little Rock, AR 72205, United States. Tel.: +1 501 364 5861. E-mail address: [email protected] (B.A. Sigel). 0190-7409/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.childyouth.2013.03.012
and substance abuse (Felitti et al., 1998; Goodman, 2002; Hamblen, 1999; Myers et al., 2002; National Institute for Mental Health, 2001). Large-scale dissemination of evidence-based practices (EBPs) is one possible solution to enhance the current negative state of mental health treatment for children who have experienced trauma. Kitson, Harvey, and McCormack (1998) suggest that successful dissemination of EBPs involves interplay of three core elements: Context, Evidence, and Facilitation. Context is the combination of culture, leadership, and measurement in which the dissemination effort is to take place. Successful dissemination efforts will occur in contexts that are open to and support continuing education and effective practices, are organized, and provide feedback. Evidence is the combination of research, clinical expertise, and patient choice of/for the practice being disseminated. For example, successful dissemination efforts will use practices that have significant research support (i.e., EBPs), have high levels of consensus and consistency of view of effectiveness, and are viewed as beneficial and helpful by patients. Facilitation is the process of implementing evidence into practice and is the combination of characteristics, role, and style of the facilitator(s). Successful dissemination efforts occur when facilitation is respected, credible, empathic, authentic, collaborative, supportive, consistent, and flexible. In the spring of 2009, the Arkansas Legislature approved funding to improve screening, monitoring, and continuity of care for children who experienced trauma in Arkansas to address the psychological impact of their trauma. The funding was disseminated through the Arkansas Commission on Child Abuse, Rape, and Domestic Violence which provided the infrastructure for the program and helped facilitate a
1024
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
multidisciplinary collaboration between the University of Arkansas for Medical Sciences Psychiatric Research Institute and the Department of Pediatrics. With the infrastructure and collaboration in place, the Arkansas Building Effective Services for Trauma (AR BEST) program was created. The mission of AR BEST is to improve outcomes for traumatized children and their families in Arkansas through excellence in clinical care, training, advocacy, and research/evaluation across systems. The remainder of the paper describes the process of disseminating an EBP for PTSD symptoms in children (i.e., Trauma-Focused CognitiveBehavioral Therapy [TF-CBT]) throughout Arkansas within a conceptual framework (i.e., Kitson et al., 1998). 1.1. Context AR BEST staff reviewed state-of-the-art trauma-informed practices and compared these to the state of care in Arkansas. Staff noted a need for improvement among Arkansas' providers charged with ensuring the safety and health of children (e.g., mental health, child welfare, juvenile justice, health, education, and first responders). Specifically, efforts were needed to create a trauma-informed system of care. A trauma-informed system of care can be broadly defined as: 1) Awareness of and knowledge about the impact of traumatic stress across systems; 2) Awareness of and access to effective trauma assessment and interventions strategies that exist within and across different systems; 3) Trauma-focused education and skill-building to providers and administrators within and across key child-serving systems in order to change practice; and 4) Promoting strong collaborations across systems and disciplines (National Center for Child Traumatic Stress, 2009). In Arkansas, efforts were needed to provide a unifying and/or systematic language or approach to assessing and treating children who have experienced trauma. Few community mental health providers (MHPs) were trained in trauma-informed EBPs and thus providers had few options for referrals in these practices. The practices for training advocates (i.e., individuals who provide support and referrals for child victims and their families) in child advocacy centers (CACs) varied across the state; furthermore, child welfare, foster parents, juvenile justice, and other systems staff had little knowledge about trauma-informed care (i.e., the impact of trauma on children) and lacked options for referrals to community mental health providers trained in trauma-informed evidencedbased treatments. Finally, little training was being conducted in traumainformed care with the foster parent, educational, juvenile justice, and first responder systems. AR BEST staff decided that disseminating TF-CBT was the first step in changing trauma practices. With MHPs trained in TF-CBT, referrals could be made to these providers from child welfare, juvenile justice, and other systems, therefore, increasing communication between systems and helping to gain buy-in from leadership within these systems. Once training in TF-CBT began, these other systems received training in trauma-informed care, further enhancing communication and providing a unified and systematic language for assessing and treating children who have experienced trauma. 1.2. Evidence TF-CBT, co-developed by Judith Cohen, M.D., Anthony Mannarino, Ph.D., and Esther Deblinger, Ph.D., is an EBP for childhood symptoms of PTSD with well-established efficacy. It is a treatment consistent with the principles of cognitive-behavioral, exposure, and parenting therapies that are widely accepted by MHPs. TF-CBT is a manualized and flexible therapeutic intervention. The therapeutic components include psychoeducation about the trauma; parenting skills; development of relaxation and other coping skills; feelings identification; understanding the link between thoughts, feelings and behaviors; developing a narrative of the traumatic event which has been
experienced by the child/adolescent and processing of associated thoughts, feelings and behaviors; gradual exposure to the traumatic event with the youth learning how to manage being exposed to such event(s); conjoint parent–child work; and enhancing safety/prevention skills. In recent reviews of research on treatment for children with PTSD symptoms (i.e., Chadwick Center for Children, Families, 2004; Chadwick Center for Children and Families and Child and Adolescent Services Research Center, 2009; Saunders, Berliner, and Hanson, 2004; Silverman et al., 2008), TF-CBT was the only treatment given the highest rating (i.e., evidence-based practice) in all of the reviews. Due to TF-CBT's promising outcomes in children with a history of trauma, large-scale dissemination is under way. In 2009, 19 states were already in the process of or had completed large-scale TF-CBT dissemination projects. The methods and processes of dissemination for the majority of these projects are briefly described in Sigel, Benton, Lynch, and Kramer (in press). 1.3. Facilitation Due to substantial evidence of efficacy and effectiveness and through on-going large-scale dissemination projects, several dissemination strategies have been developed to facilitate TF-CBT dissemination with fidelity and within an implementation research framework (i.e., Kitson et al., 1998). Cohen and Mannarino (2008) have discussed three dissemination methods that have been used (web-based learning, live training plus ongoing consultation, and learning collaborative) as well as the advantages and disadvantages of each model. Web-based learning involves completing TF-CBTWeb (accessible at www.musc.edu/tfcbt), developed by investigators at the Medical University of South Carolina Crime Victims Center in collaboration with TF-CBT developers. The website includes parent and child sections for each component of the model, video examples of all of the key treatment components, printable scripts to use, handouts for parents and children, instructions on how to handle clinically challenging situations, guidelines on cultural issues, and resources and links. This method's advantages are overcoming barriers such as cost, distance, and inconvenience of having to travel to live trainings and ability to train multiple MHPs at one time. Disadvantages are lack of interaction with a live trainer and no access to on-going consultation. The two other models of TF-CBT dissemination require more intensive training and follow-up efforts. The live training plus ongoing consultation model involves in-person training in TF-CBT followed by ongoing phone or in-person consultation. The training and consultation are provided by the treatment developers or other approved trainers who monitor trainees to ensure the treatment is being used with fidelity. The advantage of this model is that practitioners receive support when they start seeing clients, which may enhance fidelity and sustainability once training is completed. The disadvantages to this method are cost and limits to the number of MHPs that can be trained. By comparison, the learning collaborative model is designed to change the larger culture in which TF-CBT is implemented with the goal of achieving buy-in across systems and enhancing long-term sustainability. TF-CBT training typically involves separate tracks for MHPs, supervisors, and senior leaders (i.e., administrators and directors) (Ebert, Amaya-Jackson, Markiewicz, and Burroughs, 2008). Each track has several in-person trainings with follow-up by phone or in person consultation over the course of a year targeting different parts of the TF-CBT model, implementation strategies, and/or dissemination issues. The contents of the in-person trainings and consultations are based on where the therapist, supervisor, and/or senior leader are in the TF-CBT model, implementation, and/or dissemination process. The trainings and consultation are also provided by the treatment developers or other approved trainers. With this model, supervisors and senior leaders are involved in the calls, which may provide additional support to MHPs learning the model from within the agency as well as change or maintain the culture required for a particular model. The inclusion of senior leaders may enhance sustainability once training has ceased. Senior leaders
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
tend to have fewer turnovers and can sustain use of the model despite front-line worker turnover. Within the learning collaborative model, additional trainings are often conducted in trauma-informed care to other stakeholders (e.g., child welfare, Department of Mental Health, CACs, juvenile justice) to help achieve buy-in across systems. A possible disadvantage of this model over the live training and consultation model is that this method may be costlier and more timeconsuming. There is also an added expense of training supervisors, senior leaders, and stakeholders as well as conducting extra live trainings. Due to these factors, this model may result in fewer MHPs being trained but may enhance sustainability and buy-in across systems. In Arkansas, the decision was made to use the TF-CBT training and consultation dissemination method instead of the web-based learning or learning collaborative methods. Due to the disadvantages of web training alone, it was determined such an approach would yield limited sustainability of TF-CBT utilization. Furthermore, as the AR BEST project was just beginning in Arkansas, sufficient collaboration across systems of care necessary to conduct learning collaboratives was not adequately in place. As such, dissemination efforts would have been significantly delayed while these collaborations were further developed and enhanced. Furthermore, results and analysis of interviews with other states disseminating TF-CBT and previous research (e.g. CATS Consortium, 2007; Cohen and Mannarino, 2008; Mental Health Services, and Policy Program, Northwestern University, 2008; Sigel et al., in press) indicated that both the learning collaborative and training and consultation methods are effective and efficient approaches for widely disseminating TF-CBT. 2. Method 2.1. Participants AR BEST targeted MHPs working at or in collaboration with Arkansas' 13 CACs as well as the network of community mental health centers (CMHCs) that serve the entire state. By targeting CAC-affiliated MHPs, it was anticipated that training in an EBP would give CACs increased resources for referrals for mental health treatment especially for the state's sexual abuse victims. Furthermore, CAC accreditation standards are now requiring CACs to have collaborative agreements with MHPs who can provide EBPs, so it was anticipated this collaborative project would be very well-received by CAC directors. It was also deemed critical to the broad dissemination mission of the AR BEST project to ensure CMHC directors and clinical staff were familiar with the principles of trauma-informed care as well as EBPs, particularly in the very rural areas of Arkansas where CMHC staff would likely be the only resources for child and adolescent trauma victims and their families. A clear project goal was to have at least one TF-CBT trained clinician in every county of the state, such that EBPs known to be effective and efficient in treating trauma is available to all children and youth in Arkansas who require such interventions. Other MHPs from across the state have been invited to participate in the AR BEST project as space and training limitations have permitted. Arkansas' TF-CBT training model requires MHPs to complete three distinct and voluntary phases of training. First, MHPs completed TF-CBTWeb and received 10 free hours of continuing education credit (CEUs). Upon completion of TF-CBTWeb, MHPs were eligible to attend an in-person, free, two-day TF-CBT basic/introductory training conducted by Dr. Mannarino in Little Rock, Arkansas. After completing the two-day training and being awarded an additional 12 continuing education credits, MHPs were then eligible to participate in a series of bi-weekly phone consultation calls over an approximately seven-month period led by treatment developers, national TF-CBT experts, or AR BEST staff. Approximately 8–12 MHPs were in each consultation call group. To be considered successful in completing the ongoing consultation calls, MHPs were expected to participate in at least 12 out of 14 or more calls and to present on at least two separate treatment
1025
cases using one or more TF-CBT components. Participants received one CEU for each full-hour call in which they participated. The MHPs that completed the entire process received a certificate of TF-CBT training completion and were listed on a roster of trained TF-CBT MHPs that was posted on the AR BEST website. Additionally, AR BEST staff has worked with TF-CBT treatment developers to ensure that they will be eligible for national TF-CBT certification (when available). Finally, in the second and subsequent years of the project, MHPs who completed the required three phases of TF-CBT training (web-based, two-day inperson training, and series of consultation calls) were invited to participate in an in-person, free, one-day advanced TF-CBT conference led by Dr. Mannarino, earning six CEUs for the full-day training. To help facilitate sustainability of the dissemination project, AR BEST staff completed the TF-CBT training process with Dr. Mannarino leading their consultation calls in addition to observing/scribing for consultation calls led by the TF-CBT co-developers and other national experts. The bi-weekly consultation calls with Dr. Mannarino focused on case consultation and using TF-CBT with fidelity for approximately seven months and then were extended one year for monthly contacts to focus on skills development for conducting and leading TF-CBT consultation calls. To further ensure sustainability of the dissemination project, outstanding therapist participants who are not AR BEST staff were nominated by the treatment developers, national experts, or AR BEST staff to become local experts. Similar to the process already described for AR BEST staff, these local experts will receive additional training in consultation call facilitation and will initially observe/scribe and subsequently co-lead consultation calls with AR BEST staff in future rounds of TF-CBT training. Gradually, use of the treatment developers and other national TF-CBT experts will be phased out of the consultation call phase of Arkansas' dissemination project, and such calls will be managed exclusively by AR BEST staff and developing local experts. 2.2. Measures The AR BEST team and collaborators identified a set of tools that would benefit MHP implementation of TF-CBT and the children served by these MHPs. We sought to develop a dual-purpose system that would provide these tools to MHPs as well as provide information back to the AR BEST team about the efforts of MHPs to disseminate TF-CBT in Arkansas. Toward this end, a web-based data collection system was designed to include the following components: 1. Clinician Registration — All MHPs attending AR BEST training in TF-CBT were required to register in the system for tracking purposes. 2. Client Registration — The Client Registration component was the starting point for MHPs wishing to use the tools provided through the website to support their implementation of TF-CBT. Here MHPs provide basic demographic information about the client being served as well as information on the nature of the trauma and the alleged perpetrator. 3. Clinical Assessment Tools and Reports — MHPs attending the two day TF-CBT training also received training to administer two child assessment tools available on the website: the Strengths and Difficulties Questionnaire (SDQ) and the UCLA PTSD Reaction Index. MHPs were encouraged to administer the measures at intake, and at 3, 6 and 12 months post-intake (sooner if the client has already completed treatment and has been discharged) to monitor the symptoms of their clients. MHPs who have registered a client receive reminders to complete the assessment and can access reports summarizing the assessment results and graphing changes in symptoms over time. The Strengths and Difficulties Questionnaire (Goodman, 2001) is a brief behavioral screening questionnaire for children. Each version has 25 items divided between five clinical scales: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems and prosocial behavior. AR BEST made two versions of the
1026
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
SDQ available on the website: the parent-report version for children up to age 12, and the adolescent report version for children 12 and older. UCLA PTSD Reaction Index for DSM-IV (Pynoos, Rodriguez, Steinberg, Stuber, and Frederick, 1998) is designed to screen for exposure to traumatic events as well as symptoms of PTSD. The items of the UCLA PTSD Reaction indices are keyed to DSM-IV criteria and can provide preliminary PTSD diagnostic information. AR BEST made two versions available on the website: parent-report version for children up to age 12, and the adolescent-report version for children 12 and older. 4. Support for Treatment Planning — The AR BEST team worked with local experts in treatment planning to provide wording for treatment plan goals and objectives that commonly would be used when a clinician is planning to implement TF-CBT with a client. MHPs can select goals and objectives for any client that they registered and print or export the treatment plan information for the clients' chart or medical record. 5. Documenting the Elements of TF-CBT — For MHPs who want to monitor the fidelity of their implementation of TF-CBT, the AR BEST team developed an option where the clinician could access the website after each session and click on the TF-CBT components that were addressed in that session. This option was based on the Trauma-focused Cognitive-Behavior Therapy (TF-CBT) Brief Practice Checklist (Deblinger, Cohen, Mannarino, Murray, and Epstein, 2007), which is designed to track the timing and implementation of specific TF-CBT components in a manner that helps MHPs and supervisors to determine whether fidelity is being adequately maintained. 2.3. Training evaluation tools TF-CBT Basic Training Evaluation: MHPs attending the 2011 and 2012 TF-CBT initial two-day trainings were asked to complete a pre- and post-test designed by the AR BEST team. The pre-test was designed to assess MHPs' initial knowledge and self-efficacy related to TF-CBT, their willingness to use new interventions, and their interest in being recognized as a clinician in the state with specialized training in TF-CBT. The post-test was designed to assess MHPs' post-training self-efficacy related to TF-CBT, their satisfaction with the training and their intention to participate in ongoing consultation opportunities. TF-CBT Advanced Training Evaluation: MHPs attending the 2011 and 2012 Advanced Training Day (for MHPs trained during the prior two years) were asked to complete a pre- and post-test evaluation. The pre-test was designed to assess MHPs' knowledge and self-efficacy related to TF-CBT after at least a year of implementation, their willingness to use new interventions, and their interest in being recognized as a clinician in the state with specialized training in TF-CBT. The post-test was designed to assess MHPs' post-training self-efficacy related to TF-CBT and their satisfaction with the training. In addition to evaluating the trainings, the AR BEST team also monitored the ongoing consultation call process. Specifically, a member of the AR BEST team participated in each call as a scribe. The scribes documented when call participants/trainees introduced a potential client who would benefit from using the TF-CBT treatment model and when a participant presented a case (defined as describing the client's specific trauma(s) and discussing components of the model used). The scribe also documented and kept a record of call attendance and the total number of presented cases by each call participant. 3. Results 3.1. MHP training TF-CBTWeb was promoted by AR BEST in the fall of 2009. Prior to that time (October, 2005–July, 2009), 47 mental health MHPs in Arkansas had
completed this training. Between August 2009 and July 2011, 567 MHPs completed this web-based training. Annually, beginning in 2010, AR BEST sponsored a two-day conference on TF-CBT for MHPs; in the second year of the project a one-day advanced conference was added for MHPs who had previously completed a minimum of 12 TF-CBT consultation calls. Over the first three years of the project, 501 providers from 76% (57 out of a total of 75) of Arkansas counties completed the web-based and two-day TF-CBT training. Following the 2010 and 2011 conferences, 139 (43%) out of 322 providers went on to complete the required number of consultation calls needed to be acknowledged by AR BEST as fully TF-CBT trained (i.e., completed web-based training, attended two-day in-person training, completed at least 12 consultations, and presented at least two cases while on the consultation calls). Consultation calls from the 2012 conference are still ongoing, with 100 participants. Of the total 139 fully trained MHPs, 19 were contracted by CACs to provide TF-CBT in 9 out of 13 (70%) CACs located throughout the state. To date, 328 TF-CBT cases have been presented during the consultation calls. These 139 fully trained MHPs represent 52% (39 out of a total of 75) of Arkansas counties. In 2011, 50 of the 56 eligible fully trained MHPs participated in the one-day advanced training while 94 of the 139 eligible fully trained MHPs (i.e., from both the 2010 and 2011 training groups) attended in 2012. Finally, six outstanding MHPs from the 2010 training group were identified by the treatment developers, other national TF-CBT experts, and/or AR BEST staff and have begun their training process to become local experts. Results from the 2011 and 2012 TF-CBT training evaluation pre-test forms indicate that almost half of all participants worked in CMHC settings (40.1% of two-day introductory training attendees and 43.2% of one-day advanced-level training participants). At the two-day introductory training, most other attendees worked for other community providers of mental health services (46.9%), were in private practice (7.4%), worked at a local hospital (3.7%) or worked in a CAC (1.7%). Of the 365 MHPs who completed the two-day introductory training in 2011–2012, 340 provided matched pre-post training evaluations. Of the 130 attending the one-day advanced training, 121 participants provided matched evaluations. Pre-test survey results showed that most conference attendees were willing to use new and different manualized interventions with their clients, with advanced-level training participants being slightly more open. For example, 80.6% of two-day introductory and 87.3% of one-day advanced-level attendees agreed that they like to use new types of therapies to help their clients. From the post-test evaluations, with regard to TF-CBT and its rationale, most participants agreed or strongly agreed that the trainings confirmed their current knowledge or increased their knowledge of TF-CBT. For example, after the two-day basic training, 96.6% agreed that they “understand the rationale, empirical support, and core components of TF-CBT” and 96.0% agreed that the conference “improved my ability to treat children and adolescents with a trauma history.” Results were similar for advanced-level training attendees. When asked about their intent to use what they learned, 78.4% of the two-day introductory training attendees and 64.0% of one-day advanced-level training participants agreed that they intend to change their current practices as a result of attending the TF-CBT training, and 77.5% of two-day introductory TF-CBT attendees intended to participate on follow-up consultation calls. MHPs perceived few barriers to implementation of TF-CBT. For example, only 6.2% of two-day introductory-level attendees and 10.3% of one-day advanced-level attendees reported that productivity requirements at their respective agencies would prevent them from participating in ongoing training and consultation. In terms of motivators for participating in ongoing training opportunities, more than half of those who attended the two-day introductory training reported that national TF-CBT certification was important to them (51.6%), followed by statewide recognition of completion through the AR BEST website (45.7%) and appointment by AR BEST as a local expert (24.8%). Almost three-fourths (72.8%) of advanced-level
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
participants stated that national certification was important. Statewide recognition for training was valuable to 69.3% of advanced-level attendees, and 35.4% of attendees of the one-day advanced training cited the importance of being appointed by AR BEST as a local expert. Both two-day basic and one-day advanced-level training participants were asked before and after the 2011 and 2012 trainings to rate how certain they were that they could perform the TF-CBT components. Confidence was rated on a scale of 0 (can't do at all) to 100 (highly certain can do). As shown in Table 1, there were significant increases in attendee's confidence to implement each of the targeted treatment steps. (Note that two-day introductory training attendees had already completed the on-line training prior to participation in the in-person training, illustrating that in-person training may significantly MHPs' confidence in their ability to implement the TF-CBT model with fidelity.)
3.2. Clinician use of TF-CBT web-based tools Data from the first full year of implementation of the AR BEST website with tools for TF-CBT implementation suggest that the most popular tool with MHPs was the client assessment options. From July 2010 to July 2011, AR BEST-trained MHPs registered 420 clients from 59 out of 75 Arkansas counties in the AR BEST system. The majority of clients receiving mental health services were female (57.6%), ranging in age from 5 to 19 years, with a mean age of 11.4 years (SD = 3.7). Registered clients were most commonly being seen in CMHCs (34.0%), by private providers (28.3%) or in CACs (21.0%). The majority of AR BEST registered clients (56.4%) had experienced sexual trauma, while others (34.0%) had witnessed violence, been physically abused (27.4%) or experienced neglect (18.1%). In most cases (79.5%), the clinician intended to use TF-CBT with the family. When they chose not to, the most common reasons were the need to focus first on crisis intervention (11.2%) or because of caregiver limitations or lack of caregiver support (7.8%). Counseling was court ordered in 16.9% of the cases. CACs were one of the most common referral source for clients getting into therapy (21%), which highlights the critical role of
1027
CACs in connecting families to mental health services. Other key referral sources included the child welfare agency (20%) and self-referrals (21%). Of the 420 clients that were registered, 282 (67%) completed the UCLA PTSD Reaction Index. There were 142 parent-reports and 142 adolescent-reports. The parent-report indicated 59.9% of children were experiencing moderate or severe symptoms of PTSD. Based on adolescent report, 64.8% were experiencing moderate or severe symptoms. Of the 420 clients that were registered, 250 (60%) completed the SDQ. There were 130 parent-reports and 120 adolescent-reports. The SDQ results suggest that about one-half of children exhibit behaviors that fall in the ‘abnormal’ range (based on the total score), meaning that the behaviors are clinically serious. Specifically, 48.3% of adolescents reported symptoms in the abnormal range, and 53.1% of parents reported symptoms in the abnormal range. Based on adolescent and parent report, clinically elevated problems on the SDQ subscales were common: Problems with peers (43% and 35% for adolescent and parent respectively), conduct problems (24% for adolescents, 55% for parents) emotional symptoms (41% for adolescent, 30% for parent), and hyperactivitity (24% for adolescents and 22% for parents). Few MHPs consistently used the treatment planning or fidelity monitoring tools. Anecdotally, many reported that while they appreciated seeing appropriate treatment plan goals and objectives they could use as a model, they were required to use their own treatment plan forms within their agency.
4. Discussion Large-scale dissemination and implementation are no easy tasks. Fixsen, Naoom, Blasé, Friedman, and Wallace (2005) argue that dissemination requires on-going, multilevel strategies, including partnerships between community providers and skilled researchers; or other ongoing training opportunities within each implementation site; and sharing of lessons learned from a variety of dissemination efforts. Furthermore, Kitson et al. (1998) suggest that successful dissemination of EBPs involves interplay of three core elements (i.e., Evidence, Facilitation, and
Table 1 Change in clinician confidence in implementing components of TF-CBT pre- and post-training. Item
Assess symptoms in a child with a history of trauma. Provide caregiver involvement in sessions. Provide psycho-education for the child and family. Explain the physiology of relaxation to the child and family. Instruct a child in relaxation skills. Teach coping skills to help with the emotional effects of trauma. Review the relationship between thoughts, feelings and behaviors with a child. Develop a story of the trauma with a child. Help a child modify cognitive distortions about trauma. Assist child in sharing trauma story with caregivers. Address safety issues with child and family. Teach social skills to a child. Work with parents on managing a child's problem behaviors. a
Basic participants (N = 340)
Advanced participants (N = 121)
Pre-testa
Post-test
Pre-test
Post-test
68.98 (sd = 68.55 (sd = 72.81 (sd = 69.76 (sd = 74.10 (sd = 71.99 (sd = 62.78 (sd = 57.27 (sd = 61.49 (sd = 60.11 (sd = 74.03 (sd = 76.87 (sd = 75.61 (sd =
84.42 (sd = 83.47 (sd = 86.46 (sd = 85.49 (sd = 87.33 (sd = 86.89 (sd = 87.06 (sd = 83.62 (sd = 83.97 (sd = 83.15 (sd = 87.86 (sd = 87.78 (sd = 86.56 (sd =
87.53 (sd = 82.25 (sd = 85.48 (sd = 86.07 (sd = 87.43 (sd = 86.45 (sd = 85.61 (sd = 81.58 (sd = 80.78 (sd = 81.31 (sd = 87.17 (sd = 87.23 (sd = 87.49 (sd =
89.47 (sd = 88.42 (sd = 90.41 (sd = 90.33 (sd = 91.40 (sd = 91.35 (sd = 89.53 (sd = 88.41 (sd = 87.63 (sd = 88.57 (sd = 90.94 (sd = 90.64 (sd = 91.00 (sd =
21.10) 22.72) 21.22) 21.93) 20.86) 20.76) 27.18) 25.02) 24.34) 24.35) 21.53) 19.97) 19.80)
Basic attendees already completed the on-line TF-CBT training prior to conference attendance. ⁎⁎⁎ p b .001.
13.65) ⁎⁎⁎ 13.35) ⁎⁎⁎ 12.82) ⁎⁎⁎ 12.61) ⁎⁎⁎ 11.60) ⁎⁎⁎ 11.60) ⁎⁎⁎ 12.64) ⁎⁎⁎ 12.85) ⁎⁎⁎ 12.77) ⁎⁎⁎ 12.88) ⁎⁎⁎ 11.94) ⁎⁎⁎ 12.22) ⁎⁎⁎ 12.98) ⁎⁎⁎
11.29) 14.25) 12.09) 10.94) 11.52) 10.58) 12.52) 15.12) 15.44) 17.20) 12.28) 12.48) 12.32)
9.38) ⁎⁎⁎ 9.93) ⁎⁎⁎ 8.80) ⁎⁎⁎ 8.37) ⁎⁎⁎ 8.50) ⁎⁎⁎ 8.22) ⁎⁎⁎ 10.98) ⁎⁎⁎ 10.78) ⁎⁎⁎ 12.20) ⁎⁎⁎ 11.70) ⁎⁎⁎ 9.83) ⁎⁎⁎ 9.39) ⁎⁎⁎ 8.72) ⁎⁎⁎
1028
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
Context). In terms of Evidence, TF-CBT is an EBP for children with symptoms of PTSD that is straightforward and consistent with principles of cognitive-behavioral, exposure and parenting therapies that are widely accepted by MHPs. In terms of Context and Facilitation, the training and consultation method was chosen to train MHPs. This method ensured on-going education, support, and feedback in an organized, respectful, credible, and collaborative manner. Additionally, in terms of Context, after initializing TF-CBT training for MHPs, trauma-informed practice trainings were conducted with other systems involved in the care of children (e.g., CAC advocates, foster parents, child welfare staff) to help create a common language among providers, increase communication, and change the culture of state to be more trauma-informed. These efforts were made to support on-going and future EBP dissemination efforts. The process of dissemination has been described and preliminary outcomes have been reported. A larger more in depth evaluation of the impact of this initiative is currently under way. However, within a relatively short period of time (3 years), several important goals have been accomplished. First, 139 MHPs have been trained in TF-CBT and over 100 MHPs are currently completing the training process. In addition, MHPs have been contracted in or in collaboration with 70% of all CACs to provide TF-CBT and six identified outstanding MHPs have started the training process to become local experts to help enhance sustainability. Secondly, MHPs reported increases in knowledge and self-efficacy in TF-CBT components as well as conducting assessment of PTSD symptoms. Thirdly, several possible motivators were confirmed by participants to help gain buy-in for large-scale dissemination efforts (e.g., national and/or state certification in TF-CBT). Finally, in the first full year of TF-CBT dissemination, 420 children were registered by trained TF-CBT MHPs. There were several difficulties and limitations that we have encountered while training MHPs. First of all, MHPs attended the free initial trainings but gaining buy-in for these MHPs to complete the entire training process was difficult. For the first two years of training we had a 43% completion rate. This rate is respectable given that training was only required for a few MHPs (i.e., MHPs affiliated with CACs). Identification of MHPs most likely to complete the training and engage in TF-CBT with fidelity is a critical next step for the AR BEST program. Low penetration into rural counties has been a concern, so outreach has been strengthened in those CMHCs with satellite offices, and another training is planned for 2014 in a less populated area of the state. Additionally, collecting outcome symptom assessment measures (i.e., UCLA PTSD Reaction Index and SDQ) to assess improvement without incentive or requirement is difficult. For the total number of clients enrolled in the AR BEST database, MHPs completed a fair percentage (i.e., 60% to 67%) of initial assessment measures, which is positive and leads one to believe that children were being adequately assessed for symptoms of PTSD and that the children with elevated traumatic stress symptoms received TF-CBT. However, MHPs rarely completed the measures at discharge from treatment. Therefore, a limitation of this study is that symptom outcome cannot be reported. Additionally, MHPs rarely completed the TF-CBT fidelity checklist. They were monitored for fidelity for the two cases presented during the consultation calls, but we do not have data beyond this checkpoint. Despite these difficulties and limitations, AR BEST has made tremendous progress in a relatively short time and provided an example of how to train MHPs in TF-CBT through a fairly rigorous training process. Feedback from the participants that both completed and did not complete the training process helped inform future training years. We attempted to overcome barriers by adding an additional consultation call so clinicians now have 15 opportunities to complete 12 calls. Also, more consultation call times were made available to clinicians in an attempt to overcome the barrier of inflexibility in the call schedule. As far as motivators, a one-day advanced training was added for those who completed the web-based training, two-day training, and 12 consultation calls. Furthermore, we provided state recognition of TF-CBT training completion and attempted to make the AR BEST training requirements consistent with
the national TF-CBT certification requirements. Finally, we offered the opportunity for exceptional MHPs to become local experts in TF-CBT. AR BEST worked to change standard practice and culture throughout the mental health system in Arkansas. Now that there is a foundation of providers trained in TF-CBT throughout Arkansas, future efforts will focus on sustaining practices (i.e., advanced trainings, annual institutes, listservs, continuing education, online forums and website access). Training in trauma-informed practices will continue between other stakeholders and with the increased communication between stakeholders and presumably more buy-in, AR BEST will now focus on expanding training to other EBPs, as well as, focusing on broader issues such as effectiveness of TF-CBT in improving mental health of children. 5. Conclusions In summary, AR BEST has been successful in a relatively short time in changing therapeutic care for children who have experienced trauma throughout Arkansas. Large-scale dissemination of EBPs is difficult and requires collaboration across stakeholders and the use of a variety of strategies. The sharing of processes and lessons learned from large-scale dissemination efforts is particularly important for future success. References Burns, B., Phillips, S., Wagner, H., Barth, R., Kolko, D., Campbell, Y., et al. (2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 960–970. CATS Consortium (2007). Implementing CBT for traumatized children and adolescents after September 11: Lessons learned from the Child and Adolescent Trauma Treatments and Services (CATS) Project. Journal of Clinical Child and Adolescent Psychology, 36, 581–592. Chadwick Center for Children and Families & Child, & Adolescent Services Research Center (2009). The California evidence-based clearinghouse for child welfare. Retrieved August 2010 from http://www.cachildwelfareclearinghouse.org/ Chadwick Center for Children, Families (2004). Closing the quality chasm in child abuse treatment: Identifying and disseminating BEST practices. San Diego, CA: Author. Cohen, J. A., & Mannarino, A. P. (2008). Disseminating and implementing trauma-focused CBT in community settings. Trauma, Violence, & Abuse, 9, 214–226. Cohen, J. A., Mannarino, A. P., & Rogal, S. (2001). Treatment practices for childhood posttraumatic stress disorder. Child Abuse & Neglect, 25, 123–135. Copeland, W., Keeler, G., Angold, A., & Costello, E. (2007). Traumatic events and posttraumatic stress in childhood. Archives of General Psychiatry, 64, 577–584. Deblinger, E., Cohen, J., Mannarino, A., Murray, L., & Epstein, C. (2007). TF-CBT brief practice checklist. . Ebert, L., Amaya-Jackson, L., Markiewicz, J., & Burroughs, J. (2008). The NCCTS learning collaborative model for the adoption & implementation of evidence-based mental health treatment: NCCTS guidelines for conducting a learning collaborative. Los Angeles, CA and Durham, NC: National Center for Child Traumatic Stress and Duke University Evidence-Based Practice Implementation Center. Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., et al. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) Study. American Journal of Preventative Medicine, 14, 245–258. Finkelhor, D., Ormrod, R., Turner, H., & Hamby, S. (2005). The victimization of children and youth: A comprehensive, national survey. Child Maltreatment, 10, 5–25. Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. Goodman, R. F. (2002). Caring for kids after trauma and death: A guide for parents and professionals. New York: The Institute for Trauma and Stress, NYU Child Study Center. Hamblen, J. (1999). Fact sheet: PTSD in children and adolescents. : National Center for PTSD. Hernandez, J. T., Lodico, M., & DiClemente, R. J. (1993). The effects of child abuse and race on risk-taking in male adolescents. Journal of the National Medical Association, 85, 593–597. Hoven, C., Duarte, C., Lucas, C., Wu, P., Mandell, D., Goodwin, R., et al. (2005). Psychopathology among New York City public school children 6 months after September 11. Archives of General Psychiatry, 62, 545–552. Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence-based practice. Quality in Health Care, 7, 149–158. Kolko, D., Cohen, J., Mannarino, A., Baumann, B., & Knudsen, K. (2009). Community treatment of child sexual abuse: A survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services Research, 36, 37–49.
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029 Mental Health Services and Policy Program, Northwestern University (2008). Evaluation of the implementation of three evidence-based practices to address trauma for children and youth who are wards of the State of Illinois, final report. . Myers, J. E. B., Berliner, L., Briere, J., Hendrix, C. T., Jenny, C., & Reid, T. A. (Eds.). (2002). The APSAC handbook on child maltreatment. Thousand Oaks, CA: Sage Publishing. National Center for Child Traumatic Stress (2009). NCCTS leadership: Traumainformed systems. Retrieved from http://www.nctsnet.org/nctsn_assets/pdfs/ TraumaInformedServices_InfoBrief_FINAL.pdf National Institute for Mental Health (2001). Helping children and adolescents cope with violence and disasters. Bethesda, MD: Author. Pynoos, R. S., Rodriguez, N., Steinberg, A. S., Stuber, M., & Frederick, C. (1998). The UCLA PTSD reaction index for DSM IV (revision 1). Los Angeles: UCLA Trauma Psychiatry Program. Ringeisen, H., Casanueva, C., Urato, M., & Stambaugh, L. (2009). Mental health service use during the transition to adulthood for adolescents reported to the child welfare system. Psychiatric Services, 60, 1084–1091.
1029
Saunders, B. E., Berliner, L., & Hanson, R. F. (Eds.). (2004). Child physical and sexual abuse: Guidelines for treatment (revised report: April 26, 2004). Charleston, SC: National Crime Victims Research and Treatment Center. Siegel, J. A., & Williams, L. M. (2003). The relationship between child sexual abuse and female delinquency and crime: a prospective study. Journal of Research in Crime and Delinquency, 40, 71–94. Sigel, B. A., Benton, A. H., Lynch, C. E., & Kramer, T. L. (2013). Characteristics of 17 Statewide Initiatives to Disseminate Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). Psychological Trauma: Theory, Research, Practice, and Policy (in press). Silverman, W., Ortiz, C., Viswesvaran, C., Burns, B., Kolko, D., Putnam, F., et al. (2008). Evidence-based psychosocial treatments for children and adolescents exposed to traumatic events. Journal of Clinical Child and Adolescent Psychology, 37, 156–183. U.S. Department of Health and Human Services (2011). Child maltreatment 2010: Reports from the States to the National Child Abuse and Neglect Data System. Washington, DC: U.S. Government Printing Office.
Contents lists available at SciVerse ScienceDirect
Children and Youth Services Review journal homepage: www.elsevier.com/locate/childyouth
Statewide dissemination of trauma-focused cognitive-behavioral therapy (TF-CBT)☆ Benjamin A. Sigel a,⁎, Teresa L. Kramer a, Nicola A. Conners-Burrow b, Janice K. Church c, Karen B. Worley c, Nicholas A. Mitrani c a b c
Psychiatric Research Institute, University of Arkansas for Medical Sciences, United States Department of Family and Preventative Medicine, University of Arkansas for Medical Sciences, United States Department of Pediatrics, University of Arkansas for Medical Sciences, United States
a r t i c l e
i n f o
Article history: Received 28 November 2012 Received in revised form 25 March 2013 Accepted 27 March 2013 Available online 3 April 2013 Keywords: Dissemination Implementation Trauma Evidence-based practices Training
a b s t r a c t Despite evidence linking childhood trauma to subsequent social, emotional, psychological, and cognitive problems, many children who have experienced trauma do not receive mental health treatment that has been proven to be effective. Large-scale dissemination of evidence-based practices (EBPs) is one possible solution to enhance the current negative state of mental health treatment for these children. This article describes a dissemination effort of an EBP (i.e., Trauma-Focused Cognitive-Behavioral Therapy [TF-CBT]) for childhood symptoms of post-traumatic stress disorder throughout Arkansas. The effort targeted mental health professionals within child advocacy centers and community mental health centers across the state. The article describes the process of dissemination and implementation. Lessons learned and recommendations for future dissemination efforts are highlighted. © 2013 Elsevier Ltd. All rights reserved.
1. Introduction Nationally, the percentage of youth exposed to some form of trauma is high, ranging from 8% to 53%, depending on the type of trauma and population studied (e.g., Copeland, Keeler, Angold, and Costello, 2007; Finkelhor, Ormrod, Turner, and Hamby, 2005; U.S. Department of Health and Human Services, 2011). Despite high rates of subsequent related post-traumatic stress disorder (PTSD), and other symptomatologies, many children who have experienced trauma either do not receive treatment or receive treatment that has not proven to be effective (Burns et al., 2004; Cohen, Mannarino, and Rogal, 2001; Kolko, Cohen, Mannarino, Baumann, and Knudsen, 2009; Ringeisen, Casanueva, Urato, and Stambaugh, 2009). Without adequate and appropriate treatment, trauma symptoms may linger or exacerbate over time, developing into other mental health problems such as internalizing or externalizing disorders (Hamblen, 1999; Hernandez, Lodico, and DiClemente, 1993; Hoven et al., 2005; Siegel and Williams, 2003). In addition, lack of treatment has the potential to increase secondary adversities such as health problems, home and foster home placement disruptions, school difficulties, social maladjustment,
☆ This research was supported by a grant from the 2009 Arkansas Revenue Stabilization Act. ⁎ Corresponding author at: University of Arkansas for Medical Sciences, Psychiatric Research Institute, 4301 Markham Street, Slot 755, Little Rock, AR 72205, United States. Tel.: +1 501 364 5861. E-mail address: [email protected] (B.A. Sigel). 0190-7409/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.childyouth.2013.03.012
and substance abuse (Felitti et al., 1998; Goodman, 2002; Hamblen, 1999; Myers et al., 2002; National Institute for Mental Health, 2001). Large-scale dissemination of evidence-based practices (EBPs) is one possible solution to enhance the current negative state of mental health treatment for children who have experienced trauma. Kitson, Harvey, and McCormack (1998) suggest that successful dissemination of EBPs involves interplay of three core elements: Context, Evidence, and Facilitation. Context is the combination of culture, leadership, and measurement in which the dissemination effort is to take place. Successful dissemination efforts will occur in contexts that are open to and support continuing education and effective practices, are organized, and provide feedback. Evidence is the combination of research, clinical expertise, and patient choice of/for the practice being disseminated. For example, successful dissemination efforts will use practices that have significant research support (i.e., EBPs), have high levels of consensus and consistency of view of effectiveness, and are viewed as beneficial and helpful by patients. Facilitation is the process of implementing evidence into practice and is the combination of characteristics, role, and style of the facilitator(s). Successful dissemination efforts occur when facilitation is respected, credible, empathic, authentic, collaborative, supportive, consistent, and flexible. In the spring of 2009, the Arkansas Legislature approved funding to improve screening, monitoring, and continuity of care for children who experienced trauma in Arkansas to address the psychological impact of their trauma. The funding was disseminated through the Arkansas Commission on Child Abuse, Rape, and Domestic Violence which provided the infrastructure for the program and helped facilitate a
1024
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
multidisciplinary collaboration between the University of Arkansas for Medical Sciences Psychiatric Research Institute and the Department of Pediatrics. With the infrastructure and collaboration in place, the Arkansas Building Effective Services for Trauma (AR BEST) program was created. The mission of AR BEST is to improve outcomes for traumatized children and their families in Arkansas through excellence in clinical care, training, advocacy, and research/evaluation across systems. The remainder of the paper describes the process of disseminating an EBP for PTSD symptoms in children (i.e., Trauma-Focused CognitiveBehavioral Therapy [TF-CBT]) throughout Arkansas within a conceptual framework (i.e., Kitson et al., 1998). 1.1. Context AR BEST staff reviewed state-of-the-art trauma-informed practices and compared these to the state of care in Arkansas. Staff noted a need for improvement among Arkansas' providers charged with ensuring the safety and health of children (e.g., mental health, child welfare, juvenile justice, health, education, and first responders). Specifically, efforts were needed to create a trauma-informed system of care. A trauma-informed system of care can be broadly defined as: 1) Awareness of and knowledge about the impact of traumatic stress across systems; 2) Awareness of and access to effective trauma assessment and interventions strategies that exist within and across different systems; 3) Trauma-focused education and skill-building to providers and administrators within and across key child-serving systems in order to change practice; and 4) Promoting strong collaborations across systems and disciplines (National Center for Child Traumatic Stress, 2009). In Arkansas, efforts were needed to provide a unifying and/or systematic language or approach to assessing and treating children who have experienced trauma. Few community mental health providers (MHPs) were trained in trauma-informed EBPs and thus providers had few options for referrals in these practices. The practices for training advocates (i.e., individuals who provide support and referrals for child victims and their families) in child advocacy centers (CACs) varied across the state; furthermore, child welfare, foster parents, juvenile justice, and other systems staff had little knowledge about trauma-informed care (i.e., the impact of trauma on children) and lacked options for referrals to community mental health providers trained in trauma-informed evidencedbased treatments. Finally, little training was being conducted in traumainformed care with the foster parent, educational, juvenile justice, and first responder systems. AR BEST staff decided that disseminating TF-CBT was the first step in changing trauma practices. With MHPs trained in TF-CBT, referrals could be made to these providers from child welfare, juvenile justice, and other systems, therefore, increasing communication between systems and helping to gain buy-in from leadership within these systems. Once training in TF-CBT began, these other systems received training in trauma-informed care, further enhancing communication and providing a unified and systematic language for assessing and treating children who have experienced trauma. 1.2. Evidence TF-CBT, co-developed by Judith Cohen, M.D., Anthony Mannarino, Ph.D., and Esther Deblinger, Ph.D., is an EBP for childhood symptoms of PTSD with well-established efficacy. It is a treatment consistent with the principles of cognitive-behavioral, exposure, and parenting therapies that are widely accepted by MHPs. TF-CBT is a manualized and flexible therapeutic intervention. The therapeutic components include psychoeducation about the trauma; parenting skills; development of relaxation and other coping skills; feelings identification; understanding the link between thoughts, feelings and behaviors; developing a narrative of the traumatic event which has been
experienced by the child/adolescent and processing of associated thoughts, feelings and behaviors; gradual exposure to the traumatic event with the youth learning how to manage being exposed to such event(s); conjoint parent–child work; and enhancing safety/prevention skills. In recent reviews of research on treatment for children with PTSD symptoms (i.e., Chadwick Center for Children, Families, 2004; Chadwick Center for Children and Families and Child and Adolescent Services Research Center, 2009; Saunders, Berliner, and Hanson, 2004; Silverman et al., 2008), TF-CBT was the only treatment given the highest rating (i.e., evidence-based practice) in all of the reviews. Due to TF-CBT's promising outcomes in children with a history of trauma, large-scale dissemination is under way. In 2009, 19 states were already in the process of or had completed large-scale TF-CBT dissemination projects. The methods and processes of dissemination for the majority of these projects are briefly described in Sigel, Benton, Lynch, and Kramer (in press). 1.3. Facilitation Due to substantial evidence of efficacy and effectiveness and through on-going large-scale dissemination projects, several dissemination strategies have been developed to facilitate TF-CBT dissemination with fidelity and within an implementation research framework (i.e., Kitson et al., 1998). Cohen and Mannarino (2008) have discussed three dissemination methods that have been used (web-based learning, live training plus ongoing consultation, and learning collaborative) as well as the advantages and disadvantages of each model. Web-based learning involves completing TF-CBTWeb (accessible at www.musc.edu/tfcbt), developed by investigators at the Medical University of South Carolina Crime Victims Center in collaboration with TF-CBT developers. The website includes parent and child sections for each component of the model, video examples of all of the key treatment components, printable scripts to use, handouts for parents and children, instructions on how to handle clinically challenging situations, guidelines on cultural issues, and resources and links. This method's advantages are overcoming barriers such as cost, distance, and inconvenience of having to travel to live trainings and ability to train multiple MHPs at one time. Disadvantages are lack of interaction with a live trainer and no access to on-going consultation. The two other models of TF-CBT dissemination require more intensive training and follow-up efforts. The live training plus ongoing consultation model involves in-person training in TF-CBT followed by ongoing phone or in-person consultation. The training and consultation are provided by the treatment developers or other approved trainers who monitor trainees to ensure the treatment is being used with fidelity. The advantage of this model is that practitioners receive support when they start seeing clients, which may enhance fidelity and sustainability once training is completed. The disadvantages to this method are cost and limits to the number of MHPs that can be trained. By comparison, the learning collaborative model is designed to change the larger culture in which TF-CBT is implemented with the goal of achieving buy-in across systems and enhancing long-term sustainability. TF-CBT training typically involves separate tracks for MHPs, supervisors, and senior leaders (i.e., administrators and directors) (Ebert, Amaya-Jackson, Markiewicz, and Burroughs, 2008). Each track has several in-person trainings with follow-up by phone or in person consultation over the course of a year targeting different parts of the TF-CBT model, implementation strategies, and/or dissemination issues. The contents of the in-person trainings and consultations are based on where the therapist, supervisor, and/or senior leader are in the TF-CBT model, implementation, and/or dissemination process. The trainings and consultation are also provided by the treatment developers or other approved trainers. With this model, supervisors and senior leaders are involved in the calls, which may provide additional support to MHPs learning the model from within the agency as well as change or maintain the culture required for a particular model. The inclusion of senior leaders may enhance sustainability once training has ceased. Senior leaders
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
tend to have fewer turnovers and can sustain use of the model despite front-line worker turnover. Within the learning collaborative model, additional trainings are often conducted in trauma-informed care to other stakeholders (e.g., child welfare, Department of Mental Health, CACs, juvenile justice) to help achieve buy-in across systems. A possible disadvantage of this model over the live training and consultation model is that this method may be costlier and more timeconsuming. There is also an added expense of training supervisors, senior leaders, and stakeholders as well as conducting extra live trainings. Due to these factors, this model may result in fewer MHPs being trained but may enhance sustainability and buy-in across systems. In Arkansas, the decision was made to use the TF-CBT training and consultation dissemination method instead of the web-based learning or learning collaborative methods. Due to the disadvantages of web training alone, it was determined such an approach would yield limited sustainability of TF-CBT utilization. Furthermore, as the AR BEST project was just beginning in Arkansas, sufficient collaboration across systems of care necessary to conduct learning collaboratives was not adequately in place. As such, dissemination efforts would have been significantly delayed while these collaborations were further developed and enhanced. Furthermore, results and analysis of interviews with other states disseminating TF-CBT and previous research (e.g. CATS Consortium, 2007; Cohen and Mannarino, 2008; Mental Health Services, and Policy Program, Northwestern University, 2008; Sigel et al., in press) indicated that both the learning collaborative and training and consultation methods are effective and efficient approaches for widely disseminating TF-CBT. 2. Method 2.1. Participants AR BEST targeted MHPs working at or in collaboration with Arkansas' 13 CACs as well as the network of community mental health centers (CMHCs) that serve the entire state. By targeting CAC-affiliated MHPs, it was anticipated that training in an EBP would give CACs increased resources for referrals for mental health treatment especially for the state's sexual abuse victims. Furthermore, CAC accreditation standards are now requiring CACs to have collaborative agreements with MHPs who can provide EBPs, so it was anticipated this collaborative project would be very well-received by CAC directors. It was also deemed critical to the broad dissemination mission of the AR BEST project to ensure CMHC directors and clinical staff were familiar with the principles of trauma-informed care as well as EBPs, particularly in the very rural areas of Arkansas where CMHC staff would likely be the only resources for child and adolescent trauma victims and their families. A clear project goal was to have at least one TF-CBT trained clinician in every county of the state, such that EBPs known to be effective and efficient in treating trauma is available to all children and youth in Arkansas who require such interventions. Other MHPs from across the state have been invited to participate in the AR BEST project as space and training limitations have permitted. Arkansas' TF-CBT training model requires MHPs to complete three distinct and voluntary phases of training. First, MHPs completed TF-CBTWeb and received 10 free hours of continuing education credit (CEUs). Upon completion of TF-CBTWeb, MHPs were eligible to attend an in-person, free, two-day TF-CBT basic/introductory training conducted by Dr. Mannarino in Little Rock, Arkansas. After completing the two-day training and being awarded an additional 12 continuing education credits, MHPs were then eligible to participate in a series of bi-weekly phone consultation calls over an approximately seven-month period led by treatment developers, national TF-CBT experts, or AR BEST staff. Approximately 8–12 MHPs were in each consultation call group. To be considered successful in completing the ongoing consultation calls, MHPs were expected to participate in at least 12 out of 14 or more calls and to present on at least two separate treatment
1025
cases using one or more TF-CBT components. Participants received one CEU for each full-hour call in which they participated. The MHPs that completed the entire process received a certificate of TF-CBT training completion and were listed on a roster of trained TF-CBT MHPs that was posted on the AR BEST website. Additionally, AR BEST staff has worked with TF-CBT treatment developers to ensure that they will be eligible for national TF-CBT certification (when available). Finally, in the second and subsequent years of the project, MHPs who completed the required three phases of TF-CBT training (web-based, two-day inperson training, and series of consultation calls) were invited to participate in an in-person, free, one-day advanced TF-CBT conference led by Dr. Mannarino, earning six CEUs for the full-day training. To help facilitate sustainability of the dissemination project, AR BEST staff completed the TF-CBT training process with Dr. Mannarino leading their consultation calls in addition to observing/scribing for consultation calls led by the TF-CBT co-developers and other national experts. The bi-weekly consultation calls with Dr. Mannarino focused on case consultation and using TF-CBT with fidelity for approximately seven months and then were extended one year for monthly contacts to focus on skills development for conducting and leading TF-CBT consultation calls. To further ensure sustainability of the dissemination project, outstanding therapist participants who are not AR BEST staff were nominated by the treatment developers, national experts, or AR BEST staff to become local experts. Similar to the process already described for AR BEST staff, these local experts will receive additional training in consultation call facilitation and will initially observe/scribe and subsequently co-lead consultation calls with AR BEST staff in future rounds of TF-CBT training. Gradually, use of the treatment developers and other national TF-CBT experts will be phased out of the consultation call phase of Arkansas' dissemination project, and such calls will be managed exclusively by AR BEST staff and developing local experts. 2.2. Measures The AR BEST team and collaborators identified a set of tools that would benefit MHP implementation of TF-CBT and the children served by these MHPs. We sought to develop a dual-purpose system that would provide these tools to MHPs as well as provide information back to the AR BEST team about the efforts of MHPs to disseminate TF-CBT in Arkansas. Toward this end, a web-based data collection system was designed to include the following components: 1. Clinician Registration — All MHPs attending AR BEST training in TF-CBT were required to register in the system for tracking purposes. 2. Client Registration — The Client Registration component was the starting point for MHPs wishing to use the tools provided through the website to support their implementation of TF-CBT. Here MHPs provide basic demographic information about the client being served as well as information on the nature of the trauma and the alleged perpetrator. 3. Clinical Assessment Tools and Reports — MHPs attending the two day TF-CBT training also received training to administer two child assessment tools available on the website: the Strengths and Difficulties Questionnaire (SDQ) and the UCLA PTSD Reaction Index. MHPs were encouraged to administer the measures at intake, and at 3, 6 and 12 months post-intake (sooner if the client has already completed treatment and has been discharged) to monitor the symptoms of their clients. MHPs who have registered a client receive reminders to complete the assessment and can access reports summarizing the assessment results and graphing changes in symptoms over time. The Strengths and Difficulties Questionnaire (Goodman, 2001) is a brief behavioral screening questionnaire for children. Each version has 25 items divided between five clinical scales: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems and prosocial behavior. AR BEST made two versions of the
1026
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
SDQ available on the website: the parent-report version for children up to age 12, and the adolescent report version for children 12 and older. UCLA PTSD Reaction Index for DSM-IV (Pynoos, Rodriguez, Steinberg, Stuber, and Frederick, 1998) is designed to screen for exposure to traumatic events as well as symptoms of PTSD. The items of the UCLA PTSD Reaction indices are keyed to DSM-IV criteria and can provide preliminary PTSD diagnostic information. AR BEST made two versions available on the website: parent-report version for children up to age 12, and the adolescent-report version for children 12 and older. 4. Support for Treatment Planning — The AR BEST team worked with local experts in treatment planning to provide wording for treatment plan goals and objectives that commonly would be used when a clinician is planning to implement TF-CBT with a client. MHPs can select goals and objectives for any client that they registered and print or export the treatment plan information for the clients' chart or medical record. 5. Documenting the Elements of TF-CBT — For MHPs who want to monitor the fidelity of their implementation of TF-CBT, the AR BEST team developed an option where the clinician could access the website after each session and click on the TF-CBT components that were addressed in that session. This option was based on the Trauma-focused Cognitive-Behavior Therapy (TF-CBT) Brief Practice Checklist (Deblinger, Cohen, Mannarino, Murray, and Epstein, 2007), which is designed to track the timing and implementation of specific TF-CBT components in a manner that helps MHPs and supervisors to determine whether fidelity is being adequately maintained. 2.3. Training evaluation tools TF-CBT Basic Training Evaluation: MHPs attending the 2011 and 2012 TF-CBT initial two-day trainings were asked to complete a pre- and post-test designed by the AR BEST team. The pre-test was designed to assess MHPs' initial knowledge and self-efficacy related to TF-CBT, their willingness to use new interventions, and their interest in being recognized as a clinician in the state with specialized training in TF-CBT. The post-test was designed to assess MHPs' post-training self-efficacy related to TF-CBT, their satisfaction with the training and their intention to participate in ongoing consultation opportunities. TF-CBT Advanced Training Evaluation: MHPs attending the 2011 and 2012 Advanced Training Day (for MHPs trained during the prior two years) were asked to complete a pre- and post-test evaluation. The pre-test was designed to assess MHPs' knowledge and self-efficacy related to TF-CBT after at least a year of implementation, their willingness to use new interventions, and their interest in being recognized as a clinician in the state with specialized training in TF-CBT. The post-test was designed to assess MHPs' post-training self-efficacy related to TF-CBT and their satisfaction with the training. In addition to evaluating the trainings, the AR BEST team also monitored the ongoing consultation call process. Specifically, a member of the AR BEST team participated in each call as a scribe. The scribes documented when call participants/trainees introduced a potential client who would benefit from using the TF-CBT treatment model and when a participant presented a case (defined as describing the client's specific trauma(s) and discussing components of the model used). The scribe also documented and kept a record of call attendance and the total number of presented cases by each call participant. 3. Results 3.1. MHP training TF-CBTWeb was promoted by AR BEST in the fall of 2009. Prior to that time (October, 2005–July, 2009), 47 mental health MHPs in Arkansas had
completed this training. Between August 2009 and July 2011, 567 MHPs completed this web-based training. Annually, beginning in 2010, AR BEST sponsored a two-day conference on TF-CBT for MHPs; in the second year of the project a one-day advanced conference was added for MHPs who had previously completed a minimum of 12 TF-CBT consultation calls. Over the first three years of the project, 501 providers from 76% (57 out of a total of 75) of Arkansas counties completed the web-based and two-day TF-CBT training. Following the 2010 and 2011 conferences, 139 (43%) out of 322 providers went on to complete the required number of consultation calls needed to be acknowledged by AR BEST as fully TF-CBT trained (i.e., completed web-based training, attended two-day in-person training, completed at least 12 consultations, and presented at least two cases while on the consultation calls). Consultation calls from the 2012 conference are still ongoing, with 100 participants. Of the total 139 fully trained MHPs, 19 were contracted by CACs to provide TF-CBT in 9 out of 13 (70%) CACs located throughout the state. To date, 328 TF-CBT cases have been presented during the consultation calls. These 139 fully trained MHPs represent 52% (39 out of a total of 75) of Arkansas counties. In 2011, 50 of the 56 eligible fully trained MHPs participated in the one-day advanced training while 94 of the 139 eligible fully trained MHPs (i.e., from both the 2010 and 2011 training groups) attended in 2012. Finally, six outstanding MHPs from the 2010 training group were identified by the treatment developers, other national TF-CBT experts, and/or AR BEST staff and have begun their training process to become local experts. Results from the 2011 and 2012 TF-CBT training evaluation pre-test forms indicate that almost half of all participants worked in CMHC settings (40.1% of two-day introductory training attendees and 43.2% of one-day advanced-level training participants). At the two-day introductory training, most other attendees worked for other community providers of mental health services (46.9%), were in private practice (7.4%), worked at a local hospital (3.7%) or worked in a CAC (1.7%). Of the 365 MHPs who completed the two-day introductory training in 2011–2012, 340 provided matched pre-post training evaluations. Of the 130 attending the one-day advanced training, 121 participants provided matched evaluations. Pre-test survey results showed that most conference attendees were willing to use new and different manualized interventions with their clients, with advanced-level training participants being slightly more open. For example, 80.6% of two-day introductory and 87.3% of one-day advanced-level attendees agreed that they like to use new types of therapies to help their clients. From the post-test evaluations, with regard to TF-CBT and its rationale, most participants agreed or strongly agreed that the trainings confirmed their current knowledge or increased their knowledge of TF-CBT. For example, after the two-day basic training, 96.6% agreed that they “understand the rationale, empirical support, and core components of TF-CBT” and 96.0% agreed that the conference “improved my ability to treat children and adolescents with a trauma history.” Results were similar for advanced-level training attendees. When asked about their intent to use what they learned, 78.4% of the two-day introductory training attendees and 64.0% of one-day advanced-level training participants agreed that they intend to change their current practices as a result of attending the TF-CBT training, and 77.5% of two-day introductory TF-CBT attendees intended to participate on follow-up consultation calls. MHPs perceived few barriers to implementation of TF-CBT. For example, only 6.2% of two-day introductory-level attendees and 10.3% of one-day advanced-level attendees reported that productivity requirements at their respective agencies would prevent them from participating in ongoing training and consultation. In terms of motivators for participating in ongoing training opportunities, more than half of those who attended the two-day introductory training reported that national TF-CBT certification was important to them (51.6%), followed by statewide recognition of completion through the AR BEST website (45.7%) and appointment by AR BEST as a local expert (24.8%). Almost three-fourths (72.8%) of advanced-level
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
participants stated that national certification was important. Statewide recognition for training was valuable to 69.3% of advanced-level attendees, and 35.4% of attendees of the one-day advanced training cited the importance of being appointed by AR BEST as a local expert. Both two-day basic and one-day advanced-level training participants were asked before and after the 2011 and 2012 trainings to rate how certain they were that they could perform the TF-CBT components. Confidence was rated on a scale of 0 (can't do at all) to 100 (highly certain can do). As shown in Table 1, there were significant increases in attendee's confidence to implement each of the targeted treatment steps. (Note that two-day introductory training attendees had already completed the on-line training prior to participation in the in-person training, illustrating that in-person training may significantly MHPs' confidence in their ability to implement the TF-CBT model with fidelity.)
3.2. Clinician use of TF-CBT web-based tools Data from the first full year of implementation of the AR BEST website with tools for TF-CBT implementation suggest that the most popular tool with MHPs was the client assessment options. From July 2010 to July 2011, AR BEST-trained MHPs registered 420 clients from 59 out of 75 Arkansas counties in the AR BEST system. The majority of clients receiving mental health services were female (57.6%), ranging in age from 5 to 19 years, with a mean age of 11.4 years (SD = 3.7). Registered clients were most commonly being seen in CMHCs (34.0%), by private providers (28.3%) or in CACs (21.0%). The majority of AR BEST registered clients (56.4%) had experienced sexual trauma, while others (34.0%) had witnessed violence, been physically abused (27.4%) or experienced neglect (18.1%). In most cases (79.5%), the clinician intended to use TF-CBT with the family. When they chose not to, the most common reasons were the need to focus first on crisis intervention (11.2%) or because of caregiver limitations or lack of caregiver support (7.8%). Counseling was court ordered in 16.9% of the cases. CACs were one of the most common referral source for clients getting into therapy (21%), which highlights the critical role of
1027
CACs in connecting families to mental health services. Other key referral sources included the child welfare agency (20%) and self-referrals (21%). Of the 420 clients that were registered, 282 (67%) completed the UCLA PTSD Reaction Index. There were 142 parent-reports and 142 adolescent-reports. The parent-report indicated 59.9% of children were experiencing moderate or severe symptoms of PTSD. Based on adolescent report, 64.8% were experiencing moderate or severe symptoms. Of the 420 clients that were registered, 250 (60%) completed the SDQ. There were 130 parent-reports and 120 adolescent-reports. The SDQ results suggest that about one-half of children exhibit behaviors that fall in the ‘abnormal’ range (based on the total score), meaning that the behaviors are clinically serious. Specifically, 48.3% of adolescents reported symptoms in the abnormal range, and 53.1% of parents reported symptoms in the abnormal range. Based on adolescent and parent report, clinically elevated problems on the SDQ subscales were common: Problems with peers (43% and 35% for adolescent and parent respectively), conduct problems (24% for adolescents, 55% for parents) emotional symptoms (41% for adolescent, 30% for parent), and hyperactivitity (24% for adolescents and 22% for parents). Few MHPs consistently used the treatment planning or fidelity monitoring tools. Anecdotally, many reported that while they appreciated seeing appropriate treatment plan goals and objectives they could use as a model, they were required to use their own treatment plan forms within their agency.
4. Discussion Large-scale dissemination and implementation are no easy tasks. Fixsen, Naoom, Blasé, Friedman, and Wallace (2005) argue that dissemination requires on-going, multilevel strategies, including partnerships between community providers and skilled researchers; or other ongoing training opportunities within each implementation site; and sharing of lessons learned from a variety of dissemination efforts. Furthermore, Kitson et al. (1998) suggest that successful dissemination of EBPs involves interplay of three core elements (i.e., Evidence, Facilitation, and
Table 1 Change in clinician confidence in implementing components of TF-CBT pre- and post-training. Item
Assess symptoms in a child with a history of trauma. Provide caregiver involvement in sessions. Provide psycho-education for the child and family. Explain the physiology of relaxation to the child and family. Instruct a child in relaxation skills. Teach coping skills to help with the emotional effects of trauma. Review the relationship between thoughts, feelings and behaviors with a child. Develop a story of the trauma with a child. Help a child modify cognitive distortions about trauma. Assist child in sharing trauma story with caregivers. Address safety issues with child and family. Teach social skills to a child. Work with parents on managing a child's problem behaviors. a
Basic participants (N = 340)
Advanced participants (N = 121)
Pre-testa
Post-test
Pre-test
Post-test
68.98 (sd = 68.55 (sd = 72.81 (sd = 69.76 (sd = 74.10 (sd = 71.99 (sd = 62.78 (sd = 57.27 (sd = 61.49 (sd = 60.11 (sd = 74.03 (sd = 76.87 (sd = 75.61 (sd =
84.42 (sd = 83.47 (sd = 86.46 (sd = 85.49 (sd = 87.33 (sd = 86.89 (sd = 87.06 (sd = 83.62 (sd = 83.97 (sd = 83.15 (sd = 87.86 (sd = 87.78 (sd = 86.56 (sd =
87.53 (sd = 82.25 (sd = 85.48 (sd = 86.07 (sd = 87.43 (sd = 86.45 (sd = 85.61 (sd = 81.58 (sd = 80.78 (sd = 81.31 (sd = 87.17 (sd = 87.23 (sd = 87.49 (sd =
89.47 (sd = 88.42 (sd = 90.41 (sd = 90.33 (sd = 91.40 (sd = 91.35 (sd = 89.53 (sd = 88.41 (sd = 87.63 (sd = 88.57 (sd = 90.94 (sd = 90.64 (sd = 91.00 (sd =
21.10) 22.72) 21.22) 21.93) 20.86) 20.76) 27.18) 25.02) 24.34) 24.35) 21.53) 19.97) 19.80)
Basic attendees already completed the on-line TF-CBT training prior to conference attendance. ⁎⁎⁎ p b .001.
13.65) ⁎⁎⁎ 13.35) ⁎⁎⁎ 12.82) ⁎⁎⁎ 12.61) ⁎⁎⁎ 11.60) ⁎⁎⁎ 11.60) ⁎⁎⁎ 12.64) ⁎⁎⁎ 12.85) ⁎⁎⁎ 12.77) ⁎⁎⁎ 12.88) ⁎⁎⁎ 11.94) ⁎⁎⁎ 12.22) ⁎⁎⁎ 12.98) ⁎⁎⁎
11.29) 14.25) 12.09) 10.94) 11.52) 10.58) 12.52) 15.12) 15.44) 17.20) 12.28) 12.48) 12.32)
9.38) ⁎⁎⁎ 9.93) ⁎⁎⁎ 8.80) ⁎⁎⁎ 8.37) ⁎⁎⁎ 8.50) ⁎⁎⁎ 8.22) ⁎⁎⁎ 10.98) ⁎⁎⁎ 10.78) ⁎⁎⁎ 12.20) ⁎⁎⁎ 11.70) ⁎⁎⁎ 9.83) ⁎⁎⁎ 9.39) ⁎⁎⁎ 8.72) ⁎⁎⁎
1028
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029
Context). In terms of Evidence, TF-CBT is an EBP for children with symptoms of PTSD that is straightforward and consistent with principles of cognitive-behavioral, exposure and parenting therapies that are widely accepted by MHPs. In terms of Context and Facilitation, the training and consultation method was chosen to train MHPs. This method ensured on-going education, support, and feedback in an organized, respectful, credible, and collaborative manner. Additionally, in terms of Context, after initializing TF-CBT training for MHPs, trauma-informed practice trainings were conducted with other systems involved in the care of children (e.g., CAC advocates, foster parents, child welfare staff) to help create a common language among providers, increase communication, and change the culture of state to be more trauma-informed. These efforts were made to support on-going and future EBP dissemination efforts. The process of dissemination has been described and preliminary outcomes have been reported. A larger more in depth evaluation of the impact of this initiative is currently under way. However, within a relatively short period of time (3 years), several important goals have been accomplished. First, 139 MHPs have been trained in TF-CBT and over 100 MHPs are currently completing the training process. In addition, MHPs have been contracted in or in collaboration with 70% of all CACs to provide TF-CBT and six identified outstanding MHPs have started the training process to become local experts to help enhance sustainability. Secondly, MHPs reported increases in knowledge and self-efficacy in TF-CBT components as well as conducting assessment of PTSD symptoms. Thirdly, several possible motivators were confirmed by participants to help gain buy-in for large-scale dissemination efforts (e.g., national and/or state certification in TF-CBT). Finally, in the first full year of TF-CBT dissemination, 420 children were registered by trained TF-CBT MHPs. There were several difficulties and limitations that we have encountered while training MHPs. First of all, MHPs attended the free initial trainings but gaining buy-in for these MHPs to complete the entire training process was difficult. For the first two years of training we had a 43% completion rate. This rate is respectable given that training was only required for a few MHPs (i.e., MHPs affiliated with CACs). Identification of MHPs most likely to complete the training and engage in TF-CBT with fidelity is a critical next step for the AR BEST program. Low penetration into rural counties has been a concern, so outreach has been strengthened in those CMHCs with satellite offices, and another training is planned for 2014 in a less populated area of the state. Additionally, collecting outcome symptom assessment measures (i.e., UCLA PTSD Reaction Index and SDQ) to assess improvement without incentive or requirement is difficult. For the total number of clients enrolled in the AR BEST database, MHPs completed a fair percentage (i.e., 60% to 67%) of initial assessment measures, which is positive and leads one to believe that children were being adequately assessed for symptoms of PTSD and that the children with elevated traumatic stress symptoms received TF-CBT. However, MHPs rarely completed the measures at discharge from treatment. Therefore, a limitation of this study is that symptom outcome cannot be reported. Additionally, MHPs rarely completed the TF-CBT fidelity checklist. They were monitored for fidelity for the two cases presented during the consultation calls, but we do not have data beyond this checkpoint. Despite these difficulties and limitations, AR BEST has made tremendous progress in a relatively short time and provided an example of how to train MHPs in TF-CBT through a fairly rigorous training process. Feedback from the participants that both completed and did not complete the training process helped inform future training years. We attempted to overcome barriers by adding an additional consultation call so clinicians now have 15 opportunities to complete 12 calls. Also, more consultation call times were made available to clinicians in an attempt to overcome the barrier of inflexibility in the call schedule. As far as motivators, a one-day advanced training was added for those who completed the web-based training, two-day training, and 12 consultation calls. Furthermore, we provided state recognition of TF-CBT training completion and attempted to make the AR BEST training requirements consistent with
the national TF-CBT certification requirements. Finally, we offered the opportunity for exceptional MHPs to become local experts in TF-CBT. AR BEST worked to change standard practice and culture throughout the mental health system in Arkansas. Now that there is a foundation of providers trained in TF-CBT throughout Arkansas, future efforts will focus on sustaining practices (i.e., advanced trainings, annual institutes, listservs, continuing education, online forums and website access). Training in trauma-informed practices will continue between other stakeholders and with the increased communication between stakeholders and presumably more buy-in, AR BEST will now focus on expanding training to other EBPs, as well as, focusing on broader issues such as effectiveness of TF-CBT in improving mental health of children. 5. Conclusions In summary, AR BEST has been successful in a relatively short time in changing therapeutic care for children who have experienced trauma throughout Arkansas. Large-scale dissemination of EBPs is difficult and requires collaboration across stakeholders and the use of a variety of strategies. The sharing of processes and lessons learned from large-scale dissemination efforts is particularly important for future success. References Burns, B., Phillips, S., Wagner, H., Barth, R., Kolko, D., Campbell, Y., et al. (2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 960–970. CATS Consortium (2007). Implementing CBT for traumatized children and adolescents after September 11: Lessons learned from the Child and Adolescent Trauma Treatments and Services (CATS) Project. Journal of Clinical Child and Adolescent Psychology, 36, 581–592. Chadwick Center for Children and Families & Child, & Adolescent Services Research Center (2009). The California evidence-based clearinghouse for child welfare. Retrieved August 2010 from http://www.cachildwelfareclearinghouse.org/ Chadwick Center for Children, Families (2004). Closing the quality chasm in child abuse treatment: Identifying and disseminating BEST practices. San Diego, CA: Author. Cohen, J. A., & Mannarino, A. P. (2008). Disseminating and implementing trauma-focused CBT in community settings. Trauma, Violence, & Abuse, 9, 214–226. Cohen, J. A., Mannarino, A. P., & Rogal, S. (2001). Treatment practices for childhood posttraumatic stress disorder. Child Abuse & Neglect, 25, 123–135. Copeland, W., Keeler, G., Angold, A., & Costello, E. (2007). Traumatic events and posttraumatic stress in childhood. Archives of General Psychiatry, 64, 577–584. Deblinger, E., Cohen, J., Mannarino, A., Murray, L., & Epstein, C. (2007). TF-CBT brief practice checklist. . Ebert, L., Amaya-Jackson, L., Markiewicz, J., & Burroughs, J. (2008). The NCCTS learning collaborative model for the adoption & implementation of evidence-based mental health treatment: NCCTS guidelines for conducting a learning collaborative. Los Angeles, CA and Durham, NC: National Center for Child Traumatic Stress and Duke University Evidence-Based Practice Implementation Center. Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., et al. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) Study. American Journal of Preventative Medicine, 14, 245–258. Finkelhor, D., Ormrod, R., Turner, H., & Hamby, S. (2005). The victimization of children and youth: A comprehensive, national survey. Child Maltreatment, 10, 5–25. Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. Goodman, R. F. (2002). Caring for kids after trauma and death: A guide for parents and professionals. New York: The Institute for Trauma and Stress, NYU Child Study Center. Hamblen, J. (1999). Fact sheet: PTSD in children and adolescents. : National Center for PTSD. Hernandez, J. T., Lodico, M., & DiClemente, R. J. (1993). The effects of child abuse and race on risk-taking in male adolescents. Journal of the National Medical Association, 85, 593–597. Hoven, C., Duarte, C., Lucas, C., Wu, P., Mandell, D., Goodwin, R., et al. (2005). Psychopathology among New York City public school children 6 months after September 11. Archives of General Psychiatry, 62, 545–552. Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence-based practice. Quality in Health Care, 7, 149–158. Kolko, D., Cohen, J., Mannarino, A., Baumann, B., & Knudsen, K. (2009). Community treatment of child sexual abuse: A survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services Research, 36, 37–49.
B.A. Sigel et al. / Children and Youth Services Review 35 (2013) 1023–1029 Mental Health Services and Policy Program, Northwestern University (2008). Evaluation of the implementation of three evidence-based practices to address trauma for children and youth who are wards of the State of Illinois, final report. . Myers, J. E. B., Berliner, L., Briere, J., Hendrix, C. T., Jenny, C., & Reid, T. A. (Eds.). (2002). The APSAC handbook on child maltreatment. Thousand Oaks, CA: Sage Publishing. National Center for Child Traumatic Stress (2009). NCCTS leadership: Traumainformed systems. Retrieved from http://www.nctsnet.org/nctsn_assets/pdfs/ TraumaInformedServices_InfoBrief_FINAL.pdf National Institute for Mental Health (2001). Helping children and adolescents cope with violence and disasters. Bethesda, MD: Author. Pynoos, R. S., Rodriguez, N., Steinberg, A. S., Stuber, M., & Frederick, C. (1998). The UCLA PTSD reaction index for DSM IV (revision 1). Los Angeles: UCLA Trauma Psychiatry Program. Ringeisen, H., Casanueva, C., Urato, M., & Stambaugh, L. (2009). Mental health service use during the transition to adulthood for adolescents reported to the child welfare system. Psychiatric Services, 60, 1084–1091.
1029
Saunders, B. E., Berliner, L., & Hanson, R. F. (Eds.). (2004). Child physical and sexual abuse: Guidelines for treatment (revised report: April 26, 2004). Charleston, SC: National Crime Victims Research and Treatment Center. Siegel, J. A., & Williams, L. M. (2003). The relationship between child sexual abuse and female delinquency and crime: a prospective study. Journal of Research in Crime and Delinquency, 40, 71–94. Sigel, B. A., Benton, A. H., Lynch, C. E., & Kramer, T. L. (2013). Characteristics of 17 Statewide Initiatives to Disseminate Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). Psychological Trauma: Theory, Research, Practice, and Policy (in press). Silverman, W., Ortiz, C., Viswesvaran, C., Burns, B., Kolko, D., Putnam, F., et al. (2008). Evidence-based psychosocial treatments for children and adolescents exposed to traumatic events. Journal of Clinical Child and Adolescent Psychology, 37, 156–183. U.S. Department of Health and Human Services (2011). Child maltreatment 2010: Reports from the States to the National Child Abuse and Neglect Data System. Washington, DC: U.S. Government Printing Office.