Evidence-based mental health interventions for traumatized youth: A statewide dissemination project

Behaviour Research and Therapy 49 (2011) 579e587

Contents lists available at ScienceDirect

Behaviour Research and Therapy journal homepage: www.elsevier.com/locate/brat

Invited Essay

Evidence-based mental health interventions for traumatized youth: A statewide dissemination project Stephen A. Wonderlich a, b, c, *, Heather K. Simonich b, Tricia Cook Myers c, Wendy LaMontagne d, JoAnne Hoesel d, Ann L. Erickson b, Myla Korbel e, Ross D. Crosby a, b a

Department of Clinical Neuroscience, University of North Dakota, School of Medicine and Health Sciences, Fargo, ND, USA Neuropsychiatric Research Institute, 120 South 8th St., Fargo, ND 58103, USA Sanford Health, Fargo, ND, USA d North Dakota Department of Human Services, Bismarck, ND, USA e Rape & Abuse Crisis Center of Fargo-Moorhead, Fargo, ND, USA b c

a r t i c l e i n f o

a b s t r a c t

Article history: Received 1 July 2011 Accepted 1 July 2011

Psychological trauma in childhood has been shown to increase a variety of psychological disturbances and psychiatric disorders. Although evidence-based treatments for children who have been traumatized exist, they are infrequently used by clinicians treating children. The present paper describes the creation of the Treatment Collaborative for Traumatized Youth (TCTY) which is a statewide partnership in North Dakota designed to disseminate efficacious treatments for traumatized children and monitor outcomes across a broad, rural, geographic expanse. The paper reviews the dissemination strategy developed by the TCTY, reports outcomes regarding both clinicians and child participants, and highlights problems identified in the project and solutions that were generated. Ó 2011 Elsevier Ltd. All rights reserved.

Keywords: Childhood trauma Dissemination Evidence-based treatment Learning collaborative

Child abuse and neglect are pervasive problems in the United States with epidemiologic data suggesting that 702,000 U.S. children experience some form of abuse in a given year (U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau, 2010). Moreover, there is increasing evidence that histories of child maltreatment increase the risk of a variety of forms of psychopathology, including depression (Chaffin, Silovsky, & Vaughn, 2005; Wiersema et al., 2009), substance use disorders (Molnar, Buka, & Kessler, 2001; Wilsnack, Vogeltanz, Klassen, & Harris, 1997), eating disorders (Wonderlich, Brewerton, Jocic, & Dansky, Abbott, 1997), anxiety disorders (Kendall-Tackett, Williams, & Finkelhor, 1993), and disturbances in personality functioning (Battle et al., 2004; Zanarini et al., 2002). Recent studies of neurotransmitter functioning, hormone function, and brain imaging further suggest that the effects of early child trauma impact neurobiologic systems which may play a role in the psychiatric and psychological problems child maltreatment victims’ experience (Cohen, Perel, DeBellis, Friedman, & Putnam, 2002). Unfortunately, few children experiencing child maltreatment receive any mental health services

* Corresponding author. Neuropsychiatric Research Institute, 120 South 8th St., Fargo, ND 58103, USA. Tel.: þ1 701 365 4910; fax: þ1 701 293 3226. E-mail address: [email protected] (S.A. Wonderlich). 0005-7967/$ e see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.brat.2011.07.003

(Kolko, Cohen, Mannarino, Baumann, & Knudsen, 2009; Kolko, Selelyo, & Brown, 1999) and evidence-based techniques which address the psychological trauma are infrequently employed when treatment is initiated (Cohen, Mannarino, & Rogal, 2001). Moreover, failure to treat victims soon after trauma may result in increasingly complicated and difficult to treat forms of psychopathology in adulthood (Cohen et al., 2002; Wonderlich et al., 1997). The present paper describes the development of the Treatment Collaborative for Traumatized Youth (TCTY). TCTY is a program which disseminates evidence-based treatments for traumatized children across the state of North Dakota in the United States. North Dakota is a rural state which borders on Canada and covers 70,704 square miles. In 2006 the TCTY was established through several foundation grants and recently has moved into a more formal collaboration with the State of North Dakota Department of Human Services to disseminate treatments. The overarching goal of this essay is to describe the evolution of the TCTY. First we will briefly review the topic of dissemination of evidence-based treatments, particularly for children who have been traumatized. Second, we will describe the phases of development of the TCTY project, including our efforts to learn and disseminate appropriate evidence-based treatments and institute data collection procedures in North Dakota. Third, we will conclude this paper with a discussion of obstacles encountered in disseminating evidence-based treatments across large geographic areas with minimal financial

580

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

support and our recommendations for others undertaking similar projects. The dissemination of evidence-based treatments As science continues to identify empirically supported treatments, the failure of these treatments to reach frontline clinicians is well recognized (National Institute of Health, 2006a, 2006b; Insel, 2009). For example, across a broad array of psychopathologies, including panic disorder, post-traumatic stress disorder, generalized anxiety disorder, and bulimia nervosa, there is a consistent pattern that the majority of patients with these diagnoses fail to receive empirically supported treatments (Craske et al., 2009; Shafran et al., 2009). Consequently, there has been a renewed interest in studying factors which may limit or enhance the dissemination of empirically supported treatments (e.g., see special issue of Behaviour Research and Therapy, 2009). The broad issue of transporting treatment that has been shown to be efficacious in laboratories or research clinics into frontline service provision settings has become an increasingly important clinical and scientific issue (McHugh, Murray, & Barlow, 2009). This movement is characterized by efforts to identify obstacles to effective dissemination (Borntrager, Chorpita, Higa, & Weisz, 2009; Shafran et al., 2009), innovative intervention formats (McHugh et al., 2009), and new models of training and supervision (e.g., Chorpita & Regan, 2009). Large scale or community based models of dissemination of evidence-based practices are emerging (e.g., Becker, Stice, Shaw, & Woda, 2009; Ruzek & Rosen, 2009; SAMHSA National Child Traumatic Stress Network, 2000). Becker et al. (2009) describe community partnership research (CPR) strategies in the dissemination of empirically supported treatment. CPR involves creating community partnerships that promote a sharing of power and decision making about health related problems and efforts to create solutions (Israel, Eng, Schulz, & Parker, 2005). Becker et al. (2009) suggest that empirically supported treatments may be beneficially disseminated through such reciprocal and collaborative partnerships in which researchers and community members (e.g., clinicians) each bring their respective areas of expertise to the dissemination effort. Through such collaboration, researchers may more effectively disseminate their interventions and clinicians remain an important part of the process. Dissemination of evidence-based treatments for maltreated children Empirically supported treatments for children who have been victims of abuse and neglect are seldom used in practice, in spite of the fact that routine services typically delivered to maltreated children do not improve mental health status (Kolko et al., 2009). Furthermore, several treatments have been shown in randomized control trials to significantly reduce trauma related symptoms in maltreated children (Chaffin, Bonner, & Hill, 2001; Cohen et al., 2001; Kolko, Baumann, & Caldwell, 2003). The need to transport evidence-based treatments to community settings and practitioners who serve maltreated children has increasingly been emphasized (Kolko et al., 2009; SAMHSA National Child Traumatic Stress Network, 2000; Saunders, Berliner, & Hanson, 2003). One means of such dissemination is the learning collaborative methodology, originally developed by the Institute for Healthcare Improvement (2003). This model has been successfully applied by the National Child Traumatic Stress Network (NCTSN) for the dissemination of mental health treatments for children who have experienced traumatic events (SAMHSA National Child Traumatic Stress Network, 2000). The NCTSN is a national network of

programs designed to enhance a variety of services to maltreated children, their families, and professionals who serve them (www. nctsn.org). The network is hierarchically organized with 1) national coordinating centers (i.e., Category 1 centers), 2) several academically oriented centers who are charged with developing and disseminating evidence-based treatments for traumatized children (i.e., Category 2 centers), and 3) a larger number of centers who attempt to implement evidence-based treatments developed by Category 2 sites in frontline community settings (i.e., Category 3 centers). The Learning Collaborative Model utilized by the NCTSN (Markiewicz, Ebert, Ling, Amaya-Jackson, & Kisiel, 2006) is a unique approach to the dissemination and adoption of trauma-specific mental health interventions. In contrast to traditional dissemination strategies which rely on one time training workshops that are increasingly thought to be ineffective (Chorpita & Regan, 2009; McHugh et al., 2009), the goal of the Learning Collaborative Model is to improve training, implementation, and sustained use of evidence-based practices. The Learning Collaborative model is an ongoing learning process that involves a series of in-person training sessions and follow-up consultation over the course of 9e18 months. In the NCTSN, teams made up of administrative and clinical staff from centers across the United States participate in the Learning Collaborative process. Furthermore, the Learning Collaborative approach encourages the exchange of experiences and ongoing feedback regarding the implementation and sustainability of these new clinical practices within diverse clinical centers. The ultimate goal of the Learning Collaborative model is to reduce the gap between the evidence-based treatments and “treatment as usual” practice in childhood trauma. Establishing a learning collaborative for trauma in North Dakota Over the last 20 years, one of the authors (SAW) has collaborated with a variety of professionals working in the area of child protection, child advocacy, and child maltreatment across the state of North Dakota. The state of North Dakota does not have an established mental health facility for traumatized children, which may be common in larger urban areas. Typically, children who have been identified as abused or neglected are evaluated by a child protection worker and possibly medical personnel following substantiated child abuse reports. The child protection worker makes mental health referrals to clinicians in the community who have expressed an interest or capability of working with these children. Various child advocacy agencies in the region have sponsored workshops in which treatment techniques are presented by experts in the field. Although some of the interventions described in the workshops are considered evidence-based, others are not, and ongoing supervision for workshop attendees has not been provided. Thus, before the development of the TCTY, there was a relatively small collection of mental health professionals in the state who treated abused and neglected children and the range of interventions provided was quite broad and often outside of the realm of evidence-based practice. Consequently, a decision was made to use the Learning Collaborative format, which had been espoused by the NCTSN, to disseminate evidence-based treatments for maltreated children across the state. The initial goals of the TCTY were: 1) identify a group of clinicians who would complete intensive training in evidence-based treatments for traumatized children; 2) create a local training team who would conduct workshops and provide ongoing consultation for clinicians in the state; and 3) provide ongoing support and training that would sustain the community of empirically supported therapists trained by the TCTY. Below, we will describe our experiences pursuing these goals.

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

Selecting evidence-based treatments to disseminate through the TCTY At the start of the TCTY, a brief, informal survey of providers and administrators frequently serving maltreated children in North Dakota was conducted and revealed that there was a need for treatments that rely on both group therapy-based and individual/ family therapy-based formats. Group-based approaches were most feasible for children living in collective institutional foster placements, while individual/family formats would be most appropriate for children living in their family of origin or individual fosterfamily environments. We identified two treatments through the NCTSN which had at least moderate evidence of efficacy (NCTSN Empirically Supported Treatments and Promising Practices, 2005). The two treatments that were selected for implementation through the TCTY were Structured Psychotherapy for Adolescents Responding to Chronic Stress (SPARCS; Derosa et al., 2005) which is rated as “Supported and Acceptable” by the NCTSN and TraumaFocused Cognitive-Behavioral Therapy (TF-CBT; Cohen, Mannarino, & Deblinger, 2006) which has the highest ranking of “Well Supported and Efficacious.” Structured Psychotherapy for Adolescents Responding to Chronic Stress (SPARCS) is a group intervention specifically designed for chronically traumatized teens, age 12e18, who continue to reside in stressful situations (e.g., group homes). SPARCS specifically targets six domains of functioning: affect regulation and impulsivity, self-perception, relationships, attention and consciousness, somatization, and struggles with purpose and meaning in life. The overall goals of the SPARCS program are to help adolescents cope more effectively in the moment, enhance selfefficacy, connect with others, cultivate awareness, and create meaning. SPARCS is delivered in 16 one-hour group therapy sessions over the course of 16 weeks. SPARCS recipients have demonstrated significant improvement of behavioral symptoms (Habib & Ross, 2006) and significant reductions in behavioral problems, and placement disruptions in a study of multiply traumatized adolescents placed in foster care (Weiner, Schneider, & Lyons, 2009). Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) is an individual, short-term treatment (12e18 sessions) that involves individual sessions with a parent and child as well as conjoint parent-child sessions. TF-CBT is appropriate for children age 4e18 who experience symptoms of post-traumatic stress disorder (PTSD), depression, anxiety, and mild to moderate behavioral problems after exposure to a traumatic life event. TF-CBT combines trauma-sensitive and cognitive-behavioral interventions to treat both the child and their family. A series of randomized controlled trials has demonstrated the superiority of TF-CBT over nondirective play therapy and supportive therapies in children who have experienced multiple traumas and those positive results were maintained over time (Cohen, Deblinger, Mannarino, & Steer, 2004; Cohen & Mannarino, 1996a; Cohen, Mannarino, & Knudsen, 2005). TF-CBT has demonstrated effectiveness in improving symptoms of PTSD, anxiety, depression, sexual behavior problems, and feelings of shame and mistrust in several randomized controlled trials. Furthermore, the parental component of TF-CBT appears to enhance these positive findings for children by improving parents’ own level of depression, emotional distress about their children’s abuse, support of the child and effective parenting practices (Cohen & Mannarino, 1996b, 2000). Adopting a learning collaborative training model Our training model for SPARCS and TF-CBT has been consistent throughout all phases of the TCTY and is based on the model

581

developed in the NCTSN, although somewhat modified because our learning collaborative was not part of the NCTSN funding network and we needed to reach across a broad geographic area with limited funds. First, we mailed announcements to local agencies and licensed professionals (e.g., social workers, counselors, psychologists) who might be interested in the training. Second, members of the incoming training class completed readings regarding child maltreatment and evidence-based treatments. Third, there was a pre-conference orientation teleconference in which members of the training class, and often administrators from their agencies, discussed the readings and were oriented to the upcoming training workshop. Fourth, the two-day face-to-face workshop on either SPARCS or TF-CBT was conducted. Fifth, following the two-day workshop, six months of teleconference supervision was conducted by the trainers from the workshop. For most members of the TCTY, who elected to be trained in both treatments, this resulted in 12 months of supervised implementation of the treatment (six months for TF-CBT and six months for SPARCS). Phases of the TCTY The TCTY began in June of 2006 and has evolved over four phases: 1) initial training provided by the developers of the evidence-based treatments (i.e. SPARCS and TF-CBT) for an initial group of North Dakota trainees, 2) creating a North Dakota evidence-based training team, 3) conducting a one-year statewide training as a pilot and demonstration project, and 4) beginning the TCTY dissemination program. Each of these phases will be described below. Phase I: importing SPARCS and TF-CBT to North Dakota Thirteen master’s or doctoral level clinicians from local trauma centers and mental health systems were selected for our initial learning collaborative. In addition to the standard pre-workshop preparation training, these clinicians met via teleconference with trainers from North Shore University Medical Center before the SPARCS workshop and Allegheny General Hospital before the TF-CBT workshop for a basic overview of each treatment model. SPARCS was developed by Derosa and colleagues at North Shore University Hospital in Manhasset, New York in 2005. Three trainers (Mandy Habib, Psy.D., Ida Dancyger, Ph.D., & Suzanne Sunday, Ph.D.) from North Shore University Hospital came to Fargo, North Dakota in the fall of 2006, where the first workshop was held for the group of 13 clinicians. The workshop focused on a review of the SPARCS treatment manual, with a strong emphasis on experiential aspects of training. After completing the workshop, the initial TCTY group of 13 clinicians began implementing SPARCS and within three months seven SPARCS groups were being conducted across the state. Throughout this time, bi-weekly teleconference calls were held with the SPARCS trainers from New York and the 13 local clinicians. Training consultation calls focused on the SPARCS groups that were being conducted in North Dakota and North Shore consultants offered consultation on clinical issues. TF-CBT is typically disseminated via books (Cohen et al., 2006), face-to-face workshops, and a web-based training program sponsored by the NCTSN. One of the developers of TF-CBT (Anthony Mannarino, Ph.D.) conducted a two-day training workshop in Fargo, North Dakota in the spring of 2007 for the group of 13 clinicians described above. The workshop was both didactic and practical with an emphasis on skill acquisition in the educational, behavioral, and trauma narrative portions of the treatment. Following the initial workshop, Dr. Mannarino conducted biweekly consultations by telephone for a period of six months with all trained clinicians. Training calls focused on addressing

582

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

clinical problems that emerged during the implementation of TFCBT with training cases and also global clinical issues related to the treatment of traumatized children. The primary objective of Phase I was to demonstrate that we could identify clinicians in the region who expressed an interest in the training, would complete the one-year training plan for SPARCS and TF-CBT, and would initiate cases in which they deliver these treatments to traumatized children. All of the clinicians successfully completed the SPARCS and TF-CBT workshops as well as the six months of telephone consultation for each treatment. Moreover, each of the clinicians initiated at least one SPARCS treatment group and also initiated at least one TF-CBT training case in their agency. At this phase in the project, there was an emphasis on learning and implementing treatments and developing an assessment protocol at each agency for trauma related characteristics of the children, psychopathology, and clinician experiences. Many of the centers and clinicians involved were not accustomed to completing standardized assessments and this posed an obstacle which we will discuss further later. However, a decision was made to require baseline assessments for screening purposes for entry into any particular protocol. In this phase of the project each child entering a SPARCS group completed the Trauma History Checklist (Habib & Labruna 2006), UCLA PTSD Reaction Index (Pynoos, Rodriguez, Steinberg, Stuber, & Frederick, 1998), and Youth Outcomes Questionnaire Self-Report (YOQ-SR, Wells, Burlingame, & Rose, 2003). The children and families receiving TF-CBT completed the Trauma History Checklist (North Shore Long Island Jewish Health System, Inc., 2006), UCLA PTSD Reaction Index (Pynoos et al., 1998), and Child Behavior Checklist Youth Self-Report (Achenbach & Rescorla, 2001).

Table 1 Phase 1 SPARCS (N ¼ 42) and TF-CBT (N ¼ 44) demographic characteristics.

SPARCS related findings in Phase I Clinicians rated the SPARCS workshop training very highly with a mean of 4.6 (ranging from 0 ¼ not at all satisfied to 5 ¼ completely satisfied) assessing their overall satisfaction with the training. Additionally, as Fig. 1 reveals, their weekly self-rating of comfort and competence in the implementation of core SPARCS techniques increased over the six months of training. During this phase, seven SPARCS groups were initiated, and 41 children were enrolled in the SPARCS group therapy protocol. The children were largely Caucasian and Native American and the majority was female. The average age was 14.8 years (see Table 1). Overall, the TCTY Phase I project demonstrated that it was feasible to train clinicians and successfully implement SPARCS groups over a broad geographic area.

Phase II: creating a local training team

TF-CBT related findings in Phase I During the second half of Phase I, forty-four children were enrolled in TF-CBT (see Table 1). Children enrolled in the TF-CBT

% of SPARCS Sessions

120

SPARCS Interventions

100 80

Psychoeducation LET'M GO

60

Mindfulness

40 20

SPARCS (N ¼ 41)

TF-CBT (N ¼ 44)

14.8 (1.8)

11.6 (3.52)

Gender Male N (%) Female N (%)

10 (24.4) 31 (75.6)

13 (29.5) 31 (70.5)

Ethnicity White N (%) African American (%) Latino N (%) Native American N (%) Pacific Islander More than one N (%)

19 0 3 18 0 1

24 5 1 10 1 3

(46.3) (0) (7.3) (43.9) (0) (2.4)

(55.5) (11.4) (2.3) (22.7) (2.3) (6.8)

tended to be younger than children in the SPARCS protocol and the majority of the children were female and Caucasian. TF-CBT training did not include ongoing assessment of clinician skill level, as in SPARCS, so we have no information on clinician skill development. However, there was a demonstration that each clinician did open, on average, three cases in which the child and family received TF-CBT with ongoing supervision. Although the TCTY had not yet demonstrated consistent ability to complete systematic longitudinal assessments over the geographic expanse of North Dakota, we had identified this as a short-term goal. However, a more pressing need was to develop our own local training team so that we could conduct our own training workshops and consultation. This became the focus for Phase II of the TCTY, which is outlined next.

Following the completion of the initial training in the treatments in Phase I, the TCTY began to establish its own local training team in Phase II. Three of the authors (H.S., T.C.M., M.K.) were selected to be trained as trainers in SPARCS and made two trips to North Shore University Hospital where they took part in a SPARCS Learning Collaborative sponsored by the NCTSN. The developers of SPARCS provided opportunities for these trainers in training to lead various aspects of the learning collaborative and receive feedback directly from the developers. Additionally, the developers of SPARCS continued to consult with these three trainers monthly over the course of six months to provide them with sufficient experience to conduct local training and consultation in SPARCS. In order to provide the TCTY with similar training potential in TF-CBT, one of the authors (T.C.M.) applied and was admitted into a “Train the Trainers” program sponsored by Allegheny General Hospital in Pittsburgh. Along with approximately 20 other clinicians from across the United States, she participated in monthly teleconference calls over the course of a year to enhance her TF-CBT training and supervision skills. Furthermore, the Train-the-Trainer group has access to a discussion board that is used regularly to consult with the developers of TF-CBT in addition to other TF-CBT trainers. Thus, by the end of 2007, the TCTY had three individuals comprehensively trained in SPARCS training and one individual trained as a TF-CBT trainer. This was a critical step as we could now more fully pursue fundamental objectives to disseminate these treatments statewide and measure the impact of the project.

il A pr

ar ch

ry ua

M

y Fe

br

ar

r

nu

be em

Ja

be D ec

em

ct O

N ov

ob

er

r

0

Age M (SD)

Phase III: creating a learning collaborative in North Dakota e a pilot study

Months Fig. 1. Percentage of sessions for which therapist reported implementing SPARCS components with moderate to advanced skill or better during the consultation period.

The TCTY embarked on its first local (i.e., North Dakota) training in evidence-based treatments for traumatized children in July of

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

2007. Once again, clinicians were recruited for the training collaborative through two approaches. First, announcements were sent to all licensed professional psychologists, social workers, and counselors in the state of North Dakota. Second, the TCTY and the State of North Dakota Division of Substance Abuse and Mental Health Services agreed to collaborate to train clinicians in evidence-based trauma treatments across all eight state-run human service centers in North Dakota. The initial plan placed at least two TCTY trained clinicians at each human service center, which provided balanced geographic coverage for the entire state of North Dakota. It is worth noting that two of the authors of this paper (S.A.W. and J.H.) had met in April of 2007 at a meeting sponsored by the National Institute of Health and the Substance Abuse and Mental Health Services Administration focusing on collaborations between universities and state government systems. This meeting initiated collaboration between the TCTY and the Department of Human Service’s Division of Substance Abuse and Mental Health, which was critical in terms of devising and sustaining a plan for statewide dissemination. The approach of advertising TCTY training to private agencies and developing the contract with the Division of Substance Abuse and Mental Health Services, resulted in 41 clinicians enrolled in a yearlong Phase III pilot study in 2007e2008 addressing the implementation of the statewide learning collaborative. Twenty four of the 41 trainee-clinicians worked for the Department of Human Services in one of the eight human service centers and attended training through the evolving collaboration with the state, while the remaining 17 trainee-clinicians worked at 8 private agencies across the state of North Dakota. Participants from private agencies paid a training fee for the 12-month training experience. Training model The training model for Phase III was identical to that used in Phase I. After identifying participants, readings were distributed and a pre-conference orientation was held via teleconference. Next, a SPARCS workshop was conducted followed by six months of consultation. Finally, a TF-CBT workshop was also conducted followed by six months of consultation. Data collection In addition to identifying clinicians who would successfully complete the training provided by local trainers and implement the treatments (similar to the original training group in Phase I), another objective for Phase III was to enter into a more complete data collection phase, particularly regarding the child participants and their outcomes. The assessment measures completed by the children receiving SPARCS included the Trauma History Checklist (North Shore Long Island Jewish Health System, Inc., 2006), UCLA PTSD Reaction Index (Pynoos et al., 1998), and Youth Outcomes Questionnaire Self-Report (YOQ-SR, Wells et al., 2003) and the participants receiving TF-CBT completed the Trauma History Checklist (North Shore Long Island Jewish Health System, Inc., 2006), UCLA PTSD Reaction Index (Pynoos et al., 1998), and the ageappropriate version of the Child Behavior Checklist (Achenbach & Rescorla, 2001). SPARCS related findings In Phase III, the TCTY successfully provided local SPARCS training and consultation which was rated favorably by trainees (see Table 2) and was comparable to ratings of the original trainers in Phase I (M ¼ 4.6; ranging from 0 ¼ not at all satisfied to 5 ¼ completely satisfied). Phase III trainees started a total of 19 groups treating a total of 128 children. Similar to the earlier phase of the project, children receiving SPARCS tended to be female (109 of 128, 85%), with an average age of 14.9 years (SD ¼ 1.45). Sixty

583

Table 2 Ratings for SPARCS and TF-CBT workshops (ranging from 0 ¼ not at all satisfied to 5 ¼ completely satisfied). Overall mean rating Phase III SPARCS 2007 (N ¼ 36) TF-CBT 2008 (N ¼ 43)

4.25 4.40

Phase IV SPARCS 2009 (N ¼ 33) TF-CBT 2010 (N ¼ 41)

4.29 4.38

nine percent (88/128) were Caucasian, and 14% were Native American. SPARCS participants averaged 6.3 (SD ¼ 3.38) traumatic events at the time of entry into the study and a PTSD Reaction Index severity score of 26.9 (SD ¼ 13.72). It is noteworthy that a PTSD Reaction Index severity score of 17e37 is indicative of subclinical PTSD and a score of 38 or higher indicates full syndrome PTSD, suggesting that the SPARCS participants displayed significant, although possibly subclinical levels of PTSD symptomatology. Similarly, at baseline, the SPARCS participants showed high levels of psychological disturbance on the Youth Outcome Questionnaire Self-Report (M ¼ 53.86, SD ¼ 32.16) which exceeded general population scores on this measure (M ¼ 34.21, SE ¼ 1.31; Wells et al., 2003) and more closely approximates typical scores for outpatient mental health populations (M ¼ 67.07, SE ¼ 2.48). Although the Phase III pilot study demonstrated an ability to recruit clinicians and provide training, initiate SPARCS groups, conduct ongoing supervision, and complete baseline assessments, a very significant problem in follow-up data collection was identified. Of the 128 SPARCS participants, only 40 provided outcome data. This was unexpected at the time and the initial analysis suggested that it was a product of several issues. For example, a significant number of the children transferred from one group home or foster placement setting to another in the middle of the SPARCS group, which made follow-up impossible. However, there was also evidence that in spite of regular discussions about data collection on the consultation calls, there was significant deficit in terms of reliable data collection at some sites. Simple problems, such as failing to check completion of all items on measures, administering the wrong measures, or simply forgetting to administer some, or all, questionnaires predominated. TF-CBT related findings At the midpoint of the Phase III year, TCTY staff provided our first local training in TF-CBT. Again participants rated the training favorably (see Table 2). In this phase of the project, 39 children were enrolled in TF-CBT. Importantly, and somewhat surprisingly, 12 of 35 clinicians failed to open a case, but attended the six months of TF-CBT consultation calls. Children receiving TF-CBT tended to be female (26 of 37, 70%), with an average age of 11. 5 (SD ¼ 3.44). Forty eight percent (18/37) were Caucasian, with 27% Native American. Similar to SPARCS recipients, TF-CBT recipients also averaged six traumatic events. PTSD Reaction Index severity scores were again consistent with subclinical levels of post-traumatic stress disorder (M ¼ 30.2, SD ¼ 11.2; cutoff for diagnosis ¼ 38). The end-of-treatment data collection problems seen in SPARCS were worse in TF-CBT, with only 3 of 29 children providing end-oftreatment data. Given this failure of outcome related data collection, coupled with the above mentioned problems collecting outcome data in SPARCS, the TCTY made a decision to thoroughly evaluate its data collection problems before proceeding in further training. A number of problems were identified and solutions generated, which will be reviewed next.

584

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

Identifying data collection problems in the TCTY Although the early phases of the TCTY project indicated that we were able to introduce and disseminate evidence-based treatments across a broad geographical area, the difficulties we encountered collecting data were greater than anticipated. This is noteworthy because the staff coordinating the TCTY have effectively collected data on numerous, multi-center research projects and randomized controlled trials previously. However, experiences in such highly controlled and heavily staffed research projects did not transfer into reliable data collection in the TCTY. Several factors seemed to contribute to this difficulty. First, drop-out rates were quite high, due a number of factors. Many of these children were living in situations in which there was a relative absence of parental or family support. Most notably, a significant fraction of the children were in foster placement and were susceptible to transfer of their placement, which often resulted in dropping out of treatment due to increased geographic distance. Similarly, in spite of the fact that many of the children and their families appeared invested in the treatment, the large geographic expanses in North Dakota and associated weather problems made reliable attendance difficult. A second factor influencing loss of data was due to the culture of some of the agencies in which the treatments were delivered. Often, clinicians were busy managing large caseloads of complicated cases and the additional task of administering psychological instruments and collecting such data was not a typical part of their job. In the absence of close on-site data monitoring, the reliable administration of the measures was compromised. The fact that data collection in the State’s Human Services Centers was also compromised, in spite of the fact that each human service center demonstrated considerable administrative “buy in” to the project, was noteworthy and prompted us to consider what systemic factors could be manipulated to enhance compliance. Finally, regarding poor outcome assessment compliance rates for TF-CBT, it is possible that the clinicians’ twelve months of training in the TCTY ended at approximately the same time that they were finishing TF-CBT cases. The discontinuation of regular consultation calls at that time, and resulting decrease in teleconference contact time, may also have contributed to a diminished likelihood of end-of-treatment data collection. These pilot study experiences in data collection made it clear that a successful evidence-based dissemination project would ultimately rely on better training of clinicians in data collection, stronger administrative support from the data collection centers, and closer data monitoring across involved sites. With these lessons learned from our Pilot, we entered Phase IVda more definitive effort to disseminate treatment and measure outcomes.

a web-based portal (i.e., Liquid Office) which provided ease of data entry and more efficient data collection. With these new procedures in place, it was hoped that the trial phase of the TCTY would not only effectively disseminate treatments, but more reliably collect data to assess various training related effects. Training model The training model for Phase IV was identical to that used in previous phases of the TCTY. After identifying participants, readings were distributed and teleconference reviews of basic information were conducted. The training year began with a SPARCS workshop which was followed by six months of supervision, a TF-CBT workshop, and six additional months of TF-CBT supervision. SPARCS related findings Twenty-two clinicians were trained in SPARCS and 14 of these trainees worked in the state system. Within the state system, seven SPARCS groups were initiated and 60 children received SPARCS therapy. Consistent with our previous experience, participants tended to be female (80%), Caucasian (72%), and only 40% were living with one or two natural parents. Based on the Trauma History Checklist, children in the group averaged 5.5 traumatic events (SD ¼ 2.55) and the most common experiences were neglect (64%), physical abuse (64%), domestic violence (53%), severe parental illness (43%), and sexual abuse (38%). Sixty-eight percent of the 60 children enrolled in SPARCS completed the Trauma Symptom Checklist for Children (TSCC; Briere, 1996) at the beginning and at the end-of-treatment, thus providing better compliance than in the Phase III pilot study. As can be seen in Fig. 2, children in the SPARCS groups displayed modest elevations on the TSCC at entry into treatment and a general tendency toward improvement, although not at a statistically significant level on any of the scales. Effects of treatment may have been limited by low scores on the TSCC at entry into the study. TF-CBT related findings Following the completion of the SPARCS treatment, trainees began six months of TF-CBT training and supervision. Forty clinicians were trained in TF-CBT and fifteen of these trainees worked in the North Dakota Department of Human Services and thus provided the data for this report. A total of six (3 of which were from the human service centers) of these 40 trainees failed to open a case, but attended the 6 months of supervision. Nineteen children were enrolled in TF-CBT at one of the eight human service centers. On average, they were 11 years of age (SD ¼ 3.85). Fifty-three percent were female, 74% were Caucasian, and 47% were currently living with one of their natural parents. In 58% of the

Phase IV: TCTY treatment trial Based on our experiences in the pilot study, several key decisions were made. The objectives of the TCTY were not simply to disseminate treatments, but to also provide outcomes relevant to those treatments. Some agencies involved in the TCTY simply did not have sufficient resources to allow data collection in a reliable fashion. Therefore, the TCTY made a decision to limit data collection to the eight state-run human service centers and worked closely with these centers to enhance data collection procedures. Clinicians at other agencies continued to be trained and were encouraged to collect and submit data, but our primary data collection efforts would be focused on agencies affiliated with the state government. Also, one of the authors (W.L.), who has an administrative post in the Department of Human Services, was designated as the statewide data monitor and attended all meetings of the TCTY as well as telephone consultations. Finally, in a very important step, the state of North Dakota included all of the measures of the TCTY in

Fig. 2. SPARCS Trauma Symptom Checklist for Children Scores at pre- and posttreatment.

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

585

Summary

Fig. 3. TF-CBT Trauma Symptom Checklist for Children Scores at 13e18 years at preand post-treatment.

cases, the natural mother was included in the therapy with the remaining cases conducted with surrogate parent figures or without an adult involved. Results of the Trauma History Checklist revealed that the children experienced 5.8 traumas (SD ¼ 2.6), on average. The most common forms of trauma included physical abuse (72%), neglect (61%), sexual abuse (50%), severe parental illness (50%), and witnessing domestic violence (44%). Twelve of the 19 children enrolled in Phase IV TF-CBT through the 8 state-run human service centers completed treatment. One of these children, however, did not provide post-treatment assessment data, leaving 11 of 19 (58%) participants with complete data. Each of the cases lost to follow-up was tracked in terms of the reason for an absence of follow-up. Two of the children voluntarily dropped out of treatment, one became involved in the legal system and was transferred, and four had changes in foster placement which removed them from their clinic, and resulted in discontinuation of treatment. As can be seen in Figs. 3 and 4, the children who did complete TF-CBT and provided followup data showed improvements across a wide range of indicators on the Trauma Symptom Checklist for Children (TSCC; Briere, 1996) and the Trauma Symptom Checklist for Young Children (TSCYC; Briere, 2005). The lines in Figs. 3 and 4 represent pre- and post-treatment scores on different scales of the TSCC. In children age 13e16 (see Fig. 3), the greatest improvements were in depression, avoidance related dissociation, and sexual concerns, although none of these change scores reached statistical significance. However, younger children, age 8e12 did show statistically significant improvement in depression (p ¼ .03), anger (p ¼ .009), and dissociation (p ¼ .002). Although this is a very small sample size, it is clinically noteworthy that these children improved on scales directly targeted by TF-CBT (PTSD arousal, dissociation/ avoidance) and that this was accomplished with very young children.

Fig. 4. TF-CBT Trauma Symptom Checklist Scores for Young Children (ages 5e12) at pre- and post-treatment.

The Treatment Collaborative for Traumatized Youth (TCTY) was developed in an effort to enhance dissemination of evidence-based treatments to children who experience post-traumatic stress after childhood traumas. This is clearly a group of children who display significant mental health related problems and often do not receive evidence-based mental health treatment. The TCTY was developed in an effort to provide such mental health services to these children in a very rural state with a large geographic area, which presents certain logistical and technical problems. Nonetheless, over a relatively short period of time we have created an effective training team and program which has trained over 100 mental health clinicians and nearly 600 children have contributed data to the TCTY. This has clearly assisted mental health professionals and policy makers to recognize the need for mental health services for these children as evidenced by the fact that our State Department of Human Services has contracted with the TCTY to help train mental health professionals in the state system. Furthermore, other unanticipated possibilities have emerged across the state, such as training county child protection workers in well defined mental health screening techniques developed by the NCTSN, which produce referrals to professionals who are members of the TCTY and provide evidence-based treatments. Also, the TCTY has initiated a program in which foster parents receive NCTSN based training in childhood trauma, its consequences, and the impact it may have on foster children. Importantly, the TCTY staff continues to focus on sustaining these practices across the state of North Dakota. In an effort to maintain this group of clinicians that are well-trained in childhood trauma interventions and committed to serving traumatized children and families, the TCTY holds an annual meeting for all members of the TCTY which provides advanced training opportunities. Furthermore, the TCTY developed a website (tcty-nd.org) that is accessible by the general public but also offers a unique login for each TCTY clinician. The “clinician only” area of the website includes a variety of resources for the TCTY members. For example, the website features stories about the local agencies involved in the TCTY in addition to popular press stories that are relevant to childhood trauma. Summaries of relevant journal articles are also posted periodically as many clinicians have described difficulty accessing scientific literature. The TCTY website also supports the implementation of SPARCS and TF-CBT by providing clinicians with numerous documents that aid in the implementation of these interventions. Furthermore, a secure discussion board is available for TCTY clinicians to consult with the TCTY trainers and other TCTY clinicians on difficult cases. In spite of these favorable developments through the TCTY, we have also identified significant problems. Most of the problems we encountered have been summarized by Proctor et al. (2009) in their discussion of implementation research. In particular, these authors point out that issues of measurement and design have considerable impact on effective implementation of evidence-based treatments. This was most clearly seen in the TCTY in terms of the difficulties we had collecting clinical outcome data. The measurement of such outcomes is critical to the success of the program and its absence threatens its viability, particularly in terms of disseminating evidence-based treatments. Certainly many clinicians and agencies in the TCTY were not familiar with data collection or even measuring outcomes using standardized objective assessment tools. Also, the nature of this population of children, with considerable movement geographically and significant psychopathology make regular, routine assessments somewhat unpredictable and difficult. Additionally, many clinicians are simply busy and it is at times easy to forget to gather critical data. We made several

586

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587

adjustments, including greater surveillance through the state system and inclusion of data collection instruments in an electronic record format. In spite of the fact that collecting the data through the mental health system’s web-based data portal enhanced feasibility of administering the tests and monitoring compliance, it did not provide immediate scoring and feedback on test scores to the clinicians. As we note below, we believe this function could greatly enhance clinician utilization of project data. We also spend substantial time in our trainings discussing the importance of assessment and data collection, which is also mentioned frequently in our consultation calls, but it remains an issue which must be monitored and addressed. Two additional strategies, which the TCTY is considering to enhance data collection are the inclusion of on-site data monitors and the use of web-based scoring systems for the measures which provide immediate scoring and feedback to clinicians and families. The presence of local data monitors is something that other large scale dissemination initiatives have adopted. For example, in a similar statewide project in Connecticut each clinical site has a data monitor technician who oversees the administration, collection, and scoring of outcome measurements (Franks & Lang, 2010). Similarly, Franks and Lang (2010) have developed a means of scoring instruments through an easily accessible web-based scoring protocol which is highly feasible, relatively inexpensive, and offers clinicians immediate feedback that is likely to be reinforcing and promote administration of clinical instruments. The use of such technology, coupled with on-site supervision will hopefully assist the TCTY in optimizing data collection in what is clearly a challenging data environment. Our experience certainly supports the observation that effective strategies derived from theories regarding implementation of mental health techniques are critical to the success of projects similar to the TCTY (e.g., Proctor et al., 2009). In summary, the TCTY involves an effective collaboration between universities, research institutes, the State Department of Human Services and a variety of other agencies across the state of North Dakota who serve children. Such community partnerships have been critical in launching such a large and complex dissemination effort. Furthermore, a learning collaborative approach to such dissemination, which has been effectively demonstrated in the NCTSN, has been instrumental in creating a system of treatment provision and data collection which was previously nonexistent. Acknowledgments In addition to ongoing support from the North Dakota Department of Human Services, this project was supported by grants from the Otto Bremer Foundation, Bush Foundation, Dakota Medical Foundation, Alex Stern Family Foundation and MDU Resources. We would also like to acknowledge all of the members of the Treatment Collaborative for Traumatized Youth and the students from Concordia College (Moorhead, MN) and North Dakota State University (Fargo, ND) for their valuable contribution to this project. References Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. Battle, C. L., Shea, M. T., Johnson, D. M., Yen, S., Zlotnick, C., Zanarini, M. C., et al. (2004). Childhood maltreatment associated with adult personality disorders: findings from the collaborative longitudinal personality disorders study. Journal of Personality Disorders, 18(2), 193e211. Becker, C. B., Stice, E., Shaw, H., & Woda, S. (2009). Use of empirically supported interventions for psychopathology: can the participatory approach move us beyond the research-to-practice gap? Behaviour Research and Therapy, 47, 265e274.

Borntrager, C., Chorpita, B. F., Higa, C., & Weisz, J. R. (2009). Provider attitudes toward evidence-based practices: are the concerns with the evidence or with the manuals? Psychiatric Services, 60(5), 1e5. Briere, J. (1996). Trauma symptom checklist for children professional manual. Odessa, FL: Psychological Assessment Resources. Briere, J. (2005). Trauma symptom checklist for young children professional manual. Odessa, FL: Psychological Assessment Resources. Chaffin, M., Bonner, B. L., & Hill, R. F. (2001). Family preservation and family support programs: child maltreatment outcomes across client risk levels and program types. Child Abuse and Neglect, 20(10), 1269e1289. Chaffin, M., Silovsky, J. F., & Vaughn, C. (2005). Temporal concordance of anxiety disorders and child sexual abuse: implications for direct versus artifactual effects of sexual abuse. Journal of Clinical Child and Adolescent Psychology, 34(2), 210e222. Chorpita, B. F., & Regan, J. (2009). Dissemination of effective mental health treatment procedures: maximizing the return on a significant investment. Behaviour Research and Therapy, 47(11), 990e993. Cohen, J. A., Deblinger, E., Mannarino, A. P., & Steer, R. A. (2004). Multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. Journal of the Academy of Child and Adolescent Psychiatry, 43, 393e402. Cohen, J. A., & Mannarino, A. P. (1996a). A treatment outcome study for sexually abused preschool children: initial findings. Journal of American Academy of Child and Adolescent Psychiatry, 35, 42e50. Cohen, J. A., & Mannarino, A. P. (1996b). Factors that mediate treatment outcome in sexually abused preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1402e1410. Cohen, J. A., & Mannarino, A. P. (2000). Predictors of treatment outcomes in sexually abused children. Child Abuse and Neglect, 24, 983e994. Cohen, J. A., Mannarino, A. P., & Deblinger, E. (2006). Treating trauma and traumatic grief in children and adolescents. New York: Guilford Press. Cohen, J. A., Mannarino, A. P., & Knudsen, K. (2005). Treating sexually abused children: one year follow-up of a randomized controlled trial. Child Abuse and Neglect, 29, 135e146. Cohen, J. A., Mannarino, A. P., & Rogal, S. S. (2001). Treatment practices for childhood PTSD. Child Abuse & Neglect, 25, 123e125. Cohen, J. A., Perel, J. M., DeBellis, M. D., Friedman, M. J., & Putnam, F. W. (2002). Treating traumatized children: clinical implications of the psychobiology of posttraumatic stress disorder. Trauma, Violence, & Abuse, 3(2), 91e108. Craske, M. G., Roy-Byrne, P. P., Stein, M. B., Sulivan, G., Sherbourne, C., & Bystritsky, A. (2009). Treatment for anxiety disorders: efficacy to effectiveness to implementation. Behaviour Research and Therapy, 47(11), 931e937. Derosa, R., Habib, M., Pelcovitz, D., Rathus, J., Sonnenklar, J., & Kaplan, S. (2005). SPARCS e Structured psychotherapy for adolescents responding to chronic stress: A trauma-focused guide. Great Neck, NY: North Shore-Long Island Jewish Health System, Inc. Franks, R., & Lang, J. (2010). Quick, cheap, easy and good: Facilitating the use of data in EBPS. Presented at the 24th Annual San Diego international conference on child and family maltreatment, San Diego, CA. Habib, M., & Labruna, V. (2006). The adolescent trauma history checklist and interview. Unpublished measure. Habib, M., & Ross, L. A. (2006). Igniting SPARCS of change in treatment: an experiential introduction to a promising practice. Presented at the 22nd Annual meeting of the ISTSS, Hollywood, CA. Insel, T. R. (2009). Translating scientific opportunity into public health impact: a strategic plan for research on mental illness. Archives of General Psychiatry, 66, 128e133. Institute for Healthcare Improvement. (2003). The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. IHI innovation series white paper. Israel, B. A., Eng, E., Schulz, A. J., & Parker, E. A. (2005). Introduction to methods in community-based participatory research for health. In B. A. Israel, E. Eng, A. J. Shulz, & E. A. Parker (Eds.), Methods in community-based participatory research for health (pp. 3e26). San Francisco: Jossey-Bass. Kendall-Tackett, K. A., Williams, L. M., & Finkelhor, D. (1993). Impact of sexual abuse on children: a review and synthesis of recent empirical studies. Psychological Bulletin, 113(1), 164e180. Kolko, D. J., Baumann, B. L., & Caldwell, N. (2003). Correlates of community service use and improvement in child abuse cases: initial findings. Child Maltreatment, 8, 273e287. Kolko, D. J., Cohen, J. A., Mannarino, A. P., Baumann, B. L., & Knudsen, K. (2009). Community treatment of child sexual abuse: a survey of practitioners in the national child traumatic stress network. Administration and Policy in Mental Health, 36, 37e49. Kolko, D. J., Selelyo, J., & Brown, E. J. (1999). The treatment histories and service involvement of physically and sexually abusive families: description, correspondence, and clinical correlates. Child Abuse and Neglect, 23(5), 459e476. Markiewicz, J., Ebert, L., Ling, D., Amaya-Jackson, L., & Kisiel, C. (2006). Learning collaborative toolkit. Los Angeles: National Center for Child Traumatic Stress. McHugh, R. K., Murray, H. W., & Barlow, D. H. (2009). Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of transdiagnostic interventions. Behaviour Research and Therapy, 47(11), 946e953. Molnar, B. E., Buka, S. L., & Kessler, R. C. (2001). Child sexual abuse and subsequent psychopathology: results from the national comorbidity survey. American Journal of Public Health, 91(5), 753e760.

S.A. Wonderlich et al. / Behaviour Research and Therapy 49 (2011) 579e587 NCTSN Empirically Supported Treatments and Promising Practices. (2005). Retrieved from http://www.nctsn.org/nctsn_assets/pdfs/promising_practices/ NCTSN_E-STable_21705.pdf. Accessed 26.05.11. NIH (National Institutes of Health). (2006a). Dissemination and implementation research in health (RO1 program announcement). Retrieved from. http://grants. nih.gov/grants/guide/pa-files/PAR-06-039.html. NIH (National Institutes of Health). (2006b). NIH roadmap request for information (RFI); translation; the science of knowledge dissemination, implementation and integration. Retrieved from. http://www.reffectcomments.org/Roadmap/PR.aspx. Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health, 36, 24e34. Pynoos, R. S., Rodriguez, N., Steinberg, A., Stuber, M., & Frederick, C. (1998). UCLA PTSD reaction index for DSM-IV. Ruzek, J. I., & Rosen, R. C. (2009). Disseminating evidence-based treatments for PTSD in organizational settings: a high priority focus area. Behaviour Research and Therapy, 47(11), 980e989. SAMHSA National Child Traumatic Stress Network. (2000). Retrieved from http:// www.nctsn.org/. Accessed 26.05.11. Saunders, B. E., Berliner, L., & Hanson, R. F. (2003). Child physical and sexual abuse: Guidelines for treatment (final report: January 15, 2003). Charleston, SC: National Crime Victims Research and Treatment Center. Shafran, R., Clark, D. M., Fairburn, C. G., Arntz, A., Barlow, D. H., Ehlers, A., et al. (2009). Mind the gap: improving the dissemination of CBT. Behaviour Research and Therapy, 47(11), 902e909.

587

U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families, & Children’s Bureau. (2010). Child maltreatment 2009. Available at http://www.acf.hhs.gov/programs/ cb/stats_research/index.htm#can Accessed 08.05.11. Weiner, A., Schneider, A., & Lyons, J. S. (2009). Evidence-based treatments for trauma among culturally diverse foster care youth: treatment retention and outcomes. Child and Youth Services Review, 31, 1199e1205. Wells, M. G., Burlingame, G. M., & Rose, P. M. (2003). Administration and scoring manual for the YOQ SR 2.0 youth outcome questionnaire self report. OQ measures, LLC. YOQ-questionnaire-self-report. Wiersema, J. E., Hovens, J. G., van Oppen, P., Giltay, E. J., van Schaik, D. J., Beekman, A. T., et al. (2009). The importance of childhood trauma and childhood life events for chronicity of depression in adults. Journal of Clinical Psychiatry, 70(7), 983e989. Wilsnack, S. A., Vogeltanz, N. D., Klassen, A. D., & Harris, T. R. (1997). Childhood sexual abuse and women’s substance abuse: national survey findings. Journal of the Study of Alcohol, 58(3), 264e271. Wonderlich, S. A., Brewerton, T. D., Jocic, Z., Dansky, B. S., & Abbott, D. W. (1997). Relationship of childhood sexual abuse and eating disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 36(8), 1107e1115. Zanarini, M. C., Yong, L., Frankenburg, F. R., Hannen, J., Reich, D. B., Marino, M. F., et al. (2002). Severity of reported childhood sexual abuse and its relationship to severity of borderline psychopathology and psychosocial impairment among borderline inpatients. Journal of Nervous and Mental Disorders, 190(6), 381e387.