Barriers to implementing evidence-based practices in addiction treatment programs: Comparing staff reports on Motivational Interviewing, Adolescent Community Reinforcement Approach, Assertive Community Treatment, and Cognitive-behavioral Therapy

Evaluation and Program Planning 34 (2011) 382–389

Contents lists available at ScienceDirect

Evaluation and Program Planning journal homepage: www.elsevier.com/locate/evalprogplan

Barriers to implementing evidence-based practices in addiction treatment programs: Comparing staff reports on Motivational Interviewing, Adolescent Community Reinforcement Approach, Assertive Community Treatment, and Cognitive-behavioral Therapy M. Amodeo, L. Lundgren, A. Cohen, D. Rose, D. Chassler *, C. Beltrame, M. D’Ippolito Center for Addictions Research and Services, Boston University, School of Social Work, 264 Bay State Road, Boston, MA 02215, United States

A R T I C L E I N F O

A B S T R A C T

Article history: Available online 21 March 2011

Purpose: This qualitative study explored barriers to implementing evidence-based practices (EBPs) in community-based addiction treatment organizations (CBOs) by comparing staff descriptions of barriers for four EBPs: Motivational Interviewing (MI), Adolescent Community Reinforcement Approach (A-CRA), Assertive Community Treatment (ACT), and Cognitive-behavioral Therapy (CBT). Methods: The CBOs received CSAT/SAMHSA funding from 2003 to 2008 to deliver services using EBPs. Phone interview responses from 172 CBO staff directly involved in EBP implementation were analyzed using content analysis, a method for making inferences and developing themes from the systematic review of participant narratives (Berelson, 1952). Results: Staff described different types of barriers to implementing each EBP. For MI, the majority of barriers involved staff resistance or organizational setting. For A-CRA, the majority of barriers involved specific characteristics of the EBP or client resistance. For CBT, the majority of barriers were associated with client resistance, and for ACT, the majority of barriers were associated with resources. Discussion: EBP designers, policy makers who support EBP dissemination and funders should include explicit strategies to address such barriers. Addiction programs proposing to use specific EBPs must consider whether their programs have the organizational capacity and community capacity to meet the demands of the EBP selected. ß 2011 Elsevier Ltd. All rights reserved.

Keywords: Evidence-based practice Community-based substance abuse treatment Motivational Interviewing (MI) Adolescent Community Reinforcement Approach (A-CRA) Assertive Community Treatment (ACT) Cognitive-behavioral Therapy (CBT)

1. Background 1.1. Importance of staff attitudes in EBP implementation Federal substance abuse agencies such as the Substance Abuse and Mental Health Services Administration (SAMHSA) and the National Institute on Drug Abuse (NIDA) have dedicated significant resources to ensuring that community-based organizations (CBOs) receiving funding from them implement evidence-based practices (EBPs) for addiction treatment. EBP dissemination and implementation in the addiction field have been goals since the Institute of Medicine Report in 1998 (IOM, 1998) calling for improvement in health care through greater use of researchbased treatments and increased researcher-practitioner collaborations. However, implementation of EBPs by community-

* Corresponding author. E-mail addresses: [email protected] (M. Amodeo), [email protected] (L. Lundgren), [email protected] (A. Cohen), [email protected] (D. Rose), [email protected] (D. Chassler), [email protected] (C. Beltrame), [email protected] (M. D’Ippolito). 0149-7189/$ – see front matter ß 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.evalprogplan.2011.02.005

based organizations has been slow. Much research has been conducted to determine attitudinal and other barriers that might be impeding effective EBP implementation. Researchers (Ball et al., 2002; Forman, Bovasso, & Woody, 2001; Fuller et al., 2007; Knudsen, Ducharme, Roman, & Link, 2005; McCarty et al., 2007; McGovern, Fox, Xie, & Drake, 2004; Rieckmann, Daley, Fuller, Thomas, & McCarty, 2007; Willenbring et al., 2004) have recognized that attitudes influence behavior and that personnel who view these practices negatively could be resistant to implementing them. Both the attitudes of those who may be asked to implement EBPs in the future, and the experiences of those currently implementing EBPs, are relevant. However, it is important for the addiction field to understand how front-line staff who are currently using EBPs experience barriers to implementing these practices if there is to be effective technology transfer. If staff experience these EBPs to be particularly burdensome, or if their perceptions of effectiveness and fit are negative, their motivation to implement them in the future is likely to be decreased. If specific EBPs are considered to be especially barrier-heavy, motivation to implement may be diminished.

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

1.2. Variation in staff attitudes depending on the EBP A number of studies have examined staff and administrator attitudes about EBP use in community addiction treatment programs (Garner, 2009). Differences have been found in staff attitudes depending on the type of practices under consideration. Mark et al. (2003), examining attitudes about naltrexone use by physicians, found that lack of knowledge and time for patient management were common concerns. Bartholomew, Joe, RowanSzal, and Simpson (2007) surveyed staff at two time points (posttraining and 6-month follow up) after a training conference on dual-diagnosis and found that barriers to using the training materials included lack of time and the fact that some staff were already using similar materials. In several studies, a portion of staff who viewed psychosocial treatments positively viewed some forms of medication-assisted treatment negatively (see Forman et al., 2001; Garner, 2009; Herbeck, Hser, & Teruya, 2008; McCarty et al., 2007; Willenbring et al., 2004). A Canadian study by Ogborne, Wild, Braun, and Newton-Taylor (1998) found high levels of support for cognitive, coping and relapse prevention approaches, and lower support for medications, Twelve-step approaches and insight-oriented psychotherapy. Staff perceptions have also differed depending on the type of psychosocial treatment. Ball et al. (2002) found variation in clinician endorsement of a range of psychosocial treatments depending on clinician educational and recovery status. A survey of 24 public addiction programs in New Hampshire (McGovern et al., 2004) found that clinicians were more motivated to adopt Cognitive Behavioral Therapy, Motivational Interviewing, Twelve-step Facilitation, and Relapse Prevention than to adopt Contingency Management and Behavioral Couple’s Therapy. The researchers concluded that ‘‘community interest [in EBPs] may be more determined by ease of implementation, fit with what clinicians believe and are already doing. . . and in response to clinician-expressed need. . .’’ (p. 310). In a study of Veterans Administration addiction treatment programs (Willenbring et al., 2004), program leaders rated anticipated barriers to implementing Contingency Management and Behavioral Couple’s Therapy higher than barriers to some other EBPs. Mental health staff in community settings (Nelson, Steele, & Mize, 2006) were asked to identify desirable EBP characteristics. Focus group interviews yielded the following characteristics: flexibility, ease of implementation, emphasis on the therapeutic relationship, previous positive experience with the EBP or EBP reported to be positive by a trusted colleague. This suggests that clinicians will be more interested in, and have more positive attitudes toward, EBPs that are more familiar and similar to the approaches they have used in the past. Conversely, staff might identify more barriers when the EBPs are less similar to the practices they have been employing. 1.3. Barriers encountered in EBP implementation Research has also focused on staff and administrator experiences in implementing EBPs (Berger et al., 2009; Brown, 2004; Godley, White, Diamond, Passetti, & Titus, 2001; Nelson & Steele, 2007; Riley, Rieckmann, & McCarty, 2008; Sheehan, Walrath, & Holden, 2007; Thomas, Wallack, Lee, McCarty, & Swift, 2003), seeking to understand the extent to which the EBPs were userfriendly and whether they presented particular challenges to implementation. For example, addressing these questions in the mental health service system, Sheehan et al. (2007) surveyed children’s mental health service providers (n = 446) to understand the extent of EBP implementation and preparatory training; the six most frequently reported EBPs were CBT, Wrap-around Services, Anger Management, Social Skills Training, Family Education and

383

Support, and Case Management. Treatment implementation and training varied by EBP. Although the study did not have a robust response rate, findings suggest (a) the lack of full treatment implementation by providers of the six practices, and (b) the lack of training for certain EBPs, especially those that are manual-guided. The lack of training for specific types of EBPs may be a barrier to EBP implementation in the children’s mental health treatment system. In a small qualitative study of community mental health practitioners working with children and families, Nelson et al. (2006) conducted focus groups to examine perceived barriers to EBP implementation. This study assessed practitioners’ perceptions of characteristics across EBPs that influence implementation in a community setting. Identified barriers were grouped into three categories: (a) characteristics of EBPs (e.g., long treatment duration—for example, 12 sessions; specialized staff competence required); (b) characteristics of practitioners and settings (e.g., limited practitioner time, lack of training and supervision), and (c) characteristics of clients (e.g., complex client presentation, client resistance, and client inconsistency in therapy). Only a few studies (Brown, 2004; Godley et al., 2001; Riley et al., 2008) have assessed barriers to specific EBPs as perceived by the front-line staff charged with implementing them. First, Brown (2004) interviewed treatment staff involved in the CSAT Methamphetamine Treatment Project which utilized the Matrix Model (Obert et al., 2005) as one form of treatment. However, this was a randomized-controlled trial in which clients were randomized into either the Matrix Model or treatment-as-usual. The barriers reported included a combination of staff responses to the randomization and to the Matrix Model itself. This made it difficult to isolate barriers faced with the Matrix Model. Second, Godley et al. (2001) interviewed 16 therapists and 3 case managers to compare practitioner reactions to five manualguided therapies in a multi-site randomized field experiment. The manual-guided therapies varied by theoretical orientation (motivational, cognitive, behavioral, and family systems), length (5 sessions versus 12 sessions), mode (group, individual) and degree of family involvement. The approaches were MET/CBT5, MET/CBT 12, Family Support Network (FSN), Adolescent Community Reinforcement Approach (ACRA), and Multidimensional Family Therapy (MDFT). Three key themes emerged from the practitioner reports: (1) Structure and Consistency were seen as positive attributes including ease of use and maintenance of focus; (2) Restrictiveness was seen as negative including limited therapist ability to respond to individual client needs. The highest percent of therapists (70%) reported restrictiveness for interventions using MET/CBT and comments were more often directed to the CBT group component. (3) Flexibility within an Intervention was seen as positive. The majority of therapists reported that the manual allowed them to meet the needs of individual clients. ‘‘The two individual-based interventions (ACRA and MDFT) and the individual-based MET that was part of MET/CBT . . .were viewed as allowing more flexibility than the group-based interventions that required certain material be covered in each session’’ (p. 413). Thus, the authors characterized manuals that were ‘‘principlebased or procedure-based’’ (e.g., ACRA, MDFT) as ‘‘flexible,’’ and manuals that were ‘‘session-based’’ as ‘‘very structured.’’ Therapists delivering the CBT interventions and those delivering Family Support Network reported barriers that interfered with adherence to the manual; these included crises that arose for clients or their families (e.g., death, rape, assault, client psychological deterioration), logistics, or lack of cooperation from the client. Finally, using interview data with clinical and administrative directors at nine program sites, Riley et al. (2008) reported on the implementation of MET/CBT 5 for adolescents: although most programs said they did not make any changes to the protocol, eight of nine programs reported substantial adaptations to the content

384

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

and/or format. These included offering individual instead of group sessions, making cultural adaptations, and modifying content due to the client’s cognitive level. Participant comments convey that they perceived that the EBP (as it was originally designed), did not fit (a) the attendance patterns of clients (who did not show up for groups, but came individually); (b) the cultural dimensions of the population (e.g., ethnicity, gender and/or use of specific drugs); and/or (c) the cognitive abilities of clients. Thus, a range of barriers to implementation of MET/CBT 5 can be inferred from the participant interviews. A goal of EBP dissemination is to have addiction programs integrate these EBPs into day-to-day program operations (Aarons, 2005), maintaining the fidelity of the practice. Negative provider responses could affect the degree of future EBP dissemination and adoption in general, and adoption of specific EBPs in particular. 2. Present study 2.1. Summary and research question Although many addiction studies have described attitudes of addiction workers toward EBPs, there are only a limited number of studies that describe the specific barriers that staff encounter when implementing a new EBP. Even fewer compare reported barriers for various EBPs. In the present study, data from semi-structured phone interviews with 172 front-line staff were analyzed using content analysis, a research method for making inferences and developing themes from the systematic review of participant narratives (Berelson, 1952). The major research question is: Do types of implementation barriers differ for MI, A-CRA, ACT and CBT? These EBPs were chosen because they are the four most common EBPs implemented in a larger national study of 330 CBOs that received funding from CSAT/SAMHSA to implement EBPs in their organizations. A description of these EBPs can be found in Appendix A. To address the major research question, we began by addressing two preliminary questions: ‘‘What barriers are most frequently reported for each of these EBPs?’’ and ‘‘What patterns or categories best describe the barriers?’’. That is, if the barriers differ among these EBPs, are there patterns that could help us describe the differences? To assist in data analysis, the qualitative categorization developed by Nelson et al. (2006) was employed and expanded. 3. Methods 3.1. Sample CBOs that had received a grant award from CSAT/SAMHSA to implement services using EBPs were identified from a publicly available web-site of all programs funded by CSAT/SAMHSA between 2003 and 2008. Three hundred and thirty program directors, one from each program, were called and asked to identify staff members who were involved in implementing their program’s EBPs. The sample for this study was 172 staff; all had been identified by their program directors as actively involved in the implementation of at least one of the EBPs on which this study focuses: MI, ACRA, ACT and CBT. The sample was predominantly female (74%) and the average age was 40 years. Slightly more than half (55%) had a Masters degree or higher, and slightly less than half (48%) had 5 or more years of experience in the field. A large number of the participants (n = 86, 50%) worked in outpatient (intensive and regular) treatment settings with an additional 37 (22%) in inpatient and residential settings. Excluded from this sample were staff in CBOs that: (1) were funded by States that received block grants from SAMHSA; (2) received grants funded through mechanisms that did not specify a need for treatment approaches to be evidence-based; (3) received grants with a substance use prevention focus, rather

than a treatment focus, and (4) did not provide at least one of the four EBPs of focus in this article. 3.2. Data collection Front-line staff identified by their program directors as involved in the implementation of EBPs participated in 45 minute phone interviews. The qualitative interview protocol was pilot-tested with a sample of 10 respondents who were all front-line staff and who also participated in the implementation of an EBP. Based on the results of this pilot test, the protocol was revised to include additional open-ended questions, to elicit targeted information, to simplify the wording of some of the questions, and to provide consistent language for interviewers to use in answering respondents’ questions. The qualitative phone interviews with the sample of 172 respondents consisted of three steps: (1) Interviewers informed respondents that the interview would be audio-taped and would be about the two key EBPs that the program director had identified as having been implemented as part of the fulfillment of the original grant proposal to CSAT/SAMHSA; (2) Respondents answered open-ended questions about the EBPs including factors facilitating and/or impeding the implementation of each, and (3) Interviewers, who were research assistants trained in interviewing techniques, recorded answers in an Interview Guide in addition to audio-taping all sessions. Data collection procedures were approved by the Boston University Institutional Review Board. 3.3. Measures The interviewer asked each staff member to respond to the statement: ‘‘Describe barriers your project encountered in providing this treatment or service.’’ Interviewers were trained to probe by noting that barriers could be external to the organization, internal to the organization, or related to the EBP itself. Additional clarifying prompts to elicit barriers included: ‘‘things that got in the way when your project tried to provide the service,’’ ‘‘challenges your organization faced in delivering the service,’’ and ‘‘things that made implementation difficult.’’ 3.4. Data analysis Respondents’ answers to questions about barriers were simultaneously audio-taped and written down by interviewers into Interview Guides. Comments of all 172 respondents were included in the data analysis. All categorizing and analyzing of data was done manually and is described in the sections below. Interview Guides were used as a proxy for audio-tapes after two coders compared the data in a sample of 12 Interview Guides with that in the audio-tapes of the same sessions and found high congruence (95% agreement) between the two data sources. In the small number of cases (6 of 172) where Interview Guide statements were unclear (due to the interviewer’s difficulty hearing or understanding the respondent’s words and accurately recording them), coders listened to the audio-taped segments (sometimes multiple times) to clarify respondent statements, then jointly reconstructed those statements. To answer the question, ‘‘What barriers are most frequently reported for each EBP?’’ an Excel spreadsheet was developed for each EBP that included all barriers named by the respondents who had implemented that EBP. (Barriers were participant responses to the statement: ‘‘Describe barriers your project encountered in providing this treatment or service.’’). Most respondents named at least one barrier and some respondents named multiple barriers. Our analysis began with the more than 350 barriers named by respondents. Examining barriers specific to each of the four EBPs, we

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

385

Table 1 Ten most frequently reported barriers for each EBP. Motivational interviewing (MI) (n = 78 respondents)

Not enough trainings on MI (13) Staff resistance (11) Staff struggle with adjusting to MI (11) Client resistance and non-participation (10) Variance in staff training and perspectives (8)

MI philosophy conflicts with provider or organization (7) Not enough time for proper clinical processes (7) Incomplete staff/staff turnover (5) Insufficient space for all services (4) Working with mandated, criminal justice population (4)

Adolescent-Community Reinforcement Approach (A-CRA) (n = 39 respondents)

Certification process was burdensome/time consuming (7) Model is not flexible or adaptable to client needs (6) Incomplete staff/staff turnover (5) Clients are difficult to reach, track, and follow-up (5) Client resistance and non-participation (4)

Providers resisted or disliked the model (4) Not enough client referrals from community (4) Parents of adolescent clients not helpful in clinical process (4) Data management is time consuming (2) Incomplete buy-in from organization (2)

Assertive Community Treatment (ACT) (n = 28 respondents)

Not enough funding to implement fully (9) Incomplete staff/staff turnover (8) Limited affordable or subsidized housing (5) Transportation (5) Difficulty building relations and communicating with collaborators (4)

Model requirements are too complex and demanding (2) Community resources (including substance abuse treatment) are lacking (4) Client resistance (3) Homeless client population is very needy (3) Coordinating all ACT services is time-consuming (2)

Cognitive-behavioral Therapy (CBT) (n = 27 respondents)

Client resistance (9) Client (individuals and family) attendance is poor (3) Transportation (3) Clients’ cognitive barriers to understanding CBT concepts (3) Clients with anti-social personality disorder (2)

Not enough time to complete treatment (2) Not enough well educated, qualified staff in our geographic area (2) Cultural/language barriers (2) Client groups often resist prescribed content for a group session (1) Staff not trained well enough(1)

identified the ten most frequently named barriers. They are displayed in Table 1. For example, for MI, the most common barrier identified was ‘‘not enough training’’ on the EBP. This was mentioned by 13 out of 78 respondents (17%). With respect to A-CRA, the most common barrier named was ‘‘the certification process’’—it was seen as ‘‘burdensome and time consuming’’ and was mentioned by 7 out of 39 respondents (18%). For ACT, 9 out of 28 respondents (32%) mentioned, as the most common barrier, ‘‘not enough funding to implement fully’’ the EBP. Finally, 9 out of 27 respondents (33%) mentioned ‘‘client resistance to participation’’ in the EBP as the most common barrier to implementation of CBT. We sought an inclusive set of categories to describe the barriers to help us answer the question, ‘‘What patterns or categories best describe the barriers?’’ We used three categories previously used by Nelson et al. (2006) and provide examples of barriers in each category: (1) EBP characteristics (e.g., its structure, content, relevance, ease of use), (2) practitioners and/or setting characteristics (e.g., too much staff turnover; staff lacked training), and (3) client population characteristics (e.g., transient clients; insufficient number of clients). For our study, two coders categorized the barriers. The first coder worked with the three categories used by Nelson et al. (2006), but found that he needed another category to capture the comments of respondents. A fourth category was added for resources (e.g., few health services in area; funding restricted). Then the second coder used the four categories but found that another was needed to capture miscellaneous barriers not classified in the other categories; a fifth category was added for other (e.g., difficulty with collaborating agencies; requirements from multiple funders). On the few discrepant items, the coders discussed their perspectives and resolved differences of opinion. 4. Findings 4.1. Most common barrier categories for each EBP Using these five categories, we grouped the data to better understand the kinds of barriers identified by respondents. Table 2 displays the five categories on the left and the ten most common

barriers within each EBP column. Italics in the left column show the categories into which the majority of responses clustered and the percent of responses captured by that category. For MI, the category of practitioner or organizational setting captures the largest percent of barriers (54%) (i.e., 54% of staff members using MI reported that they experienced barriers in this category), whereas for A-CRA, two categories (EBP characteristics and client characteristics) capture somewhat similar percents of barriers (38% and 33% respectively). For ACT, the category of resources captures a surprisingly large percent of the barriers (82%), and for CBT, client characteristics captures a substantial percent (67%) of the barriers. Note that for each of the four EBPs, respondents named barriers in multiple categories; ACT is the only EBP with barriers in all five categories. 4.2. Most common barrier theme for each EBP To understand the nature of the various barriers within the categories, we looked for themes. We regard themes as essential because they can be used as targets for action. Barrier statements are too specific and numerous to remedy individually, and barrier categories are too broad to suggest remedial action. Examining barriers in the first category of ‘‘EBP characteristics,’’ the following themes emerged: For MI, we found that ‘‘training’’ was a theme, with respondents commenting on insufficient staff training and lack of staff training. The following are examples of comments, each from a different respondent: ‘‘some staff without enough training are confused about the model,’’ and [with so much staff turnover], ‘‘a barrier is keeping staff trained,’’ and ‘‘it is easy for staff to ‘‘slip’’ from the MI framework; it requires constant clinical supervision; and a lot of time is required for appropriate clinical supervision.’’ Another theme, ‘‘philosophical differences,’’ captured what staff members said about varying views of MI among staff, and between an organization’s philosophy and the MI philosophy. Examples of respondents’ quotes are: ‘‘There is a perceived conflict between MI and the 12-step approach,’’ and, ‘‘the transition to MI from the previous theoretical orientation is difficult for some clinicians due to different backgrounds.’’

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

386

Table 2 Most common barriers distributed by barrier category. Barrier category

MI

A-CRA

ACT

CBT

(n = 78 respondents)

(n = 39 respondents)

(n = 28 respondents)

(n = 27 respondents)

EBP characteristics 38% of A-CRA respondents named barriers in this category

Not enough time for proper clinical processes

Certification process was burdensome/time consuming Model is not flexible or adaptable to client needs Data management is time consuming

Coordinating all ACT services is time-consuming Model requirements are too complex and demanding

Practitioner or organizational setting 54% of MI respondents named barriers in this category

Not enough trainings on MI Staff resistance Staff struggle with adjusting to MI Variance in staff training and perspectives

Incomplete staff/staff turnover Providers resisted or disliked the model Incomplete buy-in from organization

Incomplete staff/staff turnover

Cultural/language barriers Not enough time to complete treatment Staff not trained well enough

Clients are difficult to reach, track, and follow-up Client resistance and non-participation Parents of adolescent clients not helpful in clinical process

Client resistance Homeless client population is very needy

Client resistance Client (individuals and family) attendance is poor Clients’ cognitive barriers to understanding CBT concepts Clients with anti-social personality disorder Client groups often resist prescribed content for a group session

Not enough funding to implement fully Limited affordable or subsidized housing

Transportation Not enough well educated, qualified staff in our geographic area

MI philosophy conflicts with provider or organization Incomplete staff/staff turnover Insufficient space for all services Clients 67% of CBT respondents named barriers in this category 33% of A-CRA respondents named barriers in this category

Client resistance and non-participation Working with mandated, criminal justice population

Resources 82% of ACT respondents named barriers in this category

Transportation Community resources (including substance abuse treatment) are lacking Other

Not enough client referrals from community

For A-CRA, there were three themes. For the first, ‘‘certification,’’ examples of respondents’ comments are: ‘‘high certification requirements,’’ ‘‘certification was difficult—constant fidelity checking,’’ ‘‘having to record sessions creates rapport problems with clients.’’ The second theme was ‘‘rigidity of the model,’’ with respondents offering comments such as: ‘‘little room for diversity in procedures,’’ ‘‘very few cultural adaptations,’’ and ‘‘the developers had a very difficult time seeing beyond this one particular model, which was very specific. . .so there’s absolutely no room to bring other orientations in,’’ and ‘‘clinician doesn’t have flexibility for home visits.’’ Client non-engagement was also a theme for ACRA. Representative quotes are: ‘‘lack of participation from caregivers,’’ and ‘‘It takes a lot of follow up to get . . .adolescents to stay engaged, and sometimes it also means more follow up with the parents as well. . .so it is a challenge for those who are really not married to the program. . .’’ For ACT, the theme of ‘‘lack of concrete services’’ emerged, with respondent comments including the following: ‘‘for homeless population, lack of housing is a barrier,’’ ‘‘for co-occurring psychiatric and physical disorders, very limited resources,’’ and ‘‘lack of resources in county (no bus system, few jobs),’’ and ‘‘project had limited access to substance abuse treatment.’’ For CBT, the dominant theme was ‘‘client resistance’’ and ‘‘lack of suitability of clients.’’ Examples of quotes representing this

Difficulty building relations and communicating with collaborators

theme are: ‘‘concrete thinking—CBT more appropriate for people who have insightful thinking,’’ ‘‘clients could not stay focused because of outside relationships with relatives/friends,’’ and ‘‘in CBT groups, clients who miss the first week of the program when CBT is explained and goals are set.’’ 5. Discussion In summary, this study interviewed front-line staff in community addiction programs to identify the barriers they experienced when implementing specific EBPs. The study extended the work of a handful of existing qualitative studies that have informed this topic (Brown, 2004; Godley et al., 2001; Nelson, Steele, & Mize, 2006; Riley et al., 2008). It was not surprising that MI, CBT, A-CRA and ACT were the EBPs most frequently implemented by these respondents. These EBPs have been used for some time in the field and are gaining more visibility with time. Similar to other researchers, we found that barriers differed by EBP. 5.1. Major barrier category for each EBP and themes For the two individual counseling methods (MI and CBT), few barriers were named in the category of EBP characteristics,

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

indicating that respondents found that the EBP itself (e.g., requirements, content, delivery format, and theory) did not create major impediments. Instead, for MI, the category of provider and setting dominated, with the theme of ‘‘philosophical differences’’ reflecting staff resistance to MI and perceived conflict between organizational perspectives and this EBP. The accompanying theme of ‘‘lack of MI training’’ available to staff may show awareness that this is not a method that necessarily comes naturally to staff and does require specialized training, and perhaps an extensive amount of it. For CBT, the category of client characteristics emerged as important, with client limitations in the areas of motivation, cognitive ability and psychiatric stability highlighted. Similar to the comments about MI, these comments may show the respondents’ awareness that when CBT is used as designed, clients must have some cognitive clarity and will benefit most if they participate in monitoring their thoughts and behavior and practicing new skills. For A-CRA, which requires clinician certification, and considerable data gathering and structure, several barriers in the category of EBP characteristics were named. This is not surprising since staff must meet many A-CRA certification standards before using the method, and must develop competence in 17 skill areas. Several barriers in the client category were also named by several respondents, indicating that securing client commitment was not easy. It is likely that the family orientation of A-CRA was at least partly responsible for this. Clinicians had to engage adolescent clients as well as their parents and/or caregivers, and joint sessions between the therapist, adolescent and parents/caregivers are also required. Thus, it is not difficult to see why respondents would indicate that barriers occurred in the client category. For the community and systems-oriented ACT, in which staff members provide assistance with housing and employment, the category of resources was major. Given that many communities lack sufficient housing and employment resources, and funding to support them, the distribution of barriers into this category is not surprising. Thus, these data confirm that barriers differ by EBP and that barrier categories differ by EBP. Further, the multi-dimensional character of various EBPs (e.g., several components, focused on multiple types of clients, delivered in different formats) may predict that the barriers will be multidimensional. 5.2. Anticipating and addressing barriers Themes that emerged from the barriers and barrier categories in this study can provide guidance for action. Researchers designing and testing EBPs, and policy makers and funders who mandate that EBPs be used in the addiction and mental health fields, must anticipate beforehand and address these impediments to ensure that community organizations can accelerate their use of EBPs. For the four EBPs we examined in depth, barriers spanned more than one barrier category. Thus, when help is provided to programs to address barriers, it may need to address multiple barrier dimensions. The themes revealed by staff perceptions could help to improve EBP training by providing a focus on ways to address barriers. Training has been identified as an important factor in the transfer of EBPs into practice (Miller, Sorenson, Selzer, & Brigham, 2006; Simpson, 2002; Walters, Matson, Baer, & Ziedonis, 2005). Improved staff training could ultimately lead to improved service delivery. In addition to ways to address barriers, training could clarify whether perceived barriers are actually present and associated with particular EBPs. Some perceived barriers may be based on faulty information or misconceptions about the EBP and/or guidelines for implementing it. Trainers need to be aware of this and find ways to

387

dispel misconceptions. In their manuals, EBP designers also need to address potential barriers and equip staff to respond to them. Our sample included only CSAT/SAMHSA-funded programs that had federal support in terms of funds and technical assistance; treatment programs solely funded by states or private insurance were not included. This is important to keep in mind because CSAT/ SAMHSA-funded programs may be more advanced in their understanding and acceptance of EBPs, more able to implement EBPs, and less affected by barriers that might be daunting to other types of community programs. Depending on their organizational structure and capacity, typical CBOs may be less able to respond to certain EBP barriers; such barriers may stymie their efforts. 5.3. Study limitations Program directors identified clinical staff who implemented EBPs, possibly introducing sample bias. Some programs may have had many staff who implemented these EBPs but program directors chose two staff to be interviewed for this study. There may have been variation in the roles and involvement of these staff in EBP implementation, which could have influenced their perceptions of barriers. Further, this is an exploratory study and thus cannot show causal connections between study variables. Instead, its contribution is in identifying key factors or elements that are likely to influence efforts to utilize or implement specific EBPs. Finally, organizations that are the least or most successful in implementing EBPs might never apply for government funding. The perspectives of those organizations are not included here, possibly introducing another source of sample bias. 5.4. Lessons learned Forman et al. (2001) have encouraged researchers to conduct studies to clarify the extent to which staff attitudes affect the implementation and outcomes of new EBP approaches. This is relevant to the study discussed here because, although we can assume that perceptions and experiences of barriers affect EBP implementation, we do not know exactly how or to what extent this might be so. For example, if many barriers and/or high barrier interference is perceived, does this lead staff to implement an EBP with less fidelity, for example, modifying the EBP to make it more manageable and perhaps reducing fidelity? Does it reduce selfefficacy among staff, leading to an erosion of motivation to use the EBP at all? Alternatively, do staff members view barriers as an expected aspect of any EBP, to be addressed in a thoughtful and energetic way and to be overcome? The field needs to know the extent to which such barriers block implementation totally or are overcome in some way that allows implementation to occur. We agree with McGovern et al. (2004) that more study is needed of the barriers to specific EBPs. It is clear that large quantitative studies are needed that compare barrier interference scores for a wide range of EBPs in order to identify those EBPs with high and low scores. This would help in identifying the characteristics of the EBPs and their implementation processes that might lead to more substantial/intractable barriers or to more malleable ones. We also need studies with multivariate analyses that examine EBPs and their barrier interference scores, while controlling for staff and organizational characteristics and other factors likely to influence staff perception of barriers. Answering these questions would refine our understanding of perceived barriers, barrier ratings, and their effects on EBP implementation, and could lay the groundwork for making EBP implementation much more efficient and effective. We suggest cost analyses as a next step in examining community implementation of EBPs. As they accelerate their use of EBPs, addiction programs will need data on the resources

388

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389

required for implementing specific EBPs. Resources include staff, community services and funding. There are likely to be cost differentials for implementing various EBPs such as ACT, ACRA and CBT. For example, some EBPs require that staff be certified in the EBP prior to EBP implementation. This certification process can be costly. Given staff turnover in the field, what are the implications if organizations must then certify replacement staff? Cost projections could be of great assistance to programs in deciding which EBPs they can realistically implement. 6. Conclusion Implementation barriers need to be acknowledged explicitly and addressed by both the designers of EBPs and the policy makers and funders who mandate them. Addiction programs proposing to use specific EBPs must consider whether they have the organizational capacity, as well as the community capacity, to meet the demands of that practice. EBP dissemination to programs should include explicit strategies to address such barriers. Future research should examine actual barriers faced during implementation, as this study could only capture perceptions of barriers. Acknowledgements Funding was provided by the Robert Wood Johnson Foundation Substance Abuse Policy Research Program Grant # 65029. The authors wish to thank the many staff who participated in our interviews and completed online surveys. This research would not have been possible without their time and valuable insights. Appendix A Motivational Interviewing (MI) is a goal-directed, client-centered counseling method for eliciting behavioral change by helping clients explore and resolve ambivalence. Ambivalence is seen as the primary obstacles to behavior change, so the resolution of ambivalence is the key goal. This is an individual counseling approach and specific methods include listening reflectively, asking open-ended questions, affirming the client’s change-related statements and efforts, eliciting recognition of the gap between current behavior and desired life goals, responding to resistance without direct confrontation, and increasing the client’s self-efficacy for change (Miller & Rollnick, 2002). Researchers have had difficulty in measuring competence in the practice of MI because there is an ‘‘explicit emphasis on the spirit of the method rather than the techniques that comprise it’’ (Moyers, Martin, Manuel, Hendrickson, & Miller, 2005, p. 19). The Adolescent Community Reinforcement Approach (A-CRA) is a behavioral intervention for alcohol/drug problems that seeks to replace environmental contingencies that have supported alcohol/ drug use with pro-social activities and behaviors that support recovery. This outpatient program targets youth 12–22 years old with DSM-IV cannabis, alcohol, and/or other substance use disorders. ACRA includes guidelines for three types of sessions: adolescents alone, parents/caregivers alone, and adolescents and parents/caregivers together. According to the adolescent’s needs and self-assessment, therapists choose from among 17 A-CRA procedures that address problem-solving, communication skills, and active participation in pro-social activities. Parental/caregiver involvement is crucial (Garner et al., 2009; Chestnut Health Systems www.Chestnut.org). Assertive Community Treatment is a team approach providing psychiatric treatment, rehabilitation, and support to persons with serious and persistent mental illness. The team (e.g., social workers,

rehabilitation specialists, nurses, and psychiatrists) provides case management, initial and ongoing assessments; psychiatric services; employment and housing assistance; family support and education; and substance abuse services which are available 24 hour per day, 365 days per year. Services are provided within a person’s own home and neighborhood, local restaurants, parks and nearby stores. ACT team members are pro-active, assisting clients to participate in treatment, live independently, and recover from disability. ACT services are intended to be long-term due to the severe impairments of the clients. The team encourages all clients to participate in community employment and directly provides many vocational rehabilitation services. Collaboration in the interdisciplinary team is crucial (Drake et al., 1998). Cognitive-behavioral Therapy joins cognitive and behavioral assessment and intervention methods in the individual counseling approach. It helps clients understand the intersection of thoughts, emotions and behaviors and learn ways to interrupt automatic responses to these patterns. Methods include helping clients monitor and modify automatic thoughts, assumptions, and beliefs and acquire problem-solving, affect-regulation, and social skills. The clinician’s role includes using a functional analysis for assessment, collaborating with the client on goal-setting, practicing skills such as assertiveness, drug refusal, and anger-management, and using homework to solidify client skills (Carroll, 2002). References Aarons, G. A. (2005). Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatry Clinics of North America, 14(2), 225–238. Ball, S. A., Bachrach, K., DeCarlo, J., Farentinos, C., Keen, M., & McSherry, T. (2002). Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers. Journal of Substance Abuse Treatment, 23(4), 309–318. Bartholomew, N. G., Joe, G. W., Rowan-Szal, G. A., & Simpson, D. (2007). Counselor assessments of training and adoption barriers. Journal of Substance Abuse Treatment, 33(2), 193–199. Berger, L. K., Otto-salaj, L., Stoffel, V. C., Hernandez-Meier, J., & Gromoske, A. N. (2009). Barriers and facilitators of transferring research to Practice: An exploratory case study of Motivational Interviewing. Journal of Social Work Practice in the Addictions, 9, 145–162. Berelson, B. (1952). Content analysis in communication research. New York: Free Press. Brown, A. H. (2004). Integrating research and practice in the CSAT Methamphetamine Treatment Project. Journal of Substance Abuse Treatment, 26(2), 103–108. Carroll, K. M. (2002). A cognitive-behavioral approach: Treating cocaine addiction. Rockville, MD: NIH Publication 98-4308, National Institute on Drug Abuse. Drake, R. E., McHugo, G. J., Clark, R. E., Teague, G. B., Xie, H., Miles, K., et al. (1998). Assertive Community Treatment for patients with co-occurring severe mental illness and substance use disorder: A clinical trial. American Journal of Orthopsychiatry, 68(2), 201–215. Forman, R. F., Bovasso, G., & Woody, G. (2001). Staff beliefs about addiction treatment. Journal of Substance Abuse Treatment, 21, 1–9. Fuller, B. E., Rieckmann, T., Nunes, E. V., Miller, M., Arfken, C., Edmundson, E., et al. (2007). Organizational readiness for change and opinions toward treatment innovations. Journal of Substance Abuse Treatment, 33, 183–192. Garner, B. R. (2009). Research on the diffusion of evidence-based treatments within substance abuse treatment: A systematic review. Journal of Substance Abuse Treatment, 36, 376–399. Garner, B. R., Godley, S. H., Funk, R. R., Dennis, M. L., Smith, J. E., & Godley, M. D. (2009). Exposure to Adolescent Community Reinforcement Approach (A-CRA) treatment procedures as a mediator of the relationship between adolescent substance abuse treatment retention and outcome. Journal of Substance Abuse Treatment, 36(3), 252–264. Godley, S. H., White, W. L., Diamond, G., Passetti, L., & Titus, J. C. (2001). Therapist reactions to manual-guided therapies for the treatment of adolescent marijuana users. Clinical Psychology: Science and Practice, 8(4), 405–417. Herbeck, D. M., Hser, Y. I., & Teruya, C. (2008). Empirically supported substance abuse treatment approaches: A survey of treatment providers’ perspectives and practices. Addictive Behaviors, 33(5), 699–712. Institute of Medicine. (1998). Bridging the gap between practice and research: Forging partnerships with community-based drug and alcohol treatment. Washington, DC: National Academy Press. Knudsen, H. K., Ducharme, L. J., Roman, P. M., & Link, T. (2005). Buphrenorphine diffusion: The attitudes of substance abuse treatment counselors. Journal of Substance Abuse Treatment, 29, 95–106.

M. Amodeo et al. / Evaluation and Program Planning 34 (2011) 382–389 Mark, T. L., Kranzler, H. R., Song, X., Bransberger, P., Poole, V. H., & Crosse, S. (2003). Physicians’ opinions about medications to treat alcoholism. Addiction, 98(5), 617–626. McCarty, D., Fuller, B. E., Arfken, C., Miller, M., Nunes, E. V., & Edmundson, E. (2007). Direct care workers in the National Drug Abuse Treatment Clinical Trials Network: Characteristics, opinions, and beliefs. Psychiatric Services, 58(2), 181–190. McGovern, M. P., Fox, T. S., Xie, H., & Drake, R. E. (2004). A survey of clinical practices and readiness to adopt evidence-based practices: dissemination research in an addiction treatment system. Journal of Substance Abuse Treatment, 26, 305–312. Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change (second edition). New York: The Guilford Press. Miller, W. R., Sorenson, J. L., Selzer, J. A., & Brigham, G. S. (2006). Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment, 31, 25–39. Moyers, T. B., Martin, T., Manuel, Hendrickson, & Miller, (2005). Assessing competence in the use of motivational interviewing. Journal of Substance Abuse Treatment, 28(1), 19–26. Nelson, T. D., & Steele, R. G. (2007). Predictors of practitioner self-reported use of evidence-based practices: Practitioner training, clinical setting, and attitudes toward research. Administrative Policy in Mental Health & Mental Health Services Research, 34(4), 319–330. Nelson, T. D., Steele, R. G., & Mize, J. A. (2006). Practitioner attitudes toward evidencebased practice: Themes and challenges. Administrative Policy in Mental Health & Mental Health Services Research, 33, 398–409. Obert, J. L., Brown, A. H., Zweben, J., Christian, D., Delmhorst, J., Minsky, S., et al. (2005). When treatment meets research: clinical perspectives from the CSAT Methamphetamine Treatment Project. Journal of Substance Abuse Treatment, 28(3), 231–237. Ogborne, A. C., Wild, T. C., Braun, K., & Newton-Taylor, B. (1998). Measuring treatment process beliefs among staff of specialized addiction treatment services. Journal of Substance Abuse Treatment, 15(4), 301–312. Rieckmann, T., Daley, M., Fuller, B. E., Thomas, C. P., & McCarty, D. (2007). Client and counselor attitudes toward the use of medications for treatment of opioid dependence. Journal of Substance Abuse Treatment, 32, 207–215. Riley, K. J., Rieckmann, T., & McCarty, D. (2008). Implementation of MET/CBT 5 for adolescents. The Journal of Behavioral Health Services and Research, 35(3), 304–314. Sheehan, A. K., Walrath, C. M., & Holden, E. W. (2007). Evidence-based practice use, training and implementation in the community-based service setting: A survey of children’s mental health service providers,. Journal of Child and Family Studies, 16, 169–182. Simpson, D.D. (2002). A conceptual framework for transferring research to practice,. Journal of Substance Abuse Treatment, 22(4), 171–182. Thomas, C. P., Wallack, S. S., Lee, S., McCarty, D., & Swift, R. (2003). Research to practice: Adoption of naltrexone in alcoholism treatment. Journal of Substance Abuse Treatment, 24, 1–11. Walters, S. T., Matson, S. A., Baer, J. S., & Ziedonis, D. M. (2005). Effectiveness of workshop training for psychosocial addiction treatments: A systematic review. Journal of Substance Abuse Treatment, 29, 283–293. Willenbring, M. L., Kivlahan, D., Kenny, M., Grillo, M., Hagedorn, H., & Postier, A. (2004). Beliefs about evidence-based practices in addiction treatment: A survey of Veterans Administration program leaders. Journal of Substance Abuse Treatment, 26, 79–85. Dr. Maryann Amodeo, Co-director of the Center for Addictions Research and Services, and Professor of Clinical Practice at the Boston University School of Social Work, has more than 20 years of professional experience in the alcoholism and drug abuse fields as a clinician, educator, and researcher. Her activities have been funded by the National

389

Institute on Drug Abuse, the Robert Wood Johnson Foundation, and the Center for Substance Abuse Prevention. She has received national awards for her pioneering work in integrating substance abuse skills in the social work curriculum. For 15 years, she directed an interdisciplinary Postgraduate Certificate Program in Alcoholism and Drug Abuse from which 300 masters and doctoral-level professionals graduated. Dr. Lena Lundgren, the Director of the Center for Addictions Research and Services, is also a Professor of Welfare Policy and the Director of Research at Boston University School of Social Work. Dr. Lundgren’s research focuses on: addiction treatment and HIV prevention in community based organizations and implementation of evidence-based practices in community based addiction treatment organizations. She has collaborated with community based organizations to help provide more than $14 million for service development in the areas of HIV prevention and substance abuse treatment. Alexander Cohen, MSW, is completing his MPH at the Boston University School of Public Health. Mr. Cohen has worked in the addiction research, treatment, and prevention fields since 2006, as a clinician in school-based and outpatient programs. He has provided substance abuse treatment and prevention to adolescents, young adults, and adults with co-occurring disorders. Dylan Rose, B.A., is a recent graduate of Boston University where he studied Psychology and Philosophy. He has published research in both fields, on subjects such as memory, affect, addiction and health psychology, political psychology, motivation, forgiveness, and the philosophical foundations of social science methodology. He is also an active local arts and literature critic. He plans to pursue further training in psychology either in medicine or at the doctoral level. Deborah Chassler, MSW, Associate Director of the Center for Addictions Research and Services, is a Senior Academic Researcher at Boston University School of Social Work. She has worked with several BUSSW professors developing data analyses which have resulted in fifteen peer-reviewed journal articles covering a range of subjects including substance abuse treatment and HIV prevention. Before joining the Center, she was responsible for the implementation of several large scale projects funded by the Commonwealth of Massachusetts and the National Institute of Aging. Clelia Beltrame, BA, is a Certified Alcohol and Drug Abuse Counselor (CADAC) who has worked in the addictions field since 2005 serving vulnerable populations such as the underinsured, uninsured, and the homeless. Ms. Beltrame has worked with individuals as well as groups and is a certified medical interpreter and a certified medical interpreter trainer. She is currently a member of the advisory board for the Alcohol/Substance Abuse Counseling Certificate program. She served as a research assistant at Boston College, conducting research, counseling, and health education. She is currently a dual degree candidate at Boston University (Masters in Social Work and Public Health). Melinda D’Ippolito, BS, is currently working toward her MSW at Boston University School of Social Work and plans to pursue an MPH with a concentration in behavioral health at Boston University School of Public Health. Ms. D’Ippolito has worked with adults with mental illness and substance use disorders in a variety of settings, most recently as a case manager for individuals transitioning out of homelessness. Her research interests include trauma responses and interventions, the intersection of trauma and substance use, community preparedness for mental health services in response to violence and disasters, and food policy.