Program evaluation of the Opioid and Naloxone Education (ONE Rx) program using the RE-AIM model

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

Contents lists available at ScienceDirect

Research in Social and Administrative Pharmacy journal homepage: www.elsevier.com/locate/rsap

Program evaluation of the Opioid and Naloxone Education (ONE Rx) program using the RE-AIM model Mark A. Stranda,∗, Heidi Eukelb, Oliver Frenzelb, Elizabeth Skoyb, Jayme Steigc, Amy Werremeyerb a

Master of Public Health Program, North Dakota State University, P.O. Box 6050, Fargo, ND, 58108, USA North Dakota State University, Fargo, ND, USA c Quality Health Associates of North Dakota, Minot, ND, USA b

ARTICLE INFO

ABSTRACT

Keywords: RE-AIM Population health Opioid use disorder Community pharmacy practice Collaboration Accidental overdose

Objective: Opioid and Naloxone Education (ONE Rx) is a program that focuses on community pharmacy-based patient screening and interventions to improve population health with regard to opioid use. The objective for this paper is to describe how ONE Rx was implemented, report on the populations impact using the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) Model, and explain future implications of the program. Methods: ONE Rx is a statewide program in which pharmacists screen patients who receive an opioid prescription for the risk of opioid misuse and accidental overdose. The five domains of the RE-AIM Model were used to evaluate ONE Rx. Reach was defined as the proportion of patients receiving opioid prescriptions who completed the screening. Efficacy was defined as the proportion of individuals identified as at risk of opioid misuse or accidental overdose and who received a pharmacist intervention. Adoption was defined as the proportion of eligible community pharmacies who enrolled in ONE Rx. Implementation was defined as the proportion of pharmacies that enrolled in ONE Rx that provided at least five patient screenings. Maintenance was defined as the proportion of pharmacies that adopted ONE Rx that completed at least one screening three months after the initial provision. Results: Approximately 16.9% of all patients receiving opioid prescriptions were screened for risk of opioid misuse and accidental overdose. Of the patients screened, 97.1% of patients at risk for opioid misuse or accidental overdose received a pharmacist-led intervention. Additionally, 44.8% of the pharmacist that enrolled in ONE Rx completed at least five screenings and of those, 80.0% maintained the program three months later. Conclusions: ONE Rx demonstrated success and positive population impact. The RE-AIM Model identified strength in the areas of efficacy, adoption and maintenance, and the need for improvement in the areas of reach and implementation.

Introduction The opioid crisis has challenged the American healthcare system and society. According to the National Survey of Drug Use and Health (NSDUH), approximately 12 million people in America aged 12 years and older misused prescription pain relievers in 2016.1 The Center for Disease Control (2017) states that the supply of prescription opioids remains high in the United States, with an estimated one out of five patients with non-cancer pain or pain-related diagnoses receiving opioids, which in turn increases the likelihood of prescription drug abuse and related consequences.2 Use of opioids post-operatively often

∗

leads to chronic use,3 and the increase in opioid use disorder prevalence is highest among persons 11–25 years old.4 In 2017, an estimated 2.1 million people aged 12 or older had an opioid use disorder, which is equivalent to 3.2% of this aged population.5 Opioid use and extent of population impact continues to negatively harm our nation's health. In 2017, 68% of the more than 70,200 drug overdose deaths involved an opioid, and 130 Americans die every day from an opioid overdose.2 In addition to harmful opioid outcomes affecting our nation, it poses a tremendous economic burden. An estimated $504 billion dollars in 2015 was allocated to the economic cost of the opioid crisis, or 2.8% of the GDP.6 Florence et al., states that 73% of estimated opioid related

Corresponding author. School of Pharmacy and Department of Public Health, North Dakota State University, P.O. Box 6050, Fargo, ND, 58108, USA. E-mail address: [email protected] (M.A. Strand).

https://doi.org/10.1016/j.sapharm.2019.11.016 Received 17 July 2019; Received in revised form 18 November 2019; Accepted 29 November 2019 1551-7411/ © 2019 Elsevier Inc. All rights reserved.

Please cite this article as: Mark A. Strand, et al., Research in Social and Administrative Pharmacy, https://doi.org/10.1016/j.sapharm.2019.11.016

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

costs in the United States will be attributed to non-fatal consequences, healthcare spending, criminal justice costs, and loss of productivity due to addition services and incarceration.7 The opioid crisis and associated deaths continue to plague the nation and pose significant burdens both socially and economically. The role of community pharmacists in addressing the opioid crisis has been slow and associated with significant variability across regions and systems. A study from Minnesota found that nearly 60.0% of community pharmacists had not dispensed naloxone in the recent month.8 Another study reported naloxone was available for purchase in only 23.5% of California's 1147 surveyed community pharmacies.9 The Government Accountability Office in 2017 reported that only 3.0% of pharmacies and other entities eligible to collect unused prescription medications had volunteered to do so.10 This inertia among community pharmacists to aggressively engage with the opioid crisis is the result of concerns over need for additional resources, investment, and organizational support.11 It will require years of intensive primary prevention to reduce the incidence of opioid use disorder, and ultimately, reduce opioid overdose deaths.12 Given that 39.0% of persons who used heroin had dependence on prescription opioids beforehand,13 the work of primary prevention needs to begin with providers and pharmacists caring for patients receiving prescription opioids. There is evidence that upstream primary prevention can reduce prescription drug misuse.14 There are few community-based opioid misuse prevention programs that have been evaluated and shown to have a significant population health impact.15,16 Yet, there remains a need for studies that demonstrate the feasibility and population impact of well-designed interventions.17 The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) Model is a comprehensive approach to determine the population-level impact of a community-based intervention.18 The RE-AIM is a flexible model where the variables are defined internally, and the quality of the evaluation is dependent upon using the definitions consistently. A program or intervention that stands up to scrutiny under the RE-AIM Model can be considered sustainable, effective, generalizable and evidence-based.19 This paper uses the REAIM Model to perform program evaluation of the Opioid and Naloxone Education (ONE Rx) program, whose goal was to implement a standardized community pharmacy-based patient screening and intervention strategy to improve population health with regard to opioid use. This paper has three aims: 1) to describe the process for implementing the ONE Rx program, 2) to report on the population impact of ONE Rx using the RE-AIM domains, and 3) to explain ways in which to increase the population impact of ONE Rx.

including asthma, depression, anxiety, COPD or emphysema, sleep apnea, liver disease, or kidney disease. Based upon screening results, pharmacists provided patient-specific education and interventions using an evidence-based clinical decision-making triage tool, developed by experts on the research team. The ONE Rx training modules, workflow documents, screening tools, data collection tools, social media links, patient referral resources, and pharmacy reimbursement guidelines are available at nodakpharmacy.com/onerx/. Additional information about the impact of the training on pharmacists’ perceptions and predictive behavioral changes and the overall screening, triage, and intervention framework have been previously published.21,22 Detailed information regarding scalability, an implementation framework, and resources for implementation also have been previously published.25 North Dakota has 149 community pharmacies, all of which are independently owned. Pharmacists in North Dakota have been authorized to prescribe and dispense naloxone since April 1, 2017.26 Data source Data were collected from October 12, 2018 through June 1, 2019. The ONE Rx research team collaborated with Quality Health Associates of North Dakota to build an on-line data collection system using REDCap (Research Electronic Data Capture), a secure, web-based application designed to support data capture for research studies.27 Sixtyfour items of data were collected on each patient, with the intent to identify trends in patient risk for opioid misuse and accidental overdose, personalized patient care, continuous quality improvement of the ONE Rx program, and recording of the interventions provided to patients. The North Dakota State University Institutional Review Board approved the study tools and methods. RE-AIM definitions The RE-AIM model is used to evaluate population impact of an intervention. To do so, it combines both individual and institutional (pharmacy) values. The RE-AIM model variables are an evidence-based approach to evaluating the population impact of a public health intervention. Each of the five domains of the RE-AIM Model – Reach, Effectiveness, Adoption, Implementation and Maintenance – were operationalized as described below. Reach was defined as the proportion of persons in each intervention site who accepted the intervention. This was analyzed at the individual level. Within ONE Rx, this was operationalized as proportion of patients completing the screening among all individual patients who received an opioid prescription. The inclusion criteria for the reported numbers of prescriptions during this period of time included any prescription medication containing an opioid or opioid derivative. Buprenorphine/ naloxone was excluded as those individuals were currently active in medication-assisted treatment (MAT). To assess reach in the ONE Rx program, it was necessary to determine the number of individual patients eligible for patient screening to serve as the denominator. This information is not easily retrievable in an automated fashion, as all pharmacies in the state are independently owned and operated. Ten pharmacies who were able to provide aggregate dispensing data and who represented a homogenous sample of the 30 participating pharmacies were selected to provide dispensing data. The research team requested total opioid prescription numbers from those 10 pharmacies for a two-month period of time (April and May, 2019). The number of ONE Rx screenings in these pharmacies (n = 370) documented during this time frame were compared to the total number of opioid prescriptions filled in those 10 pharmacies (n = 2187), with both values being multiplied by three to extrapolate the values from the sample to the entire 30 implementing pharmacies. Therefore, the denominator used for the reach calculation was 6561 (2187*3) individual patients receiving an opioid in the months of April and May, 2019, and 1110

Methods Study design ONE Rx is a statewide North Dakota program that provides optional community pharmacists training and tools for screening all patients prescribed an opioid medication for risk of opioid misuse and accidental overdose.20,21 The current program expanded upon an initial pilot program previously published.21–23 Through ONE Rx, each patient receiving an opioid prescription was screened for: 1) the risk of opioid misuse using the Opioid Risk Tool (ORT) and 2) risk of accidental overdose based on age, concurrent medication and alcohol use, and disease states.24 The screening was conducted through a paper-based or electronic screening tool each patient completed prior to receiving an opioid medication. The ORT generates a score from 0 to 26, with scores of 0–3, 4–7, and 8 or higher representing low, moderate, and high risk of opioid use disorder, respectively.24 The risk of accidental opioid overdose was determined by positive response to one or more of the following indicators including age; use of benzodiazepines, muscle relaxants, over-the-counter or prescription sleep aids, cough or cold medications; alcohol use; concomitant opioid use; and co-morbidities, 2

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

Table 1 Pharmacist interventions within ONE Rx. Required Interventions Independent of Riska Medication take-back Medication take-back program was introduced and strategies for use were discussed with the patient Opioid prescription partially filled When indicated, partial filling of opioid prescription option was discussed with the patient and acted upon b Critical Interventions for At-Risk Individuals Discussed community support services Discussion of available community support services, including when to seek, was discussed with the patient Explained benefits of naloxone The benefits and low risk of having naloxone on hand was discussed with the patient Dispensed naloxone The patient received naloxone and associated training for home use Contacted provider When indicated, the provider was contacted to discuss quantity prescribed, need for opioid, or other topics related to the risk of misuse and/or accidental overdose Discussed opioid use disorder The risk of opioids in relation to opioid use disorder was discussed with the patient, relating to their individualized screening results Discussed accidental overdose The risk of opioids in relation to accidental overdose was discussed with the patient, relating to their individualized screening results a b

Intervention which should be provided to every patient receiving an opioid prescription, regardless of risk. Critical intervention which should be offered to patients identified at risk of OUD and/or accidental overdose.

screenings (370*3). Efficacy was defined as the proportion of individuals at elevated risk for a poor outcome(s) who received the appropriate services. At risk individuals were defined as having an ORT score ≥8 and/or had a concurrent medication or disease state known to increase the risk of respiratory depression. This was analyzed at the individual level. Within ONE Rx, this was operationalized as the proportion of patients with ORT score ≥8 or who were at risk of accidental overdose who received one or more of six critical pharmacist-delivered interventions: explained the benefits of naloxone, dispensed naloxone, the pharmacist contacted their provider, the pharmacist discussed opioid use disorder, the pharmacist discussed risk of accidental overdose, and/or the pharmacist discussed community support services with the patient (Table 1). Adoption was defined as the proportion of healthcare practice sites in the community who adopted the intervention. This was analyzed at the organizational level. Within ONE Rx, this was operationalized as the proportion of eligible community pharmacies in North Dakota who enrolled in the ONE Rx program. Implementation was defined as how often the intervention was actually implemented. This was analyzed at the organizational level. Within ONE Rx, this was operationalized as the proportion of ONE Rx enrolled pharmacies that completed at least five patient screenings after enrolling. Maintenance was defined as the extent to which the intervention was sustained over time. This was analyzed at the organizational level. Within ONE Rx, this was operationalized as the proportion of pharmacies who adopted ONE Rx who performed at least one patient screening in month three after adoption.

provided patient-specific education and interventions based on screening results using a clinical decision-making triage tool. In addition to the six critical interventions that were provided based on the patient's risk profile, two interventions were required for all patients, including counseling on the option to partially fill an opioid prescription and provision of in-pharmacy medication disposal. Adoption: ONE Rx is an optional program for all pharmacies in North Dakota. To increase adoption of the ONE Rx program, a marketing model was used to promote the program across the state. To help pharmacists understand how ONE Rx aligned with their professional responsibilities, the North Dakota Board of Pharmacy wrote a memo sent to all pharmacists encouraging participation in the program. All training vignettes, toolkit materials, and support documents were housed on the North Dakota Board of Pharmacy website to allow easy access for all practicing pharmacists. A state-wide press conference was organized with speakers representing the ONE Rx team, the state pharmacists’ association, the Board of Pharmacy, the Attorney General, and representatives from the North Dakota Department of Human Services. Members from the ONE Rx research team presented the pilot study results of ONE Rx at a local state public health meeting to increase opportunities for engagement with local public health entities.22 Adoption was also enhanced by a $20 reimbursement for each patient screened and provision of a free tablet device to enrolled pharmacies to allow patients to enter their screening information electronically. Implementation: Once ONE Rx was initiated in pharmacies, the ONE Rx research team provided a welcome packet, including a toolkit for workflow implementation. Support was provided to each site through individualized phone calls and scheduled on-line coaching calls to maintain high fidelity to the program specifications. The purpose of the coaching calls was to provide informational support to the implementing pharmacists, but no data was collected. Maintenance: Several methods were used to encourage pharmacies to maintain implementation of ONE Rx. A monthly newsletter with performance highlights and sharing of best practices was distributed to all trained pharmacists using email delivery service. Additionally, periodic individualized phone calls were made to pharmacies to inquire about their work with ONE Rx and provide support as needed. Finally, a ONE Rx recognition program was created to recognize pharmacy performance in the ONE Rx program, which was shared on social media.

Methods used to maximize the impact of ONE Rx The specific methods used to maximize the impact of ONE Rx are described under each of the domains of the RE-AIM Model below. Reach: To educate pharmacists on the ONE Rx program and the importance of preventing opioid misuse and accidental overdose, and to prepare them to implement ONE Rx in their community pharmacies, a free 3-h continuing education program for pharmacists was developed by the expert team.21 The ONE Rx education program was designed and delivered to encourage pharmacists to perform screening and provide interventions to every patient receiving an opioid prescription. Training topics included the science of opioid misuse and addiction, screening for opioid misuse and overdose risk, naloxone prescribing, dispensing and consultation, opioid dispensing and consultation pearls, and techniques for communicating with patients and providers.21 Efficacy: Efforts to increase the efficacy of the program began with the selection of the ORT to assess opioid misuse risk and a survey created by the research team to assess accidental overdose to stratify patient risk profiles.24 This stratification allowed for identification of individuals at risk of OUD and/or accidental overdose. Pharmacists

Statistical analysis Microsoft Office Professional Plus 2016 Excel software was used to calculate proportions and means and to create all tables and figures. Results Sixty-three of the 149 community pharmacies in North Dakota enrolled in ONE Rx, with 30 of those pharmacies considered to be active, having screened at least 5 patients at the time of analysis for this 3

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

Table 2 Characteristics of patients provided services through ONE Rx (October 12, 2019 to June 1, 2019) (n = 1685). Proportion Gender (female) Opioid use disorder risk (ORT score ≥8) Accidental overdose risk Received another opioid prescription in last 60 days

54.8% 4.5% 19.1% 44.4%

manuscript. A total of 1685 patient screenings and associated interventions for each patient had been documented. The majority of the patients cared for through the ONE Rx program were female (Table 2). Of all screenings conducted, high risk of opioid use disorder was identified in 4.5% and the risk of an accidental overdose was identified in 19.1% of patients. Many of the patients receiving opioids would be considered chronic pain patients, as evidenced by the finding that 44.4% had been prescribed an opioid in the previous 60 days. Table 3 and Fig. 1 compare the ONE Rx target and performance in each of the RE-AIM domains. Reach and efficacy were reported as individual patient measures. The individual patient reach of ONE Rx was determined by extrapolation to be 16.9%. Efficacy in ONE Rx was defined as patients at risk of accidental overdose or opioid misuse as receiving any of the six critical interventions. 97.1% of patients identified to be either at risk of opioid misuse or of accidental overdose, or both, were delivered those critical services (Table 4). Only 40.2% of patients presenting with neither of the risk conditions were delivered those six interventions. From Table 4, one can calculate an accuracy score of 66.6% ((299 + 823)/1685), which shows that patients accurately received or did not receive services 66.6% of the time. Adoption, implementation, and maintenance were reported as organizational measures, using the pharmacy as the unit of measure. The adoption rate of 45% exceeded the ONE Rx target of 25% (Table 3 and Fig. 1). The implementation target of 80% was not achieved, with only 45% of enrolled pharmacies actually implementing the program. Eighty percent of implementing pharmacies achieved maintenance, which was the program target.

Fig. 1. ONE Rx performance and program targets on the RE-AIM Model domains. Table 4 Interventions delivered and patient risk status. ORT ≥8 or Risk of Accidental Overdose

Received any of six interventions: explained benefits of naloxone, dispensed naloxone, contacted provider, discussed opioid use disorder, discussed accidental overdose, or discussed community support services

Yes No Total %

Yes

No

299 9 308 97.1%

554 823 1377 40.2%

delivered by pharmacists for the first time, this program represents a significant breakthrough in the opioid epidemic and the profession of pharmacy. The low reach does reveal the difficulty implementing pharmacies faced with performing the screening on every eligible patient every time. Many variables contribute to the inability of the ONE Rx program to achieve the target benchmark reach goal of 90%. Factors include some pharmacy staff members not having been trained for ONE Rx services, patient refusal to complete the screening, patient not physically available to complete the screening, and if the patient was not physically available, the caregiver was not confident in answering the questions on behalf of the patient. Other factors include failure to capture potential patients due to prioritization of other healthcare services (i.e., Medicare Part D assistance, vaccine blitzes). The continual monthly screening of patients receiving opioid refills, monthly opioid renewals, or end-of-life care with opioids may not be perceived by some pharmacists to optimize their time, nor benefit the patient for preventative health services. These situations would add to pharmacy staff workflow burden and, in turn, would capture unwarranted screening results less likely to benefit population health. ONE Rx training and support services also reiterated to participating pharmacists to offer the screening tool to patients at

Discussion The population impact of the ONE Rx program in the first seven months is shown by the performance in each of the domains of the REAIM Model. The ONE Rx team focused on applying the best principals of implementation science by not just prioritizing quantifiable results but concentrating on the importance of population health during opioid use.28 The reach achieved in this program was 16.9%. This was based on the number of opioid prescriptions dispensed in ten pharmacies inferred to represent the performance of all 30 participating pharmacies. As practice sites across the state likely have significant variability in opioid dispensing and patient populations, it may not be representative of all participating pharmacies. Furthermore, despite failing to achieve the target of 90%, as a novel, and rather intensive cognitive service being Table 3 ONE Rx performance in the RE-AIM domains. Domain

Reach

Efficacy

Adoption

Implementation

Maintenance

Definition

Patients receiving opioid prescriptions who completed the screening 1110 6561 90.0% 16.9%

Patients with ORT ≥8 who receive one or more of six interventions 299 308 60.0% 97.1%

Pharmacies who enroll in the ONE Rx program 67 149 25.0% 45.0%

Enrolled pharmacies that completed at least five screenings 30 67 80.0% 44.8%

Pharmacies who implemented ONE Rx who have at least one screening in month three after adoption 24 30 80.0% 80.0%

Numerator Denominator ONE Rx target ONE Rx performance

4

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

appropriate intervals and in appropriate situations where information from the screening could be useful for preventative care. This ensured a significant amount of autonomy was granted to the pharmacist and fostered the professional judgment of the pharmacist in identifying situations where the screening would be useful and beneficial to patient health. It is important to evaluate the population impact of ONE Rx within the historical context of pharmacy practice in the state. In the past, addressing the opioid crisis at the pharmacy level only included tools to determine if a patient was currently misusing opioids through the PDMP. If misuse was suspected, the pharmacist would consult with the provider to determine individualized actions for patients, such as refusal to fill or provider contractual agreements for opioids. However, this only provided for a reactive, downstream approach to the opioid crisis. The iatrogenic nature of the opioid crisis has presented all healthcare professionals with a new challenge for upstream prevention, which calls for new approaches. Efficacy among all participating pharmacies was 97.1%. In other research, education and counseling on health recommendations for patients have been reported to be implemented only 10% of the time.29 According to the CDC, pharmacists are expected to communicate to patients the risks associated with opioid use;30 however, it has been reported that pharmacists perceive barriers to communication about prescription drug misuse due to lack of confidence, training, and time availability.31–33 This shows that the program was effective for educating pharmacists, as they were able to overcome these barriers in communicating with patients and still effectively identify patients in need of interventions and deliver them to those patients. Although the ability of screening tools using patient responses to a list of questions to predict patient outcomes has been questioned,34 such tools can be effective to stratify patients for the purpose of focusing time on those at highest risk. In this study, risk stratification allowed pharmacist to focus naloxone prescribing and other services on those patients with greatest need. This increased the value of the program because the time and effort were spent on the patients who most stood to benefit from it. In many clinical studies, efficacy information is the most reliable information of the five RE-AIM variables because the treatment or the program has been tested under highly controlled conditions. As a novel pharmacy intervention, there is no existing efficacy information available on pharmacist-delivered opioid-misuse prevention interventions. Future research will need to evaluate the effectiveness of the ONE Rx method on intermediate or long-term health outcomes such as reduced emergency department visits or reduced incidence of opioid use disorder. Efficacy in ONE Rx was defined as patients receiving any of the six critical interventions – discuss naloxone, dispense naloxone, contact provider, discuss opioid use disorder, discuss risk of accidental overdose, or discuss community support services. It was not possible to determine individual-level outcomes among patients cared for using the ONE Rx approach. Therefore, the study is proposing that the successful delivery of the six critical interventions be used as a proxy for efficacy, as these interventions are services delivered by the pharmacist in response to the results of the risk stratification of the patient. Future research with claims-based data from health systems is needed. Combining reach and efficacy creates a measure of overall individual-level impact. This individual level impact was varied when combining the reach of 16.9% and the efficacy of 97.1%; that is, there are still a considerable number of patients receiving opioid prescriptions who are not being screened, for reasons explained above, but those who are, receive appropriate interventions. The adoption rate of 45.0% exceeded the target of 25.0%, showing strong interest in participating. But upon adoption, many pharmacies struggled to get started. Only 44.8% of adopting pharmacies successfully implemented ONE Rx. While it was expected that most pharmacies would implement it right after adopting, the number of implementers was a consistent three to five new pharmacies every month, showing a

gradual broadening acceptance of the program. Once implemented, ONE Rx was maintained by 80.0% of implementing pharmacies. Time is required to implement ONE Rx into the workflow and to master the technical parts of the program, including data entry using the on-line ONE Rx data capture system, and patient care skills. This may partially explain the fact that approximately 55% of adopting pharmacies were not able to overcome the inertia of current work routines to begin implementing ONE Rx. It is also possible that pharmacists-in-charge changed their mind after enrolling or external obstacles came up to prevent implementation. The pharmacies that successfully implemented the ONE Rx program were continually supported through means of coaching calls, newsletters, and personalized technical assistance. This ensured that pharmacists were heard and respected, and concerns were addressed to continue a positive disposition toward this program. The personalized support and coaching calls promoted an inclusive sharing of perspectives about challenges that were encountered and how to overcome obstacles for continued delivery of patient care activities.35 The proportion of North Dakota community pharmacies who adopted the intervention by enrolling in the ONE Rx program was nearly double the program target of 25%. This was a result of ONE Rx having been designed thoroughly, with convenient on-line access to all documents, and personal technical support for implementation. The ONE Rx team included three pharmacists with extensive community pharmacy practice experience who guided the design and rollout to be minimally disruptive to workflow. One of the most important features of ONE Rx was that pharmacies conducting this program could do so without additional staff or resource requirements.36 Relationships entailing public agencies, local organizations, and community participants are integral in addressing health problems and learning collectively to avert such problems as the opioid crisis.37 North Dakota pharmacists are well coordinated by the North Dakota Board of Pharmacy, the North Dakota Pharmacists Association, and the North Dakota State University School of Pharmacy, creating a genuinely collegial spirit across the state. ONE Rx also involved close collaboration between community pharmacies and local public health. This is representative of concerted efforts and collaboration needed to promote health initiatives for the population. A high level of maintenance suggests that once ONE Rx was incorporated into the workflow, it was likely to be maintained. The incorporation of an objective means of stratifying risk enabled viable care delivery and fostered maintenance by the ONE Rx pharmacies and patients.36 In addition to the incorporation of ONE Rx activities into the usual care process, the ONE Rx team strived to balance rigid quality control with flexible practice-based implementation. The ONE Rx team granted independent decision-making capacity during triaging and interventions by the pharmacist. This autonomy within their scope of practice during interventions balanced scientific control with practicebased recommendations and fostered partnership between the ONE Rx coordinators and pharmacists.38 Adoption and implementation together reflect the appeal of the intervention to potential pharmacies. The 45.0% rate of adoption was better than anticipated, suggesting a desire by pharmacists-in-charge to participate in ONE Rx. But the 44.8% rate of implementation was lower than expected, suggesting a reluctance to actually get started. This could, perhaps, be due to uncertainty about the program's ultimate impact, fear of offending patients, unfamiliarity with substance use disorders, or lack of available services for those identified in need. Once pharmacies did initiate activity in the program, confidence and value was evident from their feedback in the program and continued performance was evidenced by the high rate of maintenance (80%). To achieve maximum population impact, it would be necessary to achieve 100% in each of the domains of the RE-AIM model, which is not realistic. However, the RE-AIM model is a powerful tool to identify areas where performance is less than optimal, which can inform quality improvement. For ONE Rx, it is clear that increasing the reach and 5

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

implementation are necessary in order to increase population impact. One consideration for increasing reach is to report the level of reach to each of the participating pharmacies. This may or may not be welcomed. Efforts to increase implementation have been tried with moderate success. The team is currently designing research to explore the characteristics of pharmacies who have adopted, but not yet implemented, ONE Rx. Among all the naloxone-related laws implemented, only laws allowing direct dispensing of naloxone by pharmacists have been useful to reduce fatal overdoses.39 The ONE Rx program is optimally placed in community pharmacies within North Dakota due to prescriptive authority granted to pharmacists and the rural nature of the population. These factors allow pharmacists to facilitate healthcare services to communities that otherwise may be difficult to serve. Implementing ONE Rx, or a program like it, in other states would require three main forms of support: political, technical, and financial. Political support of the state pharmacist associations and boards of pharmacy lend credibility to the program and send social cues that the approach is supported. Technical support would be needed in the form of a team of consultants able to facilitate the training, provide detailing to pharmacies as they implement ONE Rx, and perform data analytics. Financial support to reimburse pharmacists for the implementation of the program would also be essential. Healthcare organizations have called for health systems to assess key determinates of health including health behaviors, mental health conditions, and social measures in addressing actionable interventions in preventative healthcare.40 The approach to patient risk stratification used in this study was designed to provide the pharmacist critical patient information in order to determine individualized care for each patient through objective assessment of risk. This ability to care for patients based on their true risk profile increased the specificity of the care provided. Interventions such as the ONE Rx program are innovative in expanding upon education and consultation techniques to promote information delivery and additional exposure to pharmacy delivered services in the promotion of public health to communities.41 The systematic nature of ONE Rx, coupled with the burdensomeness of the opioid crisis, makes for a high likelihood that some form of the program will be adopted in other states in the future. The ONE Rx program has met this challenge by providing community pharmacists with an objective and evidence-based upstream prevention program. The ONE Rx program provides for various avenues to identify individual patient potential for opioid misuse and/or accidental overdose upstream of negative outcome. The results of screenings provide pharmacists accessibility to objective information that has not been previously available in the community pharmacy setting: comorbidities, family history of substance abuse, personal history of substance abuse, psychological diagnoses, and contaminant medications or substances that may be contraindicated with the use of opioids due to risk of respiratory depression. The ONE Rx program was able to provide training and change pharmacist perceptions of and predicted behaviors associated with the opioid crisis.21 As of June 1, 2019, there was a 41.5% increase in the number of pharmacist authorized to prescribe naloxone in North Dakota, from 164 prior to ONE Rx, to 232 presently. ONE Rx equips pharmacists with objective tools to care for patients receiving opioid prescriptions and energize community pharmacists to become more proactively engaged in preventing opioid misuse and overdose.40 The program reported here is a step forward in improving population health through community pharmacy by equipping pharmacists with necessary tools, and connecting pharmacists with other agencies and partners also engaged in the opioid crisis.37,38,42

quality and consistency of the implementation of ONE Rx at the pharmacy level. Second, the efficacy measure used in ONE Rx has not been previously validated, and the delivery of interventions is only an intermediate outcome of the program, without evidence of improved long-term health outcomes of patients. Finally, for technical reasons, the calculation for reach relied on a sample of the participating pharmacies, and not all pharmacies, so it is an estimate and may not be representative of all participating pharmacies. Conclusion Population impact requires considerable effort and extensive collaboration. In order for community pharmacy to optimize population level impact, collaborations and the focus of attention will need to move beyond the individual patient focus. The ONE Rx program has successfully demonstrated that this is feasible, and by use of the RE-AIM model, has stood up to vigorous population-level evaluation. The domains of the RE-AIM framework have shown the strengths of efficacy, adoption and maintenance, and the areas for future improvement of reach and implementation. This allows the ONE Rx team to modify the ONE Rx program based on the comprehensive approach of the RE-AIM model to continue to make improvements for optimal impact. Pharmacies across the country have here a model to emulate to join in the cause to prevent opioid misuse and accidental overdose and to ameliorate the population health consequences they have wrought. CRediT authorship contribution statement Mark A. Strand: Conceptualization, Writing - original draft, Methodology, Data curation, Formal analysis, Funding acquisition. Heidi Eukel: Funding acquisition, Writing - review & editing, Methodology. Oliver Frenzel: Data curation, Writing - review & editing, Formal analysis, Project administration. Elizabeth Skoy: Methodology, Writing - review & editing, Project administration. Jayme Steig: Data curation, Writing - review & editing, Formal analysis, Project administration. Amy Werremeyer: Validation, Writing review & editing. Acknowledgements The authors report no conflicts of interest. Funding support was provided by North Department of Human Services, the Blue Cross Blue Shield Caring Foundation, and the Alex Stern Foundation. Financial support was not provided for writing of the report or decision to submit the article for publication. The authors acknowledge North Dakota State University students Sydney Mosher, and Kimberly Schaible, PharmD candidates, and John Seiffert, MPH candidate, for their collaboration in this program. Appendix A. Supplementary data Supplementary data to this article can be found online at http://10. 1016/j.sapharm.2019.11.016. References 1. National survey on drug use and health. Accessed at https://nsduhweb.rti.org/ respweb/homepage.cfm, Accessed date: 19 March 2019. 2. CDC Injury Center. Understanding the epidemic. Accessed at https://www.cdc.gov/ drugoverdose/epidemic/index.html, Accessed date: 14 January 2019. 3. Sun EC, Darnall BD, Baker LC, et al. Incidence of and risk factors for chronic opioid use among opioid-naive patients in the postoperative period. JAMA Intern. Med. 2016;176(9):1286–1293. 4. Barocas JA, White LF, Wang J, et al. Estimated prevalence of opioid use disorder in Massachusetts, 2011–2015: a capture–recapture analysis. Am J Public Health. 2018;108(12):1675–1681. 5. Substance Abuse and Mental Health Services Administration. Key Substance Use and Mental Health Indicators in the United States: Results from the 2017 National Survey on

Limitations The ONE Rx program and this evaluation have some limitations. As a statewide, community-based program, it is difficult to monitor the 6

Research in Social and Administrative Pharmacy xxx (xxxx) xxx–xxx

M.A. Strand, et al.

6.

7. 8.

9. 10. 11. 12. 13. 14. 15.

16. 17. 18. 19. 20. 21. 22.

Drug Use and Health. Rockville, MD: Center for Behavioral Health Statistics and Quality; 2018 Accessed at https://www.samhsa.gov/data/sites/default/files/cbhsqreports/NSDUHFFR2017/NSDUHFFR2017.pdf, Accessed date: 21 August 2019. Council of Economic Advisors. The Underestimated Cost of the Opioid Crisis. Washington, D.C.: Office of the White House; 2017 Accessed at https://www. whitehouse.gov/sites/whitehouse.gov/files/images/The%20Underestimated %20Cost%20of%20the%20Opioid%20CrWoSs.pdf, Accessed date: 21 August 2019. Florence C, Luo F, Xu L, et al. The economic burden of prescription opioid overdose, abuse and dependence in the United States, 2013. Med Care. 2016;54(10):901. Palombi L, Hawthorne AN, Lunos S, et al. Community pharmacist utilization of legislation that allows impact on the opioid crisis in the state of Minnesota: a mixedmethods approach. J Pharm Pract. 2019. https://doi.org/10.1177/ 0897190019841747 First published online April 14, 2019. Puzantian T, Gasper JJ. Provision of naloxone without a prescription by California pharmacists 2 years after legislation implementation. J Am Med Assoc. 2018;320(18):1933–1934. Draper D. Preventing Drug Abuse: Low Participation by Pharmacies and Other Entities as Voluntary Collectors of Unused Prescription Drugs. Washington, DC: United States Government Accountability Office; October 2017. Steinman MA, Landefeld CS. Overcoming inertia to improve medication use and prescribing. J Am Med Assoc. 2018;320(18):1867–1869. Chen Q, Larochelle MR, Weaver DT, et al. Prevention of prescription opioid misuse and projected overdose deaths in the United States. J Amer Med Assoc Network Open. 2019;2(2) e187621-e21. Peavy KM, Banta-Green CJ, Kingston S, et al. “Hooked on” prescription-type opiates prior to using heroin: results from a survey of syringe exchange clients. J Psychoact Drugs. 2012;44(3):259–265. Spoth R, Trudeau L, Shin C, et al. Longitudinal effects of universal preventive intervention on prescription drug misuse: three randomized controlled trials with late adolescents and young adults. Am J Public Health. 2013;103(4):665–672. Cucciare MA, Scarbrough CB. Opportunities for identifying and addressing unhealthy substance use in rural communities: A commentary on Cucciare et al. Subst Abuse; 2017https://doi.org/10.1177/1178221818805980 2018; published on-line, accessed , Accessed date: 18 November 2019. Nielsen S, Kowalski M, Wood P, et al. Routine opioid outcome monitoring in community pharmacy: pilot implementation study protocol. Res Soc Adm Pharm. 2018;15(8):1047–1055. Livet M, Haines ST, Curran GM, et al. Implementation science to advance care delivery: a primer for pharmacists and other health professionals. Pharmacotherapy. 2018;38(5):490–502. Estabrooks PA, Brownson RC, Pronk NP. Dissemination and implementation science for public health professionals: an overview and call to action. Prev Chronic Dis. 2018;15(E162). Estabrooks P. What is RE-AIM? Accessed at http://www.re-aim.org/about/what-isre-aim/, Accessed date: 1 July 2019. ONERx. Accessed at https://onerxproject.org/, Accessed date: 19 June 2019. Eukel HN, Skoy E, Werremeyer A, et al. Changes in pharmacists' perceptions after a training in opioid misuse and accidental overdose prevention. J Continuing Educ Health Prof. 2019;39(1):7–12. Strand MA, Eukel H, Burck S. Moving opioid misuse prevention upstream: a pilot study of community pharmacists screening for opioid misuse risk. Res Soc Adm Pharm. 2018;15:1032–1036.

23. Strand MA, Eukel H. A primary prevention approach to the opioid epidemic. Am J Public Health. 2019;109(6):861–863. 24. Webster LR, Webster RM. Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the Opioid Risk Tool. Pain Med. 2005;6(6):432–442. 25. Skoy E, Eukel H, Werremeyer A, et al. Implementation of a statewide program within community pharmacies to prevent opioid misuse and accidental overdose. J Am Pharm Assoc. 2019 First published online Oct 31, 2019. 26. NDCC 28-32-02, 43-15-10. Bismarck, ND North Dakota Century Code. Limited Prescriptive Authority for Naloxone. July 1, 2016; July 1, 2016. 27. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. 28. Northridge ME, Metcalf SS. Enhancing implementation science by applying best principles of systems science. Health Res Policy Syst. 2016;14(1):74. 29. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–2645. 30. Centers for Disease Control and Prevention. Opioid overdose: communicating with patients. Accessed at https://www.cdc.gov/drugoverdose/training/communicating/ , Accessed date: 28 June 2019. 31. Hagemeier NE, Murawski MM, Lopez NC, et al. Theoretical exploration of Tennessee community pharmacists' perceptions regarding opioid pain reliever abuse communication. Res Soc Adm Pharm. 2014;10(3):562–575. 32. Marlowe KF, Geiler R. Pharmacist's role in dispensing opioids for acute and chronic pain. J Pharm Pract. 2012;25(5):497–502. 33. Hartung DM, Hall J, Haverly SN, et al. Pharmacists' role in opioid safety: a focus group investigation. Pain Med. 2017;19(9):1799–1806. 34. Klimas J, Gorfinkel L, Fairbairn N, et al. Strategies to identify patient risks of prescription opioid addiction when initiating opioids for pain: a systematic review. J Amer Med Assoc Network Open. 2019;2(5) e193365-e65. 35. Altpeter M, Gwyther LP, Kennedy SR, et al. From evidence to practice: using the REAIM framework to adapt the REACHII caregiver intervention to the community. Dementia. 2015;14(1):104–113. 36. Planas LG. Intervention design, implementation, and evaluation. Am J Health Syst Pharm. 2008;65(19):1854–1863. 37. Kuo T, Gase LN, Inkelas M. Dissemination, implementation, and improvement science research in population health: opportunities for public health and CTSAs. Clin Transl Sci. 2015;8(6):807–813. 38. Sweet SN, Ginis KAM, Estabrooks PA, et al. Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships. Implement Sci. 2014;9(1):74. 39. Abouk R, Pacula RL, Powell D. Association between state laws facilitating pharmacy distribution of naloxone and risk of fatal overdose. JAMA Intern. Med. 2019;179(6):805–811. 40. Harden SM, Smith ML, Ory MG, et al. RE-AIM in clinical, community, and corporate settings: perspectives, strategies, and recommendations to enhance public health impact. Front Public Health. 2018;6. https://doi.org/10.3389/fpubh.2018.00071 Article 71. 41. Bratberg JP, Smothers ZPW, Collins K, et al. Pharmacists and the opioid crisis: a narrative review of pharmacists' practice roles. Pharmacotherapy. 2019 On-line first https://accpjournals.onlinelibrary.wiley.com/doi/full/10.1002/jac5.1171. 42. Blewett LA, Owen RA. Accountable care for the poor and underserved: Minnesota's Hennepin Health Model. Am J Public Health. 2015;105(4):622–624.

7