The VHA New England Medication Error Prevention Initiative as a Model for Long-Term Improvement Collaboratives

Methods, Tools, and Strategies

The VHA New England Medication Error Prevention Initiative as a Model for Long-Term Improvement Collaboratives

Timothy S. Lesar, Pharm.D. Ernest R. Anderson, Jr., R.Ph., M.S. John Fields, R.N. Deborah Saine, R.Ph., M.S. Jill Gregoire, R.N., C.C.R.N. Susan Fraser, R.Ph. Maureen Parkin, R.N. Arnold Mattis, R.N., Ed.D. For the VHA New England Medication Error Prevention Initiative

uality improvement collaboratives (QICs) have been widely adopted by health care institutions wishing to improve their performance.1–18 A number of reports have described substantial gains in target quality measures as a result of participation in QICs.2,6,7,9–17 On the other hand, not all organizations improve equally, or measurably, from QIC participation.14–16,18 Given the importance of improving the safety, quality, and efficiency of health care delivery, and the time, cost, and human resource commitment required by QICs, it is necessary to examine QIC design, structure and application to determine factors related to success.19–30 Disparate findings from reported QIC efforts raise the question: why do some QICs succeed in producing changes in participating organizations, whereas others fail to reach measurable improvements in outcome? The design, structure, context, and operation of a QIC may be important and interdependent determinants of the presence and effectiveness of the various QIC components critical for success. One of the most commonly used QIC model structures is that developed by the Institute for Healthcare Improvement and applied in the “Breakthrough Series” collaboratives (IHI-BTS). 1,2,13–16,19–29 The IHI-BTS QICs commonly

Q

February 2007

Article-at-a-Glance Background: Quality improvement collaboratives (QICs) are a widely applied strategy for implementing change in health care organizations. Alternative collaborative methodologies were compared to gain insight into the elements important for QIC success. Methods: A modified version of a previously described QIC evaluation tool was used to assess the methods and characteristics of the Medication Error Prevention Initiative (MEPI) and to compare MEPI with two other long-term ongoing QICs—the VermontOxford Network’s Neonatal Intensive Care QIC and the Northern New England Cardiovascular Disease Study Group, and the shorter-term Breakthrough Series QICs of the Institute for Healthcare Improvement (IHI). Results: The modified QIC assessment tool was a useful framework for QIC assessment and comparison. The MEPI differed in scope of topic, team members, and the method for learning about and making improvements. Conclusions: Long-term QIC methods such as those used by MEPI may be particularly applicable when QICs address broad, complex, comprehensive, or organizationwide improvement needs.

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

73

ISMP Medication Safety Self Assessment Scores for MEPI Collaborative Hospitals

http://www.nnecdsg.org).17,30 By identifying similarities and differences of alternative collaborative models, better insight into the elements potentially important for QIC success and the selection of a specific QIC structure, or complementary application of multiple structures for a given improvement initiative may be gained.

Methods Comparative Assessment Method

Figure 1. The average (horizontal line) and range (vertical line) of ISMP Medication Safety Self Assessment scores for MEPI collaborative hospitals is shown for MEPI collaborative hospitals completing assessment in 2000, 2002, 2003, and 2004. MEPI, Medication Error Prevention Initiative.

address rapid implementation of improvements in narrow “targeted” processes during a finite period. Other QICs that use a long-term continuous10 or sequential9,17,30 structure have been well described and reported to result in positive outcomes. Using structured interviews, Mills19 and Neily16 identified a number of characteristics of successful IHI-BTS quality improvement teams, whereas Horbar et al.,9 Wilson et al.,21 and Øvretviet24 proposed potential overall QIC features that are associated with success. The VHA New England Medication Error Prevention Initiative collaborative (MEPI)10 developed a long-term continuous QIC structure to successfully improve the medication use system processes in participating hospitals (Figure 1, above). This article describes a systematic assessment of MEPI and compares MEPI methods and structure with those of the IHI-BTS QIC model and of two other well described QICs that, like MEPI, use a long-term ongoing structure—the Vermont-Oxford Network’s Neonatal Intensive Care QIC (NIC/Q; http://www.vtoxford.org/home.aspx?P= quality/nicq/index.htm) and the Northern New England Cardiovascular Disease Study Group (NNECDSG;

74

February 2007

A structured QIC assessment tool, based on “seven key components” of collaborative success,22 was modified slightly to expand and delineate proposed features influencing QIC team and collaborative success, as identified by Horbar et al.,9 Mills and Weeks,19 Neily et al.,16 and Øvretviet et al.* The MEPI collaborative structure and process was evaluated by the authors against the identified “key” components to determine possibly important methodological features and the predicted strengths and weaknesses for each of these components. The opinions of the entire MEPI work team were solicited at multiple meetings in first quarter 2005 through structured and unstructured interviews and a structured group discussion. On the basis of the described assessment tool, MEPI10 was then compared with the previously described IHIBTS,22 NIC/Q,9 and NNECDSG17,30 collaborative structures and methods: ■ The IHI-BTS QICs are typically short-term focused QICs designed to create “breakthrough” changes in the targeted process area. ■ The NIC/Q QIC focuses on teaching participants fundamental “key habits for improvement.” Local organizational teams address self-selected topics for improvement, while multiorganizational “focus groups” form a QIC around specific improvements in neonate intensive care.9 ■ The NNECDSG17,30 is a regional group of hospitals performing cardiac surgery that continuously shares outcome data and identifies and shares best practices through collaborative interactions and structured site visits. * The table showing the critical dimensions and determinants of success for the seven key components can be obtained by e-mail request from the authors.

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

Table 1. General Description of the Medication Error Prevention Initiative (MEPI)* Collaborative of the VHA New England 1. Identification of broad organizational functional process with important gaps between best practices and those found in most organizations. 2. Identify a relevant established set of best practices or quality measures/goals for organizational function targeted for improvement. 3. Participating organizations/teams are recruited from limited regional area. 4. Participating organizations identify one primary and one alternate to serve as representative on collaborative “working group.” 5. Participant representative(s) attends regular (9–11 per year) meetings of the collaborative held in a centrally located site for the regional participants. 6. Each organization performs baseline “self-assessment” comparing their practices to established “best practices.” Results are tabulated for each organization and provided to all participants in a blinded fashion. High and low performers for each best practice or performance criteria are identified. The group collectively prioritizes order in which the best practices will be analyzed. 7. The working group systematically evaluates and learns about chosen set of best practices or quality

The comparison of QIC structures and methods was used to explore potentially important considerations for selecting and modifying available QIC models for specific health care organizations and quality improvement initiatives.

Results Assessment of the MEPI Collaborative using the “Seven Key Components” The description of the MEPI collaborative based on the modified seven key components is summarized in Table 1 (above). In general, the structure, methods, and factors critical to the success of MEPI were well defined within this framework. Application of this systematic QIC assessment method allowed us to identify and categorize specific component variations deemed potentially critical to success of the collaborative and determine major similarities and differences from other QIC structures.

February 2007

8.

9.

10.

11. 12.

measures/goals to establish highly specific ideas to improve their performance and meet best practices criteria. Methods for implementation and overcoming barriers are shared between organizations on an ongoing basis and codified in writing and placed in a centralized database. Participant representative takes tools, information, and lessons learned to home organization and works to implement through participating organization’s own safety improvement structures. Specific best practices are applied by each participant based on opportunity available within his or her individual organization. The pace of collaborative and specific learning efforts is determined by continual assessment of needs. Additional learning materials/presentations and relevant improvement topics are added as needed. Self-assessment is repeated by participants at set intervals; results are then shared within collaborative. Collaborative continues to meet on an ongoing and indefinite basis, driven by continued need for improvement and new developments.

* Lesar T.S., et al.: Using the ISMP Medication Safety Self-Assessment to improve medication use processes. Jt Comm J Qual Saf 29:211–226, May 2003.

Assessment of Seven Key Components The descrption of the MEPI QIC characteristics based on the seven key components is shown in Table 2 (pages 76–77). Sponsorship. A number of factors related to MEPI sponsorship were considered important to success. The QIC sponsor of MEPI (VHA New England, a regional office of the national hospital services alliance) was viewed as being “aligned” with, and under substantial control or influence of, member hospitals. Senior leaders and others within participating organizations were familiar with the sponsor. The mission, leadership, staff, and activities of the sponsor improved initial acceptance of QIC mission and development of important social dynamics, thereby promoting information and data sharing. In short, the participating organizations and the sponsor possessed preexisting business, professional, and social relationships, which allowed the MEPI activities to be

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

75

Table 2. Description of the MEPI QIC Characteristics Based on “Seven Key Components”* Sponsorship ■ Collaborative sponsored by the regional (New England) office of the VHA and offered as a service to members Topic ■ Topic of MEPI (medication safety) clearly a high priority and practical relevance for participants ■ Topic is highly complex and extremely broad in scope—the entire medication use system was the organizational function targeted ■ Many widely accepted “best practices,” goals, and recommendations for improvement available ■ Recognized method for structured organizational selfassessment and participant comparisons in comparison with accepted standards or best practices available as structural framework for the collaborative (MEPI used the ISMP Medication Safety SelfAssessment.) Ideas for Improvement ■ Ideas for improvement developed following collaborativewide review of standards/goals/best practices, current practices, and barriers to improvement ■ Specific best practices goals/standards for collaborative established by consensus ■ Specific ideas for successfully implementing best practices generated in great detail from within collaborative by members rather than relying on topic experts or consultants ■ Options for adoption and modification to “fit” individual participating organizations always considered and included ■ Consensus best practices/standards and goals as well as specific ideas for change generated on ongoing basis, creating recommendations for improvements throughout targeted function (e.g., additional recommendations for improvement developed faster than adoption of previously established recommendations by participants) ■ Participants individually chose which recommendations to implement within their organizations and when, based on opportunities, resources, and timing Participants and Teams† ■ Participating organizations enrolled from a limited geographic region served by the sponsor (New England VHA)

76

February 2007

■

■ ■

■

■

■ ■ ■ ■ ■ ■

Participating organizations varied widely in size and structure (small community as well as large teaching institutions) Team size is generally one or two individuals consistently attending meetings Background of individual representing participating organization varies—nursing, pharmacy, administration, quality management Quality improvement experience variable, but collaborative includes a number of individuals with considerable expertise and experience Representative(s) from each organization integrally connected to quality improvement structure of entire participating organization Representatives are, or have close ties with, front-line staff Structured, systematic, and comprehensive organizational self-assessment required Participants responsible for collection and submission of self-assessment data to collaborative Pace of improvements determined individually by participants, no “timeline” for improvement Organizations varied widely in ability to rapidly adopt improvements Collaboration direction, process, and priorities set by participants based on identified needs

Senior Leadership Support and Resources ■ Senior leadership support established at collaborative outset ■ Organization representative have close organizational and communication links with senior leaders ■ Collaborative reported to sponsor’s board of directors, many of whom were senior leaders at participant organizations ■ Sponsor provided ongoing presentations regarding the collaborative activities to senior leadership site visits, phone calls, presentations at meetings, retreats, special meetings, site visits) ■ Participating teams expected to (and supported in) promote leadership participation through reports and sharing of collaborative activities ■ Primary direct resource commitment for participating organization is assigned individuals and minor travel costs; limited monetary resources required to participate

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

Table 2. Description of MEPI QIC Characteristics Based on “Seven Key Components”* (continued) ■

Internal organization resources based on specific improvements targeted thus allowing progress to be made by leveraging available organization resources and structures

Pre-Work ■ Each participating organization required to form or use existing internal medication safety team to periodically complete a structured self-assessment (ISMP Medication Safety Self-Assessment) and submit data to the collaborative and to consider and actively promote implementation of ideas for improving safety generated by the MEPI Collaborative Methods and Structures for Learning About and Making Improvements ■ Frequent meetings (approximately monthly) ■ Ongoing meetings (collaborative initiated in 1999) ■ High consistency of individuals at meetings ■ Collaborative meets as a group of individuals—not as a group of individual teams ■ Priorities and discussion are “data-driven,” but nature of self-assessment data required is readily obtained by all participants.

built on and interwoven and “leveraged” with these other activities rather than be a stand-alone activity. The success and progress of MEPI could be discussed and presented at various meetings of the sponsor. On a long-term basis, participants generally viewed the sponsorship of MEPI by the regional alliance as a “positive” because it provided a sense of ongoing and long-term commitment to participating individuals and organizations. This MEPI characteristic of an “overarching” sponsor or other type of association structure closely binding QIC participants together over a long period is similar to, but broader than, that of NICQ and NNECDS but is not as consistent a factor in IHI-BTS collaboratives. Topic. As a topic for a QIC, improving medication safety is not unique. However, the organizational scope and depth of the improvement target—that of implementing scores of best practices and care standards throughout an organization’s entire medication use process—is unique. The complexity and size of the topic required systematic learning of the processes, connec-

February 2007

■ ■ ■ ■ ■ ■ ■

■ ■

Collective “learning” stressed, rather than teaching Major emphasis on both “what” and “how” to implement changes Fundamental sustainable system changes prioritized Close monitoring and feedback from participants at monthly meetings Includes learning about role of participants in developing fundamental organization improvement processes Self-assessment measurement creates objective method of comparison within and outside of collaborative Ongoing “benchmarking” and progress measurements within collaborative provided by repeat completion of self-assessment Ongoing learning of “how to” make changes promotes spread to additional targets for improvement Strong ongoing collaborative coordination and “central support” provided by sponsor limiting need for members to expend time and “resources”

* MEPI, Medication Error Prevention Initiative. The “Seven Key Components” are drawn from Wilson T., Berwick D.M., Cleary P.D.: What do collaborative projects do? Experience from seven countries. Jt Comm J Qual Saf 29:85-93, Feb. 2003. †

Adapted from the component “Participants.”

tions, and interdependencies of multiple components of the medication use process, and in effect, determined the structure of MEPI. MEPI was required to tailor collaborative and collective learning, team development, and skills development to each individual representative rather than adopt a more “prescriptive” approach, as is used in many limited-scope QICs (both short-term and long-term). The NIC/Q and NNECDSG also addressed numerous highly complex processes, but the organizational scope was limited (neonate intensive care for NIC/Q and cardiovascular surgery for NNECDSG). The broad topic approach creates a “something for everyone” atmosphere that does not eliminate organizations with different improvement needs or that are not able to address a given problem at the same time or pace as others in the collaborative (as noted in the NIC/Q QIC9). A disadvantage of the MEPI QIC choice of a broad topic was the need for long-term commitment, which is not required when QICs address limited processes or patient populations (for example, the IHI-BTS).

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

77

Ideas for Improvement. MEPI developed ideas for improvement from multiple external and internal sources. The IHI-BTS QIC model often uses or promotes substantially predetermined ideas for change, with some modification at the organizational level. In MEPI, goals for improvement were established at the collaborative’s outset, but specific ideas for improvement were not defined. Specific ideas for improvement were primarily developed by participants in multiple shared learning sessions, often generating “new knowledge” and ideas. Ideas for change emerged from demonstration of gaps between current and best practices on the basis of shared data from standard self-assessment tools and collective learning about systems, problems, and opportunities, with open and candid discussion of potential strategies to overcome barriers. This was followed by definition and understanding of (often multiple) potential best practices, and sharing explicit examples of applicable successful change processes within member organizations. The MEPI work team’s ideas for improvement were further nurtured, guided, and organized by sponsor–provided quality improvement and medication safety experts on a limited and ad hoc basis. In MEPI, specific ideas for change were developed within the larger context of the entire medication use process, thus revealing important interdependencies and interaction within systems. The process of synthesis and development of ideas for change provided individual participants with not only specific and detailed workplan options to make improvements, but also the knowledge to alter and adapt ideas to their organization’s unique culture and special circumstances. This knowledge gained was considered critical to further “spread” changes, as other opportunities are identified within their organization. A somewhat similar approach is used in the two other long-term QICs. Shorter-term QICs might be less likely to develop necessary team dynamics between its participants and achieve the high level of team performance of ongoing long-term QICs.9,17,18 A clear disadvantage of the MEPI approach to developing ideas for change is the prolonged time needed to create effective “team” dynamics and generation of the improvement ideas themselves. The task often seemed overwhelming due to its size and complexity. The MEPI method of generating change ideas and initiatives produced practical and robust improvement solutions but

78

February 2007

required persistent focus, team leadership, and facilitation skills. Although this learning method may be uniquely creative and unbounded, it is also prone to wander from the task-at-hand at any given point and requires a skillful facilitator to guide the creative process. Optimal group social dynamics that produce this type of learning may be difficult to achieve for short-term collaboratives such as the IHI-BTS. Participants. The MEPI actively sought to create a broad-based multidisciplinary collaborative by deliberately recruiting representatives from stakeholder disciplines and departments (nurses, pharmacists, quality managers, administrators, risk managers, physicians). Members had to be not only close enough to the workflow to understand it intimately but also in a position to effectively make or lead needed organizationwide changes and to communicate with senior leadership. This approach to participation avoids the need for each participating organization to provide a full complement of disciplines in teams for the collaborative, yet it produces a forum where multiple disciplines, departments, and levels of staff and management are present. Extremely high level of trust and communication developed with little “social distance” between representatives from different organizations. Participants also develop a willingness to help other organizations and a sense of responsibility for the ongoing success of the collaborative, something noted in other long-term QICs.9,17 The experience of MEPI suggests that this approach of forming a “virtual hospital staff organization” within a QIC can be successful within the context of a long-term collaborative. A disadvantage of the MEPI QIC structure is the need for long-term commitment of a small number of individuals who may need additional support and resources to fulfill other “at home” responsibilities. The trade-off for work team members occurs if the collaborative “deliverables” assists them in performing their jobs or if they make changes that would have to be made whether participating in a collaborative or not. The MEPI was not successful in sustaining voluntary or routine participation by member physicians. The absence of sustained physician commitment, likely the result of a number of factors, may primarily reflect the use of the “broad” topic of medication safety. Physicians are possibly less likely to accept ownership for processes for which they have

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

limited control and that are not central in their day-today work. It is well recognized that most hospital operations, and especially improvement efforts, suffer from lack of physician involvement.31 Both short- and longterm medical service—based QICs typically have a more narrow patient population focus, which inherently may enlist the participation of the physicians affected. Nonetheless, MEPI works to understand and promote physician participation in the medication safety efforts at every MEPI member organization, even if physician participation at MEPI meetings is sporadic. Leadership Support. Building and maintaining leadership support was an ongoing effort for MEPI, with resulting long-term commitments by the vast majority of participating organizations. Despite a prolonged timetable, MEPI demonstrated consistency and clarity of goals and deliverables and provided a measurable outcome with which senior leadership could judge success. Work team members consistently considered leadership support to be instrumental to progress and used MEPIgenerated data and ideas to nurture support. MEPI in turn promoted leadership support through multiple activities of the sponsor, the collaborative, and an MEPI organizational representative(s). A disadvantage of addressing a broad topic as in MEPI is maintaining visibility to leadership when implementing many incremental, fundamental, and diffuse improvements and not necessarily demonstrating “dramatic” concentrated improvements. The IHI-BTS QICs have built-in requirements for leadership support, and the medical service-based NIC/Q and NNECDSG naturally foster greater physician leadership involvement. Because of the broad topic of MEPI and sporadic key physician involvement, attaining and engaging active and sustained leadership support may be even more critical to the success of MEPI than other long-term QICs. Doing Pre-Work. The MEPI used a comprehensive self-assessment (Institute of Safe Medication Practices [ISMP] Medication Safety Self Assessment®32,33) for objective baseline measurement and ongoing benchmarking of medication safety processes. Self-assessment can serve as an important learning tool to educate and motivate organizations.9 MEPI participants found considerable value in using a consistent comprehensive organizational self-assessment and comparison with best practices as collaborative “pre-work” (preliminary

February 2007

work). To complete such an assessment, a multidisciplinary team must be formed with representatives from across an organization (if one did not already exist). The act of completing a self-assessment is instructive for the organization, making the entire medication process more visible, prompting reflection on current practices, and permitting goals to be set for the organization. The completion of the same type of self-assessment has been instrumental in the educational process and prioritization of improvements, and similar pre-work and ongoing self-assessment or data collection appear to be fundamental components of other QICs.9,17,30 Strategies for Learning About and Making Improvements. A major divergence of the MEPI, NIC/Q, and NNECDSG QICs from the most typical IHI BTS model is obviously the long-term and ongoing, rather than short-term and finite life of the collaborative. The collaborative duration, ongoing activities, and open structure possibly provide greater opportunity for organizationwide impact. Once the work team became a cohesive group, the structure of MEPI essentially created a self-directed process of learning and developing ideas for change within the defined framework, allowing the group to focus on and learn about issues most important to them. The MEPI collaborative is structured to achieve a set of goals but does so without necessarily having a set timeline. This flexibility is considered necessary to allow coherence within the collaborative despite vastly different organizational size, structure, technology, resources, culture, individual representative knowledge, experiences, and perspectives. Learning needs are identified and fulfilled as the collaborative progresses through its work. Emphasis is placed on shared learning, perspectives, and experiences rather than on substantial reliance on external experts or externally generated specific recommended solutions. The small group (fewer than 30 persons per meeting) of largely consistent individual participants meeting frequently (usually monthly) nurtured close social exchange. The ongoing and consensus-driven processes of MEPI allow for differences in learning styles and needs within the work team and their organizations. The continuous QIC process of MEPI develops skills of participants, providing each participating organization with individuals highly skilled in overall quality improvement,

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

79

and the knowledge and confidence to champion medication safety at their facilities. Such individual participant development is a major strategy of NIC/Q9 and a noted outcome of NNECDSG.17 The success of MEPI was more reliant on the skills of one or two representatives from each organization than QIC models designed for participation by teams.

Discussion Systematic QIC assessment demonstrates how key components of QIC structures and methods are highly interdependent. Overall, the MEPI collaborative process, as well as the similarly long-term NIC/Q and NNECDSG QICs, incorporates many of the same components and “critical dimensions” as the IHI-BTS, but some clear differences were noted. The topic of medication use was not unique, but the comprehensive scope and depth of targeted changes and approach to the topic (using the ISMP Self Assessment32,33 as a measurement tool and a framework for identifying best practices) was. A second major feature differentiating the three long-term QICs from the IHI-BTS model was the more dominant use of internal generation of knowledge and ideas. These primary differences determined the need for a third major differentiating feature—the use of a prolonged continuous structure. The long-term QIC model also appears to allow participants the benefits of learning from, and expanding on, previous efforts within the framework of such QICs. Our assessment suggests that the presence of a sponsor “binding” all participants may be an important factor promoting long-term efforts. Can QICs addressing broad based topics be successful?23,24 Our experience suggests that they can be, given the use of appropriate methods. For broad, far-reaching, comprehensive, and complex topics, and for topics with poorly defined standards of care, long-term or sequential QICs may be advantageous. For broad organizational efforts, a QIC consisting of individuals representing broad interests, and a structure based primarily on learning through collective knowledge and experience may provide additional foundations on which to identify and build specific process improvements. To achieve the necessary effective learning environment within a collaborative, however, the proper social processes need to be developed. Longer-term or continuous QIC

80

February 2007

approaches allow greater time for the development of important social dynamics between participants, enhanced knowledge and understanding by participating individuals, cultural change within organizations, and the implementation and testing of changes in complex systems.9,34,35 Certain similar structural characteristics can be identified in the successful NIC/Q,9,11 NNECDSG,17,30 and MEPI QICs—availability of reliable data across organizations, preference for collective learning and sharing over predetermined ideas for change, and provision of a number of QI target options to choose from for local reinvention/modification.34 Sustained improvement, spread of change within organizations, and continuation of internal organizational teams may all be important measures of QIC success.15,24,34 The results of our assessment of MEPI support the notion that social processes within collaboratives play a critical role in determining success.9,11,27,34,35 Leveraging social relationships already formed within pre-existing structures or by creating structures such as a hospital alliance, interest group, or a coalition (as with MEPI, NIC/Q, and NNECDSG) may be one method that QICs can use to improve participation and outcomes.35 The IHI 100,000 Lives Campaign14 also similarly promotes the engagement of local and regional health care associations in promoting improvements. When possible, these efforts should be leveraged to establish close ongoing relationships between participating organizations that can be used in future improvement efforts. A number of lessons learned in evaluating QICs are likely applicable to organizations not involved in ongoing collaboratives. Most importantly, improving medication safety in a comprehensive way may be accelerated through a systematic long-term organizationwide effort guided by use of self-assessment and continuously supported by leadership. Individuals responsible for improving medication should be given appropriate time and resources to tackle the many challenges facing them. Such an effort could be implemented through existing medication safety improvement structures within the organization. Work should address the fundamental organization safety issues uncovered through organizational self-assessment that are unlikely to be addressed by more focused-issue

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

groups. Improving the fundamental safety processes should directly improve patient safety and increase the efficiency and success of any focused efforts undertaken within the organization. Our systematic assessment of MEPI is no doubt limited by the inherently biased and subjective nature of such exercises. However, objective testing of various QIC methods is extremely difficult, given the lack of effective measures of QIC components and the many variables influencing success. Nonetheless, important lessons can still be learned. The MEPI is applying knowledge gained from this comparative self-assessment. For example, to enhance leadership support, the collaborative has prompted the appointment of a senior-level executive to each local organization medication safety team. In addition, members of the QIC sponsor’s board of directors have been invited to attend MEPI meetings as guest observers to obtain a first-hand view of the QIC. To help leverage MEPI initiatives within participating organizations, annual workshops that include local medication safety teams and senior administrators and conference calls between local teams have been instituted to provide greater opportunity for sharing, learning, and an additional level of connectivity. Many of the findings in a recently published assessment of a statewide collaborative in Massachusetts using IHI-BTS methodology36 echo our observations. Overall, the experience of conducting this self-assessment persuaded us of the need to better understand and apply the potentially powerful tool of QICs for improving patient care.

Summary QICs’ characteristics and structure appear to affect the likelihood of a success. Our assessment of MEPI strongly suggests the structure and methods for a QIC should be modified to best fit a chosen topic for improvement. Longer-term QICs using structured assessment, measurement, and collective learning may provide advantages when addressing organizationally broad,

February 2007

comprehensive, or complex topics. All collaboratives should strive to provide greater opportunity for development of important social interactions and dynamics between participants. Increased knowledge of the critical components of QICs is necessary to more effectively and efficiently apply the collaborative strategy for improvement. J The authors thank Bill Vandenberg for reviewing the manuscript and James Cook for data management and acknowledge the many contributions of the MEPI Collaborative Working Group, in which each of the following hospitals were represented: Central Maine Medical Center (Lewiston, ME); Central Vermont Medical Center (Barre, VT); Cheshire Medical Center (Keene, NH); Concord Hospital (Concord, NH); Dartmouth-Hitchcock Medical Center (Lebanon, NH); Eastern Maine Medical Center (Bangor, ME); Fletcher Allen Health Care (Burlington, VT); Frisbie Memorial Hospital (Rochester, NH); Lahey Clinic (Burlington, MA); Maine General Medical Center (Waterville, ME); Maine Medical Center (Portland, ME); New London Hospital (New London, NH); North Country Hospital (Newport, VT); Northeastern Vermont Regional (St. Johnsbury, VT); Porter Hospital (Middlebury, VT); Penobscot Bay Medical Center (Rockport, ME); Rutland Regional Medical Center (Rutland, VT); Southern Maine Medical Center (Biddeford, ME); Southern New Hampshire Medical Center (Nashua, NH); Southwestern Vermont Medical Center (Bennington, VT); and Upper Connecticut Valley Hospital, Colebrook, NH).

Timothy S. Lesar, Pharm.D., is Director of Pharmacy, Albany Medical Center, Albany, New York. Ernest R. Anderson, Jr., R.Ph., M.S., is Pharmacist, Pharmacy Department, Lahey Clinic, Burlington, Massachusetts. John Fields, R.N., formerly Project Director, Medication Error Prevention Initiative, VHA New England, and Maine Quality Solutions, Portland, Maine, is Product Manager, A4 Health Systems, Cary, North Carolina. Deborah Saine, R.Ph., M.S., formerly Director of Pharmacy, Porter Medical Center, Middlebury, Vermont, is Medication Safety Coordinator, Winchester Medical Center, Winchester, Vermont. Jill Gregoire, R.N., C.C.R.N., formerly Nurse, Critical Care Unit, Concord Hospital, Concord, New Hampshire, is Director of Quality Assurance and Improvement, Indian Stream Health Center, Colebrook, New Hampshire. Susan Fraser, R.Ph., was Medication Safety Pharmacist, Maine Medical Center, Portland, Maine. Maureen Parkin, R.N., is Quality Management Department Manager, Southern Maine Medical Center, Biddeford, Maine. Arnold Mattis, R.N, Ed.D., is Healthcare Consultant, Dover, Delaware, and Project Director, Medication Error Prevention Initiative, VHA New England, Portland. Please address reprint requests to Timothy S. Lesar, [email protected].

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations

81

References 1. Kilo C.M.: A framework for collaborative improvement: Lessons from the Institute for Healthcare Improvement Breakthrough Series. Qual Manag Health Care 6:1–13, Sep. 1998. 2. Flamm B., Berwick D.M., Kabcenell A.: Reducing cesarean section rates safely: Lessons from a “breakthrough series” collaborative. Birth 25:117–124, Jun. 1998. 3. Kilo C.M.: Improving care through collaboration. Pediatrics 103(1 suppl E):384–931, Jan. 1999. 4. Plsek P.E.: Quality improvement methods in clinical practice. Pediatrics 103:203–214, Jan. 1999. 5. Carter S., Garside P., Black A.: Multidisciplinary team working, clinical networks and chambers: Opportunities to work differently in the NHS. Qual Saf Health Care 12:25–38, Dec. 2003. 6. Leape L.L., et al.: Reducing adverse drug events: Lessons from a breakthrough series collaborative. Jt Comm J Qual Improv 26:321–331, Jun. 2000. 7. Weeks W.B., et al.: Using an improvement model to reduce adverse drug events in VA facilities. J Comm J Qual Improv 27:243–254, May 2001. 8. Wagner E.H., et al.: Quality improvement in chronic illness care: A collaborative approach. Jt Comm J Qual Improv 27:63–80, Feb. 2001. 9. Horbar J., et al.: Collaborative quality improvement for neonatal intensive care. NIC/Q Project Investigators of the Vermont Oxford Network. Pediatrics 107:14–22, Jan. 2001. 10. Lesar T.S., et al.: Using the ISMP Medication Safety SelfAssessment to improve medication use processes. Jt Comm J Qual Saf 29:211–226, May 2003. 11. Horbar J.D., et al.: Collaborative quality improvement to promote evidence based surfactant for preterm infants: A cluster randomized trial. BMJ 329:1004–1010, Oct. 30, 2004. 12. Wang A., et al.: The North Carolina experience with the diabetes health disparities collaborative. Jt Comm J Qual Saf 30:396–404, Jul. 2004. 13. Daniel D., et al.: A state-level application of the chronic illness breakthrough series: Results from two collaboratives on diabetes in Washington state. Jt Comm J Qual Saf 30:69–79, Feb. 2004. 14. Resar R., et al.: Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Jt Comm J Qual Pat Saf 31:243–248, May 2005. 15. Landon B.E., et al.: Effects of quality improvement collaborative on the outcome of care of patients with HIV infection: The EQHIV Study. Ann Intern Med 140:897–901, Jun. 1, 2004. 16. Neily J., et al.: One-year follow-up after a collaborative Breakthrough Series on reducing falls and fall-related injuries. Jt Comm J Qual Pat Saf 31:275–285, May 2005. 17. Nugent W.C.: Building and supporting sustainable improvements in cardiac surgery: The Northern New England Experience. Semin Cardiothorac Vasc Anesth 9:115–118, Jun. 2005. 18. Homer C.J., et al.: Impact of a quality improvement program on care and outcomes for children with asthma. Arch Pediatr Adolesc Med 159:464–469, May 2005. 19. Mills P.D., Weeks W.B.: Characteristics of successful quality improvement teams: Lessons from five collaborative projects in the VHA. Jt Comm J Qual Saf 29:152–162, Mar. 2004.

82

February 2007

20. Cretin S., Shortell S.M., Keeler E.B.: An evaluation of collaborative interventions to improve chronic illness care. Eval Res 28:28–51, Feb. 2004. 21. Gandhi T.K., et al.: Obstacles to collaborative quality improvement: The case of ambulatory general medical care. Int J Qual Health Care 12:115–123, Apr. 2000. 22. Wilson T., Berwick D.M., Cleary P.D.: What do collaborative projects do? Experience from seven countries. Jt Comm J Qual Saf 29:85–93, Feb. 2003. 23. Øvretveit J: How to run an effective improvement collaborative. Int J Health Care Qual Assur 15:192–196, Sep. 2002. 24. Øvretviet J., et al.: Quality collaboratives: Lessons learned from research. Qual Saf Health Care 11:345–351, Dec. 2002. 25. Grimshaw J., et al.: Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care 12:298–303, Aug. 2003. 26. Palsbo S.E., Kroll T., McNeill M.: Addressing Chronic Conditions Through Community Partnerships: A Formative Evaluation of Taking on Diabetes (Fund Report). Commonwealth Fund, Sep. 2004. http://www.cmwf.org/publications/publications_show.htm?doc_id=239779 (last accessed Nov. 30, 2006). 27. Severens J.L.: Value for money of changing healthcare services? Economic evaluation of quality improvement. Qual Saf Health Care 12:366–371, Oct. 2003. 28. Mittman B.S.: Creating the evidence base for quality improvement collaboratives. Ann Intern Med 140:897–896, Jun. 1, 2004. 29. Leatherman S.: Optimizing quality collaboratives. Qual Saf Health Care 11:307, Dec. 2002. 30. O’Connor G.T., et al.: A regional intervention to improve hospital mortality associated with coronary artery bypass graft surgery. The Northern New England Cardiovascular Disease Study Group. JAMA 275:841–846, Mar. 20, 1996. 31. VHA: Physician Hospital Relations: Forging a New Covenant. VHA Research Series, Oct. 2004. VHA, Irving, Tx. (executive summary available at https://www.vha.com/portal/server.pt/gateway/ PTARGS_0_2_1534_341_0_43/http%3B/remote.vha.com/public/ research/newcovenant.asp). 32. Smetzer J., et al.: Findings from the ISMP Medication Safety SelfAssessment for Hospitals. Jt Comm J Qual Saf 29:586–597, Nov. 2003. 33. Institute for Safe Medication Practices: ISMP Medication Safety Self-Assessment® for Hospitals. http://www.ismp.org/selfassessments/ Hospital/Intro.asp (last accessed Nov. 30, 2006). 34. Bate P., Robert G., Bevan H.: The next phase of healthcare improvement: What can we earn from social movements? Qual Saf Health Care 13:62–66, Feb. 2004. 35. Green P.L., Plsek P.E.: Coaching and leadership for the diffusion of innovation in health care: A different type of multi-organization improvement collaborative. Jt Comm J Qual Improv 28:55–71, Feb. 2002. 36. Leape L.L., et al.: Developing and implementing new safe practices: Voluntary adoption through statewide collaboratives. Qual Saf Health Care 15:289–295, Aug. 2006.

Volume 33 Number 2

Copyright 2007 Joint Commission on Accreditation of Healthcare Organizations