- Email: [email protected]
A Multihospital Safety Improvement Effort and the Dissemination of New Knowledge
Joint Commission
Journal on Quality and Safety
Performance Improvement
A Multihospital Safety Improvement Effort and the Dissemination of New Knowledge
Peter D. Mills, PhD, MS William B. Weeks, MD, MBA, CHE B.C. Surott-Kimberly, CTRS
iffusion of innovation has been defined as the “process by which an innovation is communicated through channels over time among the members of a social system.”1(p 5) Traditionally, knowledge has been disseminated in schools, internships, and residencies and then through continuing education programs. Studies on the diffusion of new medications, technological innovations, and clinical guidelines2–7 suggest that the dissemination of medical innovation is driven more by interpersonal relationships and observation of peers than by new research or available information. Adoption of innovations tends to be slowed within capitated systems,5 and the use of unsafe or discredited practices by physicians may take a long time to extinguish.2 Since 1995 the Institute for Healthcare Improvement (IHI; Boston) has facilitated multihospital quality improvement efforts, called Breakthrough Series, which improve care in the majority of participating facilities.8–10 Another multihospital dissemination strategy is that of the Northern New England Cardiovascular Group,11 which continuously compiles data and provides feedback on best practices to the participating members. Our research has shown that the Breakthrough Series model is effective within the VA system.12–14 Despite these gains, it remains unclear whether innovations can be further disseminated beyond the change teams participating in a collaborative. No study has examined this type of secondary diffusion in a health care system. We had an opportunity to examine the secondary diffusion of process innovations beyond the participating change teams during a national collaborative within the Veterans Health Administration (VHA) in 2000–2001.
D
124
March 2003
Article-at-a-Glance Background: Research on the transfer of medical technology and guidelines suggests that this transfer is driven more by interpersonal relationships than by new research or available information and that it is inconsistent, largely unsuccessful, and strongly influenced by local factors. Yet studies of collaborative, multiple-hospital improvement efforts have shown these transfers to be effective for the specific microsystems participating in the project. The diffusion of medical innovations beyond the participating teams was studied during a 2000–2001 national collaborative safety improvement effort. Methods: Twenty-two teams from Department of Veterans Affairs (VA) hospitals participated in a 9month quality improvement project designed to improve safety in high-hazard areas. Participating hospitals and other regional hospitals were contacted to determine the level of dissemination of information generated during and after the project. Results: While the participating hospitals benefited from the quality improvement effort, changes were implemented only 9% of the time on other units within the hospitals and only 2% of the time in other regional hospitals. After 12 months, there was no implementation within participating hospitals, and other regional hospitals were implementing changes 10% of the time. Discussion: Personal commitment from senior leadership, dissemination strategies that push information to clinicians, and monitoring of progress at the regional level are all needed for dissemination of complex medical information to occur.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
The Institute of Medicine’s (IOM’s) 1999 report, To Err Is Human15 heightened awareness of patient safety and medical errors in the United States. The federal government’s Quality Interagency Coordination Task Force (QuIC) used the principles of systems change outlined in the IOM report to develop a detailed set of action plans designed to reduce medical errors and their impact.16 One such action plan called for participation in an IHI Breakthrough Series Collaborative that was designed to reduce medical errors in high-hazard environments.16 (See Sidebar 1, right.) This article describes a study on teams’ diffusion of process innovations to other hospitals within their health care systems. Each team was to focus on one high-hazard area, make local process changes that could demonstrably improve patient safety, and spread those changes both to the same high-hazard area in other locations and to other high-hazard areas within the participating hospital.
Methods The QuIC Initiative The QuIC initiative (Jul 2000–May 2001) was designed to reduce medical errors in high-hazard areas—labor and delivery suites, emergency departments, operating rooms (ORs), and intensive care units (ICUs). Experts in patient safety in each of these areas and experts in quality improvement constituted the faculty. During the first learning session, teams were taught specific changes they could make to improve safety in their facilities, how to measure improvements, and quality improvement techniques, and they chose specific safety goals. During the second face-to-face meeting, teams reported on the changes they had made. The faculty continued to provide information on measurement and key changes in the four high-hazard areas. In addition, the teams were encouraged to spread their changes to other units of the hospital or other hospitals in their Veterans Integrated Service Network (VISN).* During the third face-to-face meeting, teams reported gains, continued to learn from each other, and consolidated plans to disseminate what they had learned.
Sidebar 1. Breakthrough Series Collaborative A Collaborative Breakthrough Series is designed to facilitate the achievement of best practices through face-to-face knowledge distribution at three 2-day conferences and ongoing electronic and telephone support of participating institutions during a a 9-month period. After initiative’s end, teams continue to monitor their progress and are expected to help disseminate the new improvement knowledge broadly across their institution. In addition, the teams are expected to help spread the innovations to other hospitals in their health care systems.
Each month during the collaborative, teams reported their results on a Web site and participated in collaborative calls; listserv support was available throughout the collaborative. An overview of the Breakthrough Series Collaborative model is shown in the left part of Figure 1 (p 126).
Sample and Design Forty teams from federal health care organizations participated in the QuIC initiative. The initial sample consisted of the 22 VHA teams, representing 19 VISNs. Table 1 (p 127) provides a breakdown of the hospital teams by department and area of focus for improvement. We conducted a before–after study on the 22 VHA teams. One month after the second face-to-face meeting, in which teams were first encouraged to spread their process innovations, the research team began calling the VISNs to monitor the progress of the spread of information. One month after the third face-to-face meeting, in which teams had developed diffusion plans, we again called the VISNs to monitor the progress of spread. Finally, 12 months after the third meeting, we again called the VISNs to determine if further dissemination had taken place.
Interventions to Produce Secondary Diffusion * VHA is geographically divided into 22 VISNS. VISNs, which consist of between 4 and 11 VA medical centers, are intended to help standardize care and facilitate process knowledge diffusion within and across their geographic boundaries.
March 2003
We asked the director of each VISN to choose one team that had strong clinical skills and was a leader in the VISN that could serve as a model to spread the innovations
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
125
Joint Commission
Journal on Quality and Safety
QuIC Breakthrough Series on Reducing Errors in High-Hazard Areas Time Line
Figure 1. An overview of the Breakthrough Series Collaborative model is shown in the left part of the figure. The timing of the collection of measures is represented on the right side
126
March 2003
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Table 1. The 22 VA Hospital Teams, High-Hazard Area, and Area of Focus for Improvement*
* ED, emergency department; ICU, intensive care unit; OR, operating room.
learned during the collaborative. We also asked each director to spread the information gained by its pilot team to the rest of the VISN. Toward that end, we held a 1-day conference for the VISN directors on the diffusion strategies and other strategies they could use. In addition, we directed each participating team to spread what it had learned to other units. We introduced the teams and the VISN leaders to Rogers’s model of diffusion1 and suggested strategies such as working with opinion leaders and making sure that the change they were trying to spread had a relative advantage for the clinician making the change. For example, adherence to a standard protocol for surgical site preparation reduces the rate of surgical site infection for patients and increases confidence among clinicians that the procedure has been conducted properly.
March 2003
Measures Team characteristics. We surveyed the characteristics of each improvement team and their perceptions of support from their organizations.12 The survey was based on research in team performance and organizational learning and the characteristics of high-performing health care microsystems.17–22 Each question focuses on an important area of systemic support or team performance. The team was asked to complete the questionnaire after the first and third learning sessions. Answers were on a Likert scale from 1 (strongly disagree) to 7 (strongly agree). Self-reported progress in achieving team goals. During the initiative, each team developed and implemented a local safety improvement project. After obtaining baseline measurements (for example, the rate of
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
127
Joint Commission
Journal on Quality and Safety
nosocomial infections in the ICU), teams reported monthly on the changes they had made and the measures they were monitoring. Teams that were able to sustain at least a 20% improvement from baseline for at least 2 months before the end of the collaborative were rated “successful.” Measures of diffusion of innovation. We conducted telephone interviews 1 month after the first and third face-to-face meetings to determine whether process innovations learned in the collaborative spread to other high-hazard areas within the participating team’s medical center or to the same high-hazard areas at other medical centers within the VISN. Within a participating team’s medical center, the chief nurse of one of the high-hazard areas other than the participating team’s area focus (for example, the OR or the ICU if the team was working on reducing medical errors in the emergency department) were surveyed to determine if the innovations developed in one high-hazard area spread to another. One month after the third faceto-face meeting, we called the same service that we had called in the first round. Outside a participating team’s medical center, but within the participating team’s VISN, we interviewed each chief nurse of the same high-hazard area at two randomly selected hospitals that did not participate in the collaborative. For example, if the participating team from VISN 1 was focusing on reducing errors in the OR, then OR chief nurses in two other, randomly selected, medical centers within VISN 1 were interviewed. One month after the third meeting, we called back one of the units we called in the first round and two other, randomly selected, hospitals. We called back units in the second round to track whether the call itself was influencing dissemination. Twelve months after the third face-to-face meeting we called back both the participating hospitals and other hospitals in the region to determine whether further dissemination had taken place. We called back the same unit in the participating hospital which we had contacted in the first two rounds. In addition, we called similar units in three other hospitals in the VISN (one of these was the same unit as was called in the first two rounds, and two others were randomly selected). We asked if each chief nurse knew the names of the people on the participating team, had heard about the
128
March 2003
IHI QuIC Collaborative initiative on improving safety in high-hazard areas, had received any information about the collaborative, or had begun implementing any innovations offered in the collaborative. The timing of the collection of measures is represented on the right side of Figure 1. All surveys and interviews were voluntary, and all answers were treated confidentially and presented in aggregate form only. Randomization was conducted by using a random numbers table. The Human Subjects Committees in both the VA and Dartmouth Medical School approved this proposal.
Data Analysis Twenty-five VA teams joined the collaborative; 3 of the VISNs sent two teams. Three teams dropped out of the collaborative before the beginning of the second face-toface meeting. Therefore, our sample for analysis consisted of the remaining 22 teams, representing 19 VISNs, completing the project. Three VISNs had two teams in the project; the remaining 16 VISNs had one team each. The relationship between number of teams and level of dissemination was analyzed by using a Spearman’s rho correlation. The relationships between team characteristics, team success in the project and dissemination were also analyzed by using Spearman’s Rho.
Results Figure 2 (p 129) shows characteristics of the teams before and after the collaborative. About one-third of the teams had worked together before, had worked on improvement projects before, and were familiar with measurement strategies. At the beginning of the collaborative, most of the teams reported having mutual respect and comfort expressing their opinions. By the end of the collaborative, most reported that they had learned new ideas and concepts about improving safety, shared information with other teams, had learned methods to test changes, and reported that the collaborative added value to their facilities. Sixty-eight percent of the 22 teams were able to sustain a 20% improvement for 2 months by the end of the project, and most teams reported that they used information shared with them from other teams. Figure 3 (p 130) depicts changes in the teams’ perception between the beginning and the end of the project.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Characteristics of Teams Before and After the Collaborative
Figure 2. The figure shows characteristics of the teams at the beginning and the end of the collaborative.
Teams reported improvements in understanding each other, in solving conflicts, and in their organizations’ becoming less “punitive” in handling medical incidents. At the end of the project more teams reported specific plans to disseminate information, increased frontline staff support, and shared vision of how to improve. Few teams felt that they had sufficient time to meet their aims or that their information systems provided useful information to them. Teams indicated that the project had lost some of its mandate from senior leadership, that their team leadership had less strength, and that the project was less likely to be part of the organization’s key strategic goals; however, these team perceptions were not related to dissemination. Teams that were involved early in the collaborative, as evidenced by being on preconference calls and by quickly completing their first tests of change, were more likely than other teams to be successful in the initiative (Spearman’s r = .73, p = .001 and r = .58, p = .005, respectively). Teams that reported having “sufficient resources to meet their aims” in the first face-to-face meeting were more likely to spread information to other hospitals in their VISN (Spearman’s r = .50, p = .043). Teams that reported gathering data from patients at the end of the project were more likely than others to spread information to other units within their hospital (Spearman’s r = .485, p = .041). Support from leadership as measured on our questionnaire was not correlated with success in the initiative, and measures of success for individual teams were not correlated with successful dissemination. Figure 4 (p 131) displays the measures of dissemination of information about the initiative. Within the
March 2003
participating hospital, most nurse managers on other units were not aware of the initiative—this was true for both rounds of calls, even though the same person was called back in the second and third round of calls, and all the nurse managers knew the people on the change team. By the second round of calls, 18% of nurse managers on other units in the participating hospital had heard something (25% of these from the previous call), 9% had received information about the initiative, and 9% had begun to implement changes. After 12 months, 49% had heard about the project, but only 6% reported receiving information, and there were no reports of implementation. The majority of the other hospitals in the VISN also were not aware of the initiative. By the second round of calls, 37% had heard something (9% of these from the previous call), 5% had received information, and 2% were implementing change. After 12 months, 37% had heard of the initiative, but 10% had received information and 10% reported implementing change. In two of the more “successful” VISNs (see Sidebar 2, p 132), 83% had heard of the initiative, 33% had received information, and 17% were implementing changes. We found a significant negative correlation between dissemination to other hospitals and the number of teams in the VISN participating (Spearman’s r = –.48, p = .023), which suggests that VISNs with two teams participating were less likely than VISNs with only one team to disseminate information to other hospitals in their VISN (each of the two most successful VISNs had only one participating team).
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
129
Joint Commission
Journal on Quality and Safety
Characteristics of Teams Before and After the Project
Figure 3. The figure depicts changes in the teams’ perceptions between the beginning and the end of the project.
Discussion Teams that participated in the QuIC initiative felt that they learned about reducing medical errors, and 68% of them were able to achieve sustainable reductions in medical errors, despite a perceived shortage of time and limited leadership support. Teams that were active participants early in the collaborative effort had more success than teams that were not. Despite the local success that this model produced, there was very little diffusion of medical innovation either within or between medical centers. It is very difficult to disseminate medical innovations and important changes in process throughout a health care system. Even with a clear mandate to regional leaders to spread innovations developed by the pilot teams and training to help such dissemination take place, we found very little actual implementation of the new innovations. It was surprising that implementation within participating hospitals became worse over time, while other regional hospitals improved their use of the new innovations. It may be that less attention is paid to dissemination at the local level because it is assumed that
130
March 2003
clinicians within the same hospital will more easily learn from one another. On the contrary, personally knowing the clinicians on the improvement team did not help other clinicians to take on the new information. Our findings corroborate previous reports of clinical guidelines which also found inconsistent dissemination and little clinical change.6 Teams that reported in the first face-to-face meeting that they had sufficient resources were more likely to spread information to other hospitals in their VISNs. It may well be that increased resources do promote dissemination but that other measures of leadership support—such as having a mandate from senior leadership, having enough time to complete the project, or having the project as part of the organization’s key strategic goals—were not related either to teams’ success in the project or to dissemination. In addition, teams that were locally more successful were not more likely than others to disseminate the information. Finally, VISNs with more than one team involved were actually less likely than others to disseminate information to the other nonparticipating facilities. The bottom line appears to be that
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Dissemination of Information About the Project
Figure 4. The figure displays the measures of dissemination of information about the project.
the characteristics and perceptions of the change teams had very little to do with successful secondary dissemination of the information produced in the project. Rather, it appears that successful dissemination was more associated with the personal commitment of individual regional (VISN) leaders than with either team or hospital characteristics. Consequently, in future studies we should not look to individual change teams to disseminate the information but to facility and regional leadership. The success of two VISNs illustrates this point (Sidebar 2). In both cases, senior regional leadership sent information to the other hospitals in the network, stayed personally involved, and monitored progress. Future studies should examine the characteristics of senior leaders and regional health care organizations to determine when or if they are ready to begin dissemination, and they should compare dissemination strategies. For individual teams to be successful, they needed to be organized and hit the ground running, but for information to spread beyond the team requires charismatic, committed leadership at the regional level. The idea that strong leadership is needed to spread changes is not new.1,7 According to Dixon,23 information should be “pushed” to the receivers; the receiving teams should be mandated to continue to improve, while being given a choice of what information to use, and the usage of the knowledge transfer system and improvement goals should be monitored. The VISNs that were successful in dissemination had more personal interactions, pushed the information to the users,
March 2003
worked more closely with the hospitals that received the information, and monitored improvement. This study has several limitations. We examined only 22 teams and 19 VISNs, so we were limited in our ability to perceive subtle relationships between variables. In addition, the study has a quasi-experimental design in which there was no control group. It is possible that the teams could have improved their care and disseminated information without the specific interventions that were conducted. It is also possible that the information generated in the collaborative could have been disseminated through channels that did not involve the nurse managers on the units. Another limitation is that we called back some of the same people in the second round of calls as we did in the first. This repetition could have cued some of the nurse managers and falsely increased the number of people who reported that they had heard of the project beyond what we were able to track. Finally, although there was a formal and intensive effort to help VISN leaders and teams disseminate the information, many teams reported having little time to complete their projects and some loss of senior leader support by the end of the project. Although it is common to see teams report limited time and diminishing support,12–14 these factors may have contributed to the lack of dissemination.
Recommendations on Improving Spread in Collaborative Efforts Despite its limitations, the study provides evidence that dissemination of process improvements in medical
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
131
Joint Commission
Journal on Quality and Safety
Sidebar 2. The Two Successful VISNS Dissemination to Other Units and to Other Hospitals One VISN was able to disseminate to both other units within the participating hospital and other hospitals with in the VISN. The team from this VISN was fundamentally different from other teams: It consisted of members from every hospital in the network. Each facility had a contact person who attended the face-to-face meetings and monitored the listserv for new information as it came in. The contact person then formed a smaller change team within each facility to begin testing and implementing the new information. From the beginning, that team understood that its project was to be conducted as a VISN rather than as a facility and made it a point to communicate with all the facilities in the region. This broad perspective made it possible for all the facilities in the region to stay informed about the project. All the facilities were involved each step of the way in training and learning. As new innovations were developed, they were immediately transferred to other facilities. Dissemination to Other Hospitals Another VISN was highly successful in disseminating information to other hospitals even though the participating change team dropped out of the project. In this case the senior quality manager at the VISN level took personal responsibility for gathering information from the project and pushing it out to the hospitals in his VISN. He gathered information on safety improvement strategies in the ER, ICU, and OR that were produced in the initiative and sent it, usually via e-mail, to each facility’s local quality manager. The facility-level manager then provided the information to the unit leaders in the appropriate areas. The VISN-level manager then checked back with the local facilities to see if they had distributed the information and to help remove barriers to implementing new ideas. This was the only VISN in which nonparticipating hospitals actually implemented change.
systems is difficult but not impossible. We can learn from our observations of the most successful VISNs. A health care system cannot assume that the change team by itself will have any success in dissemination; individual team success and most team characteristics were not related to dissemination. The people on the change team need to be
132
March 2003
given the support and resources necessary to accomplish their aims, and the hospitals that make up the regional group have to work as one system, with a clear plan of dissemination. The senior leaders in the region have to take personal responsibility for disseminating the information, attend the meetings themselves, push information to the people who need it, and monitor progress, using concrete measures of improvement. It is also important for each facility to which the information is being disseminated to have a specific contact person who, again, takes personal responsibility for making sure the information is received and pushed to the clinicians at the facility who need it. The local contact should not only send the information but also form a local change team to begin to test and modify the new ideas at the front line to fit them into the facility’s unique environment. A model for making improvement in health care (for example, Langley et al’s24) must be disseminated along with the change ideas so that the new teams can successfully make improvements. Finally, ongoing feedback loops are necessary: 1. The change team needs clinical process and outcome data to inform its ongoing improvement efforts and keep it on track; 2. The local leaders need monthly reports of the change teams’ progress, including changes in clinical outcomes, to inform efforts to support the team, push dissemination to other units within the facility, provide resources, and remove barriers to change; and 3. The regional leaders need quarterly reports from all the facilities on the progress of the local change teams, to give support to local leaders, push dissemination between facilities, and inform strategic planning for future improvement efforts. J
Peter D. Mills, PhD, MS, is Associate Director, Field Office, Veterans Affairs National Center for Patient Safety, White River Junction, Vermont. William B. Weeks, MD, MBA, CHE, is Assistant Professor, Departments of Psychiatry and of Community and Family Medicine, Dartmouth Medical School, Hanover, New Hampshire, and a member of The Joint Commission Journal on Quality and Safety's Editorial Advisory Board. B.C. Surott-Kimberly, CTRS, is a Social Worker, Veterans Affairs Medical Center, White River Junction, Vermont. Please address reprint requests to Peter D. Mills, PhD, MS, [email protected].
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
References 1. Rogers EM: Diffusion of Innovations 4th ed. New York: The Free Press, 1995. 2. Duffy SQ, Farley DE: The protracted demise of medical technology: The case of intermittent positive pressure breathing. Med Care 30:718–736, 1992. 3. Escarce JJ, et al: Diffusion of laparoscopic cholecystectomy among general surgeons in the United States. Med Care 33:256–271, 1995. 4. Chernew M, Fendrick AM, Hirth RA: Managed care and medical technology: Implications for cost growth. Health Aff (Millwood) 16:196–206, 1997. 5. Baker LC, Wheeler SK: Managed care and technology diffusion: The case of MRI. Health Aff (Millwood) 17:195–207, 1998. 6. Lichtman JH, et al: Can practice guidelines be transported effectively to different settings? Results from a multicenter interventional study. Jt Comm J Qual Improv 27:42–53, 2001. 7. Soumerai SB, et al: Effect of local medical opinion leaders on quality of care for acute myocardial infarction: A randomized controlled trial. JAMA 279:1358–1363, 1998. 8. Kosseff AL, Niemeie S: SSM Health Care clinical collaboratives: Improving the value of patient care in a health care system. Jt Comm J Qual Improv 27:5–19, 2001. 9. Plsek P: Collaborating across organizational boundaries to improve quality of care. Am J Infect Control 25:85–95, 1997. 10. Kilo C: Improving care through collaboration. Pediatrics 103:384–393, 1999. 11. O’Connor GT, et al: A regional intervention to improve the hospital mortality associated with coronary artery bypass surgery. JAMA 275:841–846, 1996. 12. Weeks WB, et al: Using an improvement model to reduce adverse drug events in VA facilities. Jt Comm J Qual Improv 27:243-254, 2001. 13. Volicer L, et al: Improved care for veterans suffering from dementia. Federal Practitioner, in press. 14. Weeks WB, Mills PD, Waldron J: Using an improvement model to improve the quality and timeliness of compensation and pension exams in VA facilities (unpublished manuscript). White River Junction, VT, Jul 2002.
March 2003
15. Institute of Medicine: To Err Is Human. Washington, DC: National Academy Press, 1999. 16. QuIC: Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. Report of the Quality Interagency Coordination Task Force (QuIC) to the President. Washington, DC, Feb 2000. 17. Baldrige National Quality Program: 2000 Health Care Criteria for Performance Excellence. (2003 Health Care Criteria at www.quality. nist.gov/HealthCare_Criteria.htm; last accessed Dec 19, 2002). 18. DiBella A, Nevis EC: How Organizations Learn. San Francisco: Jossey-Bass, 1998. 19. Gustafson D: Change Manager. Madison, WI: University of Wisconsin, 1998. 20. Dickinson TL, McIntyre RM: A conceptual framework for teamwork measurement. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahwah, NJ: Lawrence Erlbaum Associates, 1977, pp 19–44. 21. Kraiger K, Wenzel LH: Conceptual development and empirical evaluation of measures of shared mental models as indicators of team effectiveness. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahway, NJ: Lawrence Erlbaum Associates, 1977, pp 63–84. 22. Hallam G, Campbell D: The measurement of team performance with a standardized survey. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahwah, NJ: Lawrence Erlbaum Associates, 1977, pp 155–172. 23. Dixon NM: Common Knowledge: How Companies Thrive by Sharing What They Know. Boston: Harvard Business School Press, 2000. 24. Langley G, et al: The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass Publishers, 1996.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
133
Journal on Quality and Safety
Performance Improvement
A Multihospital Safety Improvement Effort and the Dissemination of New Knowledge
Peter D. Mills, PhD, MS William B. Weeks, MD, MBA, CHE B.C. Surott-Kimberly, CTRS
iffusion of innovation has been defined as the “process by which an innovation is communicated through channels over time among the members of a social system.”1(p 5) Traditionally, knowledge has been disseminated in schools, internships, and residencies and then through continuing education programs. Studies on the diffusion of new medications, technological innovations, and clinical guidelines2–7 suggest that the dissemination of medical innovation is driven more by interpersonal relationships and observation of peers than by new research or available information. Adoption of innovations tends to be slowed within capitated systems,5 and the use of unsafe or discredited practices by physicians may take a long time to extinguish.2 Since 1995 the Institute for Healthcare Improvement (IHI; Boston) has facilitated multihospital quality improvement efforts, called Breakthrough Series, which improve care in the majority of participating facilities.8–10 Another multihospital dissemination strategy is that of the Northern New England Cardiovascular Group,11 which continuously compiles data and provides feedback on best practices to the participating members. Our research has shown that the Breakthrough Series model is effective within the VA system.12–14 Despite these gains, it remains unclear whether innovations can be further disseminated beyond the change teams participating in a collaborative. No study has examined this type of secondary diffusion in a health care system. We had an opportunity to examine the secondary diffusion of process innovations beyond the participating change teams during a national collaborative within the Veterans Health Administration (VHA) in 2000–2001.
D
124
March 2003
Article-at-a-Glance Background: Research on the transfer of medical technology and guidelines suggests that this transfer is driven more by interpersonal relationships than by new research or available information and that it is inconsistent, largely unsuccessful, and strongly influenced by local factors. Yet studies of collaborative, multiple-hospital improvement efforts have shown these transfers to be effective for the specific microsystems participating in the project. The diffusion of medical innovations beyond the participating teams was studied during a 2000–2001 national collaborative safety improvement effort. Methods: Twenty-two teams from Department of Veterans Affairs (VA) hospitals participated in a 9month quality improvement project designed to improve safety in high-hazard areas. Participating hospitals and other regional hospitals were contacted to determine the level of dissemination of information generated during and after the project. Results: While the participating hospitals benefited from the quality improvement effort, changes were implemented only 9% of the time on other units within the hospitals and only 2% of the time in other regional hospitals. After 12 months, there was no implementation within participating hospitals, and other regional hospitals were implementing changes 10% of the time. Discussion: Personal commitment from senior leadership, dissemination strategies that push information to clinicians, and monitoring of progress at the regional level are all needed for dissemination of complex medical information to occur.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
The Institute of Medicine’s (IOM’s) 1999 report, To Err Is Human15 heightened awareness of patient safety and medical errors in the United States. The federal government’s Quality Interagency Coordination Task Force (QuIC) used the principles of systems change outlined in the IOM report to develop a detailed set of action plans designed to reduce medical errors and their impact.16 One such action plan called for participation in an IHI Breakthrough Series Collaborative that was designed to reduce medical errors in high-hazard environments.16 (See Sidebar 1, right.) This article describes a study on teams’ diffusion of process innovations to other hospitals within their health care systems. Each team was to focus on one high-hazard area, make local process changes that could demonstrably improve patient safety, and spread those changes both to the same high-hazard area in other locations and to other high-hazard areas within the participating hospital.
Methods The QuIC Initiative The QuIC initiative (Jul 2000–May 2001) was designed to reduce medical errors in high-hazard areas—labor and delivery suites, emergency departments, operating rooms (ORs), and intensive care units (ICUs). Experts in patient safety in each of these areas and experts in quality improvement constituted the faculty. During the first learning session, teams were taught specific changes they could make to improve safety in their facilities, how to measure improvements, and quality improvement techniques, and they chose specific safety goals. During the second face-to-face meeting, teams reported on the changes they had made. The faculty continued to provide information on measurement and key changes in the four high-hazard areas. In addition, the teams were encouraged to spread their changes to other units of the hospital or other hospitals in their Veterans Integrated Service Network (VISN).* During the third face-to-face meeting, teams reported gains, continued to learn from each other, and consolidated plans to disseminate what they had learned.
Sidebar 1. Breakthrough Series Collaborative A Collaborative Breakthrough Series is designed to facilitate the achievement of best practices through face-to-face knowledge distribution at three 2-day conferences and ongoing electronic and telephone support of participating institutions during a a 9-month period. After initiative’s end, teams continue to monitor their progress and are expected to help disseminate the new improvement knowledge broadly across their institution. In addition, the teams are expected to help spread the innovations to other hospitals in their health care systems.
Each month during the collaborative, teams reported their results on a Web site and participated in collaborative calls; listserv support was available throughout the collaborative. An overview of the Breakthrough Series Collaborative model is shown in the left part of Figure 1 (p 126).
Sample and Design Forty teams from federal health care organizations participated in the QuIC initiative. The initial sample consisted of the 22 VHA teams, representing 19 VISNs. Table 1 (p 127) provides a breakdown of the hospital teams by department and area of focus for improvement. We conducted a before–after study on the 22 VHA teams. One month after the second face-to-face meeting, in which teams were first encouraged to spread their process innovations, the research team began calling the VISNs to monitor the progress of the spread of information. One month after the third face-to-face meeting, in which teams had developed diffusion plans, we again called the VISNs to monitor the progress of spread. Finally, 12 months after the third meeting, we again called the VISNs to determine if further dissemination had taken place.
Interventions to Produce Secondary Diffusion * VHA is geographically divided into 22 VISNS. VISNs, which consist of between 4 and 11 VA medical centers, are intended to help standardize care and facilitate process knowledge diffusion within and across their geographic boundaries.
March 2003
We asked the director of each VISN to choose one team that had strong clinical skills and was a leader in the VISN that could serve as a model to spread the innovations
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
125
Joint Commission
Journal on Quality and Safety
QuIC Breakthrough Series on Reducing Errors in High-Hazard Areas Time Line
Figure 1. An overview of the Breakthrough Series Collaborative model is shown in the left part of the figure. The timing of the collection of measures is represented on the right side
126
March 2003
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Table 1. The 22 VA Hospital Teams, High-Hazard Area, and Area of Focus for Improvement*
* ED, emergency department; ICU, intensive care unit; OR, operating room.
learned during the collaborative. We also asked each director to spread the information gained by its pilot team to the rest of the VISN. Toward that end, we held a 1-day conference for the VISN directors on the diffusion strategies and other strategies they could use. In addition, we directed each participating team to spread what it had learned to other units. We introduced the teams and the VISN leaders to Rogers’s model of diffusion1 and suggested strategies such as working with opinion leaders and making sure that the change they were trying to spread had a relative advantage for the clinician making the change. For example, adherence to a standard protocol for surgical site preparation reduces the rate of surgical site infection for patients and increases confidence among clinicians that the procedure has been conducted properly.
March 2003
Measures Team characteristics. We surveyed the characteristics of each improvement team and their perceptions of support from their organizations.12 The survey was based on research in team performance and organizational learning and the characteristics of high-performing health care microsystems.17–22 Each question focuses on an important area of systemic support or team performance. The team was asked to complete the questionnaire after the first and third learning sessions. Answers were on a Likert scale from 1 (strongly disagree) to 7 (strongly agree). Self-reported progress in achieving team goals. During the initiative, each team developed and implemented a local safety improvement project. After obtaining baseline measurements (for example, the rate of
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
127
Joint Commission
Journal on Quality and Safety
nosocomial infections in the ICU), teams reported monthly on the changes they had made and the measures they were monitoring. Teams that were able to sustain at least a 20% improvement from baseline for at least 2 months before the end of the collaborative were rated “successful.” Measures of diffusion of innovation. We conducted telephone interviews 1 month after the first and third face-to-face meetings to determine whether process innovations learned in the collaborative spread to other high-hazard areas within the participating team’s medical center or to the same high-hazard areas at other medical centers within the VISN. Within a participating team’s medical center, the chief nurse of one of the high-hazard areas other than the participating team’s area focus (for example, the OR or the ICU if the team was working on reducing medical errors in the emergency department) were surveyed to determine if the innovations developed in one high-hazard area spread to another. One month after the third faceto-face meeting, we called the same service that we had called in the first round. Outside a participating team’s medical center, but within the participating team’s VISN, we interviewed each chief nurse of the same high-hazard area at two randomly selected hospitals that did not participate in the collaborative. For example, if the participating team from VISN 1 was focusing on reducing errors in the OR, then OR chief nurses in two other, randomly selected, medical centers within VISN 1 were interviewed. One month after the third meeting, we called back one of the units we called in the first round and two other, randomly selected, hospitals. We called back units in the second round to track whether the call itself was influencing dissemination. Twelve months after the third face-to-face meeting we called back both the participating hospitals and other hospitals in the region to determine whether further dissemination had taken place. We called back the same unit in the participating hospital which we had contacted in the first two rounds. In addition, we called similar units in three other hospitals in the VISN (one of these was the same unit as was called in the first two rounds, and two others were randomly selected). We asked if each chief nurse knew the names of the people on the participating team, had heard about the
128
March 2003
IHI QuIC Collaborative initiative on improving safety in high-hazard areas, had received any information about the collaborative, or had begun implementing any innovations offered in the collaborative. The timing of the collection of measures is represented on the right side of Figure 1. All surveys and interviews were voluntary, and all answers were treated confidentially and presented in aggregate form only. Randomization was conducted by using a random numbers table. The Human Subjects Committees in both the VA and Dartmouth Medical School approved this proposal.
Data Analysis Twenty-five VA teams joined the collaborative; 3 of the VISNs sent two teams. Three teams dropped out of the collaborative before the beginning of the second face-toface meeting. Therefore, our sample for analysis consisted of the remaining 22 teams, representing 19 VISNs, completing the project. Three VISNs had two teams in the project; the remaining 16 VISNs had one team each. The relationship between number of teams and level of dissemination was analyzed by using a Spearman’s rho correlation. The relationships between team characteristics, team success in the project and dissemination were also analyzed by using Spearman’s Rho.
Results Figure 2 (p 129) shows characteristics of the teams before and after the collaborative. About one-third of the teams had worked together before, had worked on improvement projects before, and were familiar with measurement strategies. At the beginning of the collaborative, most of the teams reported having mutual respect and comfort expressing their opinions. By the end of the collaborative, most reported that they had learned new ideas and concepts about improving safety, shared information with other teams, had learned methods to test changes, and reported that the collaborative added value to their facilities. Sixty-eight percent of the 22 teams were able to sustain a 20% improvement for 2 months by the end of the project, and most teams reported that they used information shared with them from other teams. Figure 3 (p 130) depicts changes in the teams’ perception between the beginning and the end of the project.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Characteristics of Teams Before and After the Collaborative
Figure 2. The figure shows characteristics of the teams at the beginning and the end of the collaborative.
Teams reported improvements in understanding each other, in solving conflicts, and in their organizations’ becoming less “punitive” in handling medical incidents. At the end of the project more teams reported specific plans to disseminate information, increased frontline staff support, and shared vision of how to improve. Few teams felt that they had sufficient time to meet their aims or that their information systems provided useful information to them. Teams indicated that the project had lost some of its mandate from senior leadership, that their team leadership had less strength, and that the project was less likely to be part of the organization’s key strategic goals; however, these team perceptions were not related to dissemination. Teams that were involved early in the collaborative, as evidenced by being on preconference calls and by quickly completing their first tests of change, were more likely than other teams to be successful in the initiative (Spearman’s r = .73, p = .001 and r = .58, p = .005, respectively). Teams that reported having “sufficient resources to meet their aims” in the first face-to-face meeting were more likely to spread information to other hospitals in their VISN (Spearman’s r = .50, p = .043). Teams that reported gathering data from patients at the end of the project were more likely than others to spread information to other units within their hospital (Spearman’s r = .485, p = .041). Support from leadership as measured on our questionnaire was not correlated with success in the initiative, and measures of success for individual teams were not correlated with successful dissemination. Figure 4 (p 131) displays the measures of dissemination of information about the initiative. Within the
March 2003
participating hospital, most nurse managers on other units were not aware of the initiative—this was true for both rounds of calls, even though the same person was called back in the second and third round of calls, and all the nurse managers knew the people on the change team. By the second round of calls, 18% of nurse managers on other units in the participating hospital had heard something (25% of these from the previous call), 9% had received information about the initiative, and 9% had begun to implement changes. After 12 months, 49% had heard about the project, but only 6% reported receiving information, and there were no reports of implementation. The majority of the other hospitals in the VISN also were not aware of the initiative. By the second round of calls, 37% had heard something (9% of these from the previous call), 5% had received information, and 2% were implementing change. After 12 months, 37% had heard of the initiative, but 10% had received information and 10% reported implementing change. In two of the more “successful” VISNs (see Sidebar 2, p 132), 83% had heard of the initiative, 33% had received information, and 17% were implementing changes. We found a significant negative correlation between dissemination to other hospitals and the number of teams in the VISN participating (Spearman’s r = –.48, p = .023), which suggests that VISNs with two teams participating were less likely than VISNs with only one team to disseminate information to other hospitals in their VISN (each of the two most successful VISNs had only one participating team).
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
129
Joint Commission
Journal on Quality and Safety
Characteristics of Teams Before and After the Project
Figure 3. The figure depicts changes in the teams’ perceptions between the beginning and the end of the project.
Discussion Teams that participated in the QuIC initiative felt that they learned about reducing medical errors, and 68% of them were able to achieve sustainable reductions in medical errors, despite a perceived shortage of time and limited leadership support. Teams that were active participants early in the collaborative effort had more success than teams that were not. Despite the local success that this model produced, there was very little diffusion of medical innovation either within or between medical centers. It is very difficult to disseminate medical innovations and important changes in process throughout a health care system. Even with a clear mandate to regional leaders to spread innovations developed by the pilot teams and training to help such dissemination take place, we found very little actual implementation of the new innovations. It was surprising that implementation within participating hospitals became worse over time, while other regional hospitals improved their use of the new innovations. It may be that less attention is paid to dissemination at the local level because it is assumed that
130
March 2003
clinicians within the same hospital will more easily learn from one another. On the contrary, personally knowing the clinicians on the improvement team did not help other clinicians to take on the new information. Our findings corroborate previous reports of clinical guidelines which also found inconsistent dissemination and little clinical change.6 Teams that reported in the first face-to-face meeting that they had sufficient resources were more likely to spread information to other hospitals in their VISNs. It may well be that increased resources do promote dissemination but that other measures of leadership support—such as having a mandate from senior leadership, having enough time to complete the project, or having the project as part of the organization’s key strategic goals—were not related either to teams’ success in the project or to dissemination. In addition, teams that were locally more successful were not more likely than others to disseminate the information. Finally, VISNs with more than one team involved were actually less likely than others to disseminate information to the other nonparticipating facilities. The bottom line appears to be that
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Dissemination of Information About the Project
Figure 4. The figure displays the measures of dissemination of information about the project.
the characteristics and perceptions of the change teams had very little to do with successful secondary dissemination of the information produced in the project. Rather, it appears that successful dissemination was more associated with the personal commitment of individual regional (VISN) leaders than with either team or hospital characteristics. Consequently, in future studies we should not look to individual change teams to disseminate the information but to facility and regional leadership. The success of two VISNs illustrates this point (Sidebar 2). In both cases, senior regional leadership sent information to the other hospitals in the network, stayed personally involved, and monitored progress. Future studies should examine the characteristics of senior leaders and regional health care organizations to determine when or if they are ready to begin dissemination, and they should compare dissemination strategies. For individual teams to be successful, they needed to be organized and hit the ground running, but for information to spread beyond the team requires charismatic, committed leadership at the regional level. The idea that strong leadership is needed to spread changes is not new.1,7 According to Dixon,23 information should be “pushed” to the receivers; the receiving teams should be mandated to continue to improve, while being given a choice of what information to use, and the usage of the knowledge transfer system and improvement goals should be monitored. The VISNs that were successful in dissemination had more personal interactions, pushed the information to the users,
March 2003
worked more closely with the hospitals that received the information, and monitored improvement. This study has several limitations. We examined only 22 teams and 19 VISNs, so we were limited in our ability to perceive subtle relationships between variables. In addition, the study has a quasi-experimental design in which there was no control group. It is possible that the teams could have improved their care and disseminated information without the specific interventions that were conducted. It is also possible that the information generated in the collaborative could have been disseminated through channels that did not involve the nurse managers on the units. Another limitation is that we called back some of the same people in the second round of calls as we did in the first. This repetition could have cued some of the nurse managers and falsely increased the number of people who reported that they had heard of the project beyond what we were able to track. Finally, although there was a formal and intensive effort to help VISN leaders and teams disseminate the information, many teams reported having little time to complete their projects and some loss of senior leader support by the end of the project. Although it is common to see teams report limited time and diminishing support,12–14 these factors may have contributed to the lack of dissemination.
Recommendations on Improving Spread in Collaborative Efforts Despite its limitations, the study provides evidence that dissemination of process improvements in medical
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
131
Joint Commission
Journal on Quality and Safety
Sidebar 2. The Two Successful VISNS Dissemination to Other Units and to Other Hospitals One VISN was able to disseminate to both other units within the participating hospital and other hospitals with in the VISN. The team from this VISN was fundamentally different from other teams: It consisted of members from every hospital in the network. Each facility had a contact person who attended the face-to-face meetings and monitored the listserv for new information as it came in. The contact person then formed a smaller change team within each facility to begin testing and implementing the new information. From the beginning, that team understood that its project was to be conducted as a VISN rather than as a facility and made it a point to communicate with all the facilities in the region. This broad perspective made it possible for all the facilities in the region to stay informed about the project. All the facilities were involved each step of the way in training and learning. As new innovations were developed, they were immediately transferred to other facilities. Dissemination to Other Hospitals Another VISN was highly successful in disseminating information to other hospitals even though the participating change team dropped out of the project. In this case the senior quality manager at the VISN level took personal responsibility for gathering information from the project and pushing it out to the hospitals in his VISN. He gathered information on safety improvement strategies in the ER, ICU, and OR that were produced in the initiative and sent it, usually via e-mail, to each facility’s local quality manager. The facility-level manager then provided the information to the unit leaders in the appropriate areas. The VISN-level manager then checked back with the local facilities to see if they had distributed the information and to help remove barriers to implementing new ideas. This was the only VISN in which nonparticipating hospitals actually implemented change.
systems is difficult but not impossible. We can learn from our observations of the most successful VISNs. A health care system cannot assume that the change team by itself will have any success in dissemination; individual team success and most team characteristics were not related to dissemination. The people on the change team need to be
132
March 2003
given the support and resources necessary to accomplish their aims, and the hospitals that make up the regional group have to work as one system, with a clear plan of dissemination. The senior leaders in the region have to take personal responsibility for disseminating the information, attend the meetings themselves, push information to the people who need it, and monitor progress, using concrete measures of improvement. It is also important for each facility to which the information is being disseminated to have a specific contact person who, again, takes personal responsibility for making sure the information is received and pushed to the clinicians at the facility who need it. The local contact should not only send the information but also form a local change team to begin to test and modify the new ideas at the front line to fit them into the facility’s unique environment. A model for making improvement in health care (for example, Langley et al’s24) must be disseminated along with the change ideas so that the new teams can successfully make improvements. Finally, ongoing feedback loops are necessary: 1. The change team needs clinical process and outcome data to inform its ongoing improvement efforts and keep it on track; 2. The local leaders need monthly reports of the change teams’ progress, including changes in clinical outcomes, to inform efforts to support the team, push dissemination to other units within the facility, provide resources, and remove barriers to change; and 3. The regional leaders need quarterly reports from all the facilities on the progress of the local change teams, to give support to local leaders, push dissemination between facilities, and inform strategic planning for future improvement efforts. J
Peter D. Mills, PhD, MS, is Associate Director, Field Office, Veterans Affairs National Center for Patient Safety, White River Junction, Vermont. William B. Weeks, MD, MBA, CHE, is Assistant Professor, Departments of Psychiatry and of Community and Family Medicine, Dartmouth Medical School, Hanover, New Hampshire, and a member of The Joint Commission Journal on Quality and Safety's Editorial Advisory Board. B.C. Surott-Kimberly, CTRS, is a Social Worker, Veterans Affairs Medical Center, White River Junction, Vermont. Please address reprint requests to Peter D. Mills, PhD, MS, [email protected].
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
References 1. Rogers EM: Diffusion of Innovations 4th ed. New York: The Free Press, 1995. 2. Duffy SQ, Farley DE: The protracted demise of medical technology: The case of intermittent positive pressure breathing. Med Care 30:718–736, 1992. 3. Escarce JJ, et al: Diffusion of laparoscopic cholecystectomy among general surgeons in the United States. Med Care 33:256–271, 1995. 4. Chernew M, Fendrick AM, Hirth RA: Managed care and medical technology: Implications for cost growth. Health Aff (Millwood) 16:196–206, 1997. 5. Baker LC, Wheeler SK: Managed care and technology diffusion: The case of MRI. Health Aff (Millwood) 17:195–207, 1998. 6. Lichtman JH, et al: Can practice guidelines be transported effectively to different settings? Results from a multicenter interventional study. Jt Comm J Qual Improv 27:42–53, 2001. 7. Soumerai SB, et al: Effect of local medical opinion leaders on quality of care for acute myocardial infarction: A randomized controlled trial. JAMA 279:1358–1363, 1998. 8. Kosseff AL, Niemeie S: SSM Health Care clinical collaboratives: Improving the value of patient care in a health care system. Jt Comm J Qual Improv 27:5–19, 2001. 9. Plsek P: Collaborating across organizational boundaries to improve quality of care. Am J Infect Control 25:85–95, 1997. 10. Kilo C: Improving care through collaboration. Pediatrics 103:384–393, 1999. 11. O’Connor GT, et al: A regional intervention to improve the hospital mortality associated with coronary artery bypass surgery. JAMA 275:841–846, 1996. 12. Weeks WB, et al: Using an improvement model to reduce adverse drug events in VA facilities. Jt Comm J Qual Improv 27:243-254, 2001. 13. Volicer L, et al: Improved care for veterans suffering from dementia. Federal Practitioner, in press. 14. Weeks WB, Mills PD, Waldron J: Using an improvement model to improve the quality and timeliness of compensation and pension exams in VA facilities (unpublished manuscript). White River Junction, VT, Jul 2002.
March 2003
15. Institute of Medicine: To Err Is Human. Washington, DC: National Academy Press, 1999. 16. QuIC: Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. Report of the Quality Interagency Coordination Task Force (QuIC) to the President. Washington, DC, Feb 2000. 17. Baldrige National Quality Program: 2000 Health Care Criteria for Performance Excellence. (2003 Health Care Criteria at www.quality. nist.gov/HealthCare_Criteria.htm; last accessed Dec 19, 2002). 18. DiBella A, Nevis EC: How Organizations Learn. San Francisco: Jossey-Bass, 1998. 19. Gustafson D: Change Manager. Madison, WI: University of Wisconsin, 1998. 20. Dickinson TL, McIntyre RM: A conceptual framework for teamwork measurement. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahwah, NJ: Lawrence Erlbaum Associates, 1977, pp 19–44. 21. Kraiger K, Wenzel LH: Conceptual development and empirical evaluation of measures of shared mental models as indicators of team effectiveness. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahway, NJ: Lawrence Erlbaum Associates, 1977, pp 63–84. 22. Hallam G, Campbell D: The measurement of team performance with a standardized survey. In Brannick MT, Salas E, Prince C (eds): Team Performance Assessment and Measurement: Theory, Methods, and Applications. Mahwah, NJ: Lawrence Erlbaum Associates, 1977, pp 155–172. 23. Dixon NM: Common Knowledge: How Companies Thrive by Sharing What They Know. Boston: Harvard Business School Press, 2000. 24. Langley G, et al: The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco: Jossey-Bass Publishers, 1996.
Volume 29 Number 3
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
133