- Email: [email protected]
When Implementation Fails: The Case of a Nursing Guideline for Fall Prevention
Evidence-Based Medicine
When Implementation Fails: The Case of a Nursing Guideline for Fall Prevention t is well established that implementing guidelines can be very difficult.1 No magic bullet or step-bystep implementation plan is available, neither is any single implementation strategy superior in achieving change.2 Given the mixture of professional and organizational factors that seem to be crucial to successful implementation, it is advisable that multiple strategies be used.3 Considerable effort has been put in developing structured step-by-step implementation approaches, yet in practice, health care systems seem to behave as complex adaptive systems.4 Little guidance can be obtained from the literature on why implementation might fail in spite of a well-prepared guideline and a sound implementation strategy.5 At the Academic Medical Center (AMC) in Amsterdam, the Netherlands, a teaching hospital with 1,000 beds, a nursing guideline was developed in 1993 on prevention of patient falls. In a 1993 study, a 30% reduction in the incidence of falls (in terms of the number of falls per 1,000 patient days) on six wards was reported.6 This result was widely disseminated within the hospital and held as promise for solving a major nursing-related problem, but an effort to implement the guideline into daily practice throughout the hospital failed. A renewed effort to implement the guideline in two wards was made in 1999. Yet this effort failed, too; in this article we will explore why. Additional details about the implementation model followed can be found elsewhere.7
I
Theoretical Framework During the last few years, a multifaceted approach has been advocated for changing physician behavior. Grol and colleagues developed a model for implementing
152
March 2006
Jelle van der Helm, M.Sc. Astrid Goossens, Ph.D., R.N. Patrick Bossuyt, Ph.D.
Article-at-a-Glance Background: Implementing guidelines can be very difficult. No magic bullet or step-by-step implementation plan is available, neither is any single implementation strategy superior. At the Academic Medical Center (AMC) in Amsterdam, a nursing guideline was developed in 1993 on prevention of patient falls. Falls decreased by 30% on six wards, yet an effort to implement the guideline into daily practice throughout the hospital failed. A renewed effort was made to implement the guideline in two wards (neurology and internal medicine) in 1999. Implementing the Guideline: Preparations were made for implementation in the two wards. Barriers to change were identified and solutions were translated into day-to-day activities in the wards. The intervention period covered 18 months (January 2000–June 2001). A mix of implementation strategies was used, including a local consensus process, educational activities, and active support and feedback to management and staff. Results: In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward, the incidence target of 11% was met in five months. Discussion: Barriers to change and enabling factors may only become apparent during the implementation process itself. A strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implementation strategy, and voluntarily cooperating nurses are no guarantees for success.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
European Foundation for Quality Management (EFQM) Excellence Model
Setting
The implementation activities took place in an internal medicine ward and a neurology ward. The neurology ward has 32 beds staffed by one senior nurse and 33 nurses and has 850 admissions per year. The internal medicine ward is staffed by one senior nurse and 55 nurses and consists of 60 beds; there are 1,500 admissions per year. These wards voluntarily participated in the study in an effort to reduce the number of falls. Both wards had been involved in the development of Figure 1. The EFQM model depicts the organizational context for achieving the guideline in 1993, when a 30% quality improvement. Used by permission of the European Foundation for reduction in falls was reported; Quality Management, Brussels. however, this reduction was not sustained. For example, in 1998 the internal medicine change that focuses on identifying and resolving barriers 8 ward and neurology ward reported a total of 100 falls to to change. It has been shown to be effective, primarily 9,10 the incident report committee, which means that every in changing the behavior of general practitioners. 3.5 days one patient experienced a fall in one of these Implementation involves the actual use of developed two wards. Because it is well known that not all falls are knowledge. Knowledge is often conceptualized as some11 reported, the project team assumed that the actual numthing that can be caught, managed, and distributed. Yet ber of falls was even higher. other recent approaches suggest that its use and mean12,13 ing are influenced by the organizational context. The European Foundation for Quality Management (EFQM) Outline Excellence Model, as shown in Figure 1 (above), is freAn implementation study was conducted, starting quently used in Europe to describe the organizational with a pilot study of three months (July–September context for achieving quality improvement.14–16 1999). The pilot was used as a benchmark to enable According to the model, customer satisfaction, peodetermination of the goal for implementation. From ple satisfaction, and impact on society are achieved September 1999 until January 2000, preparations through leadership, policy and strategy, people, partnerwere made for implementation in the two wards. In ships and resources, processes, and feedback of the this pre-intervention period, barriers to change were results to the organization—all of which comprise the identified and solutions were translated into day-toorganizational context. day activities in the wards. The intervention period covered 18 months (January 2000–June 2001). At proImplementing the Guideline ject’s end, a questionnaire was developed to evaluate Establishing the Project Team the results. In July 1999 a project team was formed that consisted of Because the initial study in 1993 was performed in the four nurses and the senior (charge) nurse for each of the same wards and the nurses told us that reintroduction of wards (internal medicine and neurology), a clinical epithe guideline had not been planned, the project team demiologist and research nurse [A.G.], and a quality offiassumed that a 30% reduction in falls, as had been seen cer [J.v.d.H.]—as the project leader. in 1993, would be feasible. Inspection of the nursing files
March 2006
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
153
and incident report forms indicated a baseline incidence of 9.1 falls per 1,000 patient days in the pilot study in the internal medicine ward; a reduction of 30% would result in a target of 6.0 falls per 1,000 patient days. In the neurology ward, a 30% reduction from the baseline incidence of 15.9 falls per 1,000 patient days in the pilot study would result in 11.0 falls.
Pre-Intervention Activities Revising the Guideline and Identifying and Resolving Barriers to Change The project team, which met every two weeks, updated the falls guideline on the basis of recent developments.17–20 All nurses on the two wards were invited to comment on the revised guideline; the members of the project team subsequently approved the guideline in November 1999. The revised guideline was printed on a small laminated card. The guideline was expected not to result in much additional work because many of the activities it specified (for example, completing the assessment tool and use of the incident registration) should have already been part of the daily work routine. The renewed guideline included the following provisions: ■ Identification of high-risk patients on the basis of a recent fall (less than 6 months ago), unrest or disorientation, hazardous behavior, or refusal to accept advice or orders from nurses. ■ Countermeasures—informing the patient and relatives about the increased risk of falling and taking preventive actions (for example, bed in the lowest position, side rails raised). In the case of the patient’s unrest or agitation, relatives were to be asked to participate in creating a structured environment (for example, a regular place for personal belongings), and, if necessary, using a restraining waist belt. The project team discussed the reasons behind the failed implementation of the guideline in 1993 and asked the nurses in the wards for comments. A clear consensus emerged that the problems were as follows: ■ Dissemination depended on a nurse’s taking action, that is, having to ask for the guideline; nurses did not take the time for this because their daily patient care responsibilities took priority.
154
March 2006
■
Patients considered at high risk for falls were to be designated with a red sticker on the lower end of the bed; this was considered stigmatization of the patient. ■ Having to complete a risk identification index every day was considered too much work. Other barriers to change that were mentioned were as follows: ■ The senior nurses of both wards thought that the problem of falls had lessened since 1993. However, the pre-intervention study showed that the incidence of falls had almost doubled from 1993 to 1999. ■ The absence of a system to systematically identify and report fall incidents in either of the two wards ■ The requirement to complete the incident report form following each fall, which was considered too much work and, according to the nurses, contained many irrelevant questions ■ Lack of top management’s active support for implementation of the guideline ■ Absence or improper functioning of equipment (for example, bed railings that did not fit the bed, cables for televisions found lying on the ground, dangerous steps to toilet entrances) To facilitate the adoption of the guideline, we (the project team) took the following actions: ■ In cooperation with the incident report committee, shortened the incident report form ■ Ensured that every incident was reported on an incident report form and in the nursing file ■ Added a question in the assessment tool to find out whether the patient had experienced a fall in the previous six months ■ Added a question in the daily patient workload classification system to check whether the patient had fallen or had experienced an increased risk of falling ■ Required that patients with an increased risk of falling on the ward be listed on the patient overview board ■ Held discussions about falls in regular team meetings ■ Added the guideline to the orientation program for new employees ■ Ensured a check-up of equipment twice a year ■ Earmarked resources to address hazards ■ Reported monthly on patient falls to the wards’ senior management
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Intervention Period
Data Collection
Once we had revised the guideline and the identified barriers to its implementation, we began the intervention. In December 1999, we introduced the guideline at a meeting to which all nurses, staff members, and charge nurses from the two wards were invited. We explained the project’s goals and handed out the guideline cards, which were also left in the mailboxes of persons unable to attend the meeting. In addition, all nurses received a small flashlight for use during the night shift with the imprint “Fall Prevention 2000.” Staff members and sponsors of the hospital board received a pen with the same inscription accompanied by a letter asking for their support. Before the meeting, letters were sent to all staff members in the wards, including the medical and paramedical staff, informing them of the project. Posters presenting both the revised guideline and the project’s objectives were displayed in the nurses’ coffee room. We provided a separate presentation to the physicians in the neurological ward, during which we asked them to inform the nurses about results of neurological tests and drug prescriptions that could increase the risk of falling. Finally, an article announcing the guideline’s implementation was published in the medical center’s biweekly periodical. We provided the senior nurses and senior management with monthly feedback on fall incidents in their wards and informed the nursing managers, heads of staff, and senior management regularly in writing about the project’s progress. We made a continuous effort to address the identified barriers to change. For example, to lessen hazards we requested check-ups of the brakes of bedside commodes, contacted the technical service for putting TV cables into conduits, and inquired about the planned introduction of new beds in the hospital because many bed railings did not fit properly. To understand the outcome of the implementation process we developed a questionnaire and sent it in January 2001 to all nurses in both of the wards. It contained 25 questions regarding actual knowledge of the guideline, faith in or doubts about the guideline, the importance of the guideline, and possible improvements to the guideline.
The primary outcome was the incidence of falls in both wards. Because we knew from the 1993 study that only half of the fall incidents had been reported on the incident report form, we calculated the fall rate on the basis of the nursing file after a patient’s discharge and unique outcomes reported on the incident report forms. The main secondary outcomes were as follows: ■ Percentage of falls reported by incident report forms ■ Completion of the assessment tool ■ Average number of falls per fallen patient We held several discussions about a focus on injuries due to falls. However, because of the small percentage of severe fractures and potential danger of every fall incident, we decided to consider each fall—regardless of the type of injury—a serious problem. Reviewing nursing files to detect falls turned out to be very time consuming. Therefore, we experimented with an electronic incident-reporting system that was based on the existing system for daily workload measurements. We added two fields to indicate whether the patient was at increased risk of falling and whether the patient had experienced a fall in the last 24 hours. This allowed us to verify the reliability of the number of falls reported on incident report forms without having to read the nursing files. It also provided a mechanism for the nurses for a daily reminder to check the patients’ risk of falling so an intervention could be started, if indicated. Observation data on the implementation process was collected by reviewing the project leader’s and team members’ memos from feedback sessions and other meetings.
March 2006
Results Primary Outcome Figure 2 (page 156) shows the incidence of falls in both wards as extracted from nursing files and incident reports. The monthly figures did not show a reduction in falls, much less meet the target, in either ward. In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward the, incidence target of 11% was met in five months.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
155
Secondary Outcomes
Incidence of Falls in the Internal Medicine and Neurology Wards, January 2000–June 2001
For the neurology ward, an upward trend in percentage of falls reported on incident report forms was evident (Figure 3, page 157). During the intervention period, an average of 67% of the falls were reported on incident report forms, with substantial monthly variation (31%–100%). For the internal medicine ward, an average of 48% of the falls were reported on monthly incident report forms (monthly variation, 17%–100%). During the intervention period, the question added to the assessFigure 2. The monthly data did not show a reduction in falls in either ward ment tool which pertained to a fall for the 18-month intervention. in the previous six months was completed for an average of 80% of the patients on The changes we proposed—removing TV cables, the neurology ward and for an average of 55% of changing dangerous steps in bathrooms, and ensuring the patients on the internal medicine ward (Figure 4, that side railings fit the beds—were not carried out page 158). because of the costs involved and the long implementaThe two added fields in the workload classification tion times. system were completed in the neurology ward for nearly Although the project team had reached consensus 100% of the days. For the internal medicine ward, it was about the barriers to guideline implementation and the less common to use this system on a daily basis before necessary actions to overcome them, putting words the start of the project. Its use was even discouraged into action largely failed. Ward nurses cited their heavy because the ward’s clinical managers did not see its workloads and the belief that “falls can never be comvalue. Use of the system increased at the start of the pletely prevented.” The senior nurses stated that they project but decreased four months later. By the end of were unable to put more effort into implementing the the intervention, fields for risk estimations were being guideline, and as a result, the project was ended in completed for < 20% of the patients. June 2001. The average number of falls per fallen patient, indicating whether the number of multiple falls had been Discussion reduced, showed only a slight change during the interResults indicated that neither the internal medicine vention period for the internal medicine ward. nor the neurology ward was able to achieve a consisThe results of the questionnaire are summarized in tent or lasting decrease in the number of falls, nor Table 1 (page 159). was either ward able to address the contributing factors to reduce the number of falls. It was apparent Actions that the medical center and the senior nurses were Patients at increased risk of falls were not consistentnot sufficiently supportive enough to effect the ly listed on the patient overview board on each of the work process we wanted. The nature of the organizatwo wards. It was said to be too difficult to keep the tional context, as represented in the EFQM Excellence overview board up to date because of patient transfers Model (Figure 1), may help illuminate the project’s to other beds or rooms. lessons.
156
March 2006
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Monthly Percentages of Falls as Reported on Incident Report Forms, January 2000–June 2001
Figure 3. Both the internal medicine and neurology wards showed substantial variation in the monthly percentage of falls reported on incident report forms.
that the actual efforts to implement the guideline were insufficient. The postintervention questionnaire indicated that a majority of the nurses agreed that it was their responsibility for taking preventive actions regarding falls and that guidelines are a useful quality improvement tool (Table 1), but many nurses looked for problem-solving solutions outside their immediate responsibility. In addition, as stated, many nurses expressed the belief that “patients will always fall.” Reward mechanisms for nurses’ use of the guideline were unclear. In addition, physicians only inquired about the number of serious fractures due to falls and thus (unintentionally) minimized the problem.
Leadership
Policy and Strategy
We faced a complex situation reflecting the multi-factorial nature of fall prevention, the unclear nature of nurses’ perceived responsibilities, and difficulty in reaching a consensus with nurses, who worked on different shifts and often did not respond to letters or emails, on actions to be taken. Addressing the barriers to guideline implementation requires support from both the ward management and the medical center’s senior management. Although the clinical ward management underlined the importance of implementing the guideline at the outset of the project, the actual support given was too weak to be effective. Some managers expressed doubt about the project’s chances for success to the project leader, stating that implementation “had already failed before.” Ward staff often regarded improvement activities as unwanted additional work that hindered daily operations. The two senior nurses often displayed a delegating rather than a directive management style,21 for example, in terms of ensuring that the risk assessment tool was completed or all incidents reported.
The wards’ senior nurses and clinical management were primarily concerned with stability and daily operations. Additional efforts necessary to implement the guideline seemed to be perceived as an addition to the workload and sometimes as interfering with daily operations. Three months after the intervention period, a change in policy had taken place in the internal medicine ward. The recently appointed medical chief made stabilization of the ward’s financial position a priority, with a focus on its work processes and capacity and throughput of patients, therefore distracting attention from the implementation of the guideline. Crossdepartmental collaboration was often difficult. For instance, the medical center’s technical department, which had its own priorities, did not carry out the placement of TV cables into conduits.
People Although we were unaware of any disagreement about the importance of the guideline and nurses’ responsibility to offer patients a safe environment, results suggest
March 2006
Partnerships and Resources A lack of help and resources appeared to impede the project. For example, the incident report committee did not provide feedback to the individual wards, and electronic systems for incident reporting were not available. The absence of a reliable information system on falls forced us to expend considerable effort to obtain
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
157
the necessary information. When it became apparent that actions were not being taken regarding the TV cable conduits, bed railings, and bathroom steps, nurses told us that the medical center did not take the falls problem seriously, which therefore undermined their own motivation to contribute to the project’s success.
Monthly Percentages of Fall Risk Questions Completed on the Assessment Tool, January 2000–June 2001
Processes Many processes, such as the use of the incident report forms, assessment tool, and orientation of new employees, which we thought would facilitate guideline implementation, were fragmented and otherwise inadequate. Imperfect work processes also caused workload fluctuations and distracted attention from the implementation activities.
Figure 4. The question about a fall in the previous six months was completed for an average of 80% of the patients (neurology ward) and 55% of the patients (internal medicine ward).
Innovation and Learning This study of guideline implementation shows that a strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implementation strategy, and voluntarily cooperating nurses are no guarantees for success. Leadership support of front-line staff is widely considered a crucial success factor for quality improvement interventions.22 Yet focusing only on leadership neglects the influences of other forces within the ward and within the overall organization. Experience with a variety of projects suggests that an “organizational” check of the environment (with the EFQM Excellence Model as a framework) in which implementation is to occur, and which is only rarely carried out, can be useful in revealing barriers to change. This kind of check can enable use of countermeasures to help ensure the project’s eventual success. The course of the project suggests that implementation has to involve all levels and functions of the organization which play a role and include feedback loops in
158
March 2006
case of unpredicted barriers to change (for example, as in this case, a change of focus in the internal medicine ward). Cultural barriers (such as the difficulty of crossdepartmental collaboration), which played a major role, only became visible once implementation began. The question is whether a large organization such as a medical center is too complex to make changes with a linear implementation model.4 Research suggests that the inability to manage improvement programs as a dynamic process is one of the main determinants of program failure.23
Reflections Examples of failed efforts to implement guidelines are scarce, inevitably reflecting a publication bias. Our initial failure in 1993 with implementation of a falls guideline led us to attempt to understand why we failed, and we may have learned more from that experience than we would have learned if the implementation had been a success. Although we were able to identify a group of early adopters,24 the question arises as to whether one implementation strategy is sufficient for the remaining staff. It might be necessary to gear activities, for example, to those persons not “present at the creation” of the
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Table 1. Results of the Questionnaire (January 2001) Regarding Knowledge of the Guideline and Attitude Toward Implementation*
Response rate Able to recall at least 2 out of 3 predictive fall factors Statements: Falls are not a big problem in the ward It is the responsibility of nurses to take preventive actions No change in working method is required Additional staff is required for application of the guideline There is enough support from the management for guideline implementation IRF registration is important for prevention of falls Absence of proper equipment in the wards is an important obstacle to meet the requirements of the guideline Guidelines are a useful tool for quality improvement in the wards
Neurology (n = 32) 97% 31%
Internal medicine (n = 20) 59% 35%
22% 62% 91% 15% 44% 75%
10% 62% 95% 20% 53% 75%
62%
55%
66%
50%
* IRF, incident report form.
project but whose participation will be needed for successful implementation. It is widely recognized that success requires leadership to play a key role in the process of change.25 Assignment of accountability is necessary to solve a problem where “everybody is involved and therefore nobody is responsible.” Some guidelines, especially those, that like the falls guideline, concern prevention, might not be perceived as sufficiently “challenging” or “inspiring.” Yet we believe that implementation of guidelines should be viewed as part of a wider range of quality improvement initiatives within an organization supportive of learning and improvement. J
This project was funded by the Hospital Board of Directors through the Center for Clinical Practice Guidelines of the Academic Medical Center in Amsterdam. The authors thank J.A. Tutuarima for collecting the data; M.A.E.G. Linskens, M. Sengkerij-Koolstra, E. Diesbergen, M.A.P.M. van den Bergh, N. Diggele, A.C. Brakenhoff, F. Stitou Laaroussi, and P.J. Bakker for their cooperation; and J.C. Wyatt for commenting on the manuscript.
Jelle van der Helm, M.Sc., is Consultant for Quality Improvement Projects, Academic Medical Center at the University of Amsterdam, Amsterdam, The Netherlands; Astrid Goossens, Ph.D., R.N., and Patrick Bossuyt, Ph.D., are Clinical Epidemiologists. Please address correspondence to Jelle van der Helm, [email protected].
References 1. Wensing M., van der Weijden T., Grol R.: Implementing guidelines and innovations in general practice: Which interventions are effective? Br J Gen Pract 48:991–997, Feb. 1998. 2. Oxman A.D., et al.: No magic bullets: A systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J 153:1423–1431, Nov. 15, 1995. 3. Solberg L., et al.: Lessons from experienced guideline implementers: Attend to many factors and use multiple strategies. Jt Comm J Qual Improv 26:171–178, Apr. 2000. 4. Plsek P.E., Greenhalgh T.: The challenge of complexity in health care. BMJ 323:625–628, Sep. 15, 2001. 5. Smith R.: Quality improvement reports: A new kind of article. BMJ 321:1428, Dec. 9, 1999.
March 2006
6. Tutuarima J.A., Limburg M.: Prevention of falls, introduction of guidelines for prevention of falls (internal document; in Dutch). Center for Clinical Practice Guidelines, Academic Medical Center at the University of Amsterdam. Amsterdam, 1993. 7. Semin-Goossens A., van der Helm J.M., Bossuyt P.M.: A failed model-based attempt to implement an evidence-based nursing guideline for fall prevention. J Nurs Care Qual 18:217–225, Jul.–Sep. 2003. 8. Grol R.: Beliefs and evidence in changing clinical practice. BMJ 315:418–421, Aug. 16, 1997. 9. Grol R.: Jones R.: Twenty years of implementation research. Fam Pract 17(suppl 1):S32-S35, Feb. 2000. continued
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
159
References, continued 10. Grol R.: Successes and failures in the implementation of evidencebased guidelines for clinical practice. Med Care 39(8 suppl 2):II46–54, Aug. 2001. 11. Pfeffer J., Sutton R.I.: The Knowing-Doing Gap: How Smart Companies Turn Knowledge into Action. Boston: Harvard Business School Press, 2000. 12. Andersen S.E.: Implementing a new drug record system: A qualitative study of difficulties perceived by physicians and nurses. Qual Saf Health Care 11:19–24, Mar. 2002. 13. van Harten W.H., Casparie T.F., Fisscher O.A., et al.: The evaluation of the introduction of a quality management system: A process-oriented case study in a large rehabilitation hospital. Health Policy 60:17–37, Apr. 2002. 14. Nabitz U., Klazinga N., Walburg J.: The EFQM excellence model: European and Dutch experiences with the EFQM approach in health care. Int J Qual Health Care 12:191–201, Jun. 2000. 15. Gené-Badia J., et al.: The EFQM excellence model is useful for primary health care teams. Fam Pract 18:407–409, Aug. 2001. 16. Naylor G.: Using the business excellence model to develop a strategy for a healthcare organisation. Int J Health Care Qual Assur Inc Leadersh Health Serv 12(2-3):37–44, 1999.
160
March 2006
17. Innes E.M., Turman W.G.: Evaluation of patients. Qual Rev Bull QRB 9:30–35, Feb. 1983. 18. Fife D.D., Solomon P., Stanton M.: A risk falls program: Code orange for success. Nurs Manage 15:50–53, Nov. 1984. 19. Oliver D., et al.: Development and evaluation of evidence based risk assessment tool to predict which elderly inpatients will fall: Case-control and cohort studies. BMJ 315:1049–1053, Oct. 25, 1997. 20. Evans D., et al.: Fall prevention: A systematic review. Clinical Effectiveness in Nursing 3:106–111, Sep. 1999. 21. Zirgami P., et al.: Leadership and the One Minute Manager: Increasing Effectiveness through Situational Leadership. New York: William Morrow & Co., 1999. 22. Kotter J.P.: Leading Change. Boston: Harvard Business School Press, 1996. 23. Keating et al.: Overcoming the improvement paradox. European Management Journal 17:120–134, Summer 1999. 24. Rogers E.: Diffusion of Innovations. New York: Free Press, 1983. 25. Firth-Cozens J., Mowbray D.: Leadership and the quality of care. Qual Health Care 10(suppl 2):ii3–ii7, Dec. 2001.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
When Implementation Fails: The Case of a Nursing Guideline for Fall Prevention t is well established that implementing guidelines can be very difficult.1 No magic bullet or step-bystep implementation plan is available, neither is any single implementation strategy superior in achieving change.2 Given the mixture of professional and organizational factors that seem to be crucial to successful implementation, it is advisable that multiple strategies be used.3 Considerable effort has been put in developing structured step-by-step implementation approaches, yet in practice, health care systems seem to behave as complex adaptive systems.4 Little guidance can be obtained from the literature on why implementation might fail in spite of a well-prepared guideline and a sound implementation strategy.5 At the Academic Medical Center (AMC) in Amsterdam, the Netherlands, a teaching hospital with 1,000 beds, a nursing guideline was developed in 1993 on prevention of patient falls. In a 1993 study, a 30% reduction in the incidence of falls (in terms of the number of falls per 1,000 patient days) on six wards was reported.6 This result was widely disseminated within the hospital and held as promise for solving a major nursing-related problem, but an effort to implement the guideline into daily practice throughout the hospital failed. A renewed effort to implement the guideline in two wards was made in 1999. Yet this effort failed, too; in this article we will explore why. Additional details about the implementation model followed can be found elsewhere.7
I
Theoretical Framework During the last few years, a multifaceted approach has been advocated for changing physician behavior. Grol and colleagues developed a model for implementing
152
March 2006
Jelle van der Helm, M.Sc. Astrid Goossens, Ph.D., R.N. Patrick Bossuyt, Ph.D.
Article-at-a-Glance Background: Implementing guidelines can be very difficult. No magic bullet or step-by-step implementation plan is available, neither is any single implementation strategy superior. At the Academic Medical Center (AMC) in Amsterdam, a nursing guideline was developed in 1993 on prevention of patient falls. Falls decreased by 30% on six wards, yet an effort to implement the guideline into daily practice throughout the hospital failed. A renewed effort was made to implement the guideline in two wards (neurology and internal medicine) in 1999. Implementing the Guideline: Preparations were made for implementation in the two wards. Barriers to change were identified and solutions were translated into day-to-day activities in the wards. The intervention period covered 18 months (January 2000–June 2001). A mix of implementation strategies was used, including a local consensus process, educational activities, and active support and feedback to management and staff. Results: In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward, the incidence target of 11% was met in five months. Discussion: Barriers to change and enabling factors may only become apparent during the implementation process itself. A strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implementation strategy, and voluntarily cooperating nurses are no guarantees for success.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
European Foundation for Quality Management (EFQM) Excellence Model
Setting
The implementation activities took place in an internal medicine ward and a neurology ward. The neurology ward has 32 beds staffed by one senior nurse and 33 nurses and has 850 admissions per year. The internal medicine ward is staffed by one senior nurse and 55 nurses and consists of 60 beds; there are 1,500 admissions per year. These wards voluntarily participated in the study in an effort to reduce the number of falls. Both wards had been involved in the development of Figure 1. The EFQM model depicts the organizational context for achieving the guideline in 1993, when a 30% quality improvement. Used by permission of the European Foundation for reduction in falls was reported; Quality Management, Brussels. however, this reduction was not sustained. For example, in 1998 the internal medicine change that focuses on identifying and resolving barriers 8 ward and neurology ward reported a total of 100 falls to to change. It has been shown to be effective, primarily 9,10 the incident report committee, which means that every in changing the behavior of general practitioners. 3.5 days one patient experienced a fall in one of these Implementation involves the actual use of developed two wards. Because it is well known that not all falls are knowledge. Knowledge is often conceptualized as some11 reported, the project team assumed that the actual numthing that can be caught, managed, and distributed. Yet ber of falls was even higher. other recent approaches suggest that its use and mean12,13 ing are influenced by the organizational context. The European Foundation for Quality Management (EFQM) Outline Excellence Model, as shown in Figure 1 (above), is freAn implementation study was conducted, starting quently used in Europe to describe the organizational with a pilot study of three months (July–September context for achieving quality improvement.14–16 1999). The pilot was used as a benchmark to enable According to the model, customer satisfaction, peodetermination of the goal for implementation. From ple satisfaction, and impact on society are achieved September 1999 until January 2000, preparations through leadership, policy and strategy, people, partnerwere made for implementation in the two wards. In ships and resources, processes, and feedback of the this pre-intervention period, barriers to change were results to the organization—all of which comprise the identified and solutions were translated into day-toorganizational context. day activities in the wards. The intervention period covered 18 months (January 2000–June 2001). At proImplementing the Guideline ject’s end, a questionnaire was developed to evaluate Establishing the Project Team the results. In July 1999 a project team was formed that consisted of Because the initial study in 1993 was performed in the four nurses and the senior (charge) nurse for each of the same wards and the nurses told us that reintroduction of wards (internal medicine and neurology), a clinical epithe guideline had not been planned, the project team demiologist and research nurse [A.G.], and a quality offiassumed that a 30% reduction in falls, as had been seen cer [J.v.d.H.]—as the project leader. in 1993, would be feasible. Inspection of the nursing files
March 2006
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
153
and incident report forms indicated a baseline incidence of 9.1 falls per 1,000 patient days in the pilot study in the internal medicine ward; a reduction of 30% would result in a target of 6.0 falls per 1,000 patient days. In the neurology ward, a 30% reduction from the baseline incidence of 15.9 falls per 1,000 patient days in the pilot study would result in 11.0 falls.
Pre-Intervention Activities Revising the Guideline and Identifying and Resolving Barriers to Change The project team, which met every two weeks, updated the falls guideline on the basis of recent developments.17–20 All nurses on the two wards were invited to comment on the revised guideline; the members of the project team subsequently approved the guideline in November 1999. The revised guideline was printed on a small laminated card. The guideline was expected not to result in much additional work because many of the activities it specified (for example, completing the assessment tool and use of the incident registration) should have already been part of the daily work routine. The renewed guideline included the following provisions: ■ Identification of high-risk patients on the basis of a recent fall (less than 6 months ago), unrest or disorientation, hazardous behavior, or refusal to accept advice or orders from nurses. ■ Countermeasures—informing the patient and relatives about the increased risk of falling and taking preventive actions (for example, bed in the lowest position, side rails raised). In the case of the patient’s unrest or agitation, relatives were to be asked to participate in creating a structured environment (for example, a regular place for personal belongings), and, if necessary, using a restraining waist belt. The project team discussed the reasons behind the failed implementation of the guideline in 1993 and asked the nurses in the wards for comments. A clear consensus emerged that the problems were as follows: ■ Dissemination depended on a nurse’s taking action, that is, having to ask for the guideline; nurses did not take the time for this because their daily patient care responsibilities took priority.
154
March 2006
■
Patients considered at high risk for falls were to be designated with a red sticker on the lower end of the bed; this was considered stigmatization of the patient. ■ Having to complete a risk identification index every day was considered too much work. Other barriers to change that were mentioned were as follows: ■ The senior nurses of both wards thought that the problem of falls had lessened since 1993. However, the pre-intervention study showed that the incidence of falls had almost doubled from 1993 to 1999. ■ The absence of a system to systematically identify and report fall incidents in either of the two wards ■ The requirement to complete the incident report form following each fall, which was considered too much work and, according to the nurses, contained many irrelevant questions ■ Lack of top management’s active support for implementation of the guideline ■ Absence or improper functioning of equipment (for example, bed railings that did not fit the bed, cables for televisions found lying on the ground, dangerous steps to toilet entrances) To facilitate the adoption of the guideline, we (the project team) took the following actions: ■ In cooperation with the incident report committee, shortened the incident report form ■ Ensured that every incident was reported on an incident report form and in the nursing file ■ Added a question in the assessment tool to find out whether the patient had experienced a fall in the previous six months ■ Added a question in the daily patient workload classification system to check whether the patient had fallen or had experienced an increased risk of falling ■ Required that patients with an increased risk of falling on the ward be listed on the patient overview board ■ Held discussions about falls in regular team meetings ■ Added the guideline to the orientation program for new employees ■ Ensured a check-up of equipment twice a year ■ Earmarked resources to address hazards ■ Reported monthly on patient falls to the wards’ senior management
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Intervention Period
Data Collection
Once we had revised the guideline and the identified barriers to its implementation, we began the intervention. In December 1999, we introduced the guideline at a meeting to which all nurses, staff members, and charge nurses from the two wards were invited. We explained the project’s goals and handed out the guideline cards, which were also left in the mailboxes of persons unable to attend the meeting. In addition, all nurses received a small flashlight for use during the night shift with the imprint “Fall Prevention 2000.” Staff members and sponsors of the hospital board received a pen with the same inscription accompanied by a letter asking for their support. Before the meeting, letters were sent to all staff members in the wards, including the medical and paramedical staff, informing them of the project. Posters presenting both the revised guideline and the project’s objectives were displayed in the nurses’ coffee room. We provided a separate presentation to the physicians in the neurological ward, during which we asked them to inform the nurses about results of neurological tests and drug prescriptions that could increase the risk of falling. Finally, an article announcing the guideline’s implementation was published in the medical center’s biweekly periodical. We provided the senior nurses and senior management with monthly feedback on fall incidents in their wards and informed the nursing managers, heads of staff, and senior management regularly in writing about the project’s progress. We made a continuous effort to address the identified barriers to change. For example, to lessen hazards we requested check-ups of the brakes of bedside commodes, contacted the technical service for putting TV cables into conduits, and inquired about the planned introduction of new beds in the hospital because many bed railings did not fit properly. To understand the outcome of the implementation process we developed a questionnaire and sent it in January 2001 to all nurses in both of the wards. It contained 25 questions regarding actual knowledge of the guideline, faith in or doubts about the guideline, the importance of the guideline, and possible improvements to the guideline.
The primary outcome was the incidence of falls in both wards. Because we knew from the 1993 study that only half of the fall incidents had been reported on the incident report form, we calculated the fall rate on the basis of the nursing file after a patient’s discharge and unique outcomes reported on the incident report forms. The main secondary outcomes were as follows: ■ Percentage of falls reported by incident report forms ■ Completion of the assessment tool ■ Average number of falls per fallen patient We held several discussions about a focus on injuries due to falls. However, because of the small percentage of severe fractures and potential danger of every fall incident, we decided to consider each fall—regardless of the type of injury—a serious problem. Reviewing nursing files to detect falls turned out to be very time consuming. Therefore, we experimented with an electronic incident-reporting system that was based on the existing system for daily workload measurements. We added two fields to indicate whether the patient was at increased risk of falling and whether the patient had experienced a fall in the last 24 hours. This allowed us to verify the reliability of the number of falls reported on incident report forms without having to read the nursing files. It also provided a mechanism for the nurses for a daily reminder to check the patients’ risk of falling so an intervention could be started, if indicated. Observation data on the implementation process was collected by reviewing the project leader’s and team members’ memos from feedback sessions and other meetings.
March 2006
Results Primary Outcome Figure 2 (page 156) shows the incidence of falls in both wards as extracted from nursing files and incident reports. The monthly figures did not show a reduction in falls, much less meet the target, in either ward. In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward the, incidence target of 11% was met in five months.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
155
Secondary Outcomes
Incidence of Falls in the Internal Medicine and Neurology Wards, January 2000–June 2001
For the neurology ward, an upward trend in percentage of falls reported on incident report forms was evident (Figure 3, page 157). During the intervention period, an average of 67% of the falls were reported on incident report forms, with substantial monthly variation (31%–100%). For the internal medicine ward, an average of 48% of the falls were reported on monthly incident report forms (monthly variation, 17%–100%). During the intervention period, the question added to the assessFigure 2. The monthly data did not show a reduction in falls in either ward ment tool which pertained to a fall for the 18-month intervention. in the previous six months was completed for an average of 80% of the patients on The changes we proposed—removing TV cables, the neurology ward and for an average of 55% of changing dangerous steps in bathrooms, and ensuring the patients on the internal medicine ward (Figure 4, that side railings fit the beds—were not carried out page 158). because of the costs involved and the long implementaThe two added fields in the workload classification tion times. system were completed in the neurology ward for nearly Although the project team had reached consensus 100% of the days. For the internal medicine ward, it was about the barriers to guideline implementation and the less common to use this system on a daily basis before necessary actions to overcome them, putting words the start of the project. Its use was even discouraged into action largely failed. Ward nurses cited their heavy because the ward’s clinical managers did not see its workloads and the belief that “falls can never be comvalue. Use of the system increased at the start of the pletely prevented.” The senior nurses stated that they project but decreased four months later. By the end of were unable to put more effort into implementing the the intervention, fields for risk estimations were being guideline, and as a result, the project was ended in completed for < 20% of the patients. June 2001. The average number of falls per fallen patient, indicating whether the number of multiple falls had been Discussion reduced, showed only a slight change during the interResults indicated that neither the internal medicine vention period for the internal medicine ward. nor the neurology ward was able to achieve a consisThe results of the questionnaire are summarized in tent or lasting decrease in the number of falls, nor Table 1 (page 159). was either ward able to address the contributing factors to reduce the number of falls. It was apparent Actions that the medical center and the senior nurses were Patients at increased risk of falls were not consistentnot sufficiently supportive enough to effect the ly listed on the patient overview board on each of the work process we wanted. The nature of the organizatwo wards. It was said to be too difficult to keep the tional context, as represented in the EFQM Excellence overview board up to date because of patient transfers Model (Figure 1), may help illuminate the project’s to other beds or rooms. lessons.
156
March 2006
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Monthly Percentages of Falls as Reported on Incident Report Forms, January 2000–June 2001
Figure 3. Both the internal medicine and neurology wards showed substantial variation in the monthly percentage of falls reported on incident report forms.
that the actual efforts to implement the guideline were insufficient. The postintervention questionnaire indicated that a majority of the nurses agreed that it was their responsibility for taking preventive actions regarding falls and that guidelines are a useful quality improvement tool (Table 1), but many nurses looked for problem-solving solutions outside their immediate responsibility. In addition, as stated, many nurses expressed the belief that “patients will always fall.” Reward mechanisms for nurses’ use of the guideline were unclear. In addition, physicians only inquired about the number of serious fractures due to falls and thus (unintentionally) minimized the problem.
Leadership
Policy and Strategy
We faced a complex situation reflecting the multi-factorial nature of fall prevention, the unclear nature of nurses’ perceived responsibilities, and difficulty in reaching a consensus with nurses, who worked on different shifts and often did not respond to letters or emails, on actions to be taken. Addressing the barriers to guideline implementation requires support from both the ward management and the medical center’s senior management. Although the clinical ward management underlined the importance of implementing the guideline at the outset of the project, the actual support given was too weak to be effective. Some managers expressed doubt about the project’s chances for success to the project leader, stating that implementation “had already failed before.” Ward staff often regarded improvement activities as unwanted additional work that hindered daily operations. The two senior nurses often displayed a delegating rather than a directive management style,21 for example, in terms of ensuring that the risk assessment tool was completed or all incidents reported.
The wards’ senior nurses and clinical management were primarily concerned with stability and daily operations. Additional efforts necessary to implement the guideline seemed to be perceived as an addition to the workload and sometimes as interfering with daily operations. Three months after the intervention period, a change in policy had taken place in the internal medicine ward. The recently appointed medical chief made stabilization of the ward’s financial position a priority, with a focus on its work processes and capacity and throughput of patients, therefore distracting attention from the implementation of the guideline. Crossdepartmental collaboration was often difficult. For instance, the medical center’s technical department, which had its own priorities, did not carry out the placement of TV cables into conduits.
People Although we were unaware of any disagreement about the importance of the guideline and nurses’ responsibility to offer patients a safe environment, results suggest
March 2006
Partnerships and Resources A lack of help and resources appeared to impede the project. For example, the incident report committee did not provide feedback to the individual wards, and electronic systems for incident reporting were not available. The absence of a reliable information system on falls forced us to expend considerable effort to obtain
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
157
the necessary information. When it became apparent that actions were not being taken regarding the TV cable conduits, bed railings, and bathroom steps, nurses told us that the medical center did not take the falls problem seriously, which therefore undermined their own motivation to contribute to the project’s success.
Monthly Percentages of Fall Risk Questions Completed on the Assessment Tool, January 2000–June 2001
Processes Many processes, such as the use of the incident report forms, assessment tool, and orientation of new employees, which we thought would facilitate guideline implementation, were fragmented and otherwise inadequate. Imperfect work processes also caused workload fluctuations and distracted attention from the implementation activities.
Figure 4. The question about a fall in the previous six months was completed for an average of 80% of the patients (neurology ward) and 55% of the patients (internal medicine ward).
Innovation and Learning This study of guideline implementation shows that a strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implementation strategy, and voluntarily cooperating nurses are no guarantees for success. Leadership support of front-line staff is widely considered a crucial success factor for quality improvement interventions.22 Yet focusing only on leadership neglects the influences of other forces within the ward and within the overall organization. Experience with a variety of projects suggests that an “organizational” check of the environment (with the EFQM Excellence Model as a framework) in which implementation is to occur, and which is only rarely carried out, can be useful in revealing barriers to change. This kind of check can enable use of countermeasures to help ensure the project’s eventual success. The course of the project suggests that implementation has to involve all levels and functions of the organization which play a role and include feedback loops in
158
March 2006
case of unpredicted barriers to change (for example, as in this case, a change of focus in the internal medicine ward). Cultural barriers (such as the difficulty of crossdepartmental collaboration), which played a major role, only became visible once implementation began. The question is whether a large organization such as a medical center is too complex to make changes with a linear implementation model.4 Research suggests that the inability to manage improvement programs as a dynamic process is one of the main determinants of program failure.23
Reflections Examples of failed efforts to implement guidelines are scarce, inevitably reflecting a publication bias. Our initial failure in 1993 with implementation of a falls guideline led us to attempt to understand why we failed, and we may have learned more from that experience than we would have learned if the implementation had been a success. Although we were able to identify a group of early adopters,24 the question arises as to whether one implementation strategy is sufficient for the remaining staff. It might be necessary to gear activities, for example, to those persons not “present at the creation” of the
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
Table 1. Results of the Questionnaire (January 2001) Regarding Knowledge of the Guideline and Attitude Toward Implementation*
Response rate Able to recall at least 2 out of 3 predictive fall factors Statements: Falls are not a big problem in the ward It is the responsibility of nurses to take preventive actions No change in working method is required Additional staff is required for application of the guideline There is enough support from the management for guideline implementation IRF registration is important for prevention of falls Absence of proper equipment in the wards is an important obstacle to meet the requirements of the guideline Guidelines are a useful tool for quality improvement in the wards
Neurology (n = 32) 97% 31%
Internal medicine (n = 20) 59% 35%
22% 62% 91% 15% 44% 75%
10% 62% 95% 20% 53% 75%
62%
55%
66%
50%
* IRF, incident report form.
project but whose participation will be needed for successful implementation. It is widely recognized that success requires leadership to play a key role in the process of change.25 Assignment of accountability is necessary to solve a problem where “everybody is involved and therefore nobody is responsible.” Some guidelines, especially those, that like the falls guideline, concern prevention, might not be perceived as sufficiently “challenging” or “inspiring.” Yet we believe that implementation of guidelines should be viewed as part of a wider range of quality improvement initiatives within an organization supportive of learning and improvement. J
This project was funded by the Hospital Board of Directors through the Center for Clinical Practice Guidelines of the Academic Medical Center in Amsterdam. The authors thank J.A. Tutuarima for collecting the data; M.A.E.G. Linskens, M. Sengkerij-Koolstra, E. Diesbergen, M.A.P.M. van den Bergh, N. Diggele, A.C. Brakenhoff, F. Stitou Laaroussi, and P.J. Bakker for their cooperation; and J.C. Wyatt for commenting on the manuscript.
Jelle van der Helm, M.Sc., is Consultant for Quality Improvement Projects, Academic Medical Center at the University of Amsterdam, Amsterdam, The Netherlands; Astrid Goossens, Ph.D., R.N., and Patrick Bossuyt, Ph.D., are Clinical Epidemiologists. Please address correspondence to Jelle van der Helm, [email protected].
References 1. Wensing M., van der Weijden T., Grol R.: Implementing guidelines and innovations in general practice: Which interventions are effective? Br J Gen Pract 48:991–997, Feb. 1998. 2. Oxman A.D., et al.: No magic bullets: A systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J 153:1423–1431, Nov. 15, 1995. 3. Solberg L., et al.: Lessons from experienced guideline implementers: Attend to many factors and use multiple strategies. Jt Comm J Qual Improv 26:171–178, Apr. 2000. 4. Plsek P.E., Greenhalgh T.: The challenge of complexity in health care. BMJ 323:625–628, Sep. 15, 2001. 5. Smith R.: Quality improvement reports: A new kind of article. BMJ 321:1428, Dec. 9, 1999.
March 2006
6. Tutuarima J.A., Limburg M.: Prevention of falls, introduction of guidelines for prevention of falls (internal document; in Dutch). Center for Clinical Practice Guidelines, Academic Medical Center at the University of Amsterdam. Amsterdam, 1993. 7. Semin-Goossens A., van der Helm J.M., Bossuyt P.M.: A failed model-based attempt to implement an evidence-based nursing guideline for fall prevention. J Nurs Care Qual 18:217–225, Jul.–Sep. 2003. 8. Grol R.: Beliefs and evidence in changing clinical practice. BMJ 315:418–421, Aug. 16, 1997. 9. Grol R.: Jones R.: Twenty years of implementation research. Fam Pract 17(suppl 1):S32-S35, Feb. 2000. continued
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations
159
References, continued 10. Grol R.: Successes and failures in the implementation of evidencebased guidelines for clinical practice. Med Care 39(8 suppl 2):II46–54, Aug. 2001. 11. Pfeffer J., Sutton R.I.: The Knowing-Doing Gap: How Smart Companies Turn Knowledge into Action. Boston: Harvard Business School Press, 2000. 12. Andersen S.E.: Implementing a new drug record system: A qualitative study of difficulties perceived by physicians and nurses. Qual Saf Health Care 11:19–24, Mar. 2002. 13. van Harten W.H., Casparie T.F., Fisscher O.A., et al.: The evaluation of the introduction of a quality management system: A process-oriented case study in a large rehabilitation hospital. Health Policy 60:17–37, Apr. 2002. 14. Nabitz U., Klazinga N., Walburg J.: The EFQM excellence model: European and Dutch experiences with the EFQM approach in health care. Int J Qual Health Care 12:191–201, Jun. 2000. 15. Gené-Badia J., et al.: The EFQM excellence model is useful for primary health care teams. Fam Pract 18:407–409, Aug. 2001. 16. Naylor G.: Using the business excellence model to develop a strategy for a healthcare organisation. Int J Health Care Qual Assur Inc Leadersh Health Serv 12(2-3):37–44, 1999.
160
March 2006
17. Innes E.M., Turman W.G.: Evaluation of patients. Qual Rev Bull QRB 9:30–35, Feb. 1983. 18. Fife D.D., Solomon P., Stanton M.: A risk falls program: Code orange for success. Nurs Manage 15:50–53, Nov. 1984. 19. Oliver D., et al.: Development and evaluation of evidence based risk assessment tool to predict which elderly inpatients will fall: Case-control and cohort studies. BMJ 315:1049–1053, Oct. 25, 1997. 20. Evans D., et al.: Fall prevention: A systematic review. Clinical Effectiveness in Nursing 3:106–111, Sep. 1999. 21. Zirgami P., et al.: Leadership and the One Minute Manager: Increasing Effectiveness through Situational Leadership. New York: William Morrow & Co., 1999. 22. Kotter J.P.: Leading Change. Boston: Harvard Business School Press, 1996. 23. Keating et al.: Overcoming the improvement paradox. European Management Journal 17:120–134, Summer 1999. 24. Rogers E.: Diffusion of Innovations. New York: Free Press, 1983. 25. Firth-Cozens J., Mowbray D.: Leadership and the quality of care. Qual Health Care 10(suppl 2):ii3–ii7, Dec. 2001.
Volume 32 Number 3
Copyright 2006 Joint Commission on Accreditation of Healthcare Organizations