- Email: [email protected]
Introducing Practice-Based Learning and Improvement ACGME Core Competencies into a Family Medicine Residency Curriculum
Joint Commission
Journal on Quality and Safety
Health Professions Education
Introducing Practice-Based Learning and Improvement ACGME Core Competencies into a Family Medicine Residency Curriculum
Mary Thoesen Coleman, MD, PhD Soraya Nasraty, MD Michael Ostapchuk, MD Stephen Wheeler, MD Stephen Looney, PhD Sandra Rhodes
he ability to learn from and improve on the care of patients in one’s practice is recognized as an important skill for family physicians. Practice-based learning and improvement is one of the six general competencies approved by the Accreditation Council for Graduate Medical Education (ACGME; Chicago) in its program requirements for residency.1,2 The 2001 strategic plan for the Society for Teachers of Family Medicine (Leawood, Kan) made quality improvement (QI) one of its five major goals and established a task force whose primary focus is the integration of QI into residency education and practice.3 The American Academy of Family Physicians (Leawood, Kan) also recommends continuous quality improvement and total quality management as core knowledge and skills.4 Yet introduction of practice-based learning and improvement into family medicine residency training5–7 and other residency training programs8–16 has been limited. This article describes how a family medicine residency training program at the Department of Family and Community Medicine at the University of Louisville incorporated improvement into the daily work of residents, faculty, and staff at their ambulatory care sites and simultaneously addressed assessment of ACGME competencies related to practice-based learning and improvement. The family medicine residency program addressed the following specific ACGME competencies related to practice-based learning and improvement:
T
238
May 2003
Article-at-a-Glance Background: The Accreditation Council for Graduate Medical Education (ACGME) recommends integrating improvement activities into residency training. A curricular change was designed at the Department of Family and Community Medicine, University of Louisville, to address selected ACGME competencies by incorporating practice-based improvement activities into the routine clinical work of family medicine residents. Methods: Teams of residents, faculty, and office staff completed clinical improvement projects at three ambulatory care training sites. Residents were given academic credit for participation in team meetings. After 6 months, residents presented results to faculty, medical students, other residents, and staff from all three training sites. Residents, staff, and faculty were recognized for their participation. Program evaluation: Resident teams demonstrated ACGME competencies in practice-based improvement: Chart audits indicated improvement in clinical projects; quality improvement tools demonstrated analysis of root causes and understanding of the process; plan-do-studyact cycle worksheets demonstrated the change process. Conclusions: Improvement activities that affect patient care and demonstrate selected ACGME competencies can be successfully incorporated into the daily work of family medicine residents.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
1. Analyze practice experience and perform practicebased improvement activities using a systematic methodology; 2. Locate, appraise, and assimilate evidence from scientific studies related to patients’ health problems; 3. Obtain and use information about their own population of patients and the larger population from which their patients are drawn; and 4. Facilitate learning of students and other health professionals.2
Curriculum The plan to incorporate improvement into the curriculum of the family medicine residency program was spearheaded in fall 1999 by the director of the QI curriculum of the department, a faculty member with QI training [M.T.C.]. The plan consisted of 1. providing the needed knowledge for improvement work; 2. engendering support from leaders and participants; 3. creating a structure to facilitate improvement activities; and 4. supplying incentives to encourage and reward success. In July 2000 teams composed of residents, faculty, and staff from each site were asked to initiate and demonstrate improvement in clinical foci of their choice for 6 months. Subsequent 6-month cycles provided opportunities to choose other clinical foci and to improve the processes. To provide teams with the needed knowledge and skills to accomplish clinical improvement goals, the family medicine residency program introduced QI tools, processes, and philosophy using interactive 1-hour monthly sessions that were incorporated into the weekly residency didactic core conference series (Table 1, p 240, teaching methods for Objectives 1 and 2). The director of the QI curriculum chose topics based on a framework that includes eight domains of improvement.17 Staff from the clinical sites were invited to attend the seven monthly sessions, which included team-building exercises; application of flowcharts to process analysis; use of nominal group technique for decision making; performance of a literature search; understanding of variation; and use of run charts, bar
May 2003
graphs, and histograms for data display. Teaching methods for didactic sessions included lecture and discussion, small-group activities, panel presentations, and role playing. Residents’ experiential learning was designed to occur at team meetings during which planning, execution, and evaluation of improvement projects occurred. Residents were informed that assessment of their projects would include demonstration of teamwork, application of evidence-based medicine, and implementation of a plan-do-study-act (PDSA) learning cycle and measured clinical outcomes (Table 1, teaching and assessment strategies for objectives 1, 2, 3). On the basis of recommendations from the Breakthrough Series sponsored by the Institute for Healthcare Improvement (IHI; Boston),18 teams were encouraged to create specific goals with thresholds that would challenge the system. Support for integrating improvement activities into the curriculum was developed using several strategies. The director of the improvement curriculum sought and obtained approval from leadership by specifically seeking approval from the chairperson of the department and the director of the residency program for proposed changes. Clinical directors were asked to participate in the projects and benefited from having time allotted for team meetings that alternated between staff meetings for clinical operations and improvement work. Familiarizing residents with ACGME requirements and then explicitly linking improvement activities to demonstration of ACGME competencies was used to foster support from the residents and faculty. Time for the team meetings was secured on Thursday afternoons (3:30–5:00 PM) after core conferences (12:00–3:00 PM), when available clinical hours were already limited. An analysis of the patient volume seen by residents in the ambulatory care sites on Thursday afternoons after conference sessions suggested a limited impact on revenue and service. Incentives for resident participation included academic credit for participation in team meetings. Residents are required to attend a certain percentage of core conference sessions; the residency director approved credit for participation in team meetings. In addition, each resident team was required to give a
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
239
Joint Commission
Journal on Quality and Safety
Table 1. Practice-Based Learning and Improvement ACGME Competencies: Teaching Methods and Assessment Strategies* ACGME Objective 1. Analyze practice experience and perform practicebased improvement activities using systematic methodology
2. Locate, appraise, and assimilate evidence from scientific studies related to patients' health problems.
3. Obtain and use information about own population of patients and larger population from which those patients are drawn. 4. Facilitate learning of students and other health care professionals.
Teaching Method Didactic sessions on flowcharts/PDSA cycles. Experiential: Teams create flowchart to identify root causes of problems in practice. Teams design learning cycle to make changes in practice. Teams determine and measure outcomes to demonstrate effect of change. Didactic session on performing literature search. Residents complete literature search to document evidence for intervention.
Teams perform chart audits in own practices. Teams may perform comparison audits of other practices. Residents prepare presentations for sharing with group.
Assessment Strategy Panel of experts evaluates resident’s presentation for presence of (1) Flowchart demonstrating identification of root causes. (2) Completion of PDSA cycle worksheet used for intervention. (3) Outcomes of change intervention.
Results Flowcharts of change processes presented for Sites 2, 3. PDSA cycles presented for Sites 1, 2, 3. Outcomes presented for change intervention for sites 1, 2, 3 (Figure 1, p 243).
Panel of experts evaluates for presence of critical review of literature supporting the intervention, including (1) list of keywords used in search, (2) databases searched, and (3) results. Panel of experts evaluates for presence of clinical outcomes using chart audit data from baseline and postintervention chart audits. Panel of experts evaluates for presentation skills.
Keywords identified for Sites 1, 2, 3. Databases searched identified for Sites 1, 2, 3. Results presented for Sites 1, 3.
Chart audit data (Figure 1) presented for Sites 1, 2, 3.
Presentation for Sites 1, 2, 3 Sites 1, 2 presented by many residents; Site 3 presented by single resident.
ACGME, Accreditation Council for Graduate Medical Education; PDSA, plan-do-study-act.
presentation at the completion of the project to a large group (Table 1, teaching method and assessment strategy for Objective 4), including evaluators from key QI positions within hospital institutions and the chairperson of the department. The team that was deemed to perform best was promised a dinner party. Each team was told to choose a project that it considered important to improving clinical care at its site. It was also instructed in the use of criteria such as feasibility and impact and in the use of multivoting to reach consensus.
240
May 2003
Individual Clinical Improvement Projects at Training Sites Site 1 Team. The team consisted of 4 faculty, 12 residents, and 5 office staff. Background. Incomplete medication lists in the chart were making it difficult to determine if refills for medications were appropriate. The time it took staff to research the appropriateness markedly decreased efficiency in authorizing refills. The team was also concerned that incomplete medication lists might be leading to errors that could affect patient safety.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Literature search for evidence. Residents searched the following resources for evidence that updating medication lists would improve care: ■ Ovid, an online medical information service; ■ The online journal Evidence-Based Medicine, 1991–2000; ■ MEDLINE, 1966–2000; ■ The Web site for the American Academy of Family Practice, www.aafp.org; and ■ The journal Family Practice Management. They used the following terms as keywords: medications, medication list, outpatient charts, clinic charts, medication list update ambulatory care, QI in outpatient care, drug list, current therapy list, prescriptions, medications, and charts. The search identified five articles of interest which indicated that completion of medication lists was important in efforts to improve care.19–23 General aim. Improve efficiency and reduce errors by increasing completion of medication lists. Specific goal. Improve completion of medication lists by 30% within 3 months. Intervention. After some discussion the team determined that the likely root cause for incomplete lists was the failure to remember to complete them. The team created a prompt on the progress note consisting of a check box to remind providers to update the medication list. Method of measuring performance. 100 baseline and 50 follow-up, randomly selected charts were reviewed for documentation of chronic illness medications, refill authorizations, and medication allergies.
Site 2 Team. The team consisted of 5 faculty, 2 nurse practitioners, 1 physician assistant, 6 residents, and 10 office staff. Background. The adult data summary sheet had been previously developed as an accessible, clinically relevant chart face sheet presenting key information in eight areas: problem list, past history, family history, social history, advanced directive, special tests/procedures, preventive tests, and immunizations. Providers believed its routine use would improve ambulatory care, continuity of care, communication among providers, routine health maintenance, and efficient transfer of information to the inpatient team when a patient is
May 2003
hospitalized. Furthermore, they felt its irregular use was compromising services. Literature search for evidence. Residents used the same literature sources as Site 1 in a searched for evidence that chart summary sheets improve patient care. Keywords used were charts, outpatient charts, clinic charts, office charts, medical record, data sheet, summary sheet, and face sheet. The search did not substantiate the hypothesis that regular use of a data summary sheet improved communication or care. Benchmarking. An audit of 36 charts (convenience sample) showed a completion rate of 14%. An audit of 100 charts at an affiliated practice of faculty and residents found a completion rate of 11%. Charts were included if the patient had had three or more prior visits to the site. General aim. Improve communications among providers and routine health maintenance of patients by increasing the routine completion of the adult data summary sheets. Specific goal. Increase completion of adult data summary sheets by 60% during the study period of August 10–November 30, 2000. Completion was defined as having 6 or more (of the 8 possible) areas addressed on any given sheet. Intervention. Providers were reminded of the importance of the summary sheets during clinic meetings. Medical assistants developed a specific stamp for use on each current progress note to remind providers to complete the summary sheet. Staff members finding incomplete sheets returned the charts to the provider for completion. Method for measuring performance. A convenience sample of 35 charts of patients who had been seen at least once during the study period was selected for review by a staff member with no known bias.
Site 3 Team. The team consisted of 2 faculty, 6 residents, and 5 office staff. Background. The team chose to improve microalbumin screening for its potential impact on patient care and its capability for accurate measurement. Literature search for evidence. The residents searched for evidence that microalbumin screening
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
241
Joint Commission
Journal on Quality and Safety
would improve care of patients with diabetes. They also searched for different tests available for microalbuminuria that could be used in the office setting. The following keywords were used in the Ovid and MEDLINE 1966–2000 searches: diabetes mellitus, screening tests for diabetes, microalbumin, rapid screening tests for micro albumin, and diabetic nephropathy. Eleven articles24–34 supported the importance of microalbumin as the first clinical sign of nephropathy and the leading cause of end-stage renal disease in the United States. Evidence indicated that improved glycemic control and medication choices in patients with microalbuminuria could slow the progression of renal disease. General aim. Improve care of diabetic patients by increasing microalbumin urine screening among diabetic patients. Specific goal. Increase the number of diabetic patients screened annually for microalbuminuria by 50% over baseline. Intervention. The team chose to modify the office process for identification of diabetic patients and institute use of a microalbumin dipstick (Micral test) for the office setting. Method for measuring performance. Two hundred and three charts from a computer-generated list of patients with the diagnosis diabetes mellitus were pulled and reviewed. Of the 203 charts, 151 were used in the study. Fifty-two charts were excluded because the patient had not been seen in the past year, had died, or was shown to be wrongly diagnosed. A flowsheet was utilized to show if the patient had documented proteinuria. If not, the chart was reviewed to see if the patient had had a test for microalbuminuria. After 2 months of use of the Micral test, the charts were again reviewed. Statistical analysis of project outcomes. Because different samples of patients were used for the baseline and postintervention chart audits at Sites 1 and 2, Fisher’s exact test was used to compare the proportion of charts with the desired outcomes at baseline and postintervention.35 Because the same sample of 151 patients was used for both the baseline and postintervention chart audits at Site 3, McNemar’s test was used to compare the two proportions.
242
May 2003
Program Evaluation We evaluated the introduction of practice-based learning into the curriculum with the following measures: ■ Resident presentations documenting processes and outcomes that addressed objectives related to ACGME competencies; ■ Ongoing level of participation of staff, faculty, and residents at team meetings dedicated to planning, execution, and evaluation of improvement projects; and ■ Participant answers to questions about the impact and value of the program. ACGME competencies were demonstrated by completion of products required for presentation of the improvement projects. A panel consisting of two QI professionals from the university hospital and the chairperson of the Department of Family and Community Medicine were provided with a worksheet to rank the teams in the areas related to completion of the objectives. The panel members independently scored teams on their level of achievement of the objectives, using 0 if the objective was not met, 1 if the objective was met in part, and 2 if the objective was completely met. Table 1 summarizes assessment strategies and results for the ACGME competencies associated with this study. Figure 1 (p 243) summarizes the outcomes of the improvement projects, and Figure 2 (p 244) portrays the level of participation in team meetings and surveying participants at the completion of a 6-month cycle.
Attendance Attendance at meetings by staff, faculty, and residents varied throughout the 6 months. Six of the 11 faculty members regularly participated. Second- and third-year residents participated more regularly than did first-year residents. At site 1, resident participation ranged from 2 to 7, with the average number of residents present greater than 4. At Site 2, resident participation ranged from 2 to 4, with average attendance at almost 3. At Site 3, resident participation ranged from 1 to 4, with average attendance at 2. In general, the work was shared among the residents, with individuals pairing or individually taking responsibility for chart audits, literature searches, intervention processes, and meetings. In addition, resident teams were required to give presentations.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Results for the Improvement Projects at Three Training Sites Site 1 Aim: Improve completion of adult data summary sheets by 60%. Results: Completion of lists increased from 10% to 44% (p < 0.001). Site 2 Aim: Increase the number of diabetic patients screened annually for microalbuminuria by 50% > baseline. Results: Completion of Adult data summary sheets increased from 14% to 40% (p < 0.017). Site 3 Aim: Improve medication list completion to 30% > baseline. Results: Baseline screening of patients with negative urinalysis increased from 5% to 29% (p < 0.001). Figure 1. Each ambulatory care training site selected an improvement project with its own aim and intervention. Chart audits at each site were conducted to assess the success of the interventions.
Figure 3 (p 244) summarizes the program evaluation by participants. In general, participants indicated that the projects were very good or excellent in developing teamwork and facilitating presentation skills. The majority agreed with statements that the projects improved clinical practice, helped to meet ACGME goals, and applied to the real world. Response rates for residents were 19 of 24; for faculty, 5 of 6 who participated; and for staff, 13 of 20.
Discussion This article reports the efforts of a family medicine training program at incorporating improvement work into the daily clinical activity of residents. It describes clinical projects chosen and initiated by teams of staff, faculty, and residents at three separate ambulatory care training sites and their resulting changes. It demonstrates a method for assessing ACGME competencies related to practice-based learning and improvement. It illustrates improvement successes similar to
May 2003
efforts achieved elsewhere. For example, the percentage increase in the screening of diabetic patients for microalbuminuria is not unlike the increases in screening from 8% to 27%36 and 10% to 33%37 obtained in other improvement initiatives.
Successful Driving Forces in Incorporating Improvement into Daily Work The support of the department chair, the vice chair of clinical affairs, the residency program director, and the training site clinical directors was essential to making possible the changes in clinic schedules that permitted team meetings. The team meetings served as the primary vehicle for reflecting on opportunities and developing strategies for improving daily work. The adoption of new competencies for graduating residents by the ACGME fueled interest in improvement projects as a means for addressing those competencies. The actual presentation of improvement project results provided opportunities for assessment of competencies for teams of residents.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
243
Joint Commission
Journal on Quality and Safety
Percentage of Team Members in Attendance
Figure 2. Each ambulatory training site monitored the attendance of faculty, residents, and staff at the five to six team meetings devoted to improvement projects during the 6-month project.
Percentage of Participants Rating Value As Very Good or Excellent
Figure 3. Faculty, staff, and residents evaluated the 6-month improvement projects on six elements related to the goals of introducing quality improvement into clinical practice and demonstrating ACGME competencies, using a Likert scale rating value from 1 (poor) to 5 (excellent). ACGME, Accreditation Council for Graduate Medical Education.
244
May 2003
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Staff members at the clinical sites welcomed team meetings to address improvement of care, including issues related to the projects and to general office operations. It is likely that the awarding of academic credit for attendance and evaluation of projects by the chair strengthened the residents’ participation. The successful completion of the first set of projects supported further improvement activities in the subsequent 6 months. The chair also recognized faculty participation with a letter of appreciation.
Barriers to Incorporating Improvement into Daily Work Despite a lack of familiarity with QI principles, residents were reluctant to “sacrifice” valuable curricular sessions to learn to use QI tools. Skills in performing literature searches varied among residents. Faculty familiarity with QI tools and processes was also limited. Attendance of team members was variable. At most meetings, one or two faculty members and staff members were in attendance. All residents were encouraged to attend, but it was understood that first-year residents with intensive, acute hospital duties were excused. Travel demands of several “away” rotations prohibited attendance. Residents not scheduled for patient care half days at the clinical sites on the same day as team meetings had to make special efforts to attend and participate. Understandably, clinical obligations for staff at times took precedence over attendance. Patient care activities did not always finish by the time of the meeting, causing some staff and providers to arrive late. The need for data collection without dedicated personnel or an electronic medical record necessitated limiting the frequency and volume of data collection. Finally, residents were not always successful in finding evidence for the intervention (for example, none was found for completion of adult data summary sheets).
Lessons Learned Faculty leadership is important to success. Although 11 faculty members were encouraged to participate, 6 were active participants. After the first series of projects, the chair committed additional funds for continuing medical education to faculty who assumed leadership roles in improvement projects. Faculty development sessions devoted to QI skills were initiated.
May 2003
Staff and resident involvement is also essential. Office staff pulled appropriate charts for review by staff, residents, and faculty who performed chart audits during portions of team meetings. To facilitate staff participation, phone calls during meetings were channeled to voice mail or to an answering service. To compensate for lateness caused by unfinished competing clinical duties, meeting agendas were adjusted. To optimize resident participation, the family practice medicine training program elected to incorporate team meetings into the core clinical conference time, a time already designated as a priority by most rotations. To diminish the perception that residents were sacrificing valuable curricular time for QI, didactic sessions devoted solely to QI as a topic were discontinued after the first 6-month cycle. Instead, QI tools and philosophy were embedded into didactic sessions, introducing new professional knowledge. Because some teams were unable to find evidence in the literature to support their interventions, teams in the second cycle were advised to select interventions from established evidence-based guidelines. In addition, medical librarians were brought in as regular contributors to teach residents how to improve literature searches. Rewarding only one site was counterproductive. The team members whose teams had put forth sustained effort and had been successful but not judged as the best during presentation of their projects were disappointed. In a subsequent series of projects, each team was rewarded on the basis of achievement in one of three areas: patient care improvement, collaboration, or presentation skills. Each site was awarded a laminated certificate and a catered lunch. The primary learner outcome was achievement of improved clinical care for patients at the site and was not assessed for patients of an individual learner. It is not unusual to use provider or patient group data for measures of improvement.6 Achievement of ACGME competencies was assessed on the basis of the products demonstrated in a group presentation as a result of team effort. For example, one resident may have developed the flowchart that analyzed the clinical process, another may have done the literature search, and another the chart audit. The design of this curriculum did not include individual assessment of achievement of ACGME com-
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
245
Joint Commission
Journal on Quality and Safety
petencies. The team of residents was evaluated on its presentation of the group clinical improvement project. In the future, assessment measures may be desirable for both individual and group efforts. On the basis of the lessons learned, we recommend that other residency programs ■ involve faculty members in improving their own practices as part of faculty development; ■ create systems and modify schedules that facilitate participation of staff and residents; ■ provide librarian assistance or training in performing literature searches; ■ reward successes appropriately; and ■ develop individual as well as team measures of improvement. In summary, practice-based learning and improvement activities can be incorporated into the daily work of residents in training. Improvement activities can be facilitated by providing knowledge of QI tools and philosophy to residents and the clinical staff at the ambulatory care sites, by providing a structure that supports team meetings in which patient care teams can reflect on and change daily work, and by creating an educational process that acknowledges efforts with academic credit
and provides recognition for both faculty members and residents. Such a process can simultaneously meet selected competency requirements of the ACGME. We need to expand evaluation methods to capture individual as well as team improvement efforts. J The results of this study were presented December 10, 2001, at the 14th Annual National Forum on Quality Improvement in Health Care, International Scientific Symposium, Orlando, Florida, and April 29, 2001, at the Annual Spring Meeting of the Society for Teachers in Family and Community Medicine, Denver.
Mary Thoesen Coleman, MD, PhD, is Senior Vice Chair of Clinical Affairs, Department of Family and Community Medicine, Med Center One, University of Louisville, Louisville. Soraya Nasraty, MD, is Medical Director, University of Louisville Family Medicine at Newburg, Louisville. Michael Ostapchuk, MD, is Medical Director, University of Louisville Family Medicine at Iroquois, Louisville. Stephen Wheeler, MD, is Director of Family Medicine Residency, University of Louisville. Stephen Looney, PhD, is Professor, Department of Bioinformatics and Biostatistics, School of Public Health and Information Sciences, University of Louisville. Sandra Rhodes was Residency Curriculum Coordinator, University of Louisville Family Medicine. Please address reprint requests to Mary Thoesen Coleman, MD, PhD, [email protected].
References 1. Accreditation Council for Graduate Medical Education: Program Requirements for Residency Education in Family Practice. www.acgme.org (accessed Feb 20, 2003). 2. Accreditation Council for Graduate Medical Education: ACGME Outcome Project. ACGME General Competencies, Version 1.3. www.acgme.org/outcome/comp/compFull.asp (accessed Feb 20, 2003). 3. Bogdewic S: A message from the president: Board approves strategic plan for STFM. STFM Messenger 21(2):1, Apr 2001. 4. American Academy of Family Physicians: Recommended Curriculum Guidelines for Family Practice Residents: Practice Management. www.aafp.org/eduguide.xml (accessed Feb 20, 2003). 5. Spencer E, et al: Tools to improve documentation of smoking status: Continuous quality improvement and electronic medical records. Arch Fam Med 8:18–22, 1999. 6. Sutherland JE, et al: Diabetes management quality improvement in a family medicine residency program. J Am Board Fam Pract 14:243–251, 2001. 7. Leshan LA, et al: Increasing clinical prevention efforts in a family practice residency program through CQI methods. Jt Comm J Qual Improv 23:391–400, 1997. 8. Fox CH, Mahoney MC: Improving diabetes preventive care in a family practice residency program: A case study in continuous quality improvement. Fam Med 30:441–445, 1998.
246
May 2003
9. Welsh CH, Pedot R, Anderson RJ: Use of morning report to enhance adverse event detection. J Gen Intern Med 11:454–460, 1996. 10. O’Neil AC, et al: Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 119:370–376, 1993. 11. Weingart S: House officer education and organizational obstacles to quality improvement. Jt Comm J Qual Improv 22:640–646, 1996. 12. Weingart S: A house officer–sponsored quality improvement initiative: Leadership lessons and liabilities. Jt Comm J Qual Improv 24:371–378, 1998. 13. Eliastam M, Mizrahi T: Quality improvement, housestaff, and the role of chief residents. Acad Med 71:670–674, 1996. 14. Headrick LA, et al: Continuous quality improvement and the education of the generalist physician. Acad Med 70(1 suppl):S104–109, 1995. 15. Headrick LA, Richardson A, Priebe GP: Continuous improvement learning for residents. Pediatrics 101:768-773, 1998. 16. Parenti CM, et al: Reduction of unnecessary intravenous catheter use: Internal medicine house staff participate in a successful quality improvement project. Arch Intern Med 154:1829–1832, 1994. 17. Batalden PB, Splaine M, Baker R: Eight Domains for the Improvement of Health Care. Boston: Institute for Healthcare Improvement, 1998. 18. Roessner J (ed): Setting Aims in Reducing Delays and Waiting Times Throughout the Healthcare System. Breakthrough Series Guides. Boston: Institute for Healthcare Improvement, 1996.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
References, continued 19. Haynes RB, et al: Interventions for helping patients to follow prescriptions for medications. Cochrane Database Syst Rev 2:CD000011, 2002. 20. The College of Physicians and Surgeons of Ontario: Elements of a primary care record. In A Guide to Current Medical Record Keeping Practices: A Focus on Record Keeping in the Office-Based Setting, 2000. www.cpso.on.ca/Publications/medrecbk.htm (accessed Feb 25, 2002). 21. Health tips: What to include on medical forms. Mayo Clin Health Lett 17(8):3, 1999. 22. Silva TD, Schenkel EP, Mengue SS: Information level about drugs prescribed to ambulatory patients in a university hospital. Cadernos de Saude Publica 16:449–455, 2000. (Portuguese) 23. Levin MW: How a FP computerized his practice—On his own. Fam Pract Manag 7(6):43–46, 2000. 24. Aiell JH: Preventing diabetic nephropathy: The role of primary care. Nurse Pract 23(2):12–28, 1998. 25. Bennett PH, et al: Screening and management of microalbuminuria in patients with diabetes mellitus: Recommendations to the Scientific Advisory Board from an ad hoc committee of the Council on Diabetes Mellitus of the National Kidney Foundation. Am J Kidney Dis 25:107–112, 1995. 26. Bilous R: Diabetes: Should you be screening for microalbuminuria? Practitioner 239:343–345, 1995. 27. Fernandez Fernandez I, et al: Rapid screening test evaluation for microalbuminuria in diabetes mellitus. Acta Diabetologica 35:199–202, 1998.
May 2003
28. Gilbert RE, et al: Microalbuminuria: Prognostic and therapeutic implications in diabetes mellitus. Diabet Med 11:636–645, 1994. 29. Jensen JE, et al: The MICRAL test for diabetic microalbuminuria: Predictive values as a function of prevalence. Scand J Clin Lab Invest 56:117–122, 1996. 30. Jiang YD, et al: Role of an outpatient clinic in screening chronic complications of diabetes: A model for diabetes managed care. J Formos Med Assoc 97:521–527, 1998. 31. Konen JC, Shihabi ZK: Microalbuminuria and diabetes mellitus. Am Fam Physician 48:1421–1428, 1993. 32. Nagoshi MH, Miyahira Y: The clinical significance of microalbuminuria in diabetes mellitus. Hawaii Med J 49:417–420, 1990. 33. Viberti G, Chaturvedi N: Angiotensin converting enzyme inhibitors in diabetic patients with microalbuminuria or normoalbuminuria. Kidney Int Suppl 63:S32–S35, 1997. 34. White B: Using flow sheets to improve diabetes care. Fam Pract Manag 7(6):60–62, 2000. 35. Dawson B, Trapp RG: Basic and Clinical Biostatistics, 3rd ed. New York: Lange Medical Books, McGraw-Hill, 2001. 36. Montori VM, et al: The impact of planned care and a diabetes electronic management system on community-based diabetes care: The Mayo Health System Diabetes Translation Project. Diabetes Care 25:1952–1957, 2002. 37. Petitti DB, et al: Evaluation of the effect of performance monitoring and feedback on care process, utilization, and outcome. Diabetes Care 23:192–196, 2000.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
247
Journal on Quality and Safety
Health Professions Education
Introducing Practice-Based Learning and Improvement ACGME Core Competencies into a Family Medicine Residency Curriculum
Mary Thoesen Coleman, MD, PhD Soraya Nasraty, MD Michael Ostapchuk, MD Stephen Wheeler, MD Stephen Looney, PhD Sandra Rhodes
he ability to learn from and improve on the care of patients in one’s practice is recognized as an important skill for family physicians. Practice-based learning and improvement is one of the six general competencies approved by the Accreditation Council for Graduate Medical Education (ACGME; Chicago) in its program requirements for residency.1,2 The 2001 strategic plan for the Society for Teachers of Family Medicine (Leawood, Kan) made quality improvement (QI) one of its five major goals and established a task force whose primary focus is the integration of QI into residency education and practice.3 The American Academy of Family Physicians (Leawood, Kan) also recommends continuous quality improvement and total quality management as core knowledge and skills.4 Yet introduction of practice-based learning and improvement into family medicine residency training5–7 and other residency training programs8–16 has been limited. This article describes how a family medicine residency training program at the Department of Family and Community Medicine at the University of Louisville incorporated improvement into the daily work of residents, faculty, and staff at their ambulatory care sites and simultaneously addressed assessment of ACGME competencies related to practice-based learning and improvement. The family medicine residency program addressed the following specific ACGME competencies related to practice-based learning and improvement:
T
238
May 2003
Article-at-a-Glance Background: The Accreditation Council for Graduate Medical Education (ACGME) recommends integrating improvement activities into residency training. A curricular change was designed at the Department of Family and Community Medicine, University of Louisville, to address selected ACGME competencies by incorporating practice-based improvement activities into the routine clinical work of family medicine residents. Methods: Teams of residents, faculty, and office staff completed clinical improvement projects at three ambulatory care training sites. Residents were given academic credit for participation in team meetings. After 6 months, residents presented results to faculty, medical students, other residents, and staff from all three training sites. Residents, staff, and faculty were recognized for their participation. Program evaluation: Resident teams demonstrated ACGME competencies in practice-based improvement: Chart audits indicated improvement in clinical projects; quality improvement tools demonstrated analysis of root causes and understanding of the process; plan-do-studyact cycle worksheets demonstrated the change process. Conclusions: Improvement activities that affect patient care and demonstrate selected ACGME competencies can be successfully incorporated into the daily work of family medicine residents.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
1. Analyze practice experience and perform practicebased improvement activities using a systematic methodology; 2. Locate, appraise, and assimilate evidence from scientific studies related to patients’ health problems; 3. Obtain and use information about their own population of patients and the larger population from which their patients are drawn; and 4. Facilitate learning of students and other health professionals.2
Curriculum The plan to incorporate improvement into the curriculum of the family medicine residency program was spearheaded in fall 1999 by the director of the QI curriculum of the department, a faculty member with QI training [M.T.C.]. The plan consisted of 1. providing the needed knowledge for improvement work; 2. engendering support from leaders and participants; 3. creating a structure to facilitate improvement activities; and 4. supplying incentives to encourage and reward success. In July 2000 teams composed of residents, faculty, and staff from each site were asked to initiate and demonstrate improvement in clinical foci of their choice for 6 months. Subsequent 6-month cycles provided opportunities to choose other clinical foci and to improve the processes. To provide teams with the needed knowledge and skills to accomplish clinical improvement goals, the family medicine residency program introduced QI tools, processes, and philosophy using interactive 1-hour monthly sessions that were incorporated into the weekly residency didactic core conference series (Table 1, p 240, teaching methods for Objectives 1 and 2). The director of the QI curriculum chose topics based on a framework that includes eight domains of improvement.17 Staff from the clinical sites were invited to attend the seven monthly sessions, which included team-building exercises; application of flowcharts to process analysis; use of nominal group technique for decision making; performance of a literature search; understanding of variation; and use of run charts, bar
May 2003
graphs, and histograms for data display. Teaching methods for didactic sessions included lecture and discussion, small-group activities, panel presentations, and role playing. Residents’ experiential learning was designed to occur at team meetings during which planning, execution, and evaluation of improvement projects occurred. Residents were informed that assessment of their projects would include demonstration of teamwork, application of evidence-based medicine, and implementation of a plan-do-study-act (PDSA) learning cycle and measured clinical outcomes (Table 1, teaching and assessment strategies for objectives 1, 2, 3). On the basis of recommendations from the Breakthrough Series sponsored by the Institute for Healthcare Improvement (IHI; Boston),18 teams were encouraged to create specific goals with thresholds that would challenge the system. Support for integrating improvement activities into the curriculum was developed using several strategies. The director of the improvement curriculum sought and obtained approval from leadership by specifically seeking approval from the chairperson of the department and the director of the residency program for proposed changes. Clinical directors were asked to participate in the projects and benefited from having time allotted for team meetings that alternated between staff meetings for clinical operations and improvement work. Familiarizing residents with ACGME requirements and then explicitly linking improvement activities to demonstration of ACGME competencies was used to foster support from the residents and faculty. Time for the team meetings was secured on Thursday afternoons (3:30–5:00 PM) after core conferences (12:00–3:00 PM), when available clinical hours were already limited. An analysis of the patient volume seen by residents in the ambulatory care sites on Thursday afternoons after conference sessions suggested a limited impact on revenue and service. Incentives for resident participation included academic credit for participation in team meetings. Residents are required to attend a certain percentage of core conference sessions; the residency director approved credit for participation in team meetings. In addition, each resident team was required to give a
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
239
Joint Commission
Journal on Quality and Safety
Table 1. Practice-Based Learning and Improvement ACGME Competencies: Teaching Methods and Assessment Strategies* ACGME Objective 1. Analyze practice experience and perform practicebased improvement activities using systematic methodology
2. Locate, appraise, and assimilate evidence from scientific studies related to patients' health problems.
3. Obtain and use information about own population of patients and larger population from which those patients are drawn. 4. Facilitate learning of students and other health care professionals.
Teaching Method Didactic sessions on flowcharts/PDSA cycles. Experiential: Teams create flowchart to identify root causes of problems in practice. Teams design learning cycle to make changes in practice. Teams determine and measure outcomes to demonstrate effect of change. Didactic session on performing literature search. Residents complete literature search to document evidence for intervention.
Teams perform chart audits in own practices. Teams may perform comparison audits of other practices. Residents prepare presentations for sharing with group.
Assessment Strategy Panel of experts evaluates resident’s presentation for presence of (1) Flowchart demonstrating identification of root causes. (2) Completion of PDSA cycle worksheet used for intervention. (3) Outcomes of change intervention.
Results Flowcharts of change processes presented for Sites 2, 3. PDSA cycles presented for Sites 1, 2, 3. Outcomes presented for change intervention for sites 1, 2, 3 (Figure 1, p 243).
Panel of experts evaluates for presence of critical review of literature supporting the intervention, including (1) list of keywords used in search, (2) databases searched, and (3) results. Panel of experts evaluates for presence of clinical outcomes using chart audit data from baseline and postintervention chart audits. Panel of experts evaluates for presentation skills.
Keywords identified for Sites 1, 2, 3. Databases searched identified for Sites 1, 2, 3. Results presented for Sites 1, 3.
Chart audit data (Figure 1) presented for Sites 1, 2, 3.
Presentation for Sites 1, 2, 3 Sites 1, 2 presented by many residents; Site 3 presented by single resident.
ACGME, Accreditation Council for Graduate Medical Education; PDSA, plan-do-study-act.
presentation at the completion of the project to a large group (Table 1, teaching method and assessment strategy for Objective 4), including evaluators from key QI positions within hospital institutions and the chairperson of the department. The team that was deemed to perform best was promised a dinner party. Each team was told to choose a project that it considered important to improving clinical care at its site. It was also instructed in the use of criteria such as feasibility and impact and in the use of multivoting to reach consensus.
240
May 2003
Individual Clinical Improvement Projects at Training Sites Site 1 Team. The team consisted of 4 faculty, 12 residents, and 5 office staff. Background. Incomplete medication lists in the chart were making it difficult to determine if refills for medications were appropriate. The time it took staff to research the appropriateness markedly decreased efficiency in authorizing refills. The team was also concerned that incomplete medication lists might be leading to errors that could affect patient safety.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Literature search for evidence. Residents searched the following resources for evidence that updating medication lists would improve care: ■ Ovid, an online medical information service; ■ The online journal Evidence-Based Medicine, 1991–2000; ■ MEDLINE, 1966–2000; ■ The Web site for the American Academy of Family Practice, www.aafp.org; and ■ The journal Family Practice Management. They used the following terms as keywords: medications, medication list, outpatient charts, clinic charts, medication list update ambulatory care, QI in outpatient care, drug list, current therapy list, prescriptions, medications, and charts. The search identified five articles of interest which indicated that completion of medication lists was important in efforts to improve care.19–23 General aim. Improve efficiency and reduce errors by increasing completion of medication lists. Specific goal. Improve completion of medication lists by 30% within 3 months. Intervention. After some discussion the team determined that the likely root cause for incomplete lists was the failure to remember to complete them. The team created a prompt on the progress note consisting of a check box to remind providers to update the medication list. Method of measuring performance. 100 baseline and 50 follow-up, randomly selected charts were reviewed for documentation of chronic illness medications, refill authorizations, and medication allergies.
Site 2 Team. The team consisted of 5 faculty, 2 nurse practitioners, 1 physician assistant, 6 residents, and 10 office staff. Background. The adult data summary sheet had been previously developed as an accessible, clinically relevant chart face sheet presenting key information in eight areas: problem list, past history, family history, social history, advanced directive, special tests/procedures, preventive tests, and immunizations. Providers believed its routine use would improve ambulatory care, continuity of care, communication among providers, routine health maintenance, and efficient transfer of information to the inpatient team when a patient is
May 2003
hospitalized. Furthermore, they felt its irregular use was compromising services. Literature search for evidence. Residents used the same literature sources as Site 1 in a searched for evidence that chart summary sheets improve patient care. Keywords used were charts, outpatient charts, clinic charts, office charts, medical record, data sheet, summary sheet, and face sheet. The search did not substantiate the hypothesis that regular use of a data summary sheet improved communication or care. Benchmarking. An audit of 36 charts (convenience sample) showed a completion rate of 14%. An audit of 100 charts at an affiliated practice of faculty and residents found a completion rate of 11%. Charts were included if the patient had had three or more prior visits to the site. General aim. Improve communications among providers and routine health maintenance of patients by increasing the routine completion of the adult data summary sheets. Specific goal. Increase completion of adult data summary sheets by 60% during the study period of August 10–November 30, 2000. Completion was defined as having 6 or more (of the 8 possible) areas addressed on any given sheet. Intervention. Providers were reminded of the importance of the summary sheets during clinic meetings. Medical assistants developed a specific stamp for use on each current progress note to remind providers to complete the summary sheet. Staff members finding incomplete sheets returned the charts to the provider for completion. Method for measuring performance. A convenience sample of 35 charts of patients who had been seen at least once during the study period was selected for review by a staff member with no known bias.
Site 3 Team. The team consisted of 2 faculty, 6 residents, and 5 office staff. Background. The team chose to improve microalbumin screening for its potential impact on patient care and its capability for accurate measurement. Literature search for evidence. The residents searched for evidence that microalbumin screening
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
241
Joint Commission
Journal on Quality and Safety
would improve care of patients with diabetes. They also searched for different tests available for microalbuminuria that could be used in the office setting. The following keywords were used in the Ovid and MEDLINE 1966–2000 searches: diabetes mellitus, screening tests for diabetes, microalbumin, rapid screening tests for micro albumin, and diabetic nephropathy. Eleven articles24–34 supported the importance of microalbumin as the first clinical sign of nephropathy and the leading cause of end-stage renal disease in the United States. Evidence indicated that improved glycemic control and medication choices in patients with microalbuminuria could slow the progression of renal disease. General aim. Improve care of diabetic patients by increasing microalbumin urine screening among diabetic patients. Specific goal. Increase the number of diabetic patients screened annually for microalbuminuria by 50% over baseline. Intervention. The team chose to modify the office process for identification of diabetic patients and institute use of a microalbumin dipstick (Micral test) for the office setting. Method for measuring performance. Two hundred and three charts from a computer-generated list of patients with the diagnosis diabetes mellitus were pulled and reviewed. Of the 203 charts, 151 were used in the study. Fifty-two charts were excluded because the patient had not been seen in the past year, had died, or was shown to be wrongly diagnosed. A flowsheet was utilized to show if the patient had documented proteinuria. If not, the chart was reviewed to see if the patient had had a test for microalbuminuria. After 2 months of use of the Micral test, the charts were again reviewed. Statistical analysis of project outcomes. Because different samples of patients were used for the baseline and postintervention chart audits at Sites 1 and 2, Fisher’s exact test was used to compare the proportion of charts with the desired outcomes at baseline and postintervention.35 Because the same sample of 151 patients was used for both the baseline and postintervention chart audits at Site 3, McNemar’s test was used to compare the two proportions.
242
May 2003
Program Evaluation We evaluated the introduction of practice-based learning into the curriculum with the following measures: ■ Resident presentations documenting processes and outcomes that addressed objectives related to ACGME competencies; ■ Ongoing level of participation of staff, faculty, and residents at team meetings dedicated to planning, execution, and evaluation of improvement projects; and ■ Participant answers to questions about the impact and value of the program. ACGME competencies were demonstrated by completion of products required for presentation of the improvement projects. A panel consisting of two QI professionals from the university hospital and the chairperson of the Department of Family and Community Medicine were provided with a worksheet to rank the teams in the areas related to completion of the objectives. The panel members independently scored teams on their level of achievement of the objectives, using 0 if the objective was not met, 1 if the objective was met in part, and 2 if the objective was completely met. Table 1 summarizes assessment strategies and results for the ACGME competencies associated with this study. Figure 1 (p 243) summarizes the outcomes of the improvement projects, and Figure 2 (p 244) portrays the level of participation in team meetings and surveying participants at the completion of a 6-month cycle.
Attendance Attendance at meetings by staff, faculty, and residents varied throughout the 6 months. Six of the 11 faculty members regularly participated. Second- and third-year residents participated more regularly than did first-year residents. At site 1, resident participation ranged from 2 to 7, with the average number of residents present greater than 4. At Site 2, resident participation ranged from 2 to 4, with average attendance at almost 3. At Site 3, resident participation ranged from 1 to 4, with average attendance at 2. In general, the work was shared among the residents, with individuals pairing or individually taking responsibility for chart audits, literature searches, intervention processes, and meetings. In addition, resident teams were required to give presentations.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Results for the Improvement Projects at Three Training Sites Site 1 Aim: Improve completion of adult data summary sheets by 60%. Results: Completion of lists increased from 10% to 44% (p < 0.001). Site 2 Aim: Increase the number of diabetic patients screened annually for microalbuminuria by 50% > baseline. Results: Completion of Adult data summary sheets increased from 14% to 40% (p < 0.017). Site 3 Aim: Improve medication list completion to 30% > baseline. Results: Baseline screening of patients with negative urinalysis increased from 5% to 29% (p < 0.001). Figure 1. Each ambulatory care training site selected an improvement project with its own aim and intervention. Chart audits at each site were conducted to assess the success of the interventions.
Figure 3 (p 244) summarizes the program evaluation by participants. In general, participants indicated that the projects were very good or excellent in developing teamwork and facilitating presentation skills. The majority agreed with statements that the projects improved clinical practice, helped to meet ACGME goals, and applied to the real world. Response rates for residents were 19 of 24; for faculty, 5 of 6 who participated; and for staff, 13 of 20.
Discussion This article reports the efforts of a family medicine training program at incorporating improvement work into the daily clinical activity of residents. It describes clinical projects chosen and initiated by teams of staff, faculty, and residents at three separate ambulatory care training sites and their resulting changes. It demonstrates a method for assessing ACGME competencies related to practice-based learning and improvement. It illustrates improvement successes similar to
May 2003
efforts achieved elsewhere. For example, the percentage increase in the screening of diabetic patients for microalbuminuria is not unlike the increases in screening from 8% to 27%36 and 10% to 33%37 obtained in other improvement initiatives.
Successful Driving Forces in Incorporating Improvement into Daily Work The support of the department chair, the vice chair of clinical affairs, the residency program director, and the training site clinical directors was essential to making possible the changes in clinic schedules that permitted team meetings. The team meetings served as the primary vehicle for reflecting on opportunities and developing strategies for improving daily work. The adoption of new competencies for graduating residents by the ACGME fueled interest in improvement projects as a means for addressing those competencies. The actual presentation of improvement project results provided opportunities for assessment of competencies for teams of residents.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
243
Joint Commission
Journal on Quality and Safety
Percentage of Team Members in Attendance
Figure 2. Each ambulatory training site monitored the attendance of faculty, residents, and staff at the five to six team meetings devoted to improvement projects during the 6-month project.
Percentage of Participants Rating Value As Very Good or Excellent
Figure 3. Faculty, staff, and residents evaluated the 6-month improvement projects on six elements related to the goals of introducing quality improvement into clinical practice and demonstrating ACGME competencies, using a Likert scale rating value from 1 (poor) to 5 (excellent). ACGME, Accreditation Council for Graduate Medical Education.
244
May 2003
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
Staff members at the clinical sites welcomed team meetings to address improvement of care, including issues related to the projects and to general office operations. It is likely that the awarding of academic credit for attendance and evaluation of projects by the chair strengthened the residents’ participation. The successful completion of the first set of projects supported further improvement activities in the subsequent 6 months. The chair also recognized faculty participation with a letter of appreciation.
Barriers to Incorporating Improvement into Daily Work Despite a lack of familiarity with QI principles, residents were reluctant to “sacrifice” valuable curricular sessions to learn to use QI tools. Skills in performing literature searches varied among residents. Faculty familiarity with QI tools and processes was also limited. Attendance of team members was variable. At most meetings, one or two faculty members and staff members were in attendance. All residents were encouraged to attend, but it was understood that first-year residents with intensive, acute hospital duties were excused. Travel demands of several “away” rotations prohibited attendance. Residents not scheduled for patient care half days at the clinical sites on the same day as team meetings had to make special efforts to attend and participate. Understandably, clinical obligations for staff at times took precedence over attendance. Patient care activities did not always finish by the time of the meeting, causing some staff and providers to arrive late. The need for data collection without dedicated personnel or an electronic medical record necessitated limiting the frequency and volume of data collection. Finally, residents were not always successful in finding evidence for the intervention (for example, none was found for completion of adult data summary sheets).
Lessons Learned Faculty leadership is important to success. Although 11 faculty members were encouraged to participate, 6 were active participants. After the first series of projects, the chair committed additional funds for continuing medical education to faculty who assumed leadership roles in improvement projects. Faculty development sessions devoted to QI skills were initiated.
May 2003
Staff and resident involvement is also essential. Office staff pulled appropriate charts for review by staff, residents, and faculty who performed chart audits during portions of team meetings. To facilitate staff participation, phone calls during meetings were channeled to voice mail or to an answering service. To compensate for lateness caused by unfinished competing clinical duties, meeting agendas were adjusted. To optimize resident participation, the family practice medicine training program elected to incorporate team meetings into the core clinical conference time, a time already designated as a priority by most rotations. To diminish the perception that residents were sacrificing valuable curricular time for QI, didactic sessions devoted solely to QI as a topic were discontinued after the first 6-month cycle. Instead, QI tools and philosophy were embedded into didactic sessions, introducing new professional knowledge. Because some teams were unable to find evidence in the literature to support their interventions, teams in the second cycle were advised to select interventions from established evidence-based guidelines. In addition, medical librarians were brought in as regular contributors to teach residents how to improve literature searches. Rewarding only one site was counterproductive. The team members whose teams had put forth sustained effort and had been successful but not judged as the best during presentation of their projects were disappointed. In a subsequent series of projects, each team was rewarded on the basis of achievement in one of three areas: patient care improvement, collaboration, or presentation skills. Each site was awarded a laminated certificate and a catered lunch. The primary learner outcome was achievement of improved clinical care for patients at the site and was not assessed for patients of an individual learner. It is not unusual to use provider or patient group data for measures of improvement.6 Achievement of ACGME competencies was assessed on the basis of the products demonstrated in a group presentation as a result of team effort. For example, one resident may have developed the flowchart that analyzed the clinical process, another may have done the literature search, and another the chart audit. The design of this curriculum did not include individual assessment of achievement of ACGME com-
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
245
Joint Commission
Journal on Quality and Safety
petencies. The team of residents was evaluated on its presentation of the group clinical improvement project. In the future, assessment measures may be desirable for both individual and group efforts. On the basis of the lessons learned, we recommend that other residency programs ■ involve faculty members in improving their own practices as part of faculty development; ■ create systems and modify schedules that facilitate participation of staff and residents; ■ provide librarian assistance or training in performing literature searches; ■ reward successes appropriately; and ■ develop individual as well as team measures of improvement. In summary, practice-based learning and improvement activities can be incorporated into the daily work of residents in training. Improvement activities can be facilitated by providing knowledge of QI tools and philosophy to residents and the clinical staff at the ambulatory care sites, by providing a structure that supports team meetings in which patient care teams can reflect on and change daily work, and by creating an educational process that acknowledges efforts with academic credit
and provides recognition for both faculty members and residents. Such a process can simultaneously meet selected competency requirements of the ACGME. We need to expand evaluation methods to capture individual as well as team improvement efforts. J The results of this study were presented December 10, 2001, at the 14th Annual National Forum on Quality Improvement in Health Care, International Scientific Symposium, Orlando, Florida, and April 29, 2001, at the Annual Spring Meeting of the Society for Teachers in Family and Community Medicine, Denver.
Mary Thoesen Coleman, MD, PhD, is Senior Vice Chair of Clinical Affairs, Department of Family and Community Medicine, Med Center One, University of Louisville, Louisville. Soraya Nasraty, MD, is Medical Director, University of Louisville Family Medicine at Newburg, Louisville. Michael Ostapchuk, MD, is Medical Director, University of Louisville Family Medicine at Iroquois, Louisville. Stephen Wheeler, MD, is Director of Family Medicine Residency, University of Louisville. Stephen Looney, PhD, is Professor, Department of Bioinformatics and Biostatistics, School of Public Health and Information Sciences, University of Louisville. Sandra Rhodes was Residency Curriculum Coordinator, University of Louisville Family Medicine. Please address reprint requests to Mary Thoesen Coleman, MD, PhD, [email protected].
References 1. Accreditation Council for Graduate Medical Education: Program Requirements for Residency Education in Family Practice. www.acgme.org (accessed Feb 20, 2003). 2. Accreditation Council for Graduate Medical Education: ACGME Outcome Project. ACGME General Competencies, Version 1.3. www.acgme.org/outcome/comp/compFull.asp (accessed Feb 20, 2003). 3. Bogdewic S: A message from the president: Board approves strategic plan for STFM. STFM Messenger 21(2):1, Apr 2001. 4. American Academy of Family Physicians: Recommended Curriculum Guidelines for Family Practice Residents: Practice Management. www.aafp.org/eduguide.xml (accessed Feb 20, 2003). 5. Spencer E, et al: Tools to improve documentation of smoking status: Continuous quality improvement and electronic medical records. Arch Fam Med 8:18–22, 1999. 6. Sutherland JE, et al: Diabetes management quality improvement in a family medicine residency program. J Am Board Fam Pract 14:243–251, 2001. 7. Leshan LA, et al: Increasing clinical prevention efforts in a family practice residency program through CQI methods. Jt Comm J Qual Improv 23:391–400, 1997. 8. Fox CH, Mahoney MC: Improving diabetes preventive care in a family practice residency program: A case study in continuous quality improvement. Fam Med 30:441–445, 1998.
246
May 2003
9. Welsh CH, Pedot R, Anderson RJ: Use of morning report to enhance adverse event detection. J Gen Intern Med 11:454–460, 1996. 10. O’Neil AC, et al: Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 119:370–376, 1993. 11. Weingart S: House officer education and organizational obstacles to quality improvement. Jt Comm J Qual Improv 22:640–646, 1996. 12. Weingart S: A house officer–sponsored quality improvement initiative: Leadership lessons and liabilities. Jt Comm J Qual Improv 24:371–378, 1998. 13. Eliastam M, Mizrahi T: Quality improvement, housestaff, and the role of chief residents. Acad Med 71:670–674, 1996. 14. Headrick LA, et al: Continuous quality improvement and the education of the generalist physician. Acad Med 70(1 suppl):S104–109, 1995. 15. Headrick LA, Richardson A, Priebe GP: Continuous improvement learning for residents. Pediatrics 101:768-773, 1998. 16. Parenti CM, et al: Reduction of unnecessary intravenous catheter use: Internal medicine house staff participate in a successful quality improvement project. Arch Intern Med 154:1829–1832, 1994. 17. Batalden PB, Splaine M, Baker R: Eight Domains for the Improvement of Health Care. Boston: Institute for Healthcare Improvement, 1998. 18. Roessner J (ed): Setting Aims in Reducing Delays and Waiting Times Throughout the Healthcare System. Breakthrough Series Guides. Boston: Institute for Healthcare Improvement, 1996.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
Joint Commission
Journal on Quality and Safety
References, continued 19. Haynes RB, et al: Interventions for helping patients to follow prescriptions for medications. Cochrane Database Syst Rev 2:CD000011, 2002. 20. The College of Physicians and Surgeons of Ontario: Elements of a primary care record. In A Guide to Current Medical Record Keeping Practices: A Focus on Record Keeping in the Office-Based Setting, 2000. www.cpso.on.ca/Publications/medrecbk.htm (accessed Feb 25, 2002). 21. Health tips: What to include on medical forms. Mayo Clin Health Lett 17(8):3, 1999. 22. Silva TD, Schenkel EP, Mengue SS: Information level about drugs prescribed to ambulatory patients in a university hospital. Cadernos de Saude Publica 16:449–455, 2000. (Portuguese) 23. Levin MW: How a FP computerized his practice—On his own. Fam Pract Manag 7(6):43–46, 2000. 24. Aiell JH: Preventing diabetic nephropathy: The role of primary care. Nurse Pract 23(2):12–28, 1998. 25. Bennett PH, et al: Screening and management of microalbuminuria in patients with diabetes mellitus: Recommendations to the Scientific Advisory Board from an ad hoc committee of the Council on Diabetes Mellitus of the National Kidney Foundation. Am J Kidney Dis 25:107–112, 1995. 26. Bilous R: Diabetes: Should you be screening for microalbuminuria? Practitioner 239:343–345, 1995. 27. Fernandez Fernandez I, et al: Rapid screening test evaluation for microalbuminuria in diabetes mellitus. Acta Diabetologica 35:199–202, 1998.
May 2003
28. Gilbert RE, et al: Microalbuminuria: Prognostic and therapeutic implications in diabetes mellitus. Diabet Med 11:636–645, 1994. 29. Jensen JE, et al: The MICRAL test for diabetic microalbuminuria: Predictive values as a function of prevalence. Scand J Clin Lab Invest 56:117–122, 1996. 30. Jiang YD, et al: Role of an outpatient clinic in screening chronic complications of diabetes: A model for diabetes managed care. J Formos Med Assoc 97:521–527, 1998. 31. Konen JC, Shihabi ZK: Microalbuminuria and diabetes mellitus. Am Fam Physician 48:1421–1428, 1993. 32. Nagoshi MH, Miyahira Y: The clinical significance of microalbuminuria in diabetes mellitus. Hawaii Med J 49:417–420, 1990. 33. Viberti G, Chaturvedi N: Angiotensin converting enzyme inhibitors in diabetic patients with microalbuminuria or normoalbuminuria. Kidney Int Suppl 63:S32–S35, 1997. 34. White B: Using flow sheets to improve diabetes care. Fam Pract Manag 7(6):60–62, 2000. 35. Dawson B, Trapp RG: Basic and Clinical Biostatistics, 3rd ed. New York: Lange Medical Books, McGraw-Hill, 2001. 36. Montori VM, et al: The impact of planned care and a diabetes electronic management system on community-based diabetes care: The Mayo Health System Diabetes Translation Project. Diabetes Care 25:1952–1957, 2002. 37. Petitti DB, et al: Evaluation of the effect of performance monitoring and feedback on care process, utilization, and outcome. Diabetes Care 23:192–196, 2000.
Volume 29 Number 5
Copyright 2003 Joint Commission on Accreditation of Healthcare Organizations
247