- Email: [email protected]
A method for teaching continuous quality improvement to student pharmacists through a practical application project
Available online at www.sciencedirect.com
Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86 http://www.pharmacyteaching.com
A method for teaching continuous quality improvement to student pharmacists through a practical application project Susan J. Skledar, BS MPHa,b,*, Teresa P. McKaveney, BSa a
University of Pittsburgh School of Pharmacy, Pittsburgh, PA b University of Pittsburgh Medical Center, Pittsburgh, PA
Abstract Objective: To provide future pharmacists with the skills and knowledge they need to use the continuous quality improvement (CQI) process to solve problems in a variety of practice settings. Material and methods: At the University of Pittsburgh School of Pharmacy, the theory of CQI is presented to third-year pharmacy students in two lectures, followed by a practicum assignment. Student groups select a pharmacy, medical, or other identified problem and devise a solution through application of CQI principles. A member of each group presents the completed project to the class and a panel of CQI experts who judge project quality. Practice innovation: Students apply learned CQI principles through a problem-based practicum assignment in which they create a plan to solve a real health care or non– health care problem. Main outcome measure: Student learning was assessed through the project presentation and examination questions. A voluntary formative evaluation examined student attitudes toward the CQI module, applicability of CQI to pharmacy practice, and the most effective strategy for learning CQI concepts. Results: Mean scores for presentations were 93%, reflecting a high level of ability to apply CQI principles. In the formative evaluation, 80% of students reported that lectures were informative or necessary; however, all responding students reported learning more through practicum experience as opposed to the lecture alone. Ninety-seven percent of students were able to provide examples of CQI opportunities in their career interest area. Conclusion: This structured learning opportunity teaches students a systematic approach to identifying and solving system problems by applying CQI principles. © 2009 Elsevier Inc. All rights reserved. Keywords: Quality improvement; Pharmacy education; Practical application; Student pharmacists
Introduction Systems for assessing quality of care, identifying areas for improvement and determining a means to create improvements are being developed and applied within health care to support
* Corresponding author. Susan J. Skledar, BS MPH, Associate Professor, Department of Pharmacy and Therapeutics, Director of the Drug Use and Disease State Management Program, University of Pittsburgh School of Pharmacy and University of Pittsburgh Medical Center, 302 Scaife Hall, 200 Lothrop Street, Pittsburgh, PA 15213. E-mail address: [email protected].
the provision of quality patient care.1– 4 Continuous quality improvement (CQI) provides a structured, organizational process for involving personnel in planning and executing a continual flow of improvements, with the goal of providing quality health care that meets or exceeds expectations.5 Demonstrating the use of CQI methods to improve patient outcomes and increase safety and treatment efficiency is currently a requirement to maintain hospital and managed care organization accreditation status. Agencies and regulatory groups such as The Joint Commission, the National Committee on Quality Assurance, and the Centers for Medicare and Medicaid Services have established national quality measures for health care institutions and managed care organizations.
1877-1297/09/$ – see front matter © 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.cptl.2009.10.003
80
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
A new collaborative group, the Pharmacy Quality Alliance (PQA), was formed in 2006 with the mission “to improve health care quality and patient safety through a collaborative process in which the key stakeholders agree on a strategy for measuring performance at the pharmacy and pharmacist-levels; collecting data in the least burdensome way; and reporting meaningful information to consumers, pharmacists, employers, payers, and other healthcare decision-makers to help make informed choices, improve outcomes and stimulate the development of new payment models.”6 The interdisciplinary members of the PQA will be working toward development of educational programs, performance metrics, and data-reporting mechanisms for pharmacists and pharmacy practice sites across the United States. The data provided by the performance metrics (e.g., diabetes-based measures) are key sources for consumers and practitioners to compare quality of services. Because CQI maximizes the quality and efficiency of care for patients and providers, this improvement process can be directed toward clinical or administrative processes within the health care system.7 Interdisciplinary CQI teams now include pharmacists, whose expertise in drug therapy management serves to improve the care of patients with conditions such as pneumonia, heart failure, and diabetes. Pharmacists can also apply expertise toward improvements in medication systems and various steps in the medication use process, such as dispensing and medication administration. Health care quality measures that are routinely monitored include: (1) clinical outcomes such as length of stay, readmissions, adverse drug events, or mortality; (2) process measures such as medication delivery turnaround time, client satisfaction, and types and/or causes of medication errors; and (3) economic measures such as drug expenditures, drug cost per patient day, and total treatment cost. In its standards for the Doctor of Pharmacy degree, the Accreditation Council for Pharmacy Education (ACPE) puts emphasis on professional competencies, patient safety, interprofessional teamwork, and evaluation/assessment. Standard 12 denotes the importance of the interprofessional health care team in making evidence-based decisions, managing medication use systems, improving therapeutic outcomes, and developing health policy.8 The accompanying guideline highlights the need for pharmacy graduates to use quality improvement strategies to assess and address change, use problem-solving techniques, apply methods of outcome monitoring, and develop health policy, all of which relate to understanding CQI methodology. Limited published data exist regarding the teaching of continuous quality improvement in pharmacy schools.9 –12 Jackson reports a quality assurance course at Midwestern University Chicago College of Pharmacy in which pharmacy students work with preceptors longitudinally over a semester to develop and implement a plan to reduce medication errors in a clinical setting.13 This focused experience demonstrated to students the great potential that pharmacists have to improve medication safety in their workplace. A recent review of health care basic
concepts for clinicians noted that “front-line health care professionals will be most effective in optimally improving quality and performance in their environment if they first appreciate the characteristics and tools available for enhancing quality of care . . .”14 Our paper reports an approach to teaching CQI that focuses on teaching CQI principles and application tools to students at the University of Pittsburgh School of Pharmacy through lectures and a broad health care application of learned concepts in a group project.
Design The CQI learning module entitled “Continuous Quality Improvement: Theory and Application to Practice” is part of a three-credit course on Health Care Outcomes and Pharmacoeconomics in the Profession of Pharmacy series. This required course, taken by pharmacy students in their third professional year, is designed to develop competence in health care decision-making through an understanding of economic, clinical, and humanistic outcomes. Specific to the CQI portion of the course, the student is expected to be able to describe the principles and processes of CQI, compare and contrast methods of measuring quality in health care, and understand the impact of CQI on the provision of health care services. The educational objectives for the CQI segment are for students to learn: (1) to evaluate patient- and population-specific data, quality assurance strategies, and research processes to identify problems and create possible solutions that optimize therapeutic outcomes of the medication use process; and (2) to develop patientcentered and population-based drug therapy decisions on the basis of quality assurance strategies, medication use review, and patient safety data. The theory of CQI is presented in two 2-hour lectures. Lecture one covers CQI theory and the explanation of different performance improvement models. Students are taught about the different organizations and agencies that regulate quality in health care, sources for health care quality indicator measures, how to distinguish between quality improvement and research, and several different quality improvement models in use today. The model that is explained in detail to the students is the “FOCUS-PDCA” performance improvement model, which is a stepwise approach to teaching how to design, implement, and evaluate a quality improvement initiative.15 Lecture two includes two example CQI presentations, as well as a critique/discussion of the presentation during the class. The first example presentation, entitled “Collaborative Quality Improvement of Patient Education,” and presented in the FOCUSPDCA format, was implemented at the University of Pittsburgh Medical Center (UPMC) by one of the School of Pharmacy faculty as part of an interdisciplinary care team.3 This quality improvement project won the award for Patient Safety at the UPMC annual Quality and Innovation Fair. Presented by the expert faculty member, this project exemplifies the role of the pharmacist on the problem-solving
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
team in a real-life healthcare setting. The second example presentation, a student CQI project from the previous year, allows the class to see the translation of CQI principles into a student-generated project. As part of the CQI module, students are required to complete a group practicum assignment project outside of class to reinforce the material presented in the didactic lectures. For the project, each student group cooperatively applies CQI principles to assess a problem and then proposes a solution that could be applied. There are two weeks between the first lecture and the practicum presentation (4-hour) timeslot for the students to work through CQI project development. The second lecture (when examples are presented) allows time for student group discussion of projects with the course faculty (Ms. Skledar). Groups are also able to meet with the course faculty if any questions arise before the group presentations. Each group must select a topic to investigate, design quality indicator measures, and develop an action plan. To assist in generating ideas for topics and to encourage a focus on current health care problems, students are directed to websites of national quality organizations and health care associations, such as the National Quality Forum,16 the Institute for Healthcare Improvement,17 the Centers for Disease Control and Prevention,18 and the American Society of Health-System Pharmacists.19 In the academic year 2007– 2008, students were also directed to the PQA website for pharmacy-specific quality metric ideas.6 Student groups are required to turn in at least two articles or citations from a published source that are related to their chosen topic to either demonstrate the need for improvement or present a solution to their problem. To show applicability of the CQI methods outside of pharmacy practice and health care, a limited number of randomly selected student groups are given the opportunity to choose a topic that is not related to health care. This approach provides a broader application of the process and achieves the goal of the class, which is to understand how to use CQI principles for process improvement. Group topics must be approved by the course instructor before commencing work on the project. During the preparation time for the practicum, each student group is required to prepare and submit as many as 15 slides for the practicum presentation. One or two members of the group give a 5-minute verbal presentation of their project (excerpted from the slides) to the class and an interdisciplinary CQI panel of three to four quality experts from UPMC. The panel members evaluate the presentations and interact with the class to stimulate discussion and ask questions about the presentations. Members of the interdisciplinary panel vary each year but have included the UPMC Medication Safety Manager, Advanced Practice Nurses in Pain Management and Internal Medicine, clinical pharmacy faculty, and the Co-Chairs of the UPMC Quality and Innovation Fair (School of Pharmacy faculty member, Ms. Skledar, and a registered nurse who is the Senior Quality Improvement Coordinator for UPMC) to bring a wealth of
81
experience and a broader perspective to the students. Because interdisciplinary collaboration is a fundamental step in CQI models, encouraging diversity in panel members has provided thought-provoking questions and discussion on the topics. Presentations are assessed for application of the CQI process according to a standardized evaluation form that describes the elements of the FOCUS-PDCA method (Appendix 1). Students are able to observe and ask questions about the way the CQI process was applied in each group’s project, allowing them an opportunity to learn from the work of the other groups. The ability outcomes for this CQI module that relate directly to the practicum include demonstrating the ability to contribute effectively to a group project that requires a final written or oral presentation. Collaboration with colleagues and completion of self-directed learning assignments on time are necessary components for successful practicum performance. Researching processes to identify problems and create health care solutions are also part of the practicum experience.
Assessment methods The impact of the CQI portion of this course was evaluated by the scoring on the practicum presentations and the formative evaluation. The scoring rubric used for the practicum presentation assessed student understanding and ability to apply CQI methods to a real-life health care or non– health care topic. Presentation skills were evaluated, noting the ability to communicate clearly and stay within time limits. Presenters for each group were chosen in class on the day of the presentation, which required all group members to be knowledgeable of the work. The entire group participated in the question and answer session with the panel. A formative evaluation was administered during practicum presentations (Spring 2006 term) to assess attitudes of students toward the CQI module itself, its applicability to pharmacy practice, and the elements of the module that were most effective for them in learning concepts. Seven open-ended items, shown in Appendix 2, allowed students to freely express their opinions about the lectures, the practicum, the most effective sources for learning, and the most difficult part of the practicum. Students were asked to note their future area of pharmacy practice and how they thought CQI could be used in that focus area. Participation in the formative evaluation exercise was voluntary and the evaluation was administered in a confidential manner that would not affect the students’ course grade. An additional assessment for the CQI component of the course included a total of 10 true-false or multiple-choice questions on the midterm examination (3 questions) and final examination (7 questions). These questions assessed knowledge of the FOCUS-PDCA methodology and steps in the CQI development process. A concept map was created by each group to demonstrate understanding of the links between CQI, pharmacists,
82
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
Table 1 Group scores for CQI presentations for each FOCUS-PDCA element Presentation elements—FOCUS-PDCA (possible score) Groups (N ⫽ 16)
F (2.0)
O (2.0)
C (2.0)
U (2.0)
S (2.0)
P (2.0)
D (3.0)
C (3.0)
A (2.0)
Verbal (5.0)
Map (5.0)
Total (30.0)
Range Mean SD
1.7–2.0 2.0 0.1
1.0–2.0 1.8 0.4
1.5–2.0 1.8 0.2
2.0 2.0 0.0
1.0–2.0 1.9 0.3
1.0–2.0 1.9 0.3
2.3–3.0 2.6 0.30
2.7–3.0 2.8 0.3
1.7–2.0 1.9 0.3
4.0–5.0 4.8 0.3
4.0–5.0 4.6 0.5
25.7–29.7 27.9 1.3
CQI, continuous quality improvement; FOCUS-PDCA, find, organize, clarify, uncover, start–plan, do, check, act.
and patient care. A concept map is an evaluation technique that can help to track the thinking of students on a concept or issue and help to develop thinking skills.20 –22 Students were instructed that there are many ways to construct this map, and they were encouraged to record free-flowing ideas relating CQI to the role of the pharmacist, with a focus on improving patient care. Results Practicum scoring The mean score of practicum presentations for the 16 graded groups was 27.9 ⫹ 1.3 SD points of 30 possible points (93%). Scores for each group in the individual elements are shown in Table 1. Students scored highest in the areas of defining interdisciplinary involvement, identifying the evidence base for their problem or action, and mapping out their implementation timetables. Identifying customers and their expectations, defining data sources, and describing implementation methods, FOCUS-PDCA elements “O,” “C,” and “D”, respectively, were areas that showed room for improvement. Students voted during class on their favorite peer presentation, and an in-class award was given to the “Student’s Choice” for the top two presentations. The CQI panel also selected their first- and second-place choices overall. Of interest, in 2006 the class’s selection for favorite presentation and runner-up were the same two presentations selected by the panel, although the first- and second-place order was reversed. Formative evaluation A total of 76 students of 100 taking the course completed a formative evaluation, realizing a 76% response rate. The lectures were considered informative or necessary by 61 of 76 (80%) students. Fewer students responded to the question regarding the value of the practicum, with 60 of 70 (85.7%) students reporting that the practicum added value to CQI learning. All responding students (72/72) reported learning more through the practicum experience compared with the lecture. In response to the question regarding the most difficult part of the practicum, students reported that the timing of the practicum close to the spring break in the 2006 term made it difficult for some groups to get together. Beyond the timing issue, the most difficult part of the practicum project
was found in the initial planning stages: getting started (picking a topic), doing research, and choosing measurement indicators. Of the six student groups given the opportunity to select a nonpharmacy topic for the practicum, four of the six groups elected to identify and develop nonpharmacy topics. Topics that students presented in 2006 are found in Appendix 3, along with nonpharmacy topics presented in this and previous years. The open-ended “Other Comments” item at the end of the evaluation yielded comments related primarily to the use of non– health care topics. Twenty students indicated that these topics provided a fun way to learn about CQI. Three students specifically commented that they enjoyed hearing about how CQI applies to real life and indicated that this experience changed their views on pharmacy’s role in health care today. Seven students offered suggestions to improve the class, such as including health care topic ideas from experiential learning/internship preceptors, allowing more class time for questions and group work, and more inclusion of community-based quality measures, especially because a large number of students reported that they will be pursuing careers in community pharmacy. Even after the group presentations, seven students did not believe that they needed to learn about CQI. Negative comments included the opinion that CQI was for administrators and would not be encountered in their practice. However, 91% (69/76) of students’ comments reflected an interest in CQI and recognized its potential for fostering improvements. Nearly all of the students (97%) were able to provide examples of applications for CQI in their area of interest. Students pursuing careers in community practice reported examples such as improving waiting time for prescription refills, procuring medications for the underserved, and implementing medication therapy management programs. Students interested in institutional practice suggested decreasing medication delivery times, designing and implementing evidence-based protocols, and creating discharge education programs as examples of how CQI would fit into their practice sites. Scores from examination questions The midterm questions were administered after the lectures but before the practicum assignment was completed. For the 100 students taking the examination, the mean score
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
on the three CQI questions was 83%. The cumulative final examination, administered after the practicum exercise was completed, produced a mean score of 97.4% on the seven CQI questions. Concept map A total of 49 of 68 (67.6%) students felt that the concept map did not add value. They reported greater learning through the group practicum assignment versus the group concept map and thought that the concept map exercise itself was more useful in their earlier years of pharmacy school. Discussion The approach described here combines traditional classroom teaching with a student-directed health care management problem-solving exercise. This type of practical application promotes reinforcement of CQI principles while familiarizing students with CQI tools and methods. The usefulness of the project as an active learning technique is consistent with findings when this approach was used in pharmacy13 as well as other topics.23,24 In the formative evaluation, students indicated that they learned more through applying the CQI process to their chosen topic in the group project as opposed to the lectures. Students also reported that they benefited from the presentations of the other groups’ work. This voluntary reporting revealed students’ perception that the project enhanced their learning. Identifying a topic was reported as one of the more difficult aspects of the practicum. Providing a list of possible pharmacy topics and professional organization references helped the student groups in selecting a topic for their project. Students were permitted to use independently identified topics, if approved in advance by the instructor. To demonstrate the applicability of the FOCUS-PDCA model to nonpharmacy (“life” topics), a small number of groups (6/16) were given the chance to select a non– health care–related topic. Students were required to provide articles from newspapers, magazines, websites, student newspapers, university publications, and so on, to justify the selected topic. The use of nonpharmacy topics demonstrated the usefulness of CQI principles in identifying and solving problems in all walks of life. This broad application of CQI to a varied list of problems stimulated interest and enabled students to see the value of the process more clearly. Students reflected that they liked seeing how CQI applies to real life, and they learned about the application process through the practicum, with both health care and non– health care topics. The voluntary formative evaluation yielded a 76% response rate, although students did not reply to every question. The instructor reviewed the findings of the evaluation with the students in the next class, and the class then made suggestions for improvement for the upcoming year. One item on the evaluation challenged students to identify processes that could be investigated and improved through the CQI process in practice settings, where they may choose to
83
work after graduation. The students’ ability to generate these ideas for improvement demonstrated their understanding of applying CQI methods outside of the classroom and into their future (and current) practice sites. Although not the primary assessment measure, the final examination scores demonstrated increased knowledge gained by students after the practicum. This improvement suggests the value of the practicum in reinforcing understanding of CQI methodology and tools. Although not specifically mentioned by students, it is likely that the interactions/suggestions from the expert panel of judges provided additional learning for the students beyond the information that was presented in the lectures. Areas that showed room for improvement on the basis of practicum scores, such as identifying data sources and stating mechanisms to sustain their improvement actions over time, were emphasized in the lectures when the course was taught the following spring. More information regarding where data can be gathered (i.e., billing records, surveys, literature, or medical records) was also provided in subsequent years. Verbal, written, and electronic dissemination strategies for implementation were discussed with the students to demonstrate the wide array of approaches that can be used to educate health care staff. The need for sustaining the improvements over time, as well as continual reflection on outcomes and new literature, was also emphasized during the lectures and the practicum presentations. The concept map was used as a one-time exercise to measure student CQI concept understanding. According to faculty perspective and student feedback from the formative evaluation, this abstract exercise had minimal additional impact on learning. The students reported that they learned much more from the project development and listening to real-life scenarios. Therefore, the practicum will be used as the ongoing assessment of student understanding of CQI methods. In this CQI module, student pharmacists demonstrated their understanding of how this methodology could apply to health care problems and, innovatively, to non– health care problems they may encounter. Health care professionals who understand and use proven CQI tools are positioned to guide their work group toward interdisciplinary approaches to improve and achieve quality care.25 Recognition of the importance of learning CQI has been demonstrated by incorporation of these concepts into curricula in schools of medicine and nursing, as well as in medical residency training.11,26 –30 Some of these programs are designed to include multidisciplinary group interaction in solving problems, which reinforces the advantages of this type of cooperative involvement.11,29,30 Inclusion of CQI principles into the pharmacy curriculum, reinforced by practical experience, creates student awareness of the need for ongoing improvement processes and provides the tools to accomplish this goal. Teaching CQI concepts in the pharmacy curriculum is not unique, but teaching students a specific performance improvement model and having them apply these steps to a current health
84
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
care problem or, more uniquely, to a “life problem” is not well described in the pharmacy or medical literature. Summary Presenting the CQI process through a lecture format provided a knowledge base for understanding CQI principles. Through the practicum, students learned how to use the CQI process to assess the status and discover possible mechanisms for improvement in a variety of pharmacy and nonpharmacy topics. This practical application of learning reinforced the students’ understanding of CQI principles
and gave them an increased understanding of the value of the CQI process. It is advantageous to provide future pharmacists with tools to assist them in their professional careers. This structured learning opportunity equips students through a systematic approach to problem-solving using CQI tools for identifying and resolving system problems. Students learn how to apply concepts and tools, as well as appreciate the usefulness of this process. Training student pharmacists to understand the entire CQI process and how to apply it will enable them to effectively create system improvements in their future workplace.
Appendix 1: Scoring tool for CQI presentations Continuous Quality Improvement CQI Project Evaluation Form Group: ______________________________ Date: _______________ Presentation Title: _____________________________________________ CQI Element (FOCUS-PDCA Methodology15)
Point Score
Comments
FIND the process to improve (2 points) If used, state the problem and ASHP 2015 goal/objective or National Quality Foundation safe practice ● Is it focused and measurable? ORGANIZE a team that knows the process (2 points) ● Is team multidisciplinary? ● Identify guidance team members ● Identify project team members ● Identify customers and expectations CLARIFY current knowledge (2 points) ● Based on data—not opinion or guess ● Flowchart current process ● Literature/information provided to support project (2 sources) ● Other institution/site practices cited ● Identify at least 2 key quality outcome indicators (numerator, denominator, data source, and threshold included) UNDERSTAND causes of process variation (2 points) ● List of brainstormed problem causes provided SELECT the process to change (2 points) ● Causes of the problem ranked according to how much they affect the problem (e.g., 1 ⫽ high impact to 10 ⫽ low impact) ● Select (circle) root cause PLAN the improvement/change (2 points) ● State the intervention ● Show timetable for activities DO the improvement/change (3 points) ● State educational methods ● Who will do the education and how? (inservices, posters, etc.) ● State target audience for education CHECK the change (3 points) ● State results (not required since not expected to implement) ● Revise process flowchart with improvement steps ACT to hold the gains (2 points) ● State how the project will continue over time ● Define next action steps/future plans Verbal Presentation (5 points) Concept Map (5 points) OVERALL SCORE (30 POINTS) ●
ASHP, American Society of Health-System Pharmacists; CQI, Continuous Quality Improvement; FOCUS-PDCA, Find, Organize, Clarify, Understand, Select–Plan, Do, Check, Act.
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
Appendix 2: Student formative evaluation of teaching strategy Name two practice areas you are considering and how CQI could fit into these areas. The lectures were . . . The practicum was . . . The concept map was . . . I learned the most by . . . The most difficult part was . . . How can CQI be applied to your future area of pharmacy practice? Other comments?
Œ
85
Pharmacy information technology help for instructors in the classroom
Acknowledgments Sandra Kane-Gill, Pharm.D., is acknowledged for her review of this manuscript and for coordination of the Profession of Pharmacy 6 course content. Gary Stoehr, Pharm.D., is acknowledged for his assistance with the design of the formative evaluation for this project. Amy (Calabrese) Donihi, Pharm.D., is acknowledged for her help in presenting the CQI model example to the students as a model for a successful CQI project. References
Appendix 3: Student group topics presented for the practicum Healthcare Topics (obtained from provided sources) ● Patient education programs Œ Smoking cessation counseling programs Œ Discharge counseling programs Œ Medication reconciliation processes ● Applying technology to improve medication use Œ Computerized physician order entry Œ Barcoding ● Statin, beta-blocker, and aspirin use after acute myocardial infarction ● Discontinuation of prophylactic antibiotics post-surgery Œ Role of pharmacist for automatic stop-order programs ● Improving the pharmacist-patient relationship by education and counseling ● Improving hand hygiene in hospitals ● Improving end-of-life quality of care Œ Care plan and protocol development “Life Topics” (ideas generated from students) ● Yielding to pedestrians in crosswalks ● What to do when the bus passes you by Œ Options for public transportation and on-site shuttles ● Cleaning up a local “greasy spoon” restaurant in town ● Traffic congestion in the Pittsburgh tunnels ● Temperature control in the classroom and its effect on learning ● Improving student attendance by modifying block schedules ● Effective use of SOAP (Subjective-Objective-Assessment-Plan) notes ● Effective use of technology in the classroom Œ Procedures for posting course materials
1. Curtis JR, Cook DJ, Wall RJ, et al. Intensive care unit quality improvement: a “how-to” guide for the interdisciplinary team. Crit Care Med 2006;34:211– 8. 2. Culley CM, Bernardo JF, Gross PR, et al. Implementing a standardized safety procedure for continuous renal replacement therapy solutions. Am J Health Syst Pharm 2006;63:756 – 63. 3. Calabrese AT, Cholka K, Lenhart SE, et al. Pharmacist involvement in a multidisciplinary inpatient medication education program. Am J Health Syst Pharm 2003;60:1012– 8. 4. Dunwoody C, Skledar SJ, Freeman S. Changes in patientcontrolled analgesia following a meperidine overdose. Jt Comm J Qual Patient Saf 2006;32:528 –30. 5. McLaughlin CP, Kaluzny AD. Defining quality improvement. In: McLaughlin CP, Kaluzny AD, eds. Continuous Quality Improvement in: Health Care, 3rd ed. Sudbury, MA: Jones and Bartlett Publishers; 2006:3– 40. 6. Pharmacy Quality Alliance. Home page. Available at: http:// www.pqaalliance.org/. Accessed October 11, 2009. 7. Linder JC. Outcomes measurement: compliance tool or strategic initiative? Health Care Manage Rev 1991;16:21–33. 8. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines. Available at: http://www.acpeaccredit.org/standards/default.asp. Accessed October 10, 2009. 9. Klapp D. Quality assurance training for pharmacy students. Am J Health Sys Pharm 1980;37:922– 4. 10. Strand LM, Morley PC. Evolving health care systems: academic implications for teaching methodologies with emphasis on administration and practice. Am J Pharm Educ 1987;51:402– 6. 11. Knapp DE, Michocki RJ, Wiser TW. A flexible module for teaching quality assurance and drug prescribing review concepts. Am J Pharm Educ 1986;50:274 –7. 12. Horsburgh M, Merry A, Seddon M, et al. Educating for healthcare quality improvement in an interprofessional learning environment: a New Zealand initiative. J Interprof Care 2006;20: 555–7. 13. Jackson TL. Application of quality assurance principles: teaching medication error reduction skills in a “real world” environment. Am J Pharm Educ 2004;68(1):Article 17. 14. Varkey P, Reller MK, Resar RK. Basics of quality improvement in health care. Mayo Clin Proc 2007;82:735–9. 15. Medical Risk Management Associates, LLC. Employing tools to enhance patient safety, The FOCUS-PDCA methodology. Available at: http://www.isixsigma.com/dictionary/FOCUSPDCA-823.htm. Accessed November 19, 2009.
86
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
16. National Quality Forum. National priorities for healthcare quality measurements and reporting. Available at: http:// www.qualityforum.org/Settings Priorities/Addressing National Priorities.aspx. Accessed July 23, 2009. 17. Institute for Healthcare Improvement. Topics, Institute for Healthcare Improvement. Available at: http://www.ihi.org/ihi/ topics. Accessed October 11, 2009. 18. U.S. Department of Health and Human Services, Office of Disease Prevention and Health Service Promotion. Leading health indicators, Healthy People 2010. Available at: http:// health.gov/healthypeople. Accessed October 11, 2009. 19. American Society of Health System Pharmacists. 2015 Initiative. Available at: http://www.ashp.org/s_ashp/cat1c.asp? CID⫽218&DID⫽255. Accessed October 11, 2009. 20. Novak JD. The promise of new ideas and technology for improving teaching and learning. Cell Biol Educ 2003;2:122–32. 21. West DC, Pomeroy JR, Park JK, et al. Critical thinking in graduate medical education: a role for the concept mapping assessment? JAMA 2000;284:1105–10. 22. Hill LH. Concept mapping in a pharmacy communications course to encourage meaningful student learning. Am J Pharm Educ 2004;68(5):Article 109.
23. McCarthy JP, Anderson L. Active learning techniques versus traditional teaching styles: two experiments form history and political science. Innovative Higher Education 2000;24:279 – 94. 24. Prince M. Does active learning work? A review of the research. J Eng Educ 2004;93:1–9. 25. Burhans LD. What is quality: do we agree, and does it matter? J Healthc Qual 2007;29:39 – 44, 54. 26. Teeley KH, Lowe JM, Beal J, et al. Incorporating quality improvement concepts and practice into a community health nursing course. J Nurs Educ 2006;45:86 –90. 27. Krajewski K, Siewert B, Yam S, et al. A quality assurance elective for radiology residents. Acad Radiol 2007;14:239 – 45. 28. Weingart SN, Tess A, Driver J, et al. Creating a quality improvement elective for medical house officers. J Gen Intern Med 2004;19:861–7. 29. Varkey P, Reller MK, Smith A, et al. An experiential quality improvement education initiative. Am J Med Qual 2006;21: 317–22. 30. Walrath JM, Muganlinskaya N, Shepherd M, et al. Interdisciplinary medical, nursing, and administrator education in practice: the Johns Hopkins experience. Acad Med 2006;81:744 – 8.
Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86 http://www.pharmacyteaching.com
A method for teaching continuous quality improvement to student pharmacists through a practical application project Susan J. Skledar, BS MPHa,b,*, Teresa P. McKaveney, BSa a
University of Pittsburgh School of Pharmacy, Pittsburgh, PA b University of Pittsburgh Medical Center, Pittsburgh, PA
Abstract Objective: To provide future pharmacists with the skills and knowledge they need to use the continuous quality improvement (CQI) process to solve problems in a variety of practice settings. Material and methods: At the University of Pittsburgh School of Pharmacy, the theory of CQI is presented to third-year pharmacy students in two lectures, followed by a practicum assignment. Student groups select a pharmacy, medical, or other identified problem and devise a solution through application of CQI principles. A member of each group presents the completed project to the class and a panel of CQI experts who judge project quality. Practice innovation: Students apply learned CQI principles through a problem-based practicum assignment in which they create a plan to solve a real health care or non– health care problem. Main outcome measure: Student learning was assessed through the project presentation and examination questions. A voluntary formative evaluation examined student attitudes toward the CQI module, applicability of CQI to pharmacy practice, and the most effective strategy for learning CQI concepts. Results: Mean scores for presentations were 93%, reflecting a high level of ability to apply CQI principles. In the formative evaluation, 80% of students reported that lectures were informative or necessary; however, all responding students reported learning more through practicum experience as opposed to the lecture alone. Ninety-seven percent of students were able to provide examples of CQI opportunities in their career interest area. Conclusion: This structured learning opportunity teaches students a systematic approach to identifying and solving system problems by applying CQI principles. © 2009 Elsevier Inc. All rights reserved. Keywords: Quality improvement; Pharmacy education; Practical application; Student pharmacists
Introduction Systems for assessing quality of care, identifying areas for improvement and determining a means to create improvements are being developed and applied within health care to support
* Corresponding author. Susan J. Skledar, BS MPH, Associate Professor, Department of Pharmacy and Therapeutics, Director of the Drug Use and Disease State Management Program, University of Pittsburgh School of Pharmacy and University of Pittsburgh Medical Center, 302 Scaife Hall, 200 Lothrop Street, Pittsburgh, PA 15213. E-mail address: [email protected].
the provision of quality patient care.1– 4 Continuous quality improvement (CQI) provides a structured, organizational process for involving personnel in planning and executing a continual flow of improvements, with the goal of providing quality health care that meets or exceeds expectations.5 Demonstrating the use of CQI methods to improve patient outcomes and increase safety and treatment efficiency is currently a requirement to maintain hospital and managed care organization accreditation status. Agencies and regulatory groups such as The Joint Commission, the National Committee on Quality Assurance, and the Centers for Medicare and Medicaid Services have established national quality measures for health care institutions and managed care organizations.
1877-1297/09/$ – see front matter © 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.cptl.2009.10.003
80
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
A new collaborative group, the Pharmacy Quality Alliance (PQA), was formed in 2006 with the mission “to improve health care quality and patient safety through a collaborative process in which the key stakeholders agree on a strategy for measuring performance at the pharmacy and pharmacist-levels; collecting data in the least burdensome way; and reporting meaningful information to consumers, pharmacists, employers, payers, and other healthcare decision-makers to help make informed choices, improve outcomes and stimulate the development of new payment models.”6 The interdisciplinary members of the PQA will be working toward development of educational programs, performance metrics, and data-reporting mechanisms for pharmacists and pharmacy practice sites across the United States. The data provided by the performance metrics (e.g., diabetes-based measures) are key sources for consumers and practitioners to compare quality of services. Because CQI maximizes the quality and efficiency of care for patients and providers, this improvement process can be directed toward clinical or administrative processes within the health care system.7 Interdisciplinary CQI teams now include pharmacists, whose expertise in drug therapy management serves to improve the care of patients with conditions such as pneumonia, heart failure, and diabetes. Pharmacists can also apply expertise toward improvements in medication systems and various steps in the medication use process, such as dispensing and medication administration. Health care quality measures that are routinely monitored include: (1) clinical outcomes such as length of stay, readmissions, adverse drug events, or mortality; (2) process measures such as medication delivery turnaround time, client satisfaction, and types and/or causes of medication errors; and (3) economic measures such as drug expenditures, drug cost per patient day, and total treatment cost. In its standards for the Doctor of Pharmacy degree, the Accreditation Council for Pharmacy Education (ACPE) puts emphasis on professional competencies, patient safety, interprofessional teamwork, and evaluation/assessment. Standard 12 denotes the importance of the interprofessional health care team in making evidence-based decisions, managing medication use systems, improving therapeutic outcomes, and developing health policy.8 The accompanying guideline highlights the need for pharmacy graduates to use quality improvement strategies to assess and address change, use problem-solving techniques, apply methods of outcome monitoring, and develop health policy, all of which relate to understanding CQI methodology. Limited published data exist regarding the teaching of continuous quality improvement in pharmacy schools.9 –12 Jackson reports a quality assurance course at Midwestern University Chicago College of Pharmacy in which pharmacy students work with preceptors longitudinally over a semester to develop and implement a plan to reduce medication errors in a clinical setting.13 This focused experience demonstrated to students the great potential that pharmacists have to improve medication safety in their workplace. A recent review of health care basic
concepts for clinicians noted that “front-line health care professionals will be most effective in optimally improving quality and performance in their environment if they first appreciate the characteristics and tools available for enhancing quality of care . . .”14 Our paper reports an approach to teaching CQI that focuses on teaching CQI principles and application tools to students at the University of Pittsburgh School of Pharmacy through lectures and a broad health care application of learned concepts in a group project.
Design The CQI learning module entitled “Continuous Quality Improvement: Theory and Application to Practice” is part of a three-credit course on Health Care Outcomes and Pharmacoeconomics in the Profession of Pharmacy series. This required course, taken by pharmacy students in their third professional year, is designed to develop competence in health care decision-making through an understanding of economic, clinical, and humanistic outcomes. Specific to the CQI portion of the course, the student is expected to be able to describe the principles and processes of CQI, compare and contrast methods of measuring quality in health care, and understand the impact of CQI on the provision of health care services. The educational objectives for the CQI segment are for students to learn: (1) to evaluate patient- and population-specific data, quality assurance strategies, and research processes to identify problems and create possible solutions that optimize therapeutic outcomes of the medication use process; and (2) to develop patientcentered and population-based drug therapy decisions on the basis of quality assurance strategies, medication use review, and patient safety data. The theory of CQI is presented in two 2-hour lectures. Lecture one covers CQI theory and the explanation of different performance improvement models. Students are taught about the different organizations and agencies that regulate quality in health care, sources for health care quality indicator measures, how to distinguish between quality improvement and research, and several different quality improvement models in use today. The model that is explained in detail to the students is the “FOCUS-PDCA” performance improvement model, which is a stepwise approach to teaching how to design, implement, and evaluate a quality improvement initiative.15 Lecture two includes two example CQI presentations, as well as a critique/discussion of the presentation during the class. The first example presentation, entitled “Collaborative Quality Improvement of Patient Education,” and presented in the FOCUSPDCA format, was implemented at the University of Pittsburgh Medical Center (UPMC) by one of the School of Pharmacy faculty as part of an interdisciplinary care team.3 This quality improvement project won the award for Patient Safety at the UPMC annual Quality and Innovation Fair. Presented by the expert faculty member, this project exemplifies the role of the pharmacist on the problem-solving
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
team in a real-life healthcare setting. The second example presentation, a student CQI project from the previous year, allows the class to see the translation of CQI principles into a student-generated project. As part of the CQI module, students are required to complete a group practicum assignment project outside of class to reinforce the material presented in the didactic lectures. For the project, each student group cooperatively applies CQI principles to assess a problem and then proposes a solution that could be applied. There are two weeks between the first lecture and the practicum presentation (4-hour) timeslot for the students to work through CQI project development. The second lecture (when examples are presented) allows time for student group discussion of projects with the course faculty (Ms. Skledar). Groups are also able to meet with the course faculty if any questions arise before the group presentations. Each group must select a topic to investigate, design quality indicator measures, and develop an action plan. To assist in generating ideas for topics and to encourage a focus on current health care problems, students are directed to websites of national quality organizations and health care associations, such as the National Quality Forum,16 the Institute for Healthcare Improvement,17 the Centers for Disease Control and Prevention,18 and the American Society of Health-System Pharmacists.19 In the academic year 2007– 2008, students were also directed to the PQA website for pharmacy-specific quality metric ideas.6 Student groups are required to turn in at least two articles or citations from a published source that are related to their chosen topic to either demonstrate the need for improvement or present a solution to their problem. To show applicability of the CQI methods outside of pharmacy practice and health care, a limited number of randomly selected student groups are given the opportunity to choose a topic that is not related to health care. This approach provides a broader application of the process and achieves the goal of the class, which is to understand how to use CQI principles for process improvement. Group topics must be approved by the course instructor before commencing work on the project. During the preparation time for the practicum, each student group is required to prepare and submit as many as 15 slides for the practicum presentation. One or two members of the group give a 5-minute verbal presentation of their project (excerpted from the slides) to the class and an interdisciplinary CQI panel of three to four quality experts from UPMC. The panel members evaluate the presentations and interact with the class to stimulate discussion and ask questions about the presentations. Members of the interdisciplinary panel vary each year but have included the UPMC Medication Safety Manager, Advanced Practice Nurses in Pain Management and Internal Medicine, clinical pharmacy faculty, and the Co-Chairs of the UPMC Quality and Innovation Fair (School of Pharmacy faculty member, Ms. Skledar, and a registered nurse who is the Senior Quality Improvement Coordinator for UPMC) to bring a wealth of
81
experience and a broader perspective to the students. Because interdisciplinary collaboration is a fundamental step in CQI models, encouraging diversity in panel members has provided thought-provoking questions and discussion on the topics. Presentations are assessed for application of the CQI process according to a standardized evaluation form that describes the elements of the FOCUS-PDCA method (Appendix 1). Students are able to observe and ask questions about the way the CQI process was applied in each group’s project, allowing them an opportunity to learn from the work of the other groups. The ability outcomes for this CQI module that relate directly to the practicum include demonstrating the ability to contribute effectively to a group project that requires a final written or oral presentation. Collaboration with colleagues and completion of self-directed learning assignments on time are necessary components for successful practicum performance. Researching processes to identify problems and create health care solutions are also part of the practicum experience.
Assessment methods The impact of the CQI portion of this course was evaluated by the scoring on the practicum presentations and the formative evaluation. The scoring rubric used for the practicum presentation assessed student understanding and ability to apply CQI methods to a real-life health care or non– health care topic. Presentation skills were evaluated, noting the ability to communicate clearly and stay within time limits. Presenters for each group were chosen in class on the day of the presentation, which required all group members to be knowledgeable of the work. The entire group participated in the question and answer session with the panel. A formative evaluation was administered during practicum presentations (Spring 2006 term) to assess attitudes of students toward the CQI module itself, its applicability to pharmacy practice, and the elements of the module that were most effective for them in learning concepts. Seven open-ended items, shown in Appendix 2, allowed students to freely express their opinions about the lectures, the practicum, the most effective sources for learning, and the most difficult part of the practicum. Students were asked to note their future area of pharmacy practice and how they thought CQI could be used in that focus area. Participation in the formative evaluation exercise was voluntary and the evaluation was administered in a confidential manner that would not affect the students’ course grade. An additional assessment for the CQI component of the course included a total of 10 true-false or multiple-choice questions on the midterm examination (3 questions) and final examination (7 questions). These questions assessed knowledge of the FOCUS-PDCA methodology and steps in the CQI development process. A concept map was created by each group to demonstrate understanding of the links between CQI, pharmacists,
82
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
Table 1 Group scores for CQI presentations for each FOCUS-PDCA element Presentation elements—FOCUS-PDCA (possible score) Groups (N ⫽ 16)
F (2.0)
O (2.0)
C (2.0)
U (2.0)
S (2.0)
P (2.0)
D (3.0)
C (3.0)
A (2.0)
Verbal (5.0)
Map (5.0)
Total (30.0)
Range Mean SD
1.7–2.0 2.0 0.1
1.0–2.0 1.8 0.4
1.5–2.0 1.8 0.2
2.0 2.0 0.0
1.0–2.0 1.9 0.3
1.0–2.0 1.9 0.3
2.3–3.0 2.6 0.30
2.7–3.0 2.8 0.3
1.7–2.0 1.9 0.3
4.0–5.0 4.8 0.3
4.0–5.0 4.6 0.5
25.7–29.7 27.9 1.3
CQI, continuous quality improvement; FOCUS-PDCA, find, organize, clarify, uncover, start–plan, do, check, act.
and patient care. A concept map is an evaluation technique that can help to track the thinking of students on a concept or issue and help to develop thinking skills.20 –22 Students were instructed that there are many ways to construct this map, and they were encouraged to record free-flowing ideas relating CQI to the role of the pharmacist, with a focus on improving patient care. Results Practicum scoring The mean score of practicum presentations for the 16 graded groups was 27.9 ⫹ 1.3 SD points of 30 possible points (93%). Scores for each group in the individual elements are shown in Table 1. Students scored highest in the areas of defining interdisciplinary involvement, identifying the evidence base for their problem or action, and mapping out their implementation timetables. Identifying customers and their expectations, defining data sources, and describing implementation methods, FOCUS-PDCA elements “O,” “C,” and “D”, respectively, were areas that showed room for improvement. Students voted during class on their favorite peer presentation, and an in-class award was given to the “Student’s Choice” for the top two presentations. The CQI panel also selected their first- and second-place choices overall. Of interest, in 2006 the class’s selection for favorite presentation and runner-up were the same two presentations selected by the panel, although the first- and second-place order was reversed. Formative evaluation A total of 76 students of 100 taking the course completed a formative evaluation, realizing a 76% response rate. The lectures were considered informative or necessary by 61 of 76 (80%) students. Fewer students responded to the question regarding the value of the practicum, with 60 of 70 (85.7%) students reporting that the practicum added value to CQI learning. All responding students (72/72) reported learning more through the practicum experience compared with the lecture. In response to the question regarding the most difficult part of the practicum, students reported that the timing of the practicum close to the spring break in the 2006 term made it difficult for some groups to get together. Beyond the timing issue, the most difficult part of the practicum project
was found in the initial planning stages: getting started (picking a topic), doing research, and choosing measurement indicators. Of the six student groups given the opportunity to select a nonpharmacy topic for the practicum, four of the six groups elected to identify and develop nonpharmacy topics. Topics that students presented in 2006 are found in Appendix 3, along with nonpharmacy topics presented in this and previous years. The open-ended “Other Comments” item at the end of the evaluation yielded comments related primarily to the use of non– health care topics. Twenty students indicated that these topics provided a fun way to learn about CQI. Three students specifically commented that they enjoyed hearing about how CQI applies to real life and indicated that this experience changed their views on pharmacy’s role in health care today. Seven students offered suggestions to improve the class, such as including health care topic ideas from experiential learning/internship preceptors, allowing more class time for questions and group work, and more inclusion of community-based quality measures, especially because a large number of students reported that they will be pursuing careers in community pharmacy. Even after the group presentations, seven students did not believe that they needed to learn about CQI. Negative comments included the opinion that CQI was for administrators and would not be encountered in their practice. However, 91% (69/76) of students’ comments reflected an interest in CQI and recognized its potential for fostering improvements. Nearly all of the students (97%) were able to provide examples of applications for CQI in their area of interest. Students pursuing careers in community practice reported examples such as improving waiting time for prescription refills, procuring medications for the underserved, and implementing medication therapy management programs. Students interested in institutional practice suggested decreasing medication delivery times, designing and implementing evidence-based protocols, and creating discharge education programs as examples of how CQI would fit into their practice sites. Scores from examination questions The midterm questions were administered after the lectures but before the practicum assignment was completed. For the 100 students taking the examination, the mean score
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
on the three CQI questions was 83%. The cumulative final examination, administered after the practicum exercise was completed, produced a mean score of 97.4% on the seven CQI questions. Concept map A total of 49 of 68 (67.6%) students felt that the concept map did not add value. They reported greater learning through the group practicum assignment versus the group concept map and thought that the concept map exercise itself was more useful in their earlier years of pharmacy school. Discussion The approach described here combines traditional classroom teaching with a student-directed health care management problem-solving exercise. This type of practical application promotes reinforcement of CQI principles while familiarizing students with CQI tools and methods. The usefulness of the project as an active learning technique is consistent with findings when this approach was used in pharmacy13 as well as other topics.23,24 In the formative evaluation, students indicated that they learned more through applying the CQI process to their chosen topic in the group project as opposed to the lectures. Students also reported that they benefited from the presentations of the other groups’ work. This voluntary reporting revealed students’ perception that the project enhanced their learning. Identifying a topic was reported as one of the more difficult aspects of the practicum. Providing a list of possible pharmacy topics and professional organization references helped the student groups in selecting a topic for their project. Students were permitted to use independently identified topics, if approved in advance by the instructor. To demonstrate the applicability of the FOCUS-PDCA model to nonpharmacy (“life” topics), a small number of groups (6/16) were given the chance to select a non– health care–related topic. Students were required to provide articles from newspapers, magazines, websites, student newspapers, university publications, and so on, to justify the selected topic. The use of nonpharmacy topics demonstrated the usefulness of CQI principles in identifying and solving problems in all walks of life. This broad application of CQI to a varied list of problems stimulated interest and enabled students to see the value of the process more clearly. Students reflected that they liked seeing how CQI applies to real life, and they learned about the application process through the practicum, with both health care and non– health care topics. The voluntary formative evaluation yielded a 76% response rate, although students did not reply to every question. The instructor reviewed the findings of the evaluation with the students in the next class, and the class then made suggestions for improvement for the upcoming year. One item on the evaluation challenged students to identify processes that could be investigated and improved through the CQI process in practice settings, where they may choose to
83
work after graduation. The students’ ability to generate these ideas for improvement demonstrated their understanding of applying CQI methods outside of the classroom and into their future (and current) practice sites. Although not the primary assessment measure, the final examination scores demonstrated increased knowledge gained by students after the practicum. This improvement suggests the value of the practicum in reinforcing understanding of CQI methodology and tools. Although not specifically mentioned by students, it is likely that the interactions/suggestions from the expert panel of judges provided additional learning for the students beyond the information that was presented in the lectures. Areas that showed room for improvement on the basis of practicum scores, such as identifying data sources and stating mechanisms to sustain their improvement actions over time, were emphasized in the lectures when the course was taught the following spring. More information regarding where data can be gathered (i.e., billing records, surveys, literature, or medical records) was also provided in subsequent years. Verbal, written, and electronic dissemination strategies for implementation were discussed with the students to demonstrate the wide array of approaches that can be used to educate health care staff. The need for sustaining the improvements over time, as well as continual reflection on outcomes and new literature, was also emphasized during the lectures and the practicum presentations. The concept map was used as a one-time exercise to measure student CQI concept understanding. According to faculty perspective and student feedback from the formative evaluation, this abstract exercise had minimal additional impact on learning. The students reported that they learned much more from the project development and listening to real-life scenarios. Therefore, the practicum will be used as the ongoing assessment of student understanding of CQI methods. In this CQI module, student pharmacists demonstrated their understanding of how this methodology could apply to health care problems and, innovatively, to non– health care problems they may encounter. Health care professionals who understand and use proven CQI tools are positioned to guide their work group toward interdisciplinary approaches to improve and achieve quality care.25 Recognition of the importance of learning CQI has been demonstrated by incorporation of these concepts into curricula in schools of medicine and nursing, as well as in medical residency training.11,26 –30 Some of these programs are designed to include multidisciplinary group interaction in solving problems, which reinforces the advantages of this type of cooperative involvement.11,29,30 Inclusion of CQI principles into the pharmacy curriculum, reinforced by practical experience, creates student awareness of the need for ongoing improvement processes and provides the tools to accomplish this goal. Teaching CQI concepts in the pharmacy curriculum is not unique, but teaching students a specific performance improvement model and having them apply these steps to a current health
84
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
care problem or, more uniquely, to a “life problem” is not well described in the pharmacy or medical literature. Summary Presenting the CQI process through a lecture format provided a knowledge base for understanding CQI principles. Through the practicum, students learned how to use the CQI process to assess the status and discover possible mechanisms for improvement in a variety of pharmacy and nonpharmacy topics. This practical application of learning reinforced the students’ understanding of CQI principles
and gave them an increased understanding of the value of the CQI process. It is advantageous to provide future pharmacists with tools to assist them in their professional careers. This structured learning opportunity equips students through a systematic approach to problem-solving using CQI tools for identifying and resolving system problems. Students learn how to apply concepts and tools, as well as appreciate the usefulness of this process. Training student pharmacists to understand the entire CQI process and how to apply it will enable them to effectively create system improvements in their future workplace.
Appendix 1: Scoring tool for CQI presentations Continuous Quality Improvement CQI Project Evaluation Form Group: ______________________________ Date: _______________ Presentation Title: _____________________________________________ CQI Element (FOCUS-PDCA Methodology15)
Point Score
Comments
FIND the process to improve (2 points) If used, state the problem and ASHP 2015 goal/objective or National Quality Foundation safe practice ● Is it focused and measurable? ORGANIZE a team that knows the process (2 points) ● Is team multidisciplinary? ● Identify guidance team members ● Identify project team members ● Identify customers and expectations CLARIFY current knowledge (2 points) ● Based on data—not opinion or guess ● Flowchart current process ● Literature/information provided to support project (2 sources) ● Other institution/site practices cited ● Identify at least 2 key quality outcome indicators (numerator, denominator, data source, and threshold included) UNDERSTAND causes of process variation (2 points) ● List of brainstormed problem causes provided SELECT the process to change (2 points) ● Causes of the problem ranked according to how much they affect the problem (e.g., 1 ⫽ high impact to 10 ⫽ low impact) ● Select (circle) root cause PLAN the improvement/change (2 points) ● State the intervention ● Show timetable for activities DO the improvement/change (3 points) ● State educational methods ● Who will do the education and how? (inservices, posters, etc.) ● State target audience for education CHECK the change (3 points) ● State results (not required since not expected to implement) ● Revise process flowchart with improvement steps ACT to hold the gains (2 points) ● State how the project will continue over time ● Define next action steps/future plans Verbal Presentation (5 points) Concept Map (5 points) OVERALL SCORE (30 POINTS) ●
ASHP, American Society of Health-System Pharmacists; CQI, Continuous Quality Improvement; FOCUS-PDCA, Find, Organize, Clarify, Understand, Select–Plan, Do, Check, Act.
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
Appendix 2: Student formative evaluation of teaching strategy Name two practice areas you are considering and how CQI could fit into these areas. The lectures were . . . The practicum was . . . The concept map was . . . I learned the most by . . . The most difficult part was . . . How can CQI be applied to your future area of pharmacy practice? Other comments?
Œ
85
Pharmacy information technology help for instructors in the classroom
Acknowledgments Sandra Kane-Gill, Pharm.D., is acknowledged for her review of this manuscript and for coordination of the Profession of Pharmacy 6 course content. Gary Stoehr, Pharm.D., is acknowledged for his assistance with the design of the formative evaluation for this project. Amy (Calabrese) Donihi, Pharm.D., is acknowledged for her help in presenting the CQI model example to the students as a model for a successful CQI project. References
Appendix 3: Student group topics presented for the practicum Healthcare Topics (obtained from provided sources) ● Patient education programs Œ Smoking cessation counseling programs Œ Discharge counseling programs Œ Medication reconciliation processes ● Applying technology to improve medication use Œ Computerized physician order entry Œ Barcoding ● Statin, beta-blocker, and aspirin use after acute myocardial infarction ● Discontinuation of prophylactic antibiotics post-surgery Œ Role of pharmacist for automatic stop-order programs ● Improving the pharmacist-patient relationship by education and counseling ● Improving hand hygiene in hospitals ● Improving end-of-life quality of care Œ Care plan and protocol development “Life Topics” (ideas generated from students) ● Yielding to pedestrians in crosswalks ● What to do when the bus passes you by Œ Options for public transportation and on-site shuttles ● Cleaning up a local “greasy spoon” restaurant in town ● Traffic congestion in the Pittsburgh tunnels ● Temperature control in the classroom and its effect on learning ● Improving student attendance by modifying block schedules ● Effective use of SOAP (Subjective-Objective-Assessment-Plan) notes ● Effective use of technology in the classroom Œ Procedures for posting course materials
1. Curtis JR, Cook DJ, Wall RJ, et al. Intensive care unit quality improvement: a “how-to” guide for the interdisciplinary team. Crit Care Med 2006;34:211– 8. 2. Culley CM, Bernardo JF, Gross PR, et al. Implementing a standardized safety procedure for continuous renal replacement therapy solutions. Am J Health Syst Pharm 2006;63:756 – 63. 3. Calabrese AT, Cholka K, Lenhart SE, et al. Pharmacist involvement in a multidisciplinary inpatient medication education program. Am J Health Syst Pharm 2003;60:1012– 8. 4. Dunwoody C, Skledar SJ, Freeman S. Changes in patientcontrolled analgesia following a meperidine overdose. Jt Comm J Qual Patient Saf 2006;32:528 –30. 5. McLaughlin CP, Kaluzny AD. Defining quality improvement. In: McLaughlin CP, Kaluzny AD, eds. Continuous Quality Improvement in: Health Care, 3rd ed. Sudbury, MA: Jones and Bartlett Publishers; 2006:3– 40. 6. Pharmacy Quality Alliance. Home page. Available at: http:// www.pqaalliance.org/. Accessed October 11, 2009. 7. Linder JC. Outcomes measurement: compliance tool or strategic initiative? Health Care Manage Rev 1991;16:21–33. 8. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines. Available at: http://www.acpeaccredit.org/standards/default.asp. Accessed October 10, 2009. 9. Klapp D. Quality assurance training for pharmacy students. Am J Health Sys Pharm 1980;37:922– 4. 10. Strand LM, Morley PC. Evolving health care systems: academic implications for teaching methodologies with emphasis on administration and practice. Am J Pharm Educ 1987;51:402– 6. 11. Knapp DE, Michocki RJ, Wiser TW. A flexible module for teaching quality assurance and drug prescribing review concepts. Am J Pharm Educ 1986;50:274 –7. 12. Horsburgh M, Merry A, Seddon M, et al. Educating for healthcare quality improvement in an interprofessional learning environment: a New Zealand initiative. J Interprof Care 2006;20: 555–7. 13. Jackson TL. Application of quality assurance principles: teaching medication error reduction skills in a “real world” environment. Am J Pharm Educ 2004;68(1):Article 17. 14. Varkey P, Reller MK, Resar RK. Basics of quality improvement in health care. Mayo Clin Proc 2007;82:735–9. 15. Medical Risk Management Associates, LLC. Employing tools to enhance patient safety, The FOCUS-PDCA methodology. Available at: http://www.isixsigma.com/dictionary/FOCUSPDCA-823.htm. Accessed November 19, 2009.
86
S.J. Skledar et al. / Currents in Pharmacy Teaching and Learning 1 (2009) 79 – 86
16. National Quality Forum. National priorities for healthcare quality measurements and reporting. Available at: http:// www.qualityforum.org/Settings Priorities/Addressing National Priorities.aspx. Accessed July 23, 2009. 17. Institute for Healthcare Improvement. Topics, Institute for Healthcare Improvement. Available at: http://www.ihi.org/ihi/ topics. Accessed October 11, 2009. 18. U.S. Department of Health and Human Services, Office of Disease Prevention and Health Service Promotion. Leading health indicators, Healthy People 2010. Available at: http:// health.gov/healthypeople. Accessed October 11, 2009. 19. American Society of Health System Pharmacists. 2015 Initiative. Available at: http://www.ashp.org/s_ashp/cat1c.asp? CID⫽218&DID⫽255. Accessed October 11, 2009. 20. Novak JD. The promise of new ideas and technology for improving teaching and learning. Cell Biol Educ 2003;2:122–32. 21. West DC, Pomeroy JR, Park JK, et al. Critical thinking in graduate medical education: a role for the concept mapping assessment? JAMA 2000;284:1105–10. 22. Hill LH. Concept mapping in a pharmacy communications course to encourage meaningful student learning. Am J Pharm Educ 2004;68(5):Article 109.
23. McCarthy JP, Anderson L. Active learning techniques versus traditional teaching styles: two experiments form history and political science. Innovative Higher Education 2000;24:279 – 94. 24. Prince M. Does active learning work? A review of the research. J Eng Educ 2004;93:1–9. 25. Burhans LD. What is quality: do we agree, and does it matter? J Healthc Qual 2007;29:39 – 44, 54. 26. Teeley KH, Lowe JM, Beal J, et al. Incorporating quality improvement concepts and practice into a community health nursing course. J Nurs Educ 2006;45:86 –90. 27. Krajewski K, Siewert B, Yam S, et al. A quality assurance elective for radiology residents. Acad Radiol 2007;14:239 – 45. 28. Weingart SN, Tess A, Driver J, et al. Creating a quality improvement elective for medical house officers. J Gen Intern Med 2004;19:861–7. 29. Varkey P, Reller MK, Smith A, et al. An experiential quality improvement education initiative. Am J Med Qual 2006;21: 317–22. 30. Walrath JM, Muganlinskaya N, Shepherd M, et al. Interdisciplinary medical, nursing, and administrator education in practice: the Johns Hopkins experience. Acad Med 2006;81:744 – 8.