- Email: [email protected]
Using a multisite and multi–health system pharmacy resident program model for care documentation quality improvement
SCIENCE AND PRACTICE Journal of the American Pharmacists Association 59 (2019) 862e866
Contents lists available at ScienceDirect
Journal of the American Pharmacists Association journal homepage: www.japha.org
ADVANCES IN PHARMACY PRACTICE
Using a multisite and multiehealth system pharmacy resident program model for care documentation quality improvement Sarah Schweiss*, Amy Pavelka, Oscar W. Garza, Jean Y. Moon a r t i c l e i n f o
a b s t r a c t
Article history: Received 20 December 2018 Accepted 20 July 2019 Available online 26 August 2019
Objectives: To implement and evaluate a pharmacy resident documentation peer review process. Setting: The University of Minnesota Postgraduate Year One Pharmacy Residency Program is a multisite program with 25 residents across 16 different health care organizations. Practice description: Sites within the program provide comprehensive medication management (CMM) services to patients in ambulatory care settings, including participation in the full patient care process of assessment, care plan development, follow-up, and appropriate documentation. Practice innovation: In this innovative peer review process model, residents undergo a deidentified CMM documentation review process with residents from other practice sites, exposing them to different documentation templates and perspectives. Evaluation: A workgroup of residency preceptors led by a research team developed a peer review process, which evolved through 3 phases over 2 years in response to resident, preceptor, and administration team feedback. Resident feedback was compiled and analyzed. Results: Forty-two residents responded to the survey (67% response rate); 71% found the review process to be helpful. Residents reported that the process improved their understanding of how to improve patient care documentation (74%), how to provide peer feedback (90%), and the importance of effective interprofessional communication in clinical decision making (81%). Discussion: The core perceived benefit of the peer review process was exposure to how other health systems and practitioners document CMM. Some residents participate in a peer review process at their home institutions, which may explain some of the lack of perceived benefit. Generalizability of this study is limited by being within a single residency program with a relatively small number of participants. Conclusion: Pharmacy residents found a peer review process of documentation to be helpful during their residency education. The process exposed residents to different documentation practices at various health care systems, which led to ideas of how to improve documentation and provided a foundation for how to provide peer feedback in practice. © 2019 Published by Elsevier Inc. on behalf of the American Pharmacists Association.
Objective
Setting
The primary purpose of this study was to implement and evaluate a pharmacy resident documentation peer review process in a multisite and multiehealth system program.
The University of Minnesota Postgraduate Year One (PGY-1) Pharmacy Residency Program is a multisite multiehealth system program centered on the practice of pharmaceutical care. Residents spend more than 50% of their time in ambulatory care settings and most learning experiences in the program are evaluated on a longitudinal basis. The program currently enrolls 25 PGY-1 residents, 21 in a traditional 1-year program and 4 in a 2-year extended experience (leadership emphasis), across 19 sites and 16 health care organizations in Minnesota.1 Over the course of this project, the residency class size varied from 23 to 26 residents.
Disclosure: The authors declare no relevant conflicts of interest or financial relationships. Previous presentation: American Pharmacists Association Annual Meeting, March 2015, San Diego, CA. * Correspondence: Sarah Schweiss, PharmD, Assistant Professor, Assistant Director, University of Minnesota College of Pharmacy, 223 Life Science, 1110 Kirby Drive, Duluth, MN 55812. E-mail address: [email protected] (S. Schweiss).
https://doi.org/10.1016/j.japh.2019.07.013 1544-3191/© 2019 Published by Elsevier Inc. on behalf of the American Pharmacists Association.
SCIENCE AND PRACTICE Pharmacy resident peer review process
Key Points Background: Peer review processes of documentation among clinical pharmacists have been developed, implemented, and described in the literature. These processes have been found to improve documentation consistency and quality, improve compliance with documentation standards, and allow for the sharing of “best practices” in documentation among clinical pharmacists. Literature regarding the value of peer review processes in pharmacy residency programs is lacking. Findings: Pharmacy residents found a deidentified peer review process of comprehensive medication management (CMM) care documentation to be helpful during their residency education. The process exposed residents to different CMM documentation practices at various health care institutions, which led to ideas of how to improve their clinic documentation and gave them a foundation for how to provide peer feedback. The description of this innovative peer review process adds to the literature by presenting a unique way to perform a peer review process in a multisite and multiehealth system pharmacy residency training program.
documentation standards, and allow for the sharing of “best practices” in documentation among clinical pharmacists. However, literature on peer review processes in pharmacy residency training programs is lacking. Implementation of these processes can foster further resident learning, reflection, and documentation refinement for the new practitioner. Furthermore, processes across multiple health care institutions can diversify and broaden the depth of the resident’s learning. The unique structure of our university-affiliated multisite residency program provides many robust learning opportunities for resident trainees.1 In this innovative peer review process model, residents have the opportunity to obtain feedback from fellow residents at other practice sites and institutions, who are able to offer different documentation templates and perspectives. Although not all residency programs have the same multisite model, relationships could be built across programs to mirror similar benefits.
Evaluation Phase I
Sites within the University of Minnesota PGY-1 residency program share a common commitment to providing CMM services2 to patients. Although each site offers diversity in geography, practice infrastructure, and population served, each resident is taught to provide high-quality patient care through CMM. This includes participation in the full patient care process of assessment, care plan development, follow-up evaluation, and appropriate documentation of care. Documentation of care differs from site to site within our program, owing to the number of health care organizations involved and varying electronic medical records, but documentation at each site generally follows the SOAP (Subjective, Objective, Assessment, Plan) note format. Pharmacists in our sites consistently work within interprofessional teams, utilize collaborative practice agreements, and bill for services. The consistency of the ambulatory careefocused learning experience is core to the multisite program design.
A workgroup of residency preceptors led by a research team (2 assistant program directors, 1 residency program coordinator, and 1 college of pharmacy faculty) developed a peer review process with assessment criteria. Participation in the peer review process was voluntary for residents, because about 25% of the residents already participated in peer review processes at their clinic site. In the first cohort, pharmacy residents randomly selected SOAP note documentation of CMM encounters to submit quarterly for evaluation. An assessment tool was adapted from multiple sources (e.g., local health systems, medication therapy management provider network, affiliated school rubrics) and used to evaluate the accuracy, consistency and quality of the documentation. Each documentation element (Subjective, Objective, Assessment, Plan) was scored on a 5-point scale (1 ¼ unacceptable; 2 ¼ needs improvement; 3 ¼ satisfactory; 4 ¼ very good; and 5 ¼ exemplary) and allowed for text box feedback. In addition, the overall structure and organization, grammar, and proper prioritization of drug therapy needs were evaluated with the use of the same scale in addition to a text box for feedback. The patient’s primary care provider, a pharmacist preceptor, and a peer pharmacy resident evaluated the same deidentified note. Both the pharmacist preceptor and peer resident were outside of the resident’s training site. Summary statistics were calculated to identify areas for improvement, and comments and feedback were analyzed for emergent themes. Residents were asked to submit a satisfaction survey (Appendix 1) to the program coordinator after each quarterly review.
Practice innovation
Phase II
Various peer review processes of documentation among clinical pharmacists have been developed within individual health systems, implemented, and described in the literature.3-5 These processes have been found to improve documentation consistency and quality, improve compliance with
Feedback from phase I suggested that the process should be reduced to semiannual in order to allow for a more thorough and thoughtful review (quality over quantity). With the exception of this change, the process remained the same for the second cohort.
Practice description
863
SCIENCE AND PRACTICE S. Schweiss et al. / Journal of the American Pharmacists Association 59 (2019) 862e866
Phase III Feedback from phase II indicated that residents had a desire to elicit feedback and clinical considerations on difficult patient cases. They also expressed a desire to provide a quality example of their documentation to further improve their perceived “best” documentation practices. As a result, the process was altered to allow residents to self-select notes used for review. Furthermore, although both the primary care providers and pharmacy preceptors provided valuable feedback, rating the quality of the residents’ documentation with the use of the rubric was less helpful, because most residents scored quite well overall. One of the most valuable themes reported by the residents was the opportunity to compare and contrast documentation across various sites. In addition, the research team noted that primary care provider feedback was often extremely positive, communicating support of pharmacy services, with little constructive feedback on the documentation itself. Program administration and pharmacy preceptors also found the complex process involving many reviewing parties to be tedious to manage, specifically in receiving and coordinating all the feedback in time. As a result, primary care provider and pharmacy preceptor note review and feedback were eliminated. Finally, based on the positive experiences of residents, the process moved from a voluntary process to a residency program requirement. Data analysis Resident survey feedback from the 3 phases of the project was compiled in a Microsoft Excel document by an undergraduate student employee. Percentages and averages for various questions were computed by 2 investigators independently to ensure accuracy. Percentages of respondents who answered “yes” and “no” for a given question were computed. The average amount of time it took for residents to review patient notes was calculated. Responses to all open-ended questions were analyzed by the student employee and lead investigator, with the goal to identify themes across responses. The student employee first identified themes in the qualitative responses throughout data entry. The lead investigator then reviewed the data and confirmed identified themes. This study was reviewed by the University of Minnesota Institutional Review Board (study #00005212) and was deemed to be exempt from full review, because it was determined to not be human research. Results Three residency cohorts were surveyed regarding their perceptions of the peer review process, with a total of 42 responses and a 67% response rate. Overall the residents reported that the process improved their understanding of how to improve patient care documentation (74%), how to provide peer feedback (90%), and the importance of effective interprofessional communication in clinical decision making (81%). The majority of the residents found the review process to be helpful (71%), with 29% reporting that they did not find the process helpful. Residents reported spending 15 to 60 minutes 864
reviewing each resident note submitted, with an average note review time of 23 minutes. A number of themes emerged in residents’ self-reflection responses (Table 1). The residents consistently identified a need to provide enough details in the plan and follow-up while being concise at the same time. Residents also commented that a separate summary of visit recommendations for the provider to easily access, whether at the beginning or at the end of their note (versus interspersed in the assessment/ plan section), would be helpful. Furthermore, a theme to document the communicated plan to both provider and patient was found. Finally, the residents commonly identified a need to reorganize their notes overall after undergoing the peer review process, including the need to create a standardized documentation form for clinical pharmacist use across all health care settings. Residents consistently recommended a number of changes regarding the peer review process. They recommended completing the process more thoroughly once or twice per year instead of quarterly. Residents noted that comments and written feedback were more beneficial than Likert-type scale ratings on each section. During the live debriefing sessions, residents confirmed this notion again, emphasizing that written feedback was most helpful, and that a core benefit of participating in this process was seeing how other health care institutions and practitioners document. Practice implications As pharmacists increasingly move into direct patient care roles, it is valuable to participate in a peer review process to uphold quality documentation practices. Numerous peer review processes have been described in the literature3-5 and are performed widely at various health care institutions among practitioners who provide direct patient care. Implementing a peer review process during residency training, specifically across multiple health systems, allows the resident to see examples of varying documentation practices and improve clinical decision making documentation. Furthermore, participation in this process provides valuable skills to be used in future employment. Discussion Feedback regarding the peer review process contributed to continuous evolution of the process overall. When originally implemented, residents’ peer review numeric scores were tracked over time to measure if the process was helpful in improving their documentation practices. Although the peer review process successfully improved pharmacy residents’ documentation practice scores, feedback clearly indicated that exposure to how other institutions and practitioners document was the greatest benefit perceived by the residents. They were able to incorporate new ideas and adapt their personal documentation practices as a result of this innovative peer review process. The frequency of peer review was also modified over time. Quarterly, and even semiannually, was reported to be too often and as a result the process eventually moved to annually. For residents with an internal peer review process at their home institution, this may still amount to 3 or more reviews within
SCIENCE AND PRACTICE Pharmacy resident peer review process
Table 1 Emergent themes in resident survey question responses Survey question Are there any parts of the note (e.g., style, template, structure) that you would suggest changing?
How do you think documentation and communication of patient care decision making can be improved in your practice setting?
After this review process, in your opinion, what do you anticipate will be the most significant change that will take place in your patient care documentation practice?
Answer themes (1) Provide more detailed and clear plans while being concise overall (2) Communicate summary of patient visit and recommendations in person with the patient’s provider (3) Document patient understanding and agreement to the care plan (4) Provide a clear follow-up plan (5) Create a summary of recommendations (6) Use a standardized documentation form (1) Provide more details in the plan
(2) (3) (4) (1)
Find a balance between providing enough details while being concise Create a clear follow-up plan Reorganization of note overall Provide more details in the plan
(2) Find a balance between providing enough details while being concise (3) Create a clear follow-up plan (4) Reorganization of note overall
the academic year, although others may experience only the residency program’s peer review process. However, as each resident reviews 2 peer notes and received 2 evaluations on their own notes, we felt that this provided some additional depth to the review. Although rare, some residents reported that they did not find the process helpful. We occasionally received comments indicating that they already participate in a peer review process at their home institution, which perhaps explains this response. Despite this, program administrators still feel there is an advantage to exposing residents to different types of documentation practices at clinics other than their individual training site. The debriefing sessions were led by a residency program director who used a set of open-ended questions to guide the conversation (Appendix 2). A wide variety of topics were discussed during this time, including legal implications, professional liability, preferences versus requirements, professional lenses, benefits and disadvantages of templates, how to provide and receive constructive feedback, and precepting (e.g., future precepting of student notes). These topics may provide some insight into gaps of the current residency training experience. The results of this study are limited by being within a single residency program with a relatively small number of participants. Further delineation between residents already engaged in a peer review process compared with those without may yield interesting results. Because many changes were implemented over time, it may be difficult to extrapolate the findings to our current process. In addition, the assessment tool used in the study was not validated and was adapted from various publicly available and locally sourced templates from partner health system organizations. The process was first implemented in our 2013e2014 residency cohort, before the Joint Commission of Pharmacy Practitioners’ release of the Pharmacists’ Patient Care Process (PPCP) in May 2014.6 It would be beneficial in the future to use a validated assessment tool that more closely aligns with the PPCP.
Feedback from the 3 cohorts of residents was incorporated to create the final process for our residency program, which is as follows. Residents each submit 2 deidentified self-selected patient care notes to the program coordinator annually from a predefined academic week and review 2 peer resident notes with the use of a scored rubric evaluating required elements of a patient care note and open-ended questions. The resident receives feedback via rubric from 2 peer residents. During a regularly scheduled monthly resident meeting time, residents meet in small groups (12 to 13 people) with a program director facilitator for a reflective group discussion over 45 minutes. Residents are asked to submit a satisfaction survey to the program coordinator to reflect and provide feedback once per year.
Conclusion Overall, multisite pharmacy residents found participation in a deidentified peer review process to be helpful during residency training. Specifically, the peer review process exposed residents to different documentation practices at various health care institutions, led to ideas of how to improve documentation, and provided a foundation for how to provide peer feedback in the future.
References 1. Schweiss SK, Westberg SM, Moon JY, Sorenson TD. Expanding ambulatory care pharmacy residency education through a multisite universityaffiliated model. J Pharm Pract. 2017;30(6):643e649. 2. Nace DK, Grundy P, Nielsen M, eds. The patient centered medical home: integrating comprehensive medication management to optimize patient outcomes. 2nd ed. Washington DC: Patient-Centered Primary Care Collaborative; June 3e28, 2012. Available at: https://www.pcpcc.org/sites/ default/files/media/medmanagement.pdf. Accessed June 6, 2018. 3. Milchak JL, Shanahan RL, Kerzee JA. Implementation of a peer review process to improve documentation consistency of care process indicators in the EMR in a primary care setting. J Manag Care Pharm. 2012;18(1): 46e53.
865
SCIENCE AND PRACTICE S. Schweiss et al. / Journal of the American Pharmacists Association 59 (2019) 862e866
4. Zimmerman CR, Smolarek RT, Stevenson JG. Peer review and continuous quality improvement of pharmacists’ clinical interventions. Am J Health Syst Pharm. 1997;54(15):1722e1727. 5. Cram Jr DL, Stebbins M, Eom HS, Ratto N, Sugiyama D. Peer review as a quality assurance mechanism in three pharmacist-run medicationrefill clinics. Am J Hosp Pharm. 1992;49(11):2727e2730. 6. Joint Commission of Pharmacy Practitioners. Pharmacists’ Patient Care Process; 2014. Available at: https://jcpp.net/wp-content/ uploads/2016/03/PatientCareProcess-with-supporting-organizations. pdf. Accessed August 13, 2019.
866
Sarah Schweiss, PharmD, Assistant Professor, Assistant Director, University of Minnesota College of Pharmacy, Duluth, MN Amy Pavelka, MA, Program Coordinator, University of Minnesota College of Pharmacy, Minneapolis, MN Oscar W. Garza, PhD, MBA, Assistant Professor, University of Minnesota College of Pharmacy, Minneapolis, MN Jean Y. Moon, PharmD, Associate Professor, Director, University of Minnesota College of Pharmacy, Minneapolis, MN
SCIENCE AND PRACTICE Pharmacy resident peer review process
Appendix 1. Pharmacy resident peer review: Resident satisfaction survey
Questions 1. Did the review process improve your understanding regarding: a. How to improve patient care documentation? b. How to provide peer feedback? c. The importance of effective interprofessional communication in clinical decision making? 2. Was the review process helpful? 3. On average, how long did it take to review patient notes provided by each resident?
Yes
No
How do you think documentation and communication of patient care decision making can be improved in your practice setting? After this review process, in your opinion, what do you anticipate will be the most significant change that will take place in your patient care documentation practice? General comments/suggestions: Appendix 2 Documentation peer review debrief session
___ minutes
What part of the note did you find most helpful to you for the care of the patient? Why? Are there any parts of the note (e.g., style, template, structure) that you would suggest changing? Why?
1. What did you think of the peer review process overall? How do you feel it went? 2. What are the benefits of completing a peer review process? 3. What did you learn by reviewing others’ documentation? Was there anything you were able to take away from this experience and apply to your clinic site? 4. What did you learn from the feedback you received on your documentation? Have you changed anything in your documentation practice as a result?
866.e1
Contents lists available at ScienceDirect
Journal of the American Pharmacists Association journal homepage: www.japha.org
ADVANCES IN PHARMACY PRACTICE
Using a multisite and multiehealth system pharmacy resident program model for care documentation quality improvement Sarah Schweiss*, Amy Pavelka, Oscar W. Garza, Jean Y. Moon a r t i c l e i n f o
a b s t r a c t
Article history: Received 20 December 2018 Accepted 20 July 2019 Available online 26 August 2019
Objectives: To implement and evaluate a pharmacy resident documentation peer review process. Setting: The University of Minnesota Postgraduate Year One Pharmacy Residency Program is a multisite program with 25 residents across 16 different health care organizations. Practice description: Sites within the program provide comprehensive medication management (CMM) services to patients in ambulatory care settings, including participation in the full patient care process of assessment, care plan development, follow-up, and appropriate documentation. Practice innovation: In this innovative peer review process model, residents undergo a deidentified CMM documentation review process with residents from other practice sites, exposing them to different documentation templates and perspectives. Evaluation: A workgroup of residency preceptors led by a research team developed a peer review process, which evolved through 3 phases over 2 years in response to resident, preceptor, and administration team feedback. Resident feedback was compiled and analyzed. Results: Forty-two residents responded to the survey (67% response rate); 71% found the review process to be helpful. Residents reported that the process improved their understanding of how to improve patient care documentation (74%), how to provide peer feedback (90%), and the importance of effective interprofessional communication in clinical decision making (81%). Discussion: The core perceived benefit of the peer review process was exposure to how other health systems and practitioners document CMM. Some residents participate in a peer review process at their home institutions, which may explain some of the lack of perceived benefit. Generalizability of this study is limited by being within a single residency program with a relatively small number of participants. Conclusion: Pharmacy residents found a peer review process of documentation to be helpful during their residency education. The process exposed residents to different documentation practices at various health care systems, which led to ideas of how to improve documentation and provided a foundation for how to provide peer feedback in practice. © 2019 Published by Elsevier Inc. on behalf of the American Pharmacists Association.
Objective
Setting
The primary purpose of this study was to implement and evaluate a pharmacy resident documentation peer review process in a multisite and multiehealth system program.
The University of Minnesota Postgraduate Year One (PGY-1) Pharmacy Residency Program is a multisite multiehealth system program centered on the practice of pharmaceutical care. Residents spend more than 50% of their time in ambulatory care settings and most learning experiences in the program are evaluated on a longitudinal basis. The program currently enrolls 25 PGY-1 residents, 21 in a traditional 1-year program and 4 in a 2-year extended experience (leadership emphasis), across 19 sites and 16 health care organizations in Minnesota.1 Over the course of this project, the residency class size varied from 23 to 26 residents.
Disclosure: The authors declare no relevant conflicts of interest or financial relationships. Previous presentation: American Pharmacists Association Annual Meeting, March 2015, San Diego, CA. * Correspondence: Sarah Schweiss, PharmD, Assistant Professor, Assistant Director, University of Minnesota College of Pharmacy, 223 Life Science, 1110 Kirby Drive, Duluth, MN 55812. E-mail address: [email protected] (S. Schweiss).
https://doi.org/10.1016/j.japh.2019.07.013 1544-3191/© 2019 Published by Elsevier Inc. on behalf of the American Pharmacists Association.
SCIENCE AND PRACTICE Pharmacy resident peer review process
Key Points Background: Peer review processes of documentation among clinical pharmacists have been developed, implemented, and described in the literature. These processes have been found to improve documentation consistency and quality, improve compliance with documentation standards, and allow for the sharing of “best practices” in documentation among clinical pharmacists. Literature regarding the value of peer review processes in pharmacy residency programs is lacking. Findings: Pharmacy residents found a deidentified peer review process of comprehensive medication management (CMM) care documentation to be helpful during their residency education. The process exposed residents to different CMM documentation practices at various health care institutions, which led to ideas of how to improve their clinic documentation and gave them a foundation for how to provide peer feedback. The description of this innovative peer review process adds to the literature by presenting a unique way to perform a peer review process in a multisite and multiehealth system pharmacy residency training program.
documentation standards, and allow for the sharing of “best practices” in documentation among clinical pharmacists. However, literature on peer review processes in pharmacy residency training programs is lacking. Implementation of these processes can foster further resident learning, reflection, and documentation refinement for the new practitioner. Furthermore, processes across multiple health care institutions can diversify and broaden the depth of the resident’s learning. The unique structure of our university-affiliated multisite residency program provides many robust learning opportunities for resident trainees.1 In this innovative peer review process model, residents have the opportunity to obtain feedback from fellow residents at other practice sites and institutions, who are able to offer different documentation templates and perspectives. Although not all residency programs have the same multisite model, relationships could be built across programs to mirror similar benefits.
Evaluation Phase I
Sites within the University of Minnesota PGY-1 residency program share a common commitment to providing CMM services2 to patients. Although each site offers diversity in geography, practice infrastructure, and population served, each resident is taught to provide high-quality patient care through CMM. This includes participation in the full patient care process of assessment, care plan development, follow-up evaluation, and appropriate documentation of care. Documentation of care differs from site to site within our program, owing to the number of health care organizations involved and varying electronic medical records, but documentation at each site generally follows the SOAP (Subjective, Objective, Assessment, Plan) note format. Pharmacists in our sites consistently work within interprofessional teams, utilize collaborative practice agreements, and bill for services. The consistency of the ambulatory careefocused learning experience is core to the multisite program design.
A workgroup of residency preceptors led by a research team (2 assistant program directors, 1 residency program coordinator, and 1 college of pharmacy faculty) developed a peer review process with assessment criteria. Participation in the peer review process was voluntary for residents, because about 25% of the residents already participated in peer review processes at their clinic site. In the first cohort, pharmacy residents randomly selected SOAP note documentation of CMM encounters to submit quarterly for evaluation. An assessment tool was adapted from multiple sources (e.g., local health systems, medication therapy management provider network, affiliated school rubrics) and used to evaluate the accuracy, consistency and quality of the documentation. Each documentation element (Subjective, Objective, Assessment, Plan) was scored on a 5-point scale (1 ¼ unacceptable; 2 ¼ needs improvement; 3 ¼ satisfactory; 4 ¼ very good; and 5 ¼ exemplary) and allowed for text box feedback. In addition, the overall structure and organization, grammar, and proper prioritization of drug therapy needs were evaluated with the use of the same scale in addition to a text box for feedback. The patient’s primary care provider, a pharmacist preceptor, and a peer pharmacy resident evaluated the same deidentified note. Both the pharmacist preceptor and peer resident were outside of the resident’s training site. Summary statistics were calculated to identify areas for improvement, and comments and feedback were analyzed for emergent themes. Residents were asked to submit a satisfaction survey (Appendix 1) to the program coordinator after each quarterly review.
Practice innovation
Phase II
Various peer review processes of documentation among clinical pharmacists have been developed within individual health systems, implemented, and described in the literature.3-5 These processes have been found to improve documentation consistency and quality, improve compliance with
Feedback from phase I suggested that the process should be reduced to semiannual in order to allow for a more thorough and thoughtful review (quality over quantity). With the exception of this change, the process remained the same for the second cohort.
Practice description
863
SCIENCE AND PRACTICE S. Schweiss et al. / Journal of the American Pharmacists Association 59 (2019) 862e866
Phase III Feedback from phase II indicated that residents had a desire to elicit feedback and clinical considerations on difficult patient cases. They also expressed a desire to provide a quality example of their documentation to further improve their perceived “best” documentation practices. As a result, the process was altered to allow residents to self-select notes used for review. Furthermore, although both the primary care providers and pharmacy preceptors provided valuable feedback, rating the quality of the residents’ documentation with the use of the rubric was less helpful, because most residents scored quite well overall. One of the most valuable themes reported by the residents was the opportunity to compare and contrast documentation across various sites. In addition, the research team noted that primary care provider feedback was often extremely positive, communicating support of pharmacy services, with little constructive feedback on the documentation itself. Program administration and pharmacy preceptors also found the complex process involving many reviewing parties to be tedious to manage, specifically in receiving and coordinating all the feedback in time. As a result, primary care provider and pharmacy preceptor note review and feedback were eliminated. Finally, based on the positive experiences of residents, the process moved from a voluntary process to a residency program requirement. Data analysis Resident survey feedback from the 3 phases of the project was compiled in a Microsoft Excel document by an undergraduate student employee. Percentages and averages for various questions were computed by 2 investigators independently to ensure accuracy. Percentages of respondents who answered “yes” and “no” for a given question were computed. The average amount of time it took for residents to review patient notes was calculated. Responses to all open-ended questions were analyzed by the student employee and lead investigator, with the goal to identify themes across responses. The student employee first identified themes in the qualitative responses throughout data entry. The lead investigator then reviewed the data and confirmed identified themes. This study was reviewed by the University of Minnesota Institutional Review Board (study #00005212) and was deemed to be exempt from full review, because it was determined to not be human research. Results Three residency cohorts were surveyed regarding their perceptions of the peer review process, with a total of 42 responses and a 67% response rate. Overall the residents reported that the process improved their understanding of how to improve patient care documentation (74%), how to provide peer feedback (90%), and the importance of effective interprofessional communication in clinical decision making (81%). The majority of the residents found the review process to be helpful (71%), with 29% reporting that they did not find the process helpful. Residents reported spending 15 to 60 minutes 864
reviewing each resident note submitted, with an average note review time of 23 minutes. A number of themes emerged in residents’ self-reflection responses (Table 1). The residents consistently identified a need to provide enough details in the plan and follow-up while being concise at the same time. Residents also commented that a separate summary of visit recommendations for the provider to easily access, whether at the beginning or at the end of their note (versus interspersed in the assessment/ plan section), would be helpful. Furthermore, a theme to document the communicated plan to both provider and patient was found. Finally, the residents commonly identified a need to reorganize their notes overall after undergoing the peer review process, including the need to create a standardized documentation form for clinical pharmacist use across all health care settings. Residents consistently recommended a number of changes regarding the peer review process. They recommended completing the process more thoroughly once or twice per year instead of quarterly. Residents noted that comments and written feedback were more beneficial than Likert-type scale ratings on each section. During the live debriefing sessions, residents confirmed this notion again, emphasizing that written feedback was most helpful, and that a core benefit of participating in this process was seeing how other health care institutions and practitioners document. Practice implications As pharmacists increasingly move into direct patient care roles, it is valuable to participate in a peer review process to uphold quality documentation practices. Numerous peer review processes have been described in the literature3-5 and are performed widely at various health care institutions among practitioners who provide direct patient care. Implementing a peer review process during residency training, specifically across multiple health systems, allows the resident to see examples of varying documentation practices and improve clinical decision making documentation. Furthermore, participation in this process provides valuable skills to be used in future employment. Discussion Feedback regarding the peer review process contributed to continuous evolution of the process overall. When originally implemented, residents’ peer review numeric scores were tracked over time to measure if the process was helpful in improving their documentation practices. Although the peer review process successfully improved pharmacy residents’ documentation practice scores, feedback clearly indicated that exposure to how other institutions and practitioners document was the greatest benefit perceived by the residents. They were able to incorporate new ideas and adapt their personal documentation practices as a result of this innovative peer review process. The frequency of peer review was also modified over time. Quarterly, and even semiannually, was reported to be too often and as a result the process eventually moved to annually. For residents with an internal peer review process at their home institution, this may still amount to 3 or more reviews within
SCIENCE AND PRACTICE Pharmacy resident peer review process
Table 1 Emergent themes in resident survey question responses Survey question Are there any parts of the note (e.g., style, template, structure) that you would suggest changing?
How do you think documentation and communication of patient care decision making can be improved in your practice setting?
After this review process, in your opinion, what do you anticipate will be the most significant change that will take place in your patient care documentation practice?
Answer themes (1) Provide more detailed and clear plans while being concise overall (2) Communicate summary of patient visit and recommendations in person with the patient’s provider (3) Document patient understanding and agreement to the care plan (4) Provide a clear follow-up plan (5) Create a summary of recommendations (6) Use a standardized documentation form (1) Provide more details in the plan
(2) (3) (4) (1)
Find a balance between providing enough details while being concise Create a clear follow-up plan Reorganization of note overall Provide more details in the plan
(2) Find a balance between providing enough details while being concise (3) Create a clear follow-up plan (4) Reorganization of note overall
the academic year, although others may experience only the residency program’s peer review process. However, as each resident reviews 2 peer notes and received 2 evaluations on their own notes, we felt that this provided some additional depth to the review. Although rare, some residents reported that they did not find the process helpful. We occasionally received comments indicating that they already participate in a peer review process at their home institution, which perhaps explains this response. Despite this, program administrators still feel there is an advantage to exposing residents to different types of documentation practices at clinics other than their individual training site. The debriefing sessions were led by a residency program director who used a set of open-ended questions to guide the conversation (Appendix 2). A wide variety of topics were discussed during this time, including legal implications, professional liability, preferences versus requirements, professional lenses, benefits and disadvantages of templates, how to provide and receive constructive feedback, and precepting (e.g., future precepting of student notes). These topics may provide some insight into gaps of the current residency training experience. The results of this study are limited by being within a single residency program with a relatively small number of participants. Further delineation between residents already engaged in a peer review process compared with those without may yield interesting results. Because many changes were implemented over time, it may be difficult to extrapolate the findings to our current process. In addition, the assessment tool used in the study was not validated and was adapted from various publicly available and locally sourced templates from partner health system organizations. The process was first implemented in our 2013e2014 residency cohort, before the Joint Commission of Pharmacy Practitioners’ release of the Pharmacists’ Patient Care Process (PPCP) in May 2014.6 It would be beneficial in the future to use a validated assessment tool that more closely aligns with the PPCP.
Feedback from the 3 cohorts of residents was incorporated to create the final process for our residency program, which is as follows. Residents each submit 2 deidentified self-selected patient care notes to the program coordinator annually from a predefined academic week and review 2 peer resident notes with the use of a scored rubric evaluating required elements of a patient care note and open-ended questions. The resident receives feedback via rubric from 2 peer residents. During a regularly scheduled monthly resident meeting time, residents meet in small groups (12 to 13 people) with a program director facilitator for a reflective group discussion over 45 minutes. Residents are asked to submit a satisfaction survey to the program coordinator to reflect and provide feedback once per year.
Conclusion Overall, multisite pharmacy residents found participation in a deidentified peer review process to be helpful during residency training. Specifically, the peer review process exposed residents to different documentation practices at various health care institutions, led to ideas of how to improve documentation, and provided a foundation for how to provide peer feedback in the future.
References 1. Schweiss SK, Westberg SM, Moon JY, Sorenson TD. Expanding ambulatory care pharmacy residency education through a multisite universityaffiliated model. J Pharm Pract. 2017;30(6):643e649. 2. Nace DK, Grundy P, Nielsen M, eds. The patient centered medical home: integrating comprehensive medication management to optimize patient outcomes. 2nd ed. Washington DC: Patient-Centered Primary Care Collaborative; June 3e28, 2012. Available at: https://www.pcpcc.org/sites/ default/files/media/medmanagement.pdf. Accessed June 6, 2018. 3. Milchak JL, Shanahan RL, Kerzee JA. Implementation of a peer review process to improve documentation consistency of care process indicators in the EMR in a primary care setting. J Manag Care Pharm. 2012;18(1): 46e53.
865
SCIENCE AND PRACTICE S. Schweiss et al. / Journal of the American Pharmacists Association 59 (2019) 862e866
4. Zimmerman CR, Smolarek RT, Stevenson JG. Peer review and continuous quality improvement of pharmacists’ clinical interventions. Am J Health Syst Pharm. 1997;54(15):1722e1727. 5. Cram Jr DL, Stebbins M, Eom HS, Ratto N, Sugiyama D. Peer review as a quality assurance mechanism in three pharmacist-run medicationrefill clinics. Am J Hosp Pharm. 1992;49(11):2727e2730. 6. Joint Commission of Pharmacy Practitioners. Pharmacists’ Patient Care Process; 2014. Available at: https://jcpp.net/wp-content/ uploads/2016/03/PatientCareProcess-with-supporting-organizations. pdf. Accessed August 13, 2019.
866
Sarah Schweiss, PharmD, Assistant Professor, Assistant Director, University of Minnesota College of Pharmacy, Duluth, MN Amy Pavelka, MA, Program Coordinator, University of Minnesota College of Pharmacy, Minneapolis, MN Oscar W. Garza, PhD, MBA, Assistant Professor, University of Minnesota College of Pharmacy, Minneapolis, MN Jean Y. Moon, PharmD, Associate Professor, Director, University of Minnesota College of Pharmacy, Minneapolis, MN
SCIENCE AND PRACTICE Pharmacy resident peer review process
Appendix 1. Pharmacy resident peer review: Resident satisfaction survey
Questions 1. Did the review process improve your understanding regarding: a. How to improve patient care documentation? b. How to provide peer feedback? c. The importance of effective interprofessional communication in clinical decision making? 2. Was the review process helpful? 3. On average, how long did it take to review patient notes provided by each resident?
Yes
No
How do you think documentation and communication of patient care decision making can be improved in your practice setting? After this review process, in your opinion, what do you anticipate will be the most significant change that will take place in your patient care documentation practice? General comments/suggestions: Appendix 2 Documentation peer review debrief session
___ minutes
What part of the note did you find most helpful to you for the care of the patient? Why? Are there any parts of the note (e.g., style, template, structure) that you would suggest changing? Why?
1. What did you think of the peer review process overall? How do you feel it went? 2. What are the benefits of completing a peer review process? 3. What did you learn by reviewing others’ documentation? Was there anything you were able to take away from this experience and apply to your clinic site? 4. What did you learn from the feedback you received on your documentation? Have you changed anything in your documentation practice as a result?
866.e1