- Email: [email protected]
Impact of debates on student perceptions and competency scores in the advanced pharmacy practice setting
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
Currents in Pharmacy Teaching and Learning journal homepage: www.elsevier.com/locate/cptl
Experiences in Teaching and Learning
Impact of debates on student perceptions and competency scores in the advanced pharmacy practice setting ⁎
Eliza A. Dy-Boarmana, , Ginelle A. Bryanta, Morgan S. Herringb, Kendra Y. Fosterc a
Department of Clinical Sciences, Drake University College of Pharmacy and Health Sciences, 2507 University Avenue, Des Moines, IA 50311-4505, United States Department of Pharmacy Practice and Science/Division of Applied Clinical Sciences, University of Iowa College of Pharmacy, 115 South Grand Avenue, Iowa City, IA 52242, United States c Department of Pharmacy Practice and Science/Division of Health Services Research, 115 South Grand Avenue, Iowa City, IA 52242, United States b
AR TI CLE I NF O
AB S T R A CT
Keywords: Experiential Debate Student perceptions
Background and purpose: Advanced pharmacy practice experience (APPE) students are faced with the difficult reality that there is rarely one correct answer to a patient care question. Faculty preceptors developed a clinical debate activity to provide students with an opportunity to explore pharmacy topics with competing viewpoints. Educational activity and setting: The clinical debate activity was implemented in the APPE setting as a collaboration between three faculty preceptors from Drake University and University of Iowa. Student pre-debate and post-debate survey data was collected to assess the perceived impact of clinical debates on student confidence in skills related to the debate. Students were also asked to provide which skills were developed through the debate, whether participation in the debate changed their opinion on the issue, and if debates should be used as a teaching tool. Faculty preceptor scores on midpoint and final evaluations for applicable APPE competencies were also evaluated. Findings: Forty-two students participated in a clinical debate over a 12-month period. Students demonstrated improved confidence in almost all areas assessed, and 90.5% of students felt debates should be used as a teaching tool. Assessment of faculty midpoint (pre-debate) and final (post-debate) evaluation scores revealed statistically significant improvements in competencies related to literature evaluation and communication skills. Discussion and conclusions: Clinical debates have had a positive impact on both subjective and objective results in this APPE setting. Preceptors are encouraged to consider implementing a similar activity. Debates are a useful teaching tool in developing confidence and skills.
Background and purpose The pharmacy curriculum is designed to make students “practice-ready” so that graduates are ready to deliver evidence-based patient care in a variety of practice settings. The American Council for Pharmacy Education (ACPE) Standards 2016 specifically require that graduates have the ability to evaluate scientific literature to advance population health and patient-centered care (Standard 1.1) and apply evidence-based clinical reasoning skills across the patient's lifespan (Standard 25.7).1 Additionally ACPE requires that colleges and schools of pharmacy prepare graduates to effectively communicate when interacting with individuals, ⁎
Corresponding author. E-mail addresses: [email protected] (E.A. Dy-Boarman), [email protected] (G.A. Bryant), [email protected] (M.S. Herring), [email protected] (K.Y. Foster). http://dx.doi.org/10.1016/j.cptl.2017.09.011
1877-1297/ © 2017 Published by Elsevier Inc.
Please cite this article as: Dy-Boarman, E.A., Currents in Pharmacy Teaching and Learning (2017), http://dx.doi.org/10.1016/j.cptl.2017.09.011
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
groups, and organizations (Standard 3.6), with recent emphasis on effective communication in interprofessional healthcare teams (Standard 11.3).1 While patient cases are often used to prepare students for the nuances of real-world patients, students are limited by their experience and lack of clinical judgment. Cases used throughout the didactic curriculum are often black and white and designed so that students must select a correct answer in order to gain foundational knowledge. As students progress into advanced pharmacy practice experiences (APPEs), they are faced with the reality that there is rarely one correct answer to a patient care question or scenario. In fact, there are often many valid arguments with contradicting conclusions. APPE students who are expected to establish their clinical judgment in these scenarios may find this to be challenging or frustrating. Additionally, once students establish their position based on literature that they have collected and assessed, it is often difficult for them to confidently communicate and defend their position to other healthcare providers who may have differing opinions. These precepting challenges led to the development of an activity to enhance APPE student skills in these areas. As debates have been documented as a teaching tool in other pharmacy education settings, faculty preceptors within the UnityPoint Health-Des Moines health system implemented an APPE clinical debate activity to provide APPE students with an opportunity to explore relevant adult medicine pharmacy topics with competing viewpoints.2–6 The activity was designed to challenge students to locate and assess relevant pieces of supporting literature and to defend their argument on a clinical controversy, which has the potential to translate to better literature evaluation skills in other aspects of the experience. Additionally, students were challenged to present their position in a persuasive, yet respectful manner. Practice with this type of communication is an important aspect in understanding how to make and defend recommendations as a part of an interprofessional medical team. Previously published literature on clinical debates focuses primarily on the use of debates as a teaching tool in the didactic pharmacy education setting. These debates included student debate participants ranging from first- through third-year professional pharmacy students and occurred in both required and elective courses.2–6 Data evaluating each of these experiences demonstrated increases in students’ perceived literature evaluation skills, communication skills, and critical thinking skills following participation in a debate.2–6 While the use of debates in the didactic setting has proven to be beneficial in terms of student perceptions, the authors felt that the use of debates in the APPE setting may prove to be even more valuable. APPE students have a stronger foundational knowledge, and they are immersed in practice, giving them an opportunity to gain an appreciation for some of the challenges of patient care when one right answer may not exist. Additionally, they routinely apply literature to real patients for which they are expected to justify and communicate their recommendations, often with other health care providers. To our knowledge this is one of the first studies to assess the use of structured clinical debates in the APPE setting. The primary objective of this study was to assess the perceived impact of clinical debates on student confidence in various skills needed for future practice. Additionally, this study was designed to assess student perceptions of the usefulness of clinical debates as a teaching tool. The secondary objective was to assess the impact of clinical debates on APPE evaluation competency scores related to literature evaluation and communication. Educational activity and setting This clinical debate activity was implemented in the APPE setting as a collaboration between three faculty preceptors from two academic institutions, Drake University and University of Iowa. While faculty preceptors conduct different APPEs (two ambulatory care sites and one acute care site), all are affiliated with the UnityPoint Health-Des Moines health system. Each faculty preceptor has one to two students during each five-week block. At the beginning of each experience, students are divided into two groups, given a debate topic, and assigned a specific position to defend. An attempt is made to balance the number of students in each group, and debate groups generally consist of two to three students. In order to accommodate the different schedules for each experience, students are typically given one afternoon to work with their group members once a week for three or four weeks. Students are asked to prepare a summary handout that highlights three to four main points defending their argument, literature to support each point, and a comprehensive list of references used to prepare for the debate, which they present to all attendees (including debate opponents) on the day of the debate. While students are encouraged to use primary literature and guidelines when possible, they are allowed to present information from any resource during the course of the debate. Students are also encouraged to prepare for both sides of the debate in order to provide the soundest argument for why their position is superior. Clinical debate topics are selected based on 1) the absence of a clear “correct” answer/position; 2) the existence of a body of evidence to support both sides of the debate and 3) relevance to one or more of the practice settings. Clinical debate topics utilized over the course of this study included: the long-term use of bisphosphonates, bridging for interruption of anticoagulation therapy in atrial fibrillation patients, blood pressure goals for patients age ≥ 60 years old, first line blood pressure agents in patients with diabetes, hormone replacement therapy in post-menopausal women, and testosterone replacement therapy for age-related hypogonadism. The debate typically takes place during the fourth or fifth week of the experience. All faculty preceptors attend the debates, and other APPE and introductory pharmacy practice experience (IPPE) students on experientials within the health system are invited to attend as audience members. The debate follows a modified Lincoln-Douglas format. During the debate, student teams take turns verbally presenting their argument in the form of an opening argument, rebuttal, and concluding statement with up to 10 minutes, five minutes, and two minutes for each section, respectively (Table 1). The debate concludes with questions from audience members. Student teams are evaluated by all three faculty preceptors using a non-validated rubric (Appendix A) that is provided to the students during the first few days of the experience. Evaluation areas on the rubric include: opening argument, rebuttal, literature, 2
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 1 Debate format. Debate component
Allotted time
1. 2. 3. 4. 5. 6. 7.
10 min 10 min 5 min 5 min 5 min 2 min 2 min
Group 1 opening argument Group 2 opening argument Break for groups to prepare for rebuttal Group 1 rebuttal Group 2 rebuttal Group 1 concluding statements Group 2 concluding statements
conclusion, presentation, preparedness, persuasiveness, question/answers, references, and team contribution. Student teams are rated on a one to four scale, with higher scores equating to better overall performance in each area. Each point of the one to four scale is clearly defined with descriptions in order to assist with objective evaluation. Faculty preceptors also provide open-ended comments on items performed well during the debate and items that can be improved upon for future presentations/clinical controversy scenarios. Each student's primary faculty preceptor reviews this information with him or her during final evaluations. Students provide feedback on their group members via a peer evaluation form distributed at the conclusion of the debate. Peer evaluation forms ask students to provide open-ended feedback to their peers regarding their: contribution to the literature search, participation in group meetings, synthesis of the argument, preparation of the argument, utilization of primary literature to support the argument, utilization of persuasive techniques, and demonstration of respect for group members. Peer evaluation forms are submitted to faculty preceptors. Preceptors review these to identify discrepancies in workload within a team and to provide feedback to individual students during final evaluations. Because the faculty debate rubric evaluates each group's performance as a whole (rather than one individual's performance) during the debate, peer evaluations are not factored into overall debate rubric scores. Student survey data was collected over a 12-month period to assess the perceived impact of clinical debates in an APPE setting. An 11-item pre-debate survey was developed to assess student confidence in various skills related to the debate activity. Many of these skills directly relate to the rubric used to score student performance in the debate. Pre-debate surveys were electronically distributed via the online survey platform Qualtrics™ on the first day of the experience. Similarly, a post-debate survey was created to assess confidence in those same 11 skills. Additionally, students were asked to provide input on which skills were gained/further developed as a result of the debate (research, verbal communication, written communication, teamwork, critical-thinking, and/or rebuttal), whether participation in the debate changed their opinion on the issue, and if debates should be used more frequently as a teaching tool. Students were also asked to provide open-ended responses on the ways in which the debate assignment was or was not a useful teaching tool. Post-debate surveys were distributed via the online survey platform Qualtrics™ following the debate. All APPE debate participants completing both pre-debate and post-debate surveys were included in the study. In addition to student survey data, faculty preceptors assessed scores on midpoint (pre-debate) and final (post-debate) evaluations for applicable APPE competencies. Both Drake University and University of Iowa use the same experiential evaluations and APPE competencies, and three of these competencies were selected for assessment: “critically analyzes scientific literature related to both drugs and diseases;” “communicates in a manner that values team-based decision making and respects contributions from other areas of expertise” and “communicates with confidence, clarity, respect and empathy to establish rapport and build trusting relationships.” These competencies were selected due to their direct relationship to the goals of the clinical debate activity and related ACPE Standards. T-tests were run for the analyses of all pre-debate and post-debate data using Stata® software version 14.2. A p-value of 0.05 was considered significant. This study was approved as exempt by both the Drake University and University of Iowa Institutional Review Boards. Findings A total of 42 students participated in a clinical debate over a 12-month period and completed the pre-debate and post-debate surveys. Students generally did well on the assignment, as demonstrated by scores on preceptor debate rubrics. Out of 40 possible points, average scores for each team ranged from 36 to 40 with an overall average of 38 points. When looking at the pre-debate and post-debate survey data (Table 2), the only statistically insignificant result was change in student confidence “in ability to provide a clear, succinct rationale for why an argument is superior to an alternative.” The two most statistically different results were confidence “in ability to fully adapt an argument as opposing information is presented” (mean increase of 0.8) and confidence “in the ability to formulate a conclusion based on conflicting bodies of literature” (mean increase of 0.7). Twelve students (28.6%) had a change in opinion about the debate topic after participating in this activity. Additionally, 38 (90.5%) students also felt that debates should be used as a teaching tool. Open-ended comments revealed that many students were initially uncomfortable with this project; however, most students found this activity to be a useful exercise in learning how to evaluate a clinical controversy and how to defend a position based on literature. Finally, students self-identified which of the six listed skills were further developed as a result of this clinical debate activity (Table 3). Assessment of faculty preceptor midpoint (pre-debate) and final (post-debate) evaluation scores revealed statistically significant 3
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 2 Comparison of pre-debate and post-debate survey results (N = 42). Survey statementsa
Mean pre-debate scores (Standard deviation)
Mean post-debate scores (Standard deviation)
P-value
I am confident in my ability to formulate an organized, logical argument I am confident in my ability to evaluate points from an opposing argument and provide information to refute that argument I am confident in my ability to find appropriate references to support an argument I am confident in my ability to provide a clear, succinct rationale for why an argument is superior to an alternative. I am confident in my ability to present in a self-assured, organized, and complete manner I am confident in my ability to fully adapt an argument as opposing information is presented I am confident in my ability to fully deliver information in a persuasive manner I am confident in my ability to answer questions about a particular side to an argument I am confident in my ability to summarize and appropriately document references I am confident in my ability to work in a team to defend an argument I am confident in my ability to formulate a conclusion based on conflicting bodies of literature
3.9 (0.1) 3.7 (0.1)
4.2 (0.1) 4.1 (0.1)
0.0099 0.0008
4.0 (0.1)
4.4 (0.1)
0.0005
3.6 (0.1)
3.9 (0.1)
0.0586
3.6 (0.1)
3.9 (0.1)
0.0302
3.1 (0.1)
3.9 (0.1)
< 0.0001
3.4 (0.1)
3.9 (0.1)
0.0026
3.8 (0.1)
4.2 (0.1)
0.0003
4.0 (0.1)
4.5 (0.1)
0.0016
4.0 (0.1) 3.5 (0.1)
4.4 (0.1) 4.2 (0.1)
0.0007 < 0.0001
a
Rating scale: 1 = Strongly Disagree and 5 = Strongly Agree.
Table 3 Skills that were gained or further developed as a result of the debate activity (N = 42). Skill
Number of students reporting (%)
Rebuttal skills Teamwork skills Critical-thinking skills Research (literature) skills Verbal communication skills Written communication skills
38 36 35 33 32 18
(90.5%) (85.7%) (83.3%) (78.6%) (76.2%) (42.9%)
improvements in the three APPE competencies related to literature evaluation and communication skills (Table 4). Anecdotally, faculty preceptors also observed an increase in student proficiency at collecting and analyzing literature and their ability to communicate in a persuasive manner as a result of this activity. Discussion Previously published literature primarily utilizes student perception data to demonstrate the value of debate activities as learning tools.2–6 Our findings are consistent with findings from many of these other studies. Students in our study reported increased confidence in almost all areas related to the clinical debate. Additionally, students felt that their skills in a number of areas, most notably rebuttal skills, critical-thinking skills, and teamwork skills, were enhanced as a result of the debate activity. Objective assessment data are far more limited in the current literature and is mainly focused on demonstrating increases in content knowledge/retention, rather than skills, as a result of a debate activity.5 What sets our findings apart from previously published literature is the evaluation of APPE competencies related to the debate activity. Our assessment of three key evaluation competencies related to literature evaluation skills and communication skills demonstrate positive objective findings and highlight the potential benefit of debates. While our use of objective data is limited, the results that we do present are focused on assessment of skills rather than merely content knowledge. One main strength of this activity is the direct link between the goals of the clinical debate and ACPE Standards 2016. This single activity requires APPE students to use and further develop a variety of skills including, but not limited to, literature identification and analysis, written communication, verbal communication, persuasiveness, ability to adapt, and teamwork in order to develop into a practice-ready pharmacist. Additionally, students from different experiences (acute care and ambulatory care) and academic institutions are asked to work in groups and to learn from/with one another in order to complete this activity. Faculty preceptors have noted the strength of this unique feature, as students are able to share different perspectives and experiences to achieve a common goal. In our experience, clinical debates provide a richer experience with a higher level of critical evaluation than the traditional journal club activity. There are, however, a number of aspects to consider prior to implementation. First, this activity does take 4
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 4 Comparison of faculty experiential evaluation scores before and after debate activity. Evaluation competencya
Mean midpoint (Pre-debate) evaluation scores
Mean final (Post-debate) evaluation scores
P-value
Critically analyzes scientific literature related to both drugs and diseases Communicates in a manner that values team-based decision making and respects contributions from other areas of expertise Communicates with confidence, clarity, respect and empathy to establish rapport and build trusting relationships
2.8 2.9
3.2 3.2
< 0.0001 0.0001
2.7
3.1
< 0.0001
a
Rating scale: 1 = Needs Significant Improvement, 2 = Needs Improvement, 3 = Meets Expectations, 4 = Exceed Expectations.
thoughtful, intentional planning in order to be successful for both students and preceptors. Steps taken that have helped to make debates successful in our APPE sites include providing students with a specific timeline for each component of the debate, examples or templates for what the debate should look like, and evaluation tools (e.g., rubrics) at the start of the experience. While students have found this activity to be a very beneficial learning experience, students have often commented verbally and on written evaluations that they were initially nervous about the debate. Providing clear expectations and examples to students has helped to address the potential anxiety about this activity. Finally, given that our debate groups have been comprised of students from different sites, providing students with scheduled group meeting time throughout the experience has facilitated positive group dynamics. There are a few notable limitations to our study findings. First, the sample size is relatively small, and much of our data relies on student perceptions of the activity. Objective measures were evaluated through APPE competency scores comparing midpoint and final evaluations. However, a number of other factors also likely influenced changes in these scores, including other experience activities, provider interactions, drug information questions, and general increase in comfort level and confidence over the course of a five-week experience. Additionally, despite experiential office attempts to standardize the way preceptors complete evaluations, it is possible that preceptor evaluations may vary. Faculty preceptors and sites also vary in the experiences offered to students, with one site being ambulatory care internal medicine, one ambulatory care family medicine, and one acute care internal medicine. With this, students complete different activities, and there are likely additional factors taken into consideration when evaluating APPE competencies during midpoint and final evaluations. Finally, as preceptors were involved in implementing and evaluating the debate activity, there is a potential that preceptor bias impacted evaluation scores. Despite these limitations, our study findings suggest that clinical debates have a positive impact on both subjective and objective results in the APPE setting. This activity incorporates a number of critical skills and provides an opportunity for students to further develop those skills in evaluating literature to provide clinical judgment on a topic. As such, we plan to continue using clinical debates as a part of our APPEs. Future research will focus on objectively evaluating specific skills related to the clinical debate activity and identifying the true impact of this activity outside of the other normally scheduled activities. Preceptors at other institutions are encouraged to consider implementing a similar activity, as well as develop ways in which to objectively evaluate and share those findings with the greater academy. Summary Student survey data revealed the value of this activity as a teaching tool in developing confidence in various skills. Additionally, faculty preceptor evaluation data demonstrate the potential for this activity to improve student performance in APPE competencies related to literature evaluation and communication skills. This activity has achieved our initial goals, and APPE preceptors should consider this as a potential teaching tool.
Conflicts of interest None.
Financial disclosures The research did not receive any specific grant from funding agencies in the public, commercial, or non-for-profit sectors.
5
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Appendix A. Clinical debate rubric
Category
1
Opening Argument
Some thought put into No original thoughts put Opening argument into supporting argument significantly lacking in key opening argument, but points to support argument unorganized thought process
Rebuttal
Unable to evaluate opposing team's argument; does not provide structured information to refute opposing team Lack of supporting evidence. Referenced class notes
Literature
Conclusion
Conclusion limited to statement of team's argument
Presentation
Unsure of information presenting; poorly prepared and organized presentation with inconsistency amongst presenters Unprepared to adapt argument Delivers information without effort placed on persuading others to agree with argument
Preparedness Persuasiveness
Q&A
References
Team Contribution
2
3
4
Significant thought put into opening argument with organized and logical thought process Readily evaluate points Readily evaluates Exhibits some trouble from opposing team, but points from opposing evaluating points from team and provides opposing team's argument; does not address both adequate information provides some information sides of argument to refute opposing to refute opposing argument argument References consist of Primary and tertiary Only tertiary references literature appropriately appropriate guidelines used when primary and primary literature used with minor literature available when available omissions Conclusion well Conclusion limited with no Conclusion relatively thought out and final supporting evidence well thought out but complete with does not address why rationale for why alternative is less alternative is less desirable desirable Appears comfortable Good understanding of Exhibits basic information but appears and self-assured; understanding of organized, complete information; shows lack of unsure; generally well presentation organized presentation effort in presentation and with minor omissions poorly organized
Somewhat prepared to adapt argument Somewhat able to deliver information in a persuasive manner, key points missed
Able to adapt the majority of the argument Able to deliver majority of information in a persuasive manner, all information not addressed Able to answer most questions appropriately
Fully able to adapt argument as needed Fully able to deliver information in a persuasive manner
Listed sources complete, summarized, and appropriately documented per AMA guidelines Actively participates in the debate presentation
Unable to state where information was obtained or answer questions appropriately Complete lack of reputable information and no summary included for each reference
Able to answer questions with significant prompting
Some references listed and/ or not appropriately documented. No summary included for any references
Reference list complete with minor omissions/ mistakes in citations. Summary not included for all references
Minimal participation in the debate presentation
N/A
N/A
Total Points: _______/ 40
6
Able to answer questions appropriately
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
References 1. Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree (“Standards 2016”); Published February 2015. Available at: 〈https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf〉. Accessed 19 September 2017. 2. Blackmer AB, Diez HL, Klein KC. Design, implementation, and assessment of clinical debate as an active learning tool in two elective pharmacy courses: immunizations and pediatrics. Curr Pharm Teach Learn. 2014;6(2):254–258. 3. McNamara A, Janke K, Conway J, Schweiss S. Student self-ratings of skill acquisition from a clinical controversy debate in a third year pharmaceutical care lab. Inov Pharm. 2013;4(4):130. 4. Moore KG, Clements J, Sease J, Anderson Z. The utility of clinical controversy debates in an ambulatory care elective. Curr Pharm Teach Learn. 2015;7(2):239–248. 5. Lampkin SJ, Collins C, Danison R, Lewis M. Active learning through a debate series in a first-year pharmacy self-care course. Am J Pharm Educ. 2015;79(2) [Article 25]. 6. Lin SJ, Crawford SY. An online debate series for first-year pharmacy students. Am J Pharm Educ. 2007;71(1) [Article 12].
7
Contents lists available at ScienceDirect
Currents in Pharmacy Teaching and Learning journal homepage: www.elsevier.com/locate/cptl
Experiences in Teaching and Learning
Impact of debates on student perceptions and competency scores in the advanced pharmacy practice setting ⁎
Eliza A. Dy-Boarmana, , Ginelle A. Bryanta, Morgan S. Herringb, Kendra Y. Fosterc a
Department of Clinical Sciences, Drake University College of Pharmacy and Health Sciences, 2507 University Avenue, Des Moines, IA 50311-4505, United States Department of Pharmacy Practice and Science/Division of Applied Clinical Sciences, University of Iowa College of Pharmacy, 115 South Grand Avenue, Iowa City, IA 52242, United States c Department of Pharmacy Practice and Science/Division of Health Services Research, 115 South Grand Avenue, Iowa City, IA 52242, United States b
AR TI CLE I NF O
AB S T R A CT
Keywords: Experiential Debate Student perceptions
Background and purpose: Advanced pharmacy practice experience (APPE) students are faced with the difficult reality that there is rarely one correct answer to a patient care question. Faculty preceptors developed a clinical debate activity to provide students with an opportunity to explore pharmacy topics with competing viewpoints. Educational activity and setting: The clinical debate activity was implemented in the APPE setting as a collaboration between three faculty preceptors from Drake University and University of Iowa. Student pre-debate and post-debate survey data was collected to assess the perceived impact of clinical debates on student confidence in skills related to the debate. Students were also asked to provide which skills were developed through the debate, whether participation in the debate changed their opinion on the issue, and if debates should be used as a teaching tool. Faculty preceptor scores on midpoint and final evaluations for applicable APPE competencies were also evaluated. Findings: Forty-two students participated in a clinical debate over a 12-month period. Students demonstrated improved confidence in almost all areas assessed, and 90.5% of students felt debates should be used as a teaching tool. Assessment of faculty midpoint (pre-debate) and final (post-debate) evaluation scores revealed statistically significant improvements in competencies related to literature evaluation and communication skills. Discussion and conclusions: Clinical debates have had a positive impact on both subjective and objective results in this APPE setting. Preceptors are encouraged to consider implementing a similar activity. Debates are a useful teaching tool in developing confidence and skills.
Background and purpose The pharmacy curriculum is designed to make students “practice-ready” so that graduates are ready to deliver evidence-based patient care in a variety of practice settings. The American Council for Pharmacy Education (ACPE) Standards 2016 specifically require that graduates have the ability to evaluate scientific literature to advance population health and patient-centered care (Standard 1.1) and apply evidence-based clinical reasoning skills across the patient's lifespan (Standard 25.7).1 Additionally ACPE requires that colleges and schools of pharmacy prepare graduates to effectively communicate when interacting with individuals, ⁎
Corresponding author. E-mail addresses: [email protected] (E.A. Dy-Boarman), [email protected] (G.A. Bryant), [email protected] (M.S. Herring), [email protected] (K.Y. Foster). http://dx.doi.org/10.1016/j.cptl.2017.09.011
1877-1297/ © 2017 Published by Elsevier Inc.
Please cite this article as: Dy-Boarman, E.A., Currents in Pharmacy Teaching and Learning (2017), http://dx.doi.org/10.1016/j.cptl.2017.09.011
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
groups, and organizations (Standard 3.6), with recent emphasis on effective communication in interprofessional healthcare teams (Standard 11.3).1 While patient cases are often used to prepare students for the nuances of real-world patients, students are limited by their experience and lack of clinical judgment. Cases used throughout the didactic curriculum are often black and white and designed so that students must select a correct answer in order to gain foundational knowledge. As students progress into advanced pharmacy practice experiences (APPEs), they are faced with the reality that there is rarely one correct answer to a patient care question or scenario. In fact, there are often many valid arguments with contradicting conclusions. APPE students who are expected to establish their clinical judgment in these scenarios may find this to be challenging or frustrating. Additionally, once students establish their position based on literature that they have collected and assessed, it is often difficult for them to confidently communicate and defend their position to other healthcare providers who may have differing opinions. These precepting challenges led to the development of an activity to enhance APPE student skills in these areas. As debates have been documented as a teaching tool in other pharmacy education settings, faculty preceptors within the UnityPoint Health-Des Moines health system implemented an APPE clinical debate activity to provide APPE students with an opportunity to explore relevant adult medicine pharmacy topics with competing viewpoints.2–6 The activity was designed to challenge students to locate and assess relevant pieces of supporting literature and to defend their argument on a clinical controversy, which has the potential to translate to better literature evaluation skills in other aspects of the experience. Additionally, students were challenged to present their position in a persuasive, yet respectful manner. Practice with this type of communication is an important aspect in understanding how to make and defend recommendations as a part of an interprofessional medical team. Previously published literature on clinical debates focuses primarily on the use of debates as a teaching tool in the didactic pharmacy education setting. These debates included student debate participants ranging from first- through third-year professional pharmacy students and occurred in both required and elective courses.2–6 Data evaluating each of these experiences demonstrated increases in students’ perceived literature evaluation skills, communication skills, and critical thinking skills following participation in a debate.2–6 While the use of debates in the didactic setting has proven to be beneficial in terms of student perceptions, the authors felt that the use of debates in the APPE setting may prove to be even more valuable. APPE students have a stronger foundational knowledge, and they are immersed in practice, giving them an opportunity to gain an appreciation for some of the challenges of patient care when one right answer may not exist. Additionally, they routinely apply literature to real patients for which they are expected to justify and communicate their recommendations, often with other health care providers. To our knowledge this is one of the first studies to assess the use of structured clinical debates in the APPE setting. The primary objective of this study was to assess the perceived impact of clinical debates on student confidence in various skills needed for future practice. Additionally, this study was designed to assess student perceptions of the usefulness of clinical debates as a teaching tool. The secondary objective was to assess the impact of clinical debates on APPE evaluation competency scores related to literature evaluation and communication. Educational activity and setting This clinical debate activity was implemented in the APPE setting as a collaboration between three faculty preceptors from two academic institutions, Drake University and University of Iowa. While faculty preceptors conduct different APPEs (two ambulatory care sites and one acute care site), all are affiliated with the UnityPoint Health-Des Moines health system. Each faculty preceptor has one to two students during each five-week block. At the beginning of each experience, students are divided into two groups, given a debate topic, and assigned a specific position to defend. An attempt is made to balance the number of students in each group, and debate groups generally consist of two to three students. In order to accommodate the different schedules for each experience, students are typically given one afternoon to work with their group members once a week for three or four weeks. Students are asked to prepare a summary handout that highlights three to four main points defending their argument, literature to support each point, and a comprehensive list of references used to prepare for the debate, which they present to all attendees (including debate opponents) on the day of the debate. While students are encouraged to use primary literature and guidelines when possible, they are allowed to present information from any resource during the course of the debate. Students are also encouraged to prepare for both sides of the debate in order to provide the soundest argument for why their position is superior. Clinical debate topics are selected based on 1) the absence of a clear “correct” answer/position; 2) the existence of a body of evidence to support both sides of the debate and 3) relevance to one or more of the practice settings. Clinical debate topics utilized over the course of this study included: the long-term use of bisphosphonates, bridging for interruption of anticoagulation therapy in atrial fibrillation patients, blood pressure goals for patients age ≥ 60 years old, first line blood pressure agents in patients with diabetes, hormone replacement therapy in post-menopausal women, and testosterone replacement therapy for age-related hypogonadism. The debate typically takes place during the fourth or fifth week of the experience. All faculty preceptors attend the debates, and other APPE and introductory pharmacy practice experience (IPPE) students on experientials within the health system are invited to attend as audience members. The debate follows a modified Lincoln-Douglas format. During the debate, student teams take turns verbally presenting their argument in the form of an opening argument, rebuttal, and concluding statement with up to 10 minutes, five minutes, and two minutes for each section, respectively (Table 1). The debate concludes with questions from audience members. Student teams are evaluated by all three faculty preceptors using a non-validated rubric (Appendix A) that is provided to the students during the first few days of the experience. Evaluation areas on the rubric include: opening argument, rebuttal, literature, 2
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 1 Debate format. Debate component
Allotted time
1. 2. 3. 4. 5. 6. 7.
10 min 10 min 5 min 5 min 5 min 2 min 2 min
Group 1 opening argument Group 2 opening argument Break for groups to prepare for rebuttal Group 1 rebuttal Group 2 rebuttal Group 1 concluding statements Group 2 concluding statements
conclusion, presentation, preparedness, persuasiveness, question/answers, references, and team contribution. Student teams are rated on a one to four scale, with higher scores equating to better overall performance in each area. Each point of the one to four scale is clearly defined with descriptions in order to assist with objective evaluation. Faculty preceptors also provide open-ended comments on items performed well during the debate and items that can be improved upon for future presentations/clinical controversy scenarios. Each student's primary faculty preceptor reviews this information with him or her during final evaluations. Students provide feedback on their group members via a peer evaluation form distributed at the conclusion of the debate. Peer evaluation forms ask students to provide open-ended feedback to their peers regarding their: contribution to the literature search, participation in group meetings, synthesis of the argument, preparation of the argument, utilization of primary literature to support the argument, utilization of persuasive techniques, and demonstration of respect for group members. Peer evaluation forms are submitted to faculty preceptors. Preceptors review these to identify discrepancies in workload within a team and to provide feedback to individual students during final evaluations. Because the faculty debate rubric evaluates each group's performance as a whole (rather than one individual's performance) during the debate, peer evaluations are not factored into overall debate rubric scores. Student survey data was collected over a 12-month period to assess the perceived impact of clinical debates in an APPE setting. An 11-item pre-debate survey was developed to assess student confidence in various skills related to the debate activity. Many of these skills directly relate to the rubric used to score student performance in the debate. Pre-debate surveys were electronically distributed via the online survey platform Qualtrics™ on the first day of the experience. Similarly, a post-debate survey was created to assess confidence in those same 11 skills. Additionally, students were asked to provide input on which skills were gained/further developed as a result of the debate (research, verbal communication, written communication, teamwork, critical-thinking, and/or rebuttal), whether participation in the debate changed their opinion on the issue, and if debates should be used more frequently as a teaching tool. Students were also asked to provide open-ended responses on the ways in which the debate assignment was or was not a useful teaching tool. Post-debate surveys were distributed via the online survey platform Qualtrics™ following the debate. All APPE debate participants completing both pre-debate and post-debate surveys were included in the study. In addition to student survey data, faculty preceptors assessed scores on midpoint (pre-debate) and final (post-debate) evaluations for applicable APPE competencies. Both Drake University and University of Iowa use the same experiential evaluations and APPE competencies, and three of these competencies were selected for assessment: “critically analyzes scientific literature related to both drugs and diseases;” “communicates in a manner that values team-based decision making and respects contributions from other areas of expertise” and “communicates with confidence, clarity, respect and empathy to establish rapport and build trusting relationships.” These competencies were selected due to their direct relationship to the goals of the clinical debate activity and related ACPE Standards. T-tests were run for the analyses of all pre-debate and post-debate data using Stata® software version 14.2. A p-value of 0.05 was considered significant. This study was approved as exempt by both the Drake University and University of Iowa Institutional Review Boards. Findings A total of 42 students participated in a clinical debate over a 12-month period and completed the pre-debate and post-debate surveys. Students generally did well on the assignment, as demonstrated by scores on preceptor debate rubrics. Out of 40 possible points, average scores for each team ranged from 36 to 40 with an overall average of 38 points. When looking at the pre-debate and post-debate survey data (Table 2), the only statistically insignificant result was change in student confidence “in ability to provide a clear, succinct rationale for why an argument is superior to an alternative.” The two most statistically different results were confidence “in ability to fully adapt an argument as opposing information is presented” (mean increase of 0.8) and confidence “in the ability to formulate a conclusion based on conflicting bodies of literature” (mean increase of 0.7). Twelve students (28.6%) had a change in opinion about the debate topic after participating in this activity. Additionally, 38 (90.5%) students also felt that debates should be used as a teaching tool. Open-ended comments revealed that many students were initially uncomfortable with this project; however, most students found this activity to be a useful exercise in learning how to evaluate a clinical controversy and how to defend a position based on literature. Finally, students self-identified which of the six listed skills were further developed as a result of this clinical debate activity (Table 3). Assessment of faculty preceptor midpoint (pre-debate) and final (post-debate) evaluation scores revealed statistically significant 3
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 2 Comparison of pre-debate and post-debate survey results (N = 42). Survey statementsa
Mean pre-debate scores (Standard deviation)
Mean post-debate scores (Standard deviation)
P-value
I am confident in my ability to formulate an organized, logical argument I am confident in my ability to evaluate points from an opposing argument and provide information to refute that argument I am confident in my ability to find appropriate references to support an argument I am confident in my ability to provide a clear, succinct rationale for why an argument is superior to an alternative. I am confident in my ability to present in a self-assured, organized, and complete manner I am confident in my ability to fully adapt an argument as opposing information is presented I am confident in my ability to fully deliver information in a persuasive manner I am confident in my ability to answer questions about a particular side to an argument I am confident in my ability to summarize and appropriately document references I am confident in my ability to work in a team to defend an argument I am confident in my ability to formulate a conclusion based on conflicting bodies of literature
3.9 (0.1) 3.7 (0.1)
4.2 (0.1) 4.1 (0.1)
0.0099 0.0008
4.0 (0.1)
4.4 (0.1)
0.0005
3.6 (0.1)
3.9 (0.1)
0.0586
3.6 (0.1)
3.9 (0.1)
0.0302
3.1 (0.1)
3.9 (0.1)
< 0.0001
3.4 (0.1)
3.9 (0.1)
0.0026
3.8 (0.1)
4.2 (0.1)
0.0003
4.0 (0.1)
4.5 (0.1)
0.0016
4.0 (0.1) 3.5 (0.1)
4.4 (0.1) 4.2 (0.1)
0.0007 < 0.0001
a
Rating scale: 1 = Strongly Disagree and 5 = Strongly Agree.
Table 3 Skills that were gained or further developed as a result of the debate activity (N = 42). Skill
Number of students reporting (%)
Rebuttal skills Teamwork skills Critical-thinking skills Research (literature) skills Verbal communication skills Written communication skills
38 36 35 33 32 18
(90.5%) (85.7%) (83.3%) (78.6%) (76.2%) (42.9%)
improvements in the three APPE competencies related to literature evaluation and communication skills (Table 4). Anecdotally, faculty preceptors also observed an increase in student proficiency at collecting and analyzing literature and their ability to communicate in a persuasive manner as a result of this activity. Discussion Previously published literature primarily utilizes student perception data to demonstrate the value of debate activities as learning tools.2–6 Our findings are consistent with findings from many of these other studies. Students in our study reported increased confidence in almost all areas related to the clinical debate. Additionally, students felt that their skills in a number of areas, most notably rebuttal skills, critical-thinking skills, and teamwork skills, were enhanced as a result of the debate activity. Objective assessment data are far more limited in the current literature and is mainly focused on demonstrating increases in content knowledge/retention, rather than skills, as a result of a debate activity.5 What sets our findings apart from previously published literature is the evaluation of APPE competencies related to the debate activity. Our assessment of three key evaluation competencies related to literature evaluation skills and communication skills demonstrate positive objective findings and highlight the potential benefit of debates. While our use of objective data is limited, the results that we do present are focused on assessment of skills rather than merely content knowledge. One main strength of this activity is the direct link between the goals of the clinical debate and ACPE Standards 2016. This single activity requires APPE students to use and further develop a variety of skills including, but not limited to, literature identification and analysis, written communication, verbal communication, persuasiveness, ability to adapt, and teamwork in order to develop into a practice-ready pharmacist. Additionally, students from different experiences (acute care and ambulatory care) and academic institutions are asked to work in groups and to learn from/with one another in order to complete this activity. Faculty preceptors have noted the strength of this unique feature, as students are able to share different perspectives and experiences to achieve a common goal. In our experience, clinical debates provide a richer experience with a higher level of critical evaluation than the traditional journal club activity. There are, however, a number of aspects to consider prior to implementation. First, this activity does take 4
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Table 4 Comparison of faculty experiential evaluation scores before and after debate activity. Evaluation competencya
Mean midpoint (Pre-debate) evaluation scores
Mean final (Post-debate) evaluation scores
P-value
Critically analyzes scientific literature related to both drugs and diseases Communicates in a manner that values team-based decision making and respects contributions from other areas of expertise Communicates with confidence, clarity, respect and empathy to establish rapport and build trusting relationships
2.8 2.9
3.2 3.2
< 0.0001 0.0001
2.7
3.1
< 0.0001
a
Rating scale: 1 = Needs Significant Improvement, 2 = Needs Improvement, 3 = Meets Expectations, 4 = Exceed Expectations.
thoughtful, intentional planning in order to be successful for both students and preceptors. Steps taken that have helped to make debates successful in our APPE sites include providing students with a specific timeline for each component of the debate, examples or templates for what the debate should look like, and evaluation tools (e.g., rubrics) at the start of the experience. While students have found this activity to be a very beneficial learning experience, students have often commented verbally and on written evaluations that they were initially nervous about the debate. Providing clear expectations and examples to students has helped to address the potential anxiety about this activity. Finally, given that our debate groups have been comprised of students from different sites, providing students with scheduled group meeting time throughout the experience has facilitated positive group dynamics. There are a few notable limitations to our study findings. First, the sample size is relatively small, and much of our data relies on student perceptions of the activity. Objective measures were evaluated through APPE competency scores comparing midpoint and final evaluations. However, a number of other factors also likely influenced changes in these scores, including other experience activities, provider interactions, drug information questions, and general increase in comfort level and confidence over the course of a five-week experience. Additionally, despite experiential office attempts to standardize the way preceptors complete evaluations, it is possible that preceptor evaluations may vary. Faculty preceptors and sites also vary in the experiences offered to students, with one site being ambulatory care internal medicine, one ambulatory care family medicine, and one acute care internal medicine. With this, students complete different activities, and there are likely additional factors taken into consideration when evaluating APPE competencies during midpoint and final evaluations. Finally, as preceptors were involved in implementing and evaluating the debate activity, there is a potential that preceptor bias impacted evaluation scores. Despite these limitations, our study findings suggest that clinical debates have a positive impact on both subjective and objective results in the APPE setting. This activity incorporates a number of critical skills and provides an opportunity for students to further develop those skills in evaluating literature to provide clinical judgment on a topic. As such, we plan to continue using clinical debates as a part of our APPEs. Future research will focus on objectively evaluating specific skills related to the clinical debate activity and identifying the true impact of this activity outside of the other normally scheduled activities. Preceptors at other institutions are encouraged to consider implementing a similar activity, as well as develop ways in which to objectively evaluate and share those findings with the greater academy. Summary Student survey data revealed the value of this activity as a teaching tool in developing confidence in various skills. Additionally, faculty preceptor evaluation data demonstrate the potential for this activity to improve student performance in APPE competencies related to literature evaluation and communication skills. This activity has achieved our initial goals, and APPE preceptors should consider this as a potential teaching tool.
Conflicts of interest None.
Financial disclosures The research did not receive any specific grant from funding agencies in the public, commercial, or non-for-profit sectors.
5
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
Appendix A. Clinical debate rubric
Category
1
Opening Argument
Some thought put into No original thoughts put Opening argument into supporting argument significantly lacking in key opening argument, but points to support argument unorganized thought process
Rebuttal
Unable to evaluate opposing team's argument; does not provide structured information to refute opposing team Lack of supporting evidence. Referenced class notes
Literature
Conclusion
Conclusion limited to statement of team's argument
Presentation
Unsure of information presenting; poorly prepared and organized presentation with inconsistency amongst presenters Unprepared to adapt argument Delivers information without effort placed on persuading others to agree with argument
Preparedness Persuasiveness
Q&A
References
Team Contribution
2
3
4
Significant thought put into opening argument with organized and logical thought process Readily evaluate points Readily evaluates Exhibits some trouble from opposing team, but points from opposing evaluating points from team and provides opposing team's argument; does not address both adequate information provides some information sides of argument to refute opposing to refute opposing argument argument References consist of Primary and tertiary Only tertiary references literature appropriately appropriate guidelines used when primary and primary literature used with minor literature available when available omissions Conclusion well Conclusion limited with no Conclusion relatively thought out and final supporting evidence well thought out but complete with does not address why rationale for why alternative is less alternative is less desirable desirable Appears comfortable Good understanding of Exhibits basic information but appears and self-assured; understanding of organized, complete information; shows lack of unsure; generally well presentation organized presentation effort in presentation and with minor omissions poorly organized
Somewhat prepared to adapt argument Somewhat able to deliver information in a persuasive manner, key points missed
Able to adapt the majority of the argument Able to deliver majority of information in a persuasive manner, all information not addressed Able to answer most questions appropriately
Fully able to adapt argument as needed Fully able to deliver information in a persuasive manner
Listed sources complete, summarized, and appropriately documented per AMA guidelines Actively participates in the debate presentation
Unable to state where information was obtained or answer questions appropriately Complete lack of reputable information and no summary included for each reference
Able to answer questions with significant prompting
Some references listed and/ or not appropriately documented. No summary included for any references
Reference list complete with minor omissions/ mistakes in citations. Summary not included for all references
Minimal participation in the debate presentation
N/A
N/A
Total Points: _______/ 40
6
Able to answer questions appropriately
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
E.A. Dy-Boarman et al.
References 1. Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree (“Standards 2016”); Published February 2015. Available at: 〈https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf〉. Accessed 19 September 2017. 2. Blackmer AB, Diez HL, Klein KC. Design, implementation, and assessment of clinical debate as an active learning tool in two elective pharmacy courses: immunizations and pediatrics. Curr Pharm Teach Learn. 2014;6(2):254–258. 3. McNamara A, Janke K, Conway J, Schweiss S. Student self-ratings of skill acquisition from a clinical controversy debate in a third year pharmaceutical care lab. Inov Pharm. 2013;4(4):130. 4. Moore KG, Clements J, Sease J, Anderson Z. The utility of clinical controversy debates in an ambulatory care elective. Curr Pharm Teach Learn. 2015;7(2):239–248. 5. Lampkin SJ, Collins C, Danison R, Lewis M. Active learning through a debate series in a first-year pharmacy self-care course. Am J Pharm Educ. 2015;79(2) [Article 25]. 6. Lin SJ, Crawford SY. An online debate series for first-year pharmacy students. Am J Pharm Educ. 2007;71(1) [Article 12].
7