- Email: [email protected]
Structured academic discussions through an online education-specific platform to improve Pharm.D. students learning outcomes
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
Contents lists available at ScienceDirect
Currents in Pharmacy Teaching and Learning journal homepage: www.elsevier.com/locate/cptl
Research Article
Structured academic discussions through an online education-specific platform to improve Pharm.D. students learning outcomes ⁎
Srikanth Kollurua, , James T. Varugheseb a b
Keck Graduate Institute School of Pharmacy, Claremont, CA 91739 Medicare & Retirement Appeals & Grievances United Healthcare, Sugar Land, TX 77478
A R T I C L E I N F O
ABSTRACT
Keywords: Active learning Online academic discussions Piazza Social media
Objective: To facilitate active academic discussions using an online, education-centered platform and reinforce concepts, in order to improve overall course outcomes. Materials and methods: A third year integrated pharmacotherapy course was enrolled on an online searchable platform, Piazza®, to facilitate academic discussions. Students could ask, answer, and explore content, and build on submitted answers in wiki style in collaboration. Instructor posted learning objectives, endorsed student responses with correct answers and led follow-up discussions. Review sessions were conducted on this platform before all major exams. A student t-test was used to compare class performance with those of previous years. Results: In a post-activity qualitative survey, most students appreciated the less stressful, online interaction with peers and faculty. For 15 medicinal chemistry course hours, there were 83 posts on Piazza® with 303 total contributions, 107 student responses, and 546 min of group discussion time. 94% of questions received student responses and 89% of those were endorsed by the instructor. Students enjoyed pre-exam discussions, organization of the page, and reinforcing material on complex learning objectives. This discussion forum fostered personal exploration of content by the students, which led to better performance on examinations. Conclusions: Involving the use of an online, education-centered platform for student discussions was an effective means of increasing class engagement with the course material. Student performance on exams was significantly improved in both cohorts that utilized active learning compared to the cohort without active learning (p=0.001 and p= 0.002 respectively). Piazza® can be utilized for any course and across disciplines.
Introduction Pharmacy educators are always seeking the most innovative ways to reach out to students, both in and out of the classroom, to enhance learning and retention of the material necessary for today's demanding health care needs. Technology is evolving at a rapid pace, and the speed at which the current generation of students adapting is just as quick. Guideline 11.2 of the 2007 Accreditation Council for Pharmacy Education (ACPE) Standards calls for the continual improvement of students’ critical thinking and problemsolving skills through active learning strategies throughout every curriculum.1 A variety of technologies are employed at all pharmacy schools in the United States (US) including audience response systems (ARS), learning management systems (LMS), web conferencing,
⁎
Corresponding author at: Keck Graduate Institute School of Pharmacy, Claremont, CA 91739. E-mail addresses: [email protected] (S. Kolluru), [email protected] (J.T. Varughese).
http://dx.doi.org/10.1016/j.cptl.2016.11.022
1877-1297/ © 2016 Elsevier Inc. All rights reserved.
Please cite this article as: Kolluru, S., Currents in Pharmacy Teaching and Learning (2016), http://dx.doi.org/10.1016/j.cptl.2016.11.022
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
electronic testing software, and blogging tools. This is not only to fulfill all requirements from ACPE, but also help students to graduate as pharmacists who will be better-prepared to use all types of technologies and have better pharmacy knowledge.2 Utilizing certain types of technologies have shown to improve student motivation and attention during lectures. For instance, TurningPoint® (Turning Technologies; Youngstown, OH), an audience response system, provides instantaneous feedback to the instructor with regards to student understanding of the material.3,4 A recent survey showed that 87% percent of US pharmacy schools are employing some form of active learning techniques in their curriculam.5 A variety of active learning techniques are used based on the curricular needs and the learning objectives.6 The internet has evolved a great deal in a short time and all types of social media have diffused throughout the society to connect us even further. Newer options, such as Twitter (Twitter, Inc., San Francisco, CA) and Google+ (Google, Inc.; Mountain View, CA), offer their own benefits. Facebook (Facebook, Inc., Palo Alto, CA), a very popular social media site, continues to grow and has been studied in various settings to determine how best use the networking site as a beneficial tool in education.7 With Facebook, the ability to create a personal profile, connect with others via “friending” as well as join specialized “groups,” and discuss various topics within those groups is a key to the optimal interaction between students and faculty. Utilizing Facebook as an informal learning environment, educators potentially run into the issue of student interaction being less than ideal, both in the number of members as well as the number of posts related to topics in their class groups.7 However, it does not mean that the students who used those particular groups did not appreciate educators’ efforts in creating a Facebook group to enhance learning.8 Within the short lifespan of Facebook, students have taken the initiative to utilize the social networking site for educational purposes; faculty members have seen this and are responding positively in order to maximize learning in this increasingly open and social world.9 However, the use of social media and online networking tools has not been completely positive.10 Overall, social media and privacy in the public setting still seems to be a gray area in the legal as well as ethical sense.11 With all the technology available today, educators seek more efficient and captivating ways to reach students and help reinforce key concepts without getting distracted. Learning management systems such as Blackboard® and Sakai® also provide discussion boards in different formats, but they are often not as easily accessible as social media platforms. Piazza® (Piazza Technologies, Inc., Palo Alto, CA) is a free online searchable platform created specifically for productive academic discussions. The site offers a comprehensive toolkit and a more systematic way to facilitate academic discussions online and outside of the classroom. From the instructor's view, all class pages in Piazza® include three tabs: a question and answer (Q & A) panel, a course page, and the instructor-access-only class management control panel. The course page can be customized by posting announcements, course information, syllabi, contact information, and other course resources, such as lecture files, homework assignments, and other beneficial supplemental links. Announcements could be made to bypass the course email list and reach the entire class instantaneously. The class is able to sign up via a custom weblink. It also has an option to automatically detect and log in to the platform whenever a user is logged in to Facebook via “Facebook Connect” option. This option is convenient and safe as it does not borrow any more information other than a user's name and profile picture. It does not take control of an active Facebook profile and posts onto the profiles of other people. Piazza® is not only offered as a platform on a computer browser, but also as an application (also known as an “app”) on both Apple's iOS and Google's Android mobile operating systems. This allowed students to conveniently access this discussion forum away from their computers and “on the go.” Information posted on this discussion forum is available only to people who were enrolled in the course on this platform. Students can also maintain anonymous to rest of the class. Course administrator manages everyone's access to maintain the integrity of the content. In this study, Piazza®12 was utilized to create a class page, invite a class with the custom signup link, and post questions regarding various topics for the class to answer. The major objective of this study was to facilitate active academic discussions using an online, education-centered platform to reinforce important concepts taught in a Pharm.D. course in order to improve overall course outcomes. Methods Pharmacotherapy courses offered at the Texas A & M University Health Science Center College of Pharmacy integrate pathophysiology, pharmacology, medicinal chemistry, and pharmacotherapy within eight different modules covered over students’ second and third professional years. The active learning experiment was implemented for third year professional pharmacy students within the Integrated Pharmacotherapy (IPT VI) course during the fall 2012 semester. Integrated Pharmacotherapy VI (IPT VI) was a five credit-hour course including fifteen contact hours assigned to medicinal chemistry section. Topics covered included urinary incontinence, benign prostatic hyperplasia (BPH), erectile dysfunction (ED), peptic ulcer disease (PUD), gastroesophageal reflux disease (GERD), anti-inflammatory drugs, asthma, chronic obstructive pulmonary disease (COPD), allergic rhinitis, osteoarthritis (OA), rheumatoid arthritis (RA), and gout. The IPT VI course was enrolled on the Piazza® platform. On the class page, the course syllabus, contact information, scheduled pre-exam review dates, and other needed supplementary links were made available to the students. Students were requested to sign up with a legitimate university email address. Within the dynamic Q & A panel, students had the ability and the choice to ask and answer questions anonymously or not. Students could also edit responses in “wiki”-style. With this approach, each question posed can only have one collective answer. While questions were being answered and answers were being edited, the instructor “endorsed” questions and answers that were deemed acceptable. An “endorsement” on the discussion forum worked similar to the “Like” option that Facebook employs. Endorsements would signify a well-written answer. Student-answered, instructor-endorsed responses gave the students the right idea about topics discussed in class. All the activity on the class page was offered on a real-time basis.14 A preexam review session held on this forum, was scheduled a few days before each exam. During these pre-exam review sessions, the 2
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
Fig. 1. A screen shot of an instructor question and answers provided and modified by students and the instructor.
instructor led the discussion by posting learning objectives, relevant to the upcoming exam, in the form of various questions. Students had the opportunity to further understand what each objective meant, within the context of each topic, through discussions; to encourage the positive direction of students’ learning. The instructor verified student answer submissions by clicking the "good answer" button to confirm that the answer was correct. Based on the completeness of the discussion for each learning objective posted, the instructor or students had the opportunity to lead follow-up discussion by posing additional questions until the question in discussion was fully resolved. A screenshot of the typical Piazza® discussion is shown in Fig. 1. A “Class Report” and “Class Statistics” could be obtained from this discussion forum. The class report offers a graph of activity from the date of class inception to the present, a list of the top contributors in the class, and several other statistics related to the number of questions, contributions, views, whether questions were answered or not, and average response times. The class report is brief and can be used as an attention-grabbing display about how the class was doing in terms of utilizing discussion page in their studies. An example of a class report is demonstrated in Fig. 2. The class statistics included a much more detailed version of the class report, offering a commaseparated value (CSV) file to download and use in various statistical analysis programs. At the end of the module, student perceptions and preferences towards this type of online discussion were obtained via a web-based survey instrument. The survey was created using
Fig. 2. Piazza® report for Integrated Pharmacotherapy (IPT) VI course.
3
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
Table 1 Student demographics of the Fall 2012, 2011 and 2010 classes.
Class size Average age (yrs) Age range (yrs) GPA Gender (%) Male Female Ethnicity (%) Asian Black Hispanic White Other
Fall 2011
Fall 2010
Fall 2012
79 24 18–44 3.27
89 23 18–35 3.39
88 24 19–37 3.34
52 48
53 47
51 49
34 6 25 33 2
37 2 28 31 2
28 6 41 26 1
SurveyMonkey®, an online custom survey website (www.surveymonkey.com, Palo Alto, CA). This study was exempted by the Institutional Review Board (IRB) of Texas A & M University. A student t-test was used to compare class performance with those of previous years. Results In order to compare the effectiveness of implementing structured discussion forum as a supplemental study resource, student performance was analyzed through course assessments as well as student perceptions of the platform through a survey. The medicinal chemistry portion of the course was assessed through four major examinations. Examination 1 covered peptic ulcer disease; examination 2 covered arthritis and gout; examination 3 covered asthma, chronic obstructive pulmonary disease, and allergic rhinitis; and examination 4 covered benign prostatic hyperplasia, erectile dysfunction, and urinary incontinence. All exams involved only multiple-choice questions and included questions from pathophysiology, pharmacology, medicinal chemistry, and clinical sciences for the disease that was being assessed. Since only the medicinal chemistry portion of the course was discussed, the medicinal chemistry portions of all exams were separated to compare student performance of the fall 2012 class, where active learning discussion forum was used, with fall 2010 and 2011 offerings. The content that was covered in each exam was the same in all three years. Since the learning objectives did not change during these years, true comparisons between years were possible. Student demographics showed similar average class grade point average (GPA) between these classes. Gender and ethnicity were similar between classes; however, the Hispanic population was higher and Asian population was lower in 2012 class compared to 2010 and 2011 (Table 1). The Fall 2010 class did not use any active learning outside of the Socratic Method of teaching. The Fall 2011 class received an active learning exercise through student written exam questions in a systematic way.13 In the current study, for the Fall 2012 class, a structured online discussion forum (Piazza®) was used as an additional learning tool. Student performance of the Fall 2012 class was compared to the previous two years and the results are shown in Table 2. A student t-test was used to compare student performance. Student utilization of the discussion platform is shown in Fig. 2. Student perceptions of the usefulness of online discussions, with regards to their studying, and personal preferences on Piazza® were obtained via a customized survey developed using SurveyMonkey. The rating scale of 1–5 was used representing “strongly disagree” to “strongly agree” (Table 3). A custom survey employed to the students also included an open-response item to find out in what other ways this online gathering place helped their learning and provide any additional suggestions to improve this activity in the future. This survey was conducted after all examinations were completed. Discussion Several studies have showed better student learning outcomes when basic sciences were integrated with clinical sciences, or basic sciences were made clinically relevant.15,16 Need and importance of basic sciences for making clinically sound decisions has been identified in the literature.17 Regardless, there remains a perception that basic sciences are not relevant in day-to-day clinical Table 2 Average percentage student grades in medicinal chemistry section of all exams in IPT VI course during fall 2010, 2011 and 2012.
All exams Exam 1 Exam 2 Exam 3 Exam 4
Fall 2010
Fall 2011
Fall 2012
71.6 67.5 71.9 71.1 87.9
88.2 81.8 90.1 88.2 92.7
88.4 88.9 87.0 88.0 89.8
(67) (8) (19) (24) (16)
Numbers within the parenthesis indicate the number of questions in each corresponding exam.
4
(64) (9) (26) (16) (13)
(72) (10) (29) (18) (15)
5
2 (3.5%) 2 (3.5%) 8 (14.0%)
3 (5.3%) 1 (1.8%) 4 (7.0%)
6 (10.5%) 5 (8.8%) 10 (17.5%) 1 (1.8%) 7 (12.3%) 1 (1.8%)
(10.5%) (1.8%) (1.8%) (1.8%) (3.5%)
Disagree
2 (3.5%)
6 1 1 1 2
Strongly disagree
Students’ responses, no. (%)a
4 (7.0%) 3 (5.3%) 12 (21.1%)
3 (5.3%)
13 (22.8%) 10 (17.5%) 9 (15.8%) 1 (1.8%) 15 (26.3%)
Neutral
(36.8%) (50.9%) (49.1%) (43.9%) (40.4%)
30 (52.6%) 16 (28.1%) 17 (29.8%)
23 (40.4%)
21 29 28 25 23
Agree
18 (31.6%) 35 (61.4%) 16 (28.1%)
28 (49.1%)
11 (19.3%) 12 (21.1%) 9 (15.8%) 29 (50.9%) 10 (17.5%)
Strongly agree
N=57 a The number under the rating column indicates the number of students who picked that option and the number within the parentheses is the corresponding percentage of students.
I enjoyed using Piazza® for discussing subject content. The provided instructions on how to use Piazza® were easy to follow. Piazza's® layout was organized in a way to help me foster productive discussion. Comments provided by the faculty were helpful in understanding the material better. The accessibility of Piazza® on multiple platforms and devices was beneficial to studying at any time. Discussion within Piazza® helped me reinforce important concepts taught in class lectures and provided me guidance about the concepts that I needed to focus on. I was able to understand discussion of complex material on an online format. Pre-exam reviews helped me perform better on exams. I would recommend the use of Piazza® for future class discussions, or see it used by other professors/students to use.
Statement
Table 3 Student survey on learning after utilizing Piazza® as a supplemental platform to improve their education.
4.0 4.4 3.6
4.3
3.4 3.8 3.6 4.4 3.6
Mean
4 5 4
4
4 4 4 5 4
Median
S. Kolluru, J.T. Varughese
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
practice. As a result, there is a tendency for reduced student interest in basic sciences courses of pharmacy curricula. Nevertheless, the role and need for basic sciences knowledge has increased tremendously in health care delivery in recent years, especially with the future of medicine focusing on the provision of personalized medicine.18 It requires current students to graduate with even stronger foundations in basic sciences to make better clinical decisions with scientific merit based on the best available knowledge for better patient outcomes. Educators are trying various active learning strategies to engage students more with the course content for improved student learning and retention.5 There have been a few reported active learning techniques13,16 and an assessment model19 that have helped improve student interest in the importance of basic sciences in pharmacy practice. The major objective of the current study was to find a way to improve student learning by reinforcing important concepts taught in the medicinal chemistry component of a pharmacotherapeutics course. Using the online discussion platform, pharmacy students had another supplement to study with. Although only 15 medicinal chemistry lectures were used for this particular active-learning exercise, interesting data were collected with regard to the relationship between the use of the platform and student performance. Structured online discussions were well-received by the students. Ultimately, it was observed that the students were able to learn and retain medicinal chemistry topics better with the use of this unique online platform. Similar improvement in student learning was observed with 2011 student cohort where another active learning strategy was used.13 A significant number of students, in general, were reticent to be the first ones to post questions to initiate discussion. In order to increase student engagement, during scheduled pre-exam discussion sessions, the instructor posted open-response questions derived from the detailed learning objectives that were provided to students at the beginning of each lecture. Based on instructor's observations, students were lot more receptive to answer and further discuss the questions posted by the instructor. The instructor made sure to cover most, if not all, learning objectives provided during these discussions with the expectation of improved learning and retention by the students. As determined by the student t-test, Fall 2012 students showed significant improvement in their exam performance compared to the Fall 2010 class where no active learning instruction was used (p=0.001). Mean class performance was close to a letter grade higher in all examinations (Table 2) than the Fall 2010 class and is comparable to Fall 2011, where another active learning strategy was employed (Fig. 3). The Piazza® report in Fig. 2 shows that, overall, a total of 69 questions were posted online to discuss the medicinal chemistry portion of this IPT course. Either student or instructor, or both, responded to 99% of the posted questions, totaling 303 contributions to the page. The entire group together spent around 546 minutes on this discussion page apart from the regular class room time. Ninety-four percent of the questions received student responses indicating significant student involvement in the discussions. Students worked collaboratively to answer questions in wiki style, so that each question had only one collaborative answer with multiple edits by different members of the class. Eighty-nine percent of student responses were endorsed by the instructor for correct and complete answers. If the answer was not complete, follow up discussions were continued until the question in discussion was completely resolved. This provided students personal exploration of the content. Only about 39% of the class was actively involved in asking and answering questions; however, the rest of the group was following the discussion, which was evident from the large number of views for every question. Students appreciated the way this discussion page was set up by utilizing the learning objectives to lead the discussion. It helped them understand the learning objectives clearly and perform better on the exams, as exam questions were derived from one or more of learning objectives provided. Because of this online discussion, the number of student visits during office hours dropped significantly. Because it was a common discussion page where all students can view each other's contributions, question redundancy was also reduced. Students had an opportunity to ask further details if the concept in question was not clearly addressed by the fellow classmates or a faculty. However, some faculty members felt that it was a challenge to answer student questions or endorse their answers during non-working hours. In order to address this challenge, faculty scheduled online office hours to answer questions on the discussion page. Of course, there were still times where faculty needed to be online away from office hours which may not acceptable to all faculty. But, the time spent here may be approximately close to the time a faculty member takes to answer student questions through e-mails. From the survey conducted to obtain student perceptions of the activity, most students liked online structured discussion platform as an additional study tool to aid in their learning. Sixty-six percent of the Fall 2012 class participated in the survey. Majority of the students indicated that they enjoyed using discussion page because of its online format for discussing more complex medicinal chemistry material
Fig. 3. Student performance in the course examination during fall 2012, 2011 and 2010.
6
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
of different drugs for several disease states from IPT VI. They also liked the comments made by the professor to clarify student answers as it helped them understand the material better. It helped students reinforce important concepts taught in class lectures and provided them the right guidance about the concepts that they needed to focus on, not only for exams, but for the long term learning. Some of the limitations of the study may include that this discussion platform was adopted by only one instructor of the course corresponding to 15 contact hours in a five-credit-hour course. Also, it was implemented in only one course and at a single university. Students were lot more actively discussing and asking questions few days before each exam which may be challenging for faculty take significant additional time during that time due to other commitments. It would be interesting to see similar benefits to students if implemented in multiple courses and across universities and disciplines. Conclusion An online discussion platform, Piazza®, designed to facilitate academic discussions, was implemented within a medicinal chemistry portion of an integrated pharmacotherapy course. Based on instructor's observation, students were more actively engaged in discussions outside the classroom compared to previous years. This active learning strategy improved grades and students perceived that it reinforced the important concepts of the lectures. This online platform can be adopted to any course type and across curricula. Conflict of interest None. Acknowledgements The authors would like to thank Piazza Technologies (www.piazza.com) for the support and service in executing this project. References 1. Accreditation Council for Pharmacy Education: Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (“Standards. v2.0”). Published January 2011. Available at: 〈https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf〉. Accessed January 21, 2017. 2. Monaghan MS, Cain J, Malone PM, et al. Educational technology use among US colleges and schools of pharmacy. Am J Pharm Educ. 2011;75(5) Article 87. 3. Cain J, Black EP, Rohr J. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ. 2009;73(2) Article 21. 4. Liu FC, Gettig JP, Fjortoft N. Impact of a student response system on short- and long-term learning in a drug literature evaluation course. Am J Pharm Educ. 2010;74(1) Article 6. 5. Stewart DW, Brown SD, Clavier CW, Wyatt J. Am J Pharm Educ. 2011;75(4) Article 68. 6. Active Learning. University of Michigan College of Pharmacy website. Available at: 〈https://pharmacy.umich.edu/pharmd/active-learning〉 Accessed January 21, 2017. 7. Cain J, Policastri A. Using Facebook as an informal learning environment. Active-learning processes used in US pharmacy education.. Am J Pharm Educ. 2011;75(10) Article 207. 8. Cain J. Online social networking issues within academia and pharmacy education. Am J Pharm Educ. 2008;72(1) Article 10. 9. Cain J, Fox BI. Web 2.0 and pharmacy education. Am J Pharm Educ. 2009;3(7) Article 120. 10. Cain J, Scott DR, Akers P. Pharmacy students' Facebook activity and opinions regarding accountability and e-professionalism. Am J Pharm Educ. 2009;73(6) Article 104. 11. Cain J, Fink JL. III. Legal and ethical issues regarding social media and pharmacy education. Am J Pharm Educ. 2010;74(10) Article 184. 12. Piazza Technologies, Inc., Available at: 〈http://www.piazza.com〉; Accessed January 21, 2017. 13. Kolluru S. An active-learning assignment requiring pharmacy students to write medicinal chemistry examination questions. Am J Pharm Educ. 2012;76(6) Article 112. 14. Piazza WhyWorks. Piazza Technologies, Inc., website. Available at: 〈https://piazza.com/why-piazza-works〉; Accessed January 21, 2017. 15. Alsharif NZ, Galt KA. Evaluation of an instructional model to teach clinically relevant medicinal chemistry in a campus and a distance pathway. Am J Pharm Educ. 2008;72(2) Article 31. 16. Kolluru S, Roesch DM, De la Fuente AA. A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content. Am J Pharm Educ. 2012;76(2) Article 33. 17. Woster PM. Maintaining basic science content throughout the PharmD curriculum. Am J Pharm Educ. 2013;67(3) Article 99. 18. Shastry BS. Pharmacogenetics and the concept of individualized medicine. Pharmacogenomics J. 2006;6(1):16–21. 19. Kolluru S, Lemke TL. An assessment model for multidisciplinary, team-taught integrated pharmacy courses. Am J Pharm Educ. 2012;76(8) Article 157.
7
Contents lists available at ScienceDirect
Currents in Pharmacy Teaching and Learning journal homepage: www.elsevier.com/locate/cptl
Research Article
Structured academic discussions through an online education-specific platform to improve Pharm.D. students learning outcomes ⁎
Srikanth Kollurua, , James T. Varugheseb a b
Keck Graduate Institute School of Pharmacy, Claremont, CA 91739 Medicare & Retirement Appeals & Grievances United Healthcare, Sugar Land, TX 77478
A R T I C L E I N F O
ABSTRACT
Keywords: Active learning Online academic discussions Piazza Social media
Objective: To facilitate active academic discussions using an online, education-centered platform and reinforce concepts, in order to improve overall course outcomes. Materials and methods: A third year integrated pharmacotherapy course was enrolled on an online searchable platform, Piazza®, to facilitate academic discussions. Students could ask, answer, and explore content, and build on submitted answers in wiki style in collaboration. Instructor posted learning objectives, endorsed student responses with correct answers and led follow-up discussions. Review sessions were conducted on this platform before all major exams. A student t-test was used to compare class performance with those of previous years. Results: In a post-activity qualitative survey, most students appreciated the less stressful, online interaction with peers and faculty. For 15 medicinal chemistry course hours, there were 83 posts on Piazza® with 303 total contributions, 107 student responses, and 546 min of group discussion time. 94% of questions received student responses and 89% of those were endorsed by the instructor. Students enjoyed pre-exam discussions, organization of the page, and reinforcing material on complex learning objectives. This discussion forum fostered personal exploration of content by the students, which led to better performance on examinations. Conclusions: Involving the use of an online, education-centered platform for student discussions was an effective means of increasing class engagement with the course material. Student performance on exams was significantly improved in both cohorts that utilized active learning compared to the cohort without active learning (p=0.001 and p= 0.002 respectively). Piazza® can be utilized for any course and across disciplines.
Introduction Pharmacy educators are always seeking the most innovative ways to reach out to students, both in and out of the classroom, to enhance learning and retention of the material necessary for today's demanding health care needs. Technology is evolving at a rapid pace, and the speed at which the current generation of students adapting is just as quick. Guideline 11.2 of the 2007 Accreditation Council for Pharmacy Education (ACPE) Standards calls for the continual improvement of students’ critical thinking and problemsolving skills through active learning strategies throughout every curriculum.1 A variety of technologies are employed at all pharmacy schools in the United States (US) including audience response systems (ARS), learning management systems (LMS), web conferencing,
⁎
Corresponding author at: Keck Graduate Institute School of Pharmacy, Claremont, CA 91739. E-mail addresses: [email protected] (S. Kolluru), [email protected] (J.T. Varughese).
http://dx.doi.org/10.1016/j.cptl.2016.11.022
1877-1297/ © 2016 Elsevier Inc. All rights reserved.
Please cite this article as: Kolluru, S., Currents in Pharmacy Teaching and Learning (2016), http://dx.doi.org/10.1016/j.cptl.2016.11.022
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
electronic testing software, and blogging tools. This is not only to fulfill all requirements from ACPE, but also help students to graduate as pharmacists who will be better-prepared to use all types of technologies and have better pharmacy knowledge.2 Utilizing certain types of technologies have shown to improve student motivation and attention during lectures. For instance, TurningPoint® (Turning Technologies; Youngstown, OH), an audience response system, provides instantaneous feedback to the instructor with regards to student understanding of the material.3,4 A recent survey showed that 87% percent of US pharmacy schools are employing some form of active learning techniques in their curriculam.5 A variety of active learning techniques are used based on the curricular needs and the learning objectives.6 The internet has evolved a great deal in a short time and all types of social media have diffused throughout the society to connect us even further. Newer options, such as Twitter (Twitter, Inc., San Francisco, CA) and Google+ (Google, Inc.; Mountain View, CA), offer their own benefits. Facebook (Facebook, Inc., Palo Alto, CA), a very popular social media site, continues to grow and has been studied in various settings to determine how best use the networking site as a beneficial tool in education.7 With Facebook, the ability to create a personal profile, connect with others via “friending” as well as join specialized “groups,” and discuss various topics within those groups is a key to the optimal interaction between students and faculty. Utilizing Facebook as an informal learning environment, educators potentially run into the issue of student interaction being less than ideal, both in the number of members as well as the number of posts related to topics in their class groups.7 However, it does not mean that the students who used those particular groups did not appreciate educators’ efforts in creating a Facebook group to enhance learning.8 Within the short lifespan of Facebook, students have taken the initiative to utilize the social networking site for educational purposes; faculty members have seen this and are responding positively in order to maximize learning in this increasingly open and social world.9 However, the use of social media and online networking tools has not been completely positive.10 Overall, social media and privacy in the public setting still seems to be a gray area in the legal as well as ethical sense.11 With all the technology available today, educators seek more efficient and captivating ways to reach students and help reinforce key concepts without getting distracted. Learning management systems such as Blackboard® and Sakai® also provide discussion boards in different formats, but they are often not as easily accessible as social media platforms. Piazza® (Piazza Technologies, Inc., Palo Alto, CA) is a free online searchable platform created specifically for productive academic discussions. The site offers a comprehensive toolkit and a more systematic way to facilitate academic discussions online and outside of the classroom. From the instructor's view, all class pages in Piazza® include three tabs: a question and answer (Q & A) panel, a course page, and the instructor-access-only class management control panel. The course page can be customized by posting announcements, course information, syllabi, contact information, and other course resources, such as lecture files, homework assignments, and other beneficial supplemental links. Announcements could be made to bypass the course email list and reach the entire class instantaneously. The class is able to sign up via a custom weblink. It also has an option to automatically detect and log in to the platform whenever a user is logged in to Facebook via “Facebook Connect” option. This option is convenient and safe as it does not borrow any more information other than a user's name and profile picture. It does not take control of an active Facebook profile and posts onto the profiles of other people. Piazza® is not only offered as a platform on a computer browser, but also as an application (also known as an “app”) on both Apple's iOS and Google's Android mobile operating systems. This allowed students to conveniently access this discussion forum away from their computers and “on the go.” Information posted on this discussion forum is available only to people who were enrolled in the course on this platform. Students can also maintain anonymous to rest of the class. Course administrator manages everyone's access to maintain the integrity of the content. In this study, Piazza®12 was utilized to create a class page, invite a class with the custom signup link, and post questions regarding various topics for the class to answer. The major objective of this study was to facilitate active academic discussions using an online, education-centered platform to reinforce important concepts taught in a Pharm.D. course in order to improve overall course outcomes. Methods Pharmacotherapy courses offered at the Texas A & M University Health Science Center College of Pharmacy integrate pathophysiology, pharmacology, medicinal chemistry, and pharmacotherapy within eight different modules covered over students’ second and third professional years. The active learning experiment was implemented for third year professional pharmacy students within the Integrated Pharmacotherapy (IPT VI) course during the fall 2012 semester. Integrated Pharmacotherapy VI (IPT VI) was a five credit-hour course including fifteen contact hours assigned to medicinal chemistry section. Topics covered included urinary incontinence, benign prostatic hyperplasia (BPH), erectile dysfunction (ED), peptic ulcer disease (PUD), gastroesophageal reflux disease (GERD), anti-inflammatory drugs, asthma, chronic obstructive pulmonary disease (COPD), allergic rhinitis, osteoarthritis (OA), rheumatoid arthritis (RA), and gout. The IPT VI course was enrolled on the Piazza® platform. On the class page, the course syllabus, contact information, scheduled pre-exam review dates, and other needed supplementary links were made available to the students. Students were requested to sign up with a legitimate university email address. Within the dynamic Q & A panel, students had the ability and the choice to ask and answer questions anonymously or not. Students could also edit responses in “wiki”-style. With this approach, each question posed can only have one collective answer. While questions were being answered and answers were being edited, the instructor “endorsed” questions and answers that were deemed acceptable. An “endorsement” on the discussion forum worked similar to the “Like” option that Facebook employs. Endorsements would signify a well-written answer. Student-answered, instructor-endorsed responses gave the students the right idea about topics discussed in class. All the activity on the class page was offered on a real-time basis.14 A preexam review session held on this forum, was scheduled a few days before each exam. During these pre-exam review sessions, the 2
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
Fig. 1. A screen shot of an instructor question and answers provided and modified by students and the instructor.
instructor led the discussion by posting learning objectives, relevant to the upcoming exam, in the form of various questions. Students had the opportunity to further understand what each objective meant, within the context of each topic, through discussions; to encourage the positive direction of students’ learning. The instructor verified student answer submissions by clicking the "good answer" button to confirm that the answer was correct. Based on the completeness of the discussion for each learning objective posted, the instructor or students had the opportunity to lead follow-up discussion by posing additional questions until the question in discussion was fully resolved. A screenshot of the typical Piazza® discussion is shown in Fig. 1. A “Class Report” and “Class Statistics” could be obtained from this discussion forum. The class report offers a graph of activity from the date of class inception to the present, a list of the top contributors in the class, and several other statistics related to the number of questions, contributions, views, whether questions were answered or not, and average response times. The class report is brief and can be used as an attention-grabbing display about how the class was doing in terms of utilizing discussion page in their studies. An example of a class report is demonstrated in Fig. 2. The class statistics included a much more detailed version of the class report, offering a commaseparated value (CSV) file to download and use in various statistical analysis programs. At the end of the module, student perceptions and preferences towards this type of online discussion were obtained via a web-based survey instrument. The survey was created using
Fig. 2. Piazza® report for Integrated Pharmacotherapy (IPT) VI course.
3
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
Table 1 Student demographics of the Fall 2012, 2011 and 2010 classes.
Class size Average age (yrs) Age range (yrs) GPA Gender (%) Male Female Ethnicity (%) Asian Black Hispanic White Other
Fall 2011
Fall 2010
Fall 2012
79 24 18–44 3.27
89 23 18–35 3.39
88 24 19–37 3.34
52 48
53 47
51 49
34 6 25 33 2
37 2 28 31 2
28 6 41 26 1
SurveyMonkey®, an online custom survey website (www.surveymonkey.com, Palo Alto, CA). This study was exempted by the Institutional Review Board (IRB) of Texas A & M University. A student t-test was used to compare class performance with those of previous years. Results In order to compare the effectiveness of implementing structured discussion forum as a supplemental study resource, student performance was analyzed through course assessments as well as student perceptions of the platform through a survey. The medicinal chemistry portion of the course was assessed through four major examinations. Examination 1 covered peptic ulcer disease; examination 2 covered arthritis and gout; examination 3 covered asthma, chronic obstructive pulmonary disease, and allergic rhinitis; and examination 4 covered benign prostatic hyperplasia, erectile dysfunction, and urinary incontinence. All exams involved only multiple-choice questions and included questions from pathophysiology, pharmacology, medicinal chemistry, and clinical sciences for the disease that was being assessed. Since only the medicinal chemistry portion of the course was discussed, the medicinal chemistry portions of all exams were separated to compare student performance of the fall 2012 class, where active learning discussion forum was used, with fall 2010 and 2011 offerings. The content that was covered in each exam was the same in all three years. Since the learning objectives did not change during these years, true comparisons between years were possible. Student demographics showed similar average class grade point average (GPA) between these classes. Gender and ethnicity were similar between classes; however, the Hispanic population was higher and Asian population was lower in 2012 class compared to 2010 and 2011 (Table 1). The Fall 2010 class did not use any active learning outside of the Socratic Method of teaching. The Fall 2011 class received an active learning exercise through student written exam questions in a systematic way.13 In the current study, for the Fall 2012 class, a structured online discussion forum (Piazza®) was used as an additional learning tool. Student performance of the Fall 2012 class was compared to the previous two years and the results are shown in Table 2. A student t-test was used to compare student performance. Student utilization of the discussion platform is shown in Fig. 2. Student perceptions of the usefulness of online discussions, with regards to their studying, and personal preferences on Piazza® were obtained via a customized survey developed using SurveyMonkey. The rating scale of 1–5 was used representing “strongly disagree” to “strongly agree” (Table 3). A custom survey employed to the students also included an open-response item to find out in what other ways this online gathering place helped their learning and provide any additional suggestions to improve this activity in the future. This survey was conducted after all examinations were completed. Discussion Several studies have showed better student learning outcomes when basic sciences were integrated with clinical sciences, or basic sciences were made clinically relevant.15,16 Need and importance of basic sciences for making clinically sound decisions has been identified in the literature.17 Regardless, there remains a perception that basic sciences are not relevant in day-to-day clinical Table 2 Average percentage student grades in medicinal chemistry section of all exams in IPT VI course during fall 2010, 2011 and 2012.
All exams Exam 1 Exam 2 Exam 3 Exam 4
Fall 2010
Fall 2011
Fall 2012
71.6 67.5 71.9 71.1 87.9
88.2 81.8 90.1 88.2 92.7
88.4 88.9 87.0 88.0 89.8
(67) (8) (19) (24) (16)
Numbers within the parenthesis indicate the number of questions in each corresponding exam.
4
(64) (9) (26) (16) (13)
(72) (10) (29) (18) (15)
5
2 (3.5%) 2 (3.5%) 8 (14.0%)
3 (5.3%) 1 (1.8%) 4 (7.0%)
6 (10.5%) 5 (8.8%) 10 (17.5%) 1 (1.8%) 7 (12.3%) 1 (1.8%)
(10.5%) (1.8%) (1.8%) (1.8%) (3.5%)
Disagree
2 (3.5%)
6 1 1 1 2
Strongly disagree
Students’ responses, no. (%)a
4 (7.0%) 3 (5.3%) 12 (21.1%)
3 (5.3%)
13 (22.8%) 10 (17.5%) 9 (15.8%) 1 (1.8%) 15 (26.3%)
Neutral
(36.8%) (50.9%) (49.1%) (43.9%) (40.4%)
30 (52.6%) 16 (28.1%) 17 (29.8%)
23 (40.4%)
21 29 28 25 23
Agree
18 (31.6%) 35 (61.4%) 16 (28.1%)
28 (49.1%)
11 (19.3%) 12 (21.1%) 9 (15.8%) 29 (50.9%) 10 (17.5%)
Strongly agree
N=57 a The number under the rating column indicates the number of students who picked that option and the number within the parentheses is the corresponding percentage of students.
I enjoyed using Piazza® for discussing subject content. The provided instructions on how to use Piazza® were easy to follow. Piazza's® layout was organized in a way to help me foster productive discussion. Comments provided by the faculty were helpful in understanding the material better. The accessibility of Piazza® on multiple platforms and devices was beneficial to studying at any time. Discussion within Piazza® helped me reinforce important concepts taught in class lectures and provided me guidance about the concepts that I needed to focus on. I was able to understand discussion of complex material on an online format. Pre-exam reviews helped me perform better on exams. I would recommend the use of Piazza® for future class discussions, or see it used by other professors/students to use.
Statement
Table 3 Student survey on learning after utilizing Piazza® as a supplemental platform to improve their education.
4.0 4.4 3.6
4.3
3.4 3.8 3.6 4.4 3.6
Mean
4 5 4
4
4 4 4 5 4
Median
S. Kolluru, J.T. Varughese
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
practice. As a result, there is a tendency for reduced student interest in basic sciences courses of pharmacy curricula. Nevertheless, the role and need for basic sciences knowledge has increased tremendously in health care delivery in recent years, especially with the future of medicine focusing on the provision of personalized medicine.18 It requires current students to graduate with even stronger foundations in basic sciences to make better clinical decisions with scientific merit based on the best available knowledge for better patient outcomes. Educators are trying various active learning strategies to engage students more with the course content for improved student learning and retention.5 There have been a few reported active learning techniques13,16 and an assessment model19 that have helped improve student interest in the importance of basic sciences in pharmacy practice. The major objective of the current study was to find a way to improve student learning by reinforcing important concepts taught in the medicinal chemistry component of a pharmacotherapeutics course. Using the online discussion platform, pharmacy students had another supplement to study with. Although only 15 medicinal chemistry lectures were used for this particular active-learning exercise, interesting data were collected with regard to the relationship between the use of the platform and student performance. Structured online discussions were well-received by the students. Ultimately, it was observed that the students were able to learn and retain medicinal chemistry topics better with the use of this unique online platform. Similar improvement in student learning was observed with 2011 student cohort where another active learning strategy was used.13 A significant number of students, in general, were reticent to be the first ones to post questions to initiate discussion. In order to increase student engagement, during scheduled pre-exam discussion sessions, the instructor posted open-response questions derived from the detailed learning objectives that were provided to students at the beginning of each lecture. Based on instructor's observations, students were lot more receptive to answer and further discuss the questions posted by the instructor. The instructor made sure to cover most, if not all, learning objectives provided during these discussions with the expectation of improved learning and retention by the students. As determined by the student t-test, Fall 2012 students showed significant improvement in their exam performance compared to the Fall 2010 class where no active learning instruction was used (p=0.001). Mean class performance was close to a letter grade higher in all examinations (Table 2) than the Fall 2010 class and is comparable to Fall 2011, where another active learning strategy was employed (Fig. 3). The Piazza® report in Fig. 2 shows that, overall, a total of 69 questions were posted online to discuss the medicinal chemistry portion of this IPT course. Either student or instructor, or both, responded to 99% of the posted questions, totaling 303 contributions to the page. The entire group together spent around 546 minutes on this discussion page apart from the regular class room time. Ninety-four percent of the questions received student responses indicating significant student involvement in the discussions. Students worked collaboratively to answer questions in wiki style, so that each question had only one collaborative answer with multiple edits by different members of the class. Eighty-nine percent of student responses were endorsed by the instructor for correct and complete answers. If the answer was not complete, follow up discussions were continued until the question in discussion was completely resolved. This provided students personal exploration of the content. Only about 39% of the class was actively involved in asking and answering questions; however, the rest of the group was following the discussion, which was evident from the large number of views for every question. Students appreciated the way this discussion page was set up by utilizing the learning objectives to lead the discussion. It helped them understand the learning objectives clearly and perform better on the exams, as exam questions were derived from one or more of learning objectives provided. Because of this online discussion, the number of student visits during office hours dropped significantly. Because it was a common discussion page where all students can view each other's contributions, question redundancy was also reduced. Students had an opportunity to ask further details if the concept in question was not clearly addressed by the fellow classmates or a faculty. However, some faculty members felt that it was a challenge to answer student questions or endorse their answers during non-working hours. In order to address this challenge, faculty scheduled online office hours to answer questions on the discussion page. Of course, there were still times where faculty needed to be online away from office hours which may not acceptable to all faculty. But, the time spent here may be approximately close to the time a faculty member takes to answer student questions through e-mails. From the survey conducted to obtain student perceptions of the activity, most students liked online structured discussion platform as an additional study tool to aid in their learning. Sixty-six percent of the Fall 2012 class participated in the survey. Majority of the students indicated that they enjoyed using discussion page because of its online format for discussing more complex medicinal chemistry material
Fig. 3. Student performance in the course examination during fall 2012, 2011 and 2010.
6
Currents in Pharmacy Teaching and Learning (xxxx) xxxx–xxxx
S. Kolluru, J.T. Varughese
of different drugs for several disease states from IPT VI. They also liked the comments made by the professor to clarify student answers as it helped them understand the material better. It helped students reinforce important concepts taught in class lectures and provided them the right guidance about the concepts that they needed to focus on, not only for exams, but for the long term learning. Some of the limitations of the study may include that this discussion platform was adopted by only one instructor of the course corresponding to 15 contact hours in a five-credit-hour course. Also, it was implemented in only one course and at a single university. Students were lot more actively discussing and asking questions few days before each exam which may be challenging for faculty take significant additional time during that time due to other commitments. It would be interesting to see similar benefits to students if implemented in multiple courses and across universities and disciplines. Conclusion An online discussion platform, Piazza®, designed to facilitate academic discussions, was implemented within a medicinal chemistry portion of an integrated pharmacotherapy course. Based on instructor's observation, students were more actively engaged in discussions outside the classroom compared to previous years. This active learning strategy improved grades and students perceived that it reinforced the important concepts of the lectures. This online platform can be adopted to any course type and across curricula. Conflict of interest None. Acknowledgements The authors would like to thank Piazza Technologies (www.piazza.com) for the support and service in executing this project. References 1. Accreditation Council for Pharmacy Education: Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (“Standards. v2.0”). Published January 2011. Available at: 〈https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf〉. Accessed January 21, 2017. 2. Monaghan MS, Cain J, Malone PM, et al. Educational technology use among US colleges and schools of pharmacy. Am J Pharm Educ. 2011;75(5) Article 87. 3. Cain J, Black EP, Rohr J. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ. 2009;73(2) Article 21. 4. Liu FC, Gettig JP, Fjortoft N. Impact of a student response system on short- and long-term learning in a drug literature evaluation course. Am J Pharm Educ. 2010;74(1) Article 6. 5. Stewart DW, Brown SD, Clavier CW, Wyatt J. Am J Pharm Educ. 2011;75(4) Article 68. 6. Active Learning. University of Michigan College of Pharmacy website. Available at: 〈https://pharmacy.umich.edu/pharmd/active-learning〉 Accessed January 21, 2017. 7. Cain J, Policastri A. Using Facebook as an informal learning environment. Active-learning processes used in US pharmacy education.. Am J Pharm Educ. 2011;75(10) Article 207. 8. Cain J. Online social networking issues within academia and pharmacy education. Am J Pharm Educ. 2008;72(1) Article 10. 9. Cain J, Fox BI. Web 2.0 and pharmacy education. Am J Pharm Educ. 2009;3(7) Article 120. 10. Cain J, Scott DR, Akers P. Pharmacy students' Facebook activity and opinions regarding accountability and e-professionalism. Am J Pharm Educ. 2009;73(6) Article 104. 11. Cain J, Fink JL. III. Legal and ethical issues regarding social media and pharmacy education. Am J Pharm Educ. 2010;74(10) Article 184. 12. Piazza Technologies, Inc., Available at: 〈http://www.piazza.com〉; Accessed January 21, 2017. 13. Kolluru S. An active-learning assignment requiring pharmacy students to write medicinal chemistry examination questions. Am J Pharm Educ. 2012;76(6) Article 112. 14. Piazza WhyWorks. Piazza Technologies, Inc., website. Available at: 〈https://piazza.com/why-piazza-works〉; Accessed January 21, 2017. 15. Alsharif NZ, Galt KA. Evaluation of an instructional model to teach clinically relevant medicinal chemistry in a campus and a distance pathway. Am J Pharm Educ. 2008;72(2) Article 31. 16. Kolluru S, Roesch DM, De la Fuente AA. A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content. Am J Pharm Educ. 2012;76(2) Article 33. 17. Woster PM. Maintaining basic science content throughout the PharmD curriculum. Am J Pharm Educ. 2013;67(3) Article 99. 18. Shastry BS. Pharmacogenetics and the concept of individualized medicine. Pharmacogenomics J. 2006;6(1):16–21. 19. Kolluru S, Lemke TL. An assessment model for multidisciplinary, team-taught integrated pharmacy courses. Am J Pharm Educ. 2012;76(8) Article 157.
7