- Email: [email protected]
Student perceptions of and performance in a blended foundational drug information course
Available online at www.sciencedirect.com
Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
Research article
http://www.pharmacyteaching.com
Student perceptions of and performance in a blended foundational drug information course Peter J. Hughes, PharmD, BCPSa,*, Bruce Waldrop, PhD, RPhb, Jongwha Chang, PhDc b
a Department of Pharmacy Practice, McWhorter School of Pharmacy, Birmingham, AL Department of of Pharmaceutical, Social and Administrative Sciences, McWhorter School of Pharmacy, Birmingham, AL c Birmingham, AL
Abstract Objectives: Assess and trend student opinions of online educational components utilized in a redesigned 5-week foundational drug information course. Comparison of performance on the final examination, course grade, and changes in standardized university course evaluations between 2012 and 2013 were the secondary objectives. Methods: This course used narrated video instruction coupled with the use of a face-to-face weekly laboratory session. This project consisted of pre- and post-exposure surveys to allow for paired analysis of six opinion-based survey items using a fivepoint Likert scale. Secondary objectives were compared between the 2013 and 2012 entering classes by use of paired t-tests. Results: Only matched pairs were analyzed for this project (n ¼ 65 of 127 enrolled students; 51.2%). Changes between mean pre- and post-survey results indicated a decline in student favorability for traditional lecture styles (difference ¼ 0.49 points; p o 0.05) and a decline in the importance of face-to-face interaction with instructors outside class (difference ¼ 0.46 points; p o 0.05) while demonstrating increased favorability for use of online video demonstrations as an acceptable substitute for inperson demonstration of skills (difference ¼ 0.34 points; p o 0.05) and increased rating of overall preference of online learning compared to traditional modalities (difference ¼ 0.44 points; p o 0.05). Standardized course evaluations did not differ between 2012 and 2013. Mean final exam scores significantly increased from 84.86% in 2012 to 88.99% in 2013 (p o 0.05), but no difference between mean course grades (94.03% and 93.62%, respectively; p 4 0.05). Conclusion: Blended course design is an effective format for introductory skills-based professional pharmacy courses. r 2016 Elsevier Inc. All rights reserved.
Keywords: Blended learning; Drug information; Pharmacy education
Introduction Online technology can allow for efficient delivery of facts, creating time for active learning opportunities during classtime.1 Furthermore, while evidence for use of blended Abbreviations: P1, first professional year LMS, learning management system * Corresponding author: Peter J. Hughes, PharmD, BCPS, McWhorter School of Pharmacy, 800 Lakeshore Drive, Birmingham, AL 35229. E-mail: [email protected] http://dx.doi.org/10.1016/j.cptl.2016.02.013 1877-1297/r 2016 Elsevier Inc. All rights reserved.
course or curricular designs are accessible,2 use of Internetbased activities in a foundational drug information course is uncommon based upon the published literature.3,4 Due to student feedback from annual assessments and to accomplish institutional strategic objectives, the foundational drug information course for first-year (P1) pharmacy students at McWhorter School of Pharmacy, Samford University was revised into a blended learning model for fall 2013. Prior offerings of the course were conducted in a strictly face-toface manner. “Blended learning” can be defined as the combination of different pedagogical media (e.g., technology and activities)
360
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
to create an ideal training environment.5 Adjunctive media such as technology is not to be viewed as a replacement for face-to-face interaction, but rather as a supplement to this endeavor and to also promote self-directed learning in professional students.5 The primary purpose of this project was to assess and trend opinions about student learning preferences in this five-week (one semester credit hour) course. Secondary objectives compared the mean final examination score, overall mean course grade, and changes in standardized university course evaluations between the 2012 and 2013 offerings of this course.
Methods The modified blended course was a one credit-hour required course delivered during the first five weeks of the P1 pharmacy curriculum. A blended course, for the purposes of this research, is defined as a course that consists of asynchronous delivery of didactic content coupled with face-to-face active learning/laboratory exercises. The course consisted of a series of narrated videos accessed online via a learning management system (LMS, Moodles Version 2.4.5, Perth, Australia) in conjunction with face-to-face weekly laboratory sessions to reinforce important concepts presented in the videos. Using the previous year’s course learning objectives as the basis for content development, videos were produced using a combination of screen capture software (Camtasias for Mac version 2.4.0, Techsmiths, Okemos, MI), video recording of simulated interactions, and to a lesser extent, narrated slide presentations (see Appendices A and B for course learning objectives and course calendar, respectively). Since its initial offering as a face-to-face course in the fall of 2009, the calculation of the course grade for the foundational drug information course consisted of the student laboratory grade (30%), student weekly quiz grade (30%), student final examination grade (30%), and student participation/attendance (10%). Weekly online content was segmented into multiple videos of 6–12 minutes duration, in which each video focused on specific learning outcomes. On average, students watched five videos per week, occupying approximately 85 minutes of their time. The face-to-face laboratory sessions took place Monday through Thursday afternoons, with approximately 42 students per three-hour session. These sessions were designed to allow students to reinforce and apply the material that was presented in the online videos the previous week and consisted of exercises dedicated to paraphrasing and citing references, searching drug information databases, searching PubMed/Medline and answering a drug information question utilizing the systematic approach to answering drug information questions.6 Drug information questions utilized for laboratory exercises consisted of questions originally answered by the school’s drug information center in addition to simulated questions.
In order to encourage student accountability with respect to timely completion of required online course activities and assessments in a specific order, we utilized the built-in conditional release properties within the LMS. On the first viewing of each video, playback controls were disabled to ensure that students did not “fast-forward” to the end of the video. At the conclusion of each video, students were presented with a formative single-question quiz. Incorrect answers triggered the opening of a window with formative feedback, and students were allowed to reattempt the quiz. Successful completion of the quiz served as a tracking process for completion of the activity and also as a control mechanism to reveal the following video in the series. After viewing each video and correctly answering each formative quiz question for all videos in the series, a link for a timed (10 minute) summative quiz was revealed to the student. Students were able to use their notes on summative weekly quizzes. Scrambling the question order and the answer choices discouraged collaboration among students while taking the online summative quizzes. Furthermore, students were not allowed to return to a question once it was presented. Completion of the summative quiz was required for laboratory attendance. Laboratory attendance was required in the course. Summative quizzes utilized in the 2013 course offering were identical to quizzes utilized in 2012 and accounted for 30% of the total course grade. Since examinations are not returned to students (school policy), the final exam utilized in the 2013 course offering was also identical to the final exam utilized in the 2012 course, with the exception of two questions that were re-worded for the 2013 course to improve content clarity. The exam was a paper-based exam that consisted of 40 multiple-choice questions that were equally weighted at 2.5 points per question. The final examination accounted for 30% of the total course grade. During the P1 student orientation program one week prior to the commencement of classes, P1 students were asked to complete an online “pre-exposure” survey regarding previous experiences with and perceptions of online courses. Similarly, after the completion of the course, students were asked to complete a “post-exposure” survey with a similar format as the pre-exposure survey, but also included specific questions related to course outcomes and activities. Students used a unique numeric identifier which allowed for paired comparisons of the pre- and postexposure survey results. An electronic survey (Qualtricss, Provo, UT) link for the pre-exposure survey was distributed via email to the P1 class during new student orientation and the post-exposure survey link was distributed after the course final exam through the same process. A five-point Likert scale (range: 2 ¼ strongly disagree, 1 ¼ disagree, 0 ¼ neutral, þ1 ¼ agree, and þ2 ¼ strongly agree) was used for questions related to student perceptions and attitudes. Paired t-tests were used to compare pre- and post-exposure survey data and served as the primary endpoints for this study. Standardized student course evaluation results, final examination scores, and overall
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
course score were compared between the 2012 and 2013 entering classes using unpaired t-tests. These additional comparisons served as secondary endpoints. Statistical analyses were conducted using Stata/SEs (v1.3, College Station, TX). A p o 0.05 was considered significant. The Samford University Institutional Review Board (IRB) approved this research.
Results Of 127 students in the class, 115 (90%) completed the pre-exposure survey and 88 (76%) completed the postexposure survey. Among the respondents, 77% were female, 50% had previous blended learning course experience, and 59% had previous online course experience. Only pairs who completed both the pre- and post-exposure surveys were analyzed for primary endpoint in this project (n ¼ 65; 51%). Overall, paired analysis of questions regarding student learning preferences reflected favorably on the structural changes instituted in the course. Specifically, the change between mean pre- and post-survey results for opinionbased items indicated a decline in student favorability for traditional lecture styles (difference ¼ 0.49 points; p o 0.05) and importance of face-to-face interactions with instructors outside of the classroom (difference ¼
Table 1 Student learning preferences.
Statement Statement Statement Statement Statement Statement
A B C D E F
Pre-exposure survey mean score
Post-exposure survey mean score Change
1.31 0.57 1.02 0.72 0.74 0.24
0.82 0.46 0.55 0.52 1.07 0.2
0.49 0.11 0.47 0.2 þ0.33 þ0.44
p Value o0.05 NS o0.05 NS o0.05 o0.05
Survey Statement Legend A: I consider traditional lectures where the instructor presents information to the class to be an effective learning method. B: It is important to me to have interaction with other classmates face-to-face to discuss course materials. C: It is important to have face-to-face communication with the instructor outside of the classroom. D: It is important that I have the ability to ask questions during the lecture/ class period. E: An online video demonstration of skills and techniques (i.e., how to use certain Internet databases or other skills) is an acceptable substitute to inperson demonstrations in the classroom. F:I prefer online learning compared to traditional learning modalities (i.e., traditional classroom lecture). Range: 2, strongly disagree; 1, disagree; 0, no preference; 1, agree; 2, strongly agree; negative scores indicate decreased agreement; positive scores indicate increased agreement; NS, not significant.
361
0.46 points; p o 0.05), while demonstrating increased favorability for use of online video demonstrations as an acceptable substitute for in-person demonstration of skills (difference ¼ 0.34 points; p o 0.05) and overall preference of online learning compared to traditional modalities (difference ¼ 0.44 points; p o 0.05). No statistically significant differences existed for changes of opinion in survey items assessing importance of in-class interaction with classmates and importance of the ability to ask questions during class (see Table 1). The mean score for select items on the standardized course evaluations completed by students significantly differed between the 2012 and 2013 cohorts (n ¼ 107 and 106, respectively). Mean final exam scores for the 2012 and 2013 cohorts, using the same instrument, significantly increased from 84.86% to 88.99% (p o 0.05), while mean course grades were similar between the 2012 and 2013 cohorts (94.03% and 93.62%, respectively; p 4 0.05) (see Table 2). The mean overall grade point average (GPA) and math/science entering GPAs for the fall 2012 class were 3.38 and 3.40, respectively, whereas the mean overall GPA and math/science entering GPAs for the fall 2013 class were 3.45 and 3.41, respectively. Post-hoc subgroup analyses were conducted to assess for differences to select survey items on the basis of gender and previous blended learning exposure. Significant differences were observed within the female cohort for decline in favorability for traditional lecture styles (difference = 0.57 points; p o 0.05) and increased preference for online learning compared to traditional modalities (difference = 0.38 points; p o 0.05). With male counterparts, only a significant difference in opinion was noted for improved favorability of online learning compared to traditional learning modalities (difference = 0.61 points; p o 0.05). In the context of prior blended learning exposure, consistent significant differences were noted between groups with respect to a decline in favorability of traditional lecture styles [difference (blended learning) = 0.53 points; difference (no-blended learning) = 0.44 points; p o 0.05 for both groups] and a decline in favorability of face-to-face interactions with faculty members outside of the classroom [difference (blended learning) = 0.33 points; difference (no-blended learning) = 0.68 points; p o 0.05 for both groups]. Discussion Blended or hybridized teaching models have demonstrated effectiveness in a variety pharmacy educational venues, to include elective courses,7,8 introductory practice experiences,9 stand-alone didactic courses,3,10–13 as well as student remediation courses.14 To the best knowledge of the authors, the current research is only the second known attempt to document student perceptions of and performance in a hybridized P1 foundational drug information course. The only other research in this area was published by Freeman et al.4 In that study, the authors observed and measured results of the 2005 course using different
362
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
Table 2 Final Examination and Course Grade Comparison. Final examination and course grade comparison 2012 mean score (n ¼ 121)
2013 mean score (n ¼ 127)
Change
p Value
Final exam Final grade
84.87% 94.03
88.99% 93.63
þ4.12% 0.4%
o0.05 NS
Statement Statement Statement Statement
2012 mean score (n ¼ 107) 1.63 1.62 1.71 1.61
Change þ0.06 0.02 þ0.01 þ0.02
p Value o0.05 o0.05 NS o0.05
G H I J
Course evaluation comparison 2013 mean score (n ¼ 106) 1.69 1.6 1.72 1.63
Course Evaluation Statement Legend G: Course objectives and requirements were clearly communicated to me. H: The learning opportunities provided in this course helped me to comprehend the course content. I: I know how to apply the information learned in this course. J: Course topics were presented in a logical sequence. Range: 2, strongly disagree; 1, disagree; 0, no preference; 1, agree; 2, strongly agree.
methodology by assessing student perceptions of recorded lectures and perceptons of the online LMS (WebCTs, Blackboard, Washington, DC). Analysis and correlations between online quiz peformance and final examination scores served as a secondary endpoint. A pre- and postexposure survey process was not utilized in that study, so trending of results over time was not possible, unlike the current study. Notably, upon conclusion of the course, students did not prefer asyncronous online lecture delivery compared to face-to-face delivery. Moreover, only 40% of students advocated for incorporation of online lectures in future courses based upon their experience in the foundational drug information course4 unlike the current study where the majority of students had no preference for online learning versus traditional face-to-face lectures (mean score ¼ 0.2). However, the majority of students in the course in the current study did agree at the conclusion of the course that online videos demonstrating skills are a suitable subsitute to in-person demonstration (mean score ¼ 1.07). Modifications in this foundational drug information course were favorably received and generated significant changes for opinion-based items related to student learning preferences. Similarly, Suda et al.12 discovered that the majority of students enrolled in a blended drug information and literature evaluation course agreed that the utilization of recorded videos allowed more time for meaningful face-toface interaction, application of course content and also provided equal educational value as a face-to-face lecture. However, unlike in the current study, instituted course changes did not affect examination scores or course evaluations completed by students.12 Additionally, research conducted by Lapidus et al.13 demonstrated favorable student feedback in favor of a blended course design versus traditional didactic approaches in a blended drug literature evaluation course. Specifically, 86% of students indicated some degree of agreement that the blended model improved
their learning. Despite positive student feedback regarding new course design, the authors did not detect significant academic improvement as a result of the course re-design. Similarly, McLaughlin et al.10 discovered that the majority of students enrolled in a blended pharmaceutics course preferred the flipped classroom format (85%) as compared to the traditional classroom format (15%). When comparing pre-course and post-course survey results from enrolled students, respondents significantly preferred pre-recorded lectures and felt that learning key foundational content before attending class assisted with comprehension (p o 0.001). Furthermore, students felt that watching the videos before coming to class significantly improved student participation and engagement, which led to increased understanding of course content (p o 0.001).10 Interestingly, both Suda and McLaughlin conducted research at institutions that have satellite campuses and were able to generate significant results across separate cohorts of students, whereas the university in this study only has one campus. Thus, the extrapolation of the results of this study to colleges of pharmacy that have satellite campuses would be unsupported by the current research. Utilization of a flipped classroom environment by McLaughlin et al.10 also generated significant improvement in course evaluation scores compared to the traditional classroom, similar to the current research. When comparing course evaluation scores before and after course re-design, there were significant improvements in encouraging attitudes exhibited by instructors, and instructor encouragement to be prepared for class.10 Similarly, in the current study there was consistent agreement or strong agreement for course evaluation items that related to clear communication of course objectives, presentation of course topics, the value of learning opportunities and student ability to apply learned information. In this course, structural changes led to higher final exam, but not final course, scores between comparable
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
samples, similar to research conducted by Bollmeier et al.11 where second professional year Doctor of Pharmacy students enrolled in a therapeutics course experienced an increase in final exam, but not course grades secondary to utilization of recorded lecture files. Comparisons between classes in this research are justified due to the overall similarity between incoming academic performance of the two cohorts. The rise of mean exam scores but not overall final course score in this course between the 2012 and 2013 classes is attributable to lower laboratory scores earned by the 2013 class versus the 2012 class. Emphasis was placed on engagement and participation in laboratory exercises, as an effort to improve student performance in the 2014 offering of this same course. Limitations of this research include a less than desired sample size for the primary and post-hoc analyses. With special consideration directed toward the post-hoc analyses, conclusions of student dispositions toward blended learning based upon gender or prior exposure to blended learning are hypothesis-generating only. While the survey instrument used in this study is not validated in the literature, it did undergo pilot testing in a school-wide survey in 2012. Conclusion Results indicate a significant shift in student learning preferences in favor of the blended learning model during the five-week period of observation. Furthermore, student performance was not negatively affected, but rather improved for the final examination. The blended versus face-to-face model will be utilized for delivery of lecture content for other skills-based courses in the curriculum. Conflicts of interest None. Acknowledgements Michael G. Kendrach, PharmD, FASHP.
Appendix A. Supplementary Information Supplementary data associated with this article can be found in the online version at http://dx.doi.org/10.1016/j. cptl.2016.02.013.
363
References 1. Blouin RA, Riffee WH, Robinson ET, et al. Roles of innovation in education delivery. Am J Pharm Educ. 2009; 73(8): Article 154. 2. Lancaster JW, McQueeney ML, Van Amburgh JA. Online delivery paired with in class problem-based learning…does it enhance student learning? Curr Pharm Teach Learn. 2011;3 (1):23–29. 3. Phillips JA, Gabay MP, Ficzere C, Ward KE. Curriculum and instructional methods for drug information, literature evaluation, and biostatistics: survey of US pharmacy schools. Ann Pharmacother. 2012;46(6):793–801. 4. Freeman MK, Schrimsher RH, Kendrach MG. Student perceptions of online lectures and WebCT in an introductory drug information course. Am J Pharm Educ. 2006;70(6): Article 126. 5. Bersin J. The Blended Learning Book: Best Practices, Proven Methodologies, and Lessons Learned, 1st ed., Hoboken, NJ: Pfeiffer Publishing; 2004 [XV]. 6. Calis KA, Sheehan AH. Formulating effective responses and recommendations: a structured approach. In: Malone PM, Kier KL, Stanovich JE, Malone MJ, eds. Drug Information: A Guide for Pharmacists, 5th ed., St. Louis, MO: McGraw-Hill; 2014: 36–45. 7. Crouch MA. An advanced cardiovascular pharmacotherapy course blending online and face-to-face instruction. Am J Pharm Educ. 2009;73(3): Article 51. 8. Zapantis A, Machado C, Nemire R, Leung S. An elective course in adult acute care medicine using a hybrid delivery system. Am J Pharm Educ. 2008;72(5): Article 105. 9. Ruehter V, Lindsey C, Graham M, Garavalia L. Use of online modules to enhance knowledge and skills application during an introductory pharmacy practice experience. Am J Pharm Educ. 2012;76(4): Article 69. 10. McLaughlin JE, Roth MT, Glatt DM, et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89(2): 236–243. 11. Bollmeier SG, Wenger PJ, Forinash AB. Impact of online lecture-capture on student outcomes in a therapeutics course. Am J Pharm Educ. 2010;74(7): Article 127. 12. Suda KJ, Sterling JM, Guirguis AB, Mathur SK. Student perception and academic performance after implementation of a blended learning approach to a drug information and literature evaluation course. Curr Pharm Teach Learn. 2014;6(3):367–372. 13. Lapidus M, McCord SK, McCloskey WW, Kostka-Rokosz MD. Combined use of online tutorials and hands-on group exercises in bibliographic instruction for pharmacy students. Med Ref Serv Q. 2012;31(4):383–399. 14. Leonard SN, Murphy K, Zaeem M, DiVall MV. An introductory review module for an anti-infectives therapeutics course. Am J Pharm Educ. 2012;76(7): Article 135.
Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
Research article
http://www.pharmacyteaching.com
Student perceptions of and performance in a blended foundational drug information course Peter J. Hughes, PharmD, BCPSa,*, Bruce Waldrop, PhD, RPhb, Jongwha Chang, PhDc b
a Department of Pharmacy Practice, McWhorter School of Pharmacy, Birmingham, AL Department of of Pharmaceutical, Social and Administrative Sciences, McWhorter School of Pharmacy, Birmingham, AL c Birmingham, AL
Abstract Objectives: Assess and trend student opinions of online educational components utilized in a redesigned 5-week foundational drug information course. Comparison of performance on the final examination, course grade, and changes in standardized university course evaluations between 2012 and 2013 were the secondary objectives. Methods: This course used narrated video instruction coupled with the use of a face-to-face weekly laboratory session. This project consisted of pre- and post-exposure surveys to allow for paired analysis of six opinion-based survey items using a fivepoint Likert scale. Secondary objectives were compared between the 2013 and 2012 entering classes by use of paired t-tests. Results: Only matched pairs were analyzed for this project (n ¼ 65 of 127 enrolled students; 51.2%). Changes between mean pre- and post-survey results indicated a decline in student favorability for traditional lecture styles (difference ¼ 0.49 points; p o 0.05) and a decline in the importance of face-to-face interaction with instructors outside class (difference ¼ 0.46 points; p o 0.05) while demonstrating increased favorability for use of online video demonstrations as an acceptable substitute for inperson demonstration of skills (difference ¼ 0.34 points; p o 0.05) and increased rating of overall preference of online learning compared to traditional modalities (difference ¼ 0.44 points; p o 0.05). Standardized course evaluations did not differ between 2012 and 2013. Mean final exam scores significantly increased from 84.86% in 2012 to 88.99% in 2013 (p o 0.05), but no difference between mean course grades (94.03% and 93.62%, respectively; p 4 0.05). Conclusion: Blended course design is an effective format for introductory skills-based professional pharmacy courses. r 2016 Elsevier Inc. All rights reserved.
Keywords: Blended learning; Drug information; Pharmacy education
Introduction Online technology can allow for efficient delivery of facts, creating time for active learning opportunities during classtime.1 Furthermore, while evidence for use of blended Abbreviations: P1, first professional year LMS, learning management system * Corresponding author: Peter J. Hughes, PharmD, BCPS, McWhorter School of Pharmacy, 800 Lakeshore Drive, Birmingham, AL 35229. E-mail: [email protected] http://dx.doi.org/10.1016/j.cptl.2016.02.013 1877-1297/r 2016 Elsevier Inc. All rights reserved.
course or curricular designs are accessible,2 use of Internetbased activities in a foundational drug information course is uncommon based upon the published literature.3,4 Due to student feedback from annual assessments and to accomplish institutional strategic objectives, the foundational drug information course for first-year (P1) pharmacy students at McWhorter School of Pharmacy, Samford University was revised into a blended learning model for fall 2013. Prior offerings of the course were conducted in a strictly face-toface manner. “Blended learning” can be defined as the combination of different pedagogical media (e.g., technology and activities)
360
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
to create an ideal training environment.5 Adjunctive media such as technology is not to be viewed as a replacement for face-to-face interaction, but rather as a supplement to this endeavor and to also promote self-directed learning in professional students.5 The primary purpose of this project was to assess and trend opinions about student learning preferences in this five-week (one semester credit hour) course. Secondary objectives compared the mean final examination score, overall mean course grade, and changes in standardized university course evaluations between the 2012 and 2013 offerings of this course.
Methods The modified blended course was a one credit-hour required course delivered during the first five weeks of the P1 pharmacy curriculum. A blended course, for the purposes of this research, is defined as a course that consists of asynchronous delivery of didactic content coupled with face-to-face active learning/laboratory exercises. The course consisted of a series of narrated videos accessed online via a learning management system (LMS, Moodles Version 2.4.5, Perth, Australia) in conjunction with face-to-face weekly laboratory sessions to reinforce important concepts presented in the videos. Using the previous year’s course learning objectives as the basis for content development, videos were produced using a combination of screen capture software (Camtasias for Mac version 2.4.0, Techsmiths, Okemos, MI), video recording of simulated interactions, and to a lesser extent, narrated slide presentations (see Appendices A and B for course learning objectives and course calendar, respectively). Since its initial offering as a face-to-face course in the fall of 2009, the calculation of the course grade for the foundational drug information course consisted of the student laboratory grade (30%), student weekly quiz grade (30%), student final examination grade (30%), and student participation/attendance (10%). Weekly online content was segmented into multiple videos of 6–12 minutes duration, in which each video focused on specific learning outcomes. On average, students watched five videos per week, occupying approximately 85 minutes of their time. The face-to-face laboratory sessions took place Monday through Thursday afternoons, with approximately 42 students per three-hour session. These sessions were designed to allow students to reinforce and apply the material that was presented in the online videos the previous week and consisted of exercises dedicated to paraphrasing and citing references, searching drug information databases, searching PubMed/Medline and answering a drug information question utilizing the systematic approach to answering drug information questions.6 Drug information questions utilized for laboratory exercises consisted of questions originally answered by the school’s drug information center in addition to simulated questions.
In order to encourage student accountability with respect to timely completion of required online course activities and assessments in a specific order, we utilized the built-in conditional release properties within the LMS. On the first viewing of each video, playback controls were disabled to ensure that students did not “fast-forward” to the end of the video. At the conclusion of each video, students were presented with a formative single-question quiz. Incorrect answers triggered the opening of a window with formative feedback, and students were allowed to reattempt the quiz. Successful completion of the quiz served as a tracking process for completion of the activity and also as a control mechanism to reveal the following video in the series. After viewing each video and correctly answering each formative quiz question for all videos in the series, a link for a timed (10 minute) summative quiz was revealed to the student. Students were able to use their notes on summative weekly quizzes. Scrambling the question order and the answer choices discouraged collaboration among students while taking the online summative quizzes. Furthermore, students were not allowed to return to a question once it was presented. Completion of the summative quiz was required for laboratory attendance. Laboratory attendance was required in the course. Summative quizzes utilized in the 2013 course offering were identical to quizzes utilized in 2012 and accounted for 30% of the total course grade. Since examinations are not returned to students (school policy), the final exam utilized in the 2013 course offering was also identical to the final exam utilized in the 2012 course, with the exception of two questions that were re-worded for the 2013 course to improve content clarity. The exam was a paper-based exam that consisted of 40 multiple-choice questions that were equally weighted at 2.5 points per question. The final examination accounted for 30% of the total course grade. During the P1 student orientation program one week prior to the commencement of classes, P1 students were asked to complete an online “pre-exposure” survey regarding previous experiences with and perceptions of online courses. Similarly, after the completion of the course, students were asked to complete a “post-exposure” survey with a similar format as the pre-exposure survey, but also included specific questions related to course outcomes and activities. Students used a unique numeric identifier which allowed for paired comparisons of the pre- and postexposure survey results. An electronic survey (Qualtricss, Provo, UT) link for the pre-exposure survey was distributed via email to the P1 class during new student orientation and the post-exposure survey link was distributed after the course final exam through the same process. A five-point Likert scale (range: 2 ¼ strongly disagree, 1 ¼ disagree, 0 ¼ neutral, þ1 ¼ agree, and þ2 ¼ strongly agree) was used for questions related to student perceptions and attitudes. Paired t-tests were used to compare pre- and post-exposure survey data and served as the primary endpoints for this study. Standardized student course evaluation results, final examination scores, and overall
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
course score were compared between the 2012 and 2013 entering classes using unpaired t-tests. These additional comparisons served as secondary endpoints. Statistical analyses were conducted using Stata/SEs (v1.3, College Station, TX). A p o 0.05 was considered significant. The Samford University Institutional Review Board (IRB) approved this research.
Results Of 127 students in the class, 115 (90%) completed the pre-exposure survey and 88 (76%) completed the postexposure survey. Among the respondents, 77% were female, 50% had previous blended learning course experience, and 59% had previous online course experience. Only pairs who completed both the pre- and post-exposure surveys were analyzed for primary endpoint in this project (n ¼ 65; 51%). Overall, paired analysis of questions regarding student learning preferences reflected favorably on the structural changes instituted in the course. Specifically, the change between mean pre- and post-survey results for opinionbased items indicated a decline in student favorability for traditional lecture styles (difference ¼ 0.49 points; p o 0.05) and importance of face-to-face interactions with instructors outside of the classroom (difference ¼
Table 1 Student learning preferences.
Statement Statement Statement Statement Statement Statement
A B C D E F
Pre-exposure survey mean score
Post-exposure survey mean score Change
1.31 0.57 1.02 0.72 0.74 0.24
0.82 0.46 0.55 0.52 1.07 0.2
0.49 0.11 0.47 0.2 þ0.33 þ0.44
p Value o0.05 NS o0.05 NS o0.05 o0.05
Survey Statement Legend A: I consider traditional lectures where the instructor presents information to the class to be an effective learning method. B: It is important to me to have interaction with other classmates face-to-face to discuss course materials. C: It is important to have face-to-face communication with the instructor outside of the classroom. D: It is important that I have the ability to ask questions during the lecture/ class period. E: An online video demonstration of skills and techniques (i.e., how to use certain Internet databases or other skills) is an acceptable substitute to inperson demonstrations in the classroom. F:I prefer online learning compared to traditional learning modalities (i.e., traditional classroom lecture). Range: 2, strongly disagree; 1, disagree; 0, no preference; 1, agree; 2, strongly agree; negative scores indicate decreased agreement; positive scores indicate increased agreement; NS, not significant.
361
0.46 points; p o 0.05), while demonstrating increased favorability for use of online video demonstrations as an acceptable substitute for in-person demonstration of skills (difference ¼ 0.34 points; p o 0.05) and overall preference of online learning compared to traditional modalities (difference ¼ 0.44 points; p o 0.05). No statistically significant differences existed for changes of opinion in survey items assessing importance of in-class interaction with classmates and importance of the ability to ask questions during class (see Table 1). The mean score for select items on the standardized course evaluations completed by students significantly differed between the 2012 and 2013 cohorts (n ¼ 107 and 106, respectively). Mean final exam scores for the 2012 and 2013 cohorts, using the same instrument, significantly increased from 84.86% to 88.99% (p o 0.05), while mean course grades were similar between the 2012 and 2013 cohorts (94.03% and 93.62%, respectively; p 4 0.05) (see Table 2). The mean overall grade point average (GPA) and math/science entering GPAs for the fall 2012 class were 3.38 and 3.40, respectively, whereas the mean overall GPA and math/science entering GPAs for the fall 2013 class were 3.45 and 3.41, respectively. Post-hoc subgroup analyses were conducted to assess for differences to select survey items on the basis of gender and previous blended learning exposure. Significant differences were observed within the female cohort for decline in favorability for traditional lecture styles (difference = 0.57 points; p o 0.05) and increased preference for online learning compared to traditional modalities (difference = 0.38 points; p o 0.05). With male counterparts, only a significant difference in opinion was noted for improved favorability of online learning compared to traditional learning modalities (difference = 0.61 points; p o 0.05). In the context of prior blended learning exposure, consistent significant differences were noted between groups with respect to a decline in favorability of traditional lecture styles [difference (blended learning) = 0.53 points; difference (no-blended learning) = 0.44 points; p o 0.05 for both groups] and a decline in favorability of face-to-face interactions with faculty members outside of the classroom [difference (blended learning) = 0.33 points; difference (no-blended learning) = 0.68 points; p o 0.05 for both groups]. Discussion Blended or hybridized teaching models have demonstrated effectiveness in a variety pharmacy educational venues, to include elective courses,7,8 introductory practice experiences,9 stand-alone didactic courses,3,10–13 as well as student remediation courses.14 To the best knowledge of the authors, the current research is only the second known attempt to document student perceptions of and performance in a hybridized P1 foundational drug information course. The only other research in this area was published by Freeman et al.4 In that study, the authors observed and measured results of the 2005 course using different
362
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
Table 2 Final Examination and Course Grade Comparison. Final examination and course grade comparison 2012 mean score (n ¼ 121)
2013 mean score (n ¼ 127)
Change
p Value
Final exam Final grade
84.87% 94.03
88.99% 93.63
þ4.12% 0.4%
o0.05 NS
Statement Statement Statement Statement
2012 mean score (n ¼ 107) 1.63 1.62 1.71 1.61
Change þ0.06 0.02 þ0.01 þ0.02
p Value o0.05 o0.05 NS o0.05
G H I J
Course evaluation comparison 2013 mean score (n ¼ 106) 1.69 1.6 1.72 1.63
Course Evaluation Statement Legend G: Course objectives and requirements were clearly communicated to me. H: The learning opportunities provided in this course helped me to comprehend the course content. I: I know how to apply the information learned in this course. J: Course topics were presented in a logical sequence. Range: 2, strongly disagree; 1, disagree; 0, no preference; 1, agree; 2, strongly agree.
methodology by assessing student perceptions of recorded lectures and perceptons of the online LMS (WebCTs, Blackboard, Washington, DC). Analysis and correlations between online quiz peformance and final examination scores served as a secondary endpoint. A pre- and postexposure survey process was not utilized in that study, so trending of results over time was not possible, unlike the current study. Notably, upon conclusion of the course, students did not prefer asyncronous online lecture delivery compared to face-to-face delivery. Moreover, only 40% of students advocated for incorporation of online lectures in future courses based upon their experience in the foundational drug information course4 unlike the current study where the majority of students had no preference for online learning versus traditional face-to-face lectures (mean score ¼ 0.2). However, the majority of students in the course in the current study did agree at the conclusion of the course that online videos demonstrating skills are a suitable subsitute to in-person demonstration (mean score ¼ 1.07). Modifications in this foundational drug information course were favorably received and generated significant changes for opinion-based items related to student learning preferences. Similarly, Suda et al.12 discovered that the majority of students enrolled in a blended drug information and literature evaluation course agreed that the utilization of recorded videos allowed more time for meaningful face-toface interaction, application of course content and also provided equal educational value as a face-to-face lecture. However, unlike in the current study, instituted course changes did not affect examination scores or course evaluations completed by students.12 Additionally, research conducted by Lapidus et al.13 demonstrated favorable student feedback in favor of a blended course design versus traditional didactic approaches in a blended drug literature evaluation course. Specifically, 86% of students indicated some degree of agreement that the blended model improved
their learning. Despite positive student feedback regarding new course design, the authors did not detect significant academic improvement as a result of the course re-design. Similarly, McLaughlin et al.10 discovered that the majority of students enrolled in a blended pharmaceutics course preferred the flipped classroom format (85%) as compared to the traditional classroom format (15%). When comparing pre-course and post-course survey results from enrolled students, respondents significantly preferred pre-recorded lectures and felt that learning key foundational content before attending class assisted with comprehension (p o 0.001). Furthermore, students felt that watching the videos before coming to class significantly improved student participation and engagement, which led to increased understanding of course content (p o 0.001).10 Interestingly, both Suda and McLaughlin conducted research at institutions that have satellite campuses and were able to generate significant results across separate cohorts of students, whereas the university in this study only has one campus. Thus, the extrapolation of the results of this study to colleges of pharmacy that have satellite campuses would be unsupported by the current research. Utilization of a flipped classroom environment by McLaughlin et al.10 also generated significant improvement in course evaluation scores compared to the traditional classroom, similar to the current research. When comparing course evaluation scores before and after course re-design, there were significant improvements in encouraging attitudes exhibited by instructors, and instructor encouragement to be prepared for class.10 Similarly, in the current study there was consistent agreement or strong agreement for course evaluation items that related to clear communication of course objectives, presentation of course topics, the value of learning opportunities and student ability to apply learned information. In this course, structural changes led to higher final exam, but not final course, scores between comparable
P.J. Hughes et al. / Currents in Pharmacy Teaching and Learning 8 (2016) 359–363
samples, similar to research conducted by Bollmeier et al.11 where second professional year Doctor of Pharmacy students enrolled in a therapeutics course experienced an increase in final exam, but not course grades secondary to utilization of recorded lecture files. Comparisons between classes in this research are justified due to the overall similarity between incoming academic performance of the two cohorts. The rise of mean exam scores but not overall final course score in this course between the 2012 and 2013 classes is attributable to lower laboratory scores earned by the 2013 class versus the 2012 class. Emphasis was placed on engagement and participation in laboratory exercises, as an effort to improve student performance in the 2014 offering of this same course. Limitations of this research include a less than desired sample size for the primary and post-hoc analyses. With special consideration directed toward the post-hoc analyses, conclusions of student dispositions toward blended learning based upon gender or prior exposure to blended learning are hypothesis-generating only. While the survey instrument used in this study is not validated in the literature, it did undergo pilot testing in a school-wide survey in 2012. Conclusion Results indicate a significant shift in student learning preferences in favor of the blended learning model during the five-week period of observation. Furthermore, student performance was not negatively affected, but rather improved for the final examination. The blended versus face-to-face model will be utilized for delivery of lecture content for other skills-based courses in the curriculum. Conflicts of interest None. Acknowledgements Michael G. Kendrach, PharmD, FASHP.
Appendix A. Supplementary Information Supplementary data associated with this article can be found in the online version at http://dx.doi.org/10.1016/j. cptl.2016.02.013.
363
References 1. Blouin RA, Riffee WH, Robinson ET, et al. Roles of innovation in education delivery. Am J Pharm Educ. 2009; 73(8): Article 154. 2. Lancaster JW, McQueeney ML, Van Amburgh JA. Online delivery paired with in class problem-based learning…does it enhance student learning? Curr Pharm Teach Learn. 2011;3 (1):23–29. 3. Phillips JA, Gabay MP, Ficzere C, Ward KE. Curriculum and instructional methods for drug information, literature evaluation, and biostatistics: survey of US pharmacy schools. Ann Pharmacother. 2012;46(6):793–801. 4. Freeman MK, Schrimsher RH, Kendrach MG. Student perceptions of online lectures and WebCT in an introductory drug information course. Am J Pharm Educ. 2006;70(6): Article 126. 5. Bersin J. The Blended Learning Book: Best Practices, Proven Methodologies, and Lessons Learned, 1st ed., Hoboken, NJ: Pfeiffer Publishing; 2004 [XV]. 6. Calis KA, Sheehan AH. Formulating effective responses and recommendations: a structured approach. In: Malone PM, Kier KL, Stanovich JE, Malone MJ, eds. Drug Information: A Guide for Pharmacists, 5th ed., St. Louis, MO: McGraw-Hill; 2014: 36–45. 7. Crouch MA. An advanced cardiovascular pharmacotherapy course blending online and face-to-face instruction. Am J Pharm Educ. 2009;73(3): Article 51. 8. Zapantis A, Machado C, Nemire R, Leung S. An elective course in adult acute care medicine using a hybrid delivery system. Am J Pharm Educ. 2008;72(5): Article 105. 9. Ruehter V, Lindsey C, Graham M, Garavalia L. Use of online modules to enhance knowledge and skills application during an introductory pharmacy practice experience. Am J Pharm Educ. 2012;76(4): Article 69. 10. McLaughlin JE, Roth MT, Glatt DM, et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89(2): 236–243. 11. Bollmeier SG, Wenger PJ, Forinash AB. Impact of online lecture-capture on student outcomes in a therapeutics course. Am J Pharm Educ. 2010;74(7): Article 127. 12. Suda KJ, Sterling JM, Guirguis AB, Mathur SK. Student perception and academic performance after implementation of a blended learning approach to a drug information and literature evaluation course. Curr Pharm Teach Learn. 2014;6(3):367–372. 13. Lapidus M, McCord SK, McCloskey WW, Kostka-Rokosz MD. Combined use of online tutorials and hands-on group exercises in bibliographic instruction for pharmacy students. Med Ref Serv Q. 2012;31(4):383–399. 14. Leonard SN, Murphy K, Zaeem M, DiVall MV. An introductory review module for an anti-infectives therapeutics course. Am J Pharm Educ. 2012;76(7): Article 135.