- Email: [email protected]
Effectiveness of institutional Introductory Pharmacy Practice Experiences at achieving instructional objectives
Available online at www.sciencedirect.com
Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
Short communication
http://www.pharmacyteaching.com
Effectiveness of institutional Introductory Pharmacy Practice Experiences at achieving instructional objectives Nicole L. Metzger, PharmD, BCPS, Angela O. Shogbon, PharmD, BCPS* Department of Pharmacy Practice, Mercer University College of Pharmacy, Atlanta, GA
Abstract Objective: To assess the effectiveness of four hospital-based, faculty-facilitated, Introductory Pharmacy Practice Experiences (IPPEs) at meeting instructional objectives related to the Accreditation Council for Pharmacy Education (ACPE) standards. Methods: A pre-test and a post-test were administered to students to assess their acquisition of basic institutional pharmacy knowledge and perceptions of their ability and interest in hospital pharmacy practice. The test included demographic, openended, and multiple-choice questions designed to assess objectives in line with ACPE standards. Students’ pre-test and post-test scores were compared using paired t-tests. Differences in test scores and perceptions were assessed based on year and work experience. Results: Analysis of 88 students from year one and 84 students in year two showed significant improvement in test scores from pre-test to the post-test in year one (71.4% vs. 77.0%, p o 0.01) and year two (68.4% vs. 81.7%, p o 0.001). In year one, students with prior hospital experience (30%) scored significantly higher on the pre-test but not the post-test. In year two, no difference in scores was observed for students with prior hospital experience (23.8%) compared to those without. Students reported improved confidence in their knowledge of hospital pharmacy practice at the conclusion of IPPE (p o 0.001 in both years). Students’ likelihood of pursuing a career in hospital pharmacy did not change after institutional IPPE (p ¼ 0.184 in year one, p ¼ 0.075 in year two). Conclusions: The institutional IPPE program was successful at improving students’ knowledge in basic hospital pharmacy practice. Institutional IPPE improved students’ confidence but did not change their perceived career path. r 2014 Elsevier Inc. All rights reserved.
Keywords: Introductory Pharmacy Practice Experiences; Hospitals; Institutions; Instructional objectives; Experiential education
Introduction Pharmacists are widely regarded as medication experts and can provide evidence-based, cost-effective patient care. The Accreditation Council for Pharmacy Education (ACPE) describes their vision of future pharmacy practice as one where pharmacists ensure optimal medication therapy
* Corresponding author: Angela O. Shogbon, PharmD, BCPS, Department of Pharmacy Practice, Mercer University College of Pharmacy, 3001 Mercer University Drive, Atlanta, GA 303414155. E-mail: [email protected] http://dx.doi.org/10.1016/j.cptl.2014.05.008 1877-1297/r 2014 Elsevier Inc. All rights reserved.
outcomes for patients.1 They further recommend that pharmacy education prepares students for the delivery of patient-centered care that optimizes medication use, improves therapeutic outcomes, and promotes public health initiatives.1 However, along with recognition of the pharmacist as a medication expert comes the additional accountability for patient health outcomes, and hopefully, public recognition of the pharmacist’s ability to manage medications, improve cost-effectiveness, and patient outcomes.1 As a result, ACPE has increased emphasis on experiential education, specifically Introductory Pharmacy Practice Experiences (IPPEs). IPPEs have been defined as “practice experiences offered in various practice settings
668
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
during the early sequencing of the curriculum for purposes of providing transitional experiential activities and directed exposure to pharmaceutical care.”2 Guideline 14.4 of the ACPE standards recommends that IPPE include a total of 300 hours, of which, 150 hours be equally divided between community and institutional settings.1 While obtaining community IPPE hours is feasible for many schools, institutional IPPE hours are much more difficult to secure due to the smaller number of hospitals willing to participate. The Mercer University College of Pharmacy developed a unique approach to securing institutional sites for Advanced Pharmacy Practice Experiences (APPE) and IPPEs, which included strategically placing pharmacy practice faculty members at four local hospitals to facilitate these experiences. There are several descriptions of institutional IPPE programs in the literature, each with varying designs and structure, but all with a goal to increase the preparedness of students for APPEs. Some institutional IPPE programs have involved students’ working with interprofessional patient care teams, while others have incorporated direct patient care activities such as patient interviews and provision of medication reconciliation services.3–9 These programs have typically developed IPPE objectives based on competencies derived from their curriculum committee’s interpretation of ACPE standards. Some of these programs have utilized assessment tools to evaluate students’ self-assessment of IPPE and knowledge gained, faculty’s assessment of students’ performances on IPPE, and students’ readiness for experiential rotations.3–9 Current assessment of institutional IPPE in the literature includes mostly subjective data relying on students’ and faculty’s perceptions of competency.4–8 A longitudinal IPPE paired third-year students with faculty and APPE students to interview patients and to evaluate and present patient cases. Students were surveyed before and after IPPE to assess their perceptions of achieving 21 competencies developed by the school’s curriculum committee. Students reported significant improvement following IPPE in the scores for 15 of 21 competencies.4 Another patient carecentered IPPE involved third-year students providing medication reconciliation services at a local health system. Students improved their ability to perform medication reconciliation and reported positive attitudes about the medication reconciliation process and awareness about its importance.5 In a College of Pharmacy, an extensive block IPPE program was developed utilizing the ACPE competencies as a backbone for the IPPE syllabus and learning objectives. The authors reported that student journal entries and student and preceptor perceptions survey data indicated IPPE competencies were being achieved.9 In another report, first-year, second-year, and third-year IPPE students were integrated with APPE students on an internal medicine rotation. Students were surveyed on their attitude and perceptions before and after IPPE and reported improvement in their ability to describe the role of the clinical
pharmacist, describe how a pharmacist interacts with patients, locate information in a medical record, locate the answers to drug information questions, evaluate the medical literature, and counsel patients.6,7 Lastly, in a report evaluating faculty perceptions, faculty assessed students who completed a four-week IPPE course prior to APPE more favorably on a 5-point Likert scale, compared with students who had not taken the IPPE course.8 Though these reports provide valuable information about preceptors, faculty, and students perceptions of IPPE, they are limited in their ability to demonstrate objectively what students have learned. In an effort to more accurately characterize our students’ achievement of institutional IPPE instructional objectives, an assessment tool was designed to evaluate their acquisition and application of basic hospital pharmacy practice knowledge. During the 2009 and 2010 academic years, institutional IPPE at the Mercer University College of Pharmacy was conducted longitudinally during the third professional year. Rationale and objectives Institutional IPPE is critical in the development and application of key skills required to be a pharmacist. Several pedagogical articles on institutional IPPE have reported improvement in students’ perceptions of their abilities,4–7 but there is still a paucity of reports that quantitatively assess students’ knowledge after institutional IPPE. The purpose of this study was to quantitatively assess the effectiveness of institutional IPPE at meeting instructional objectives in line with ACPE standards Appendix D by evaluating students’ performance on a pre-test compared with a post-test knowledge-based assessment. Secondary objectives included assessing students’ baseline hospital experience, determining the impact of IPPE on students’ confidence in basic hospital pharmacy practice and their likelihood of pursuing a career in hospital pharmacy, and to determine if previous hospital experience resulted in higher pre- or post-test scores. Methods Third professional year student pharmacists were sent out in groups of four to six students to one of four local hospitals for a half-day per week, for four consecutive weeks. Students completed a total of 16 hours of institutional IPPE (a four-week block) in either the fall or spring semester of their third professional year. In order to evaluate students’ performance on institutional IPPE competencies derived from the ACPE standards Appendix D, an assessment tool that provided both qualitative and quantitative measurements was designed by IPPE faculty. Students were asked about their baseline hospital pharmacy experience, confidence in knowledge of hospital pharmacy practice, and their likelihood of pursuing a career in hospital pharmacy practice in the qualitative questions. The quantitative section
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
669
Table Test questions Year one
Year two
Have you ever been inside a hospital pharmacy before today?
No change
a. Yes b. No Have you ever volunteered, interned, or worked as a technician in a hospital pharmacy?
No change
a. Yes b. No How long have you volunteered, interned, or worked in a hospital pharmacy? a. b. c. d.
Never Less than one year 1–3 years Greater than three years
Rate your agreement with the statement “I feel confident in my knowledge of hospital pharmacy practice.” a. b. c. d. e.
Strongly disagree Disagree Neutral Agree Strongly agree
Rate your agreement with the statement “I am likely to pursue a career in a hospital pharmacy.” a. b. c. d. e.
How long have you worked in a hospital pharmacy? ____________
Strongly disagree Disagree Neutral Agree Strongly agree
The reason(s) for decentralized pharmacy practice model include which of the following:
Rate your agreement with the statement “I feel confident in my knowledge of basic hospital pharmacy practice.” a. b. c. d.
Strongly disagree Disagree Agree Strongly agree
Rate your agreement with the statement “I am likely to pursue a career in a hospital pharmacy.” a. b. c. d. e.
Strongly disagree Disagree Unsure Agree Strongly agree
List three (3) advantages of a decentralized pharmacy practice model.
a. To provide specialized pharmacy services b. Foster team-based approach to patient care c. Allow for quicker and more efficient clarification of therapeutic orders d. All of the above Which of the following is an example of a satellite pharmacy? a. b. c. d. e.
Operating Room (OR) pharmacy Neonatal Intensive Care Unit (ICU) pharmacy Main inpatient pharmacy A and B None of the above
Select the method of medication distribution from the pharmacy that stores medications on the nursing unit and utilizes electronic perpetual inventory? a. b. c. d.
No change
Automated dispensing cabinets Medication carts Pneumatic tube system Robotic dispensing technology
Describe two (2) ways automated dispensing cabinets, such as Pyxiss, improve medication distribution in the hospital setting?
670
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
Table Continued Year one
Year two
How are expired medications handled?
No change
a. b. c. d.
Incineration Reverse distributor Biohazardous waste Donation to non-profit clinics
Which regulation governs the aseptic standards for compounding sterile products in a hospital? a. b. c. d.
FDA Good Manufacturing Processes NSPG 03.05.01 USP 797 Joint Commission rule 295
Which of the following is an incorrect medication order (either unapproved abbreviation or wrong)? a. b. c. d.
75 ml/minute 64 ml/minute 46 ml/minute 39 ml/minute
Which of the following would be considered a therapeutic substitution? a. b. c. d.
Changing Changing Changing Changing
A A A A
patient patient patient patient
receiving levetiracetam for seizures receiving warfarin for atrial fibrillation receiving prednisone for joint pain taking furosemide for heart failure
Medication reconciliation is performed at the following times EXCEPT? a. b. c. d.
Calculate the creatinine clearance for the patient below using the Cockcroft Gault formula and show all of your work. Patient case: A 50 4″, 80-kg, 82-year-old African American female, with a SCr ¼ 1.2 mg/dl?
No change
metoprolol to amlodipine atorvastatin to simvastatin clopidogrel to aspirin fluticasone nasal spray to fluticasone MDI
Which of the following patients should have therapeutic drug monitoring? a. b. c. d.
Describe the components of a complete medication order
Nystatin 400,000 IU four times daily Nystatin 400,000 mg swish and swallow Q6 Nystatin 400,000 mg/kg swish and swallow QD All are incorrect or inappropriate
What is the creatinine clearance for the following patient: A 50 4″, 65-kg, 80-year-old African American female, with a SCr ¼ 0.8 mg/dl? a. b. c. d.
List two (2) components of an IV room that are compliant with USP 797 Regulations?
Admission Discharge Transfer from ICU to the medical floor Transfer from medical floor A to medical floor B
A physician wants to initiate warfarin 5 mg po daily in a patient. a. What lab value(s) should be monitored for this patient? b. Explain two (2) patient counseling points that you would address with the patient
Identify two (2) errors in this patient’s medication reconciliation? Home medications
Inpatient orders
Metoprolol succinate 50 mg po daily Levothyroxine 100 mcg po daily Lisinopril 10 mg po daily Acetaminophen 500 mg po BID prn pain Glipizide 10 mg po daily
Metoprolol tartrate 50 mg po daily Levothyroxine 100 mg po daily Lisinopril 10 mg po daily Acetaminophen 500 mg IV BID prn pain Glipizide 10 mg daily
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
671
Table Continued Year one
Year two
Which of the following represent(s) “Objective” information obtained for a patient?
Which of the following represents information that would be contained in the “Objective” portion of a patient's SOAP note?
a. Patient's heart rate is 90 beats per minute and respirations 18 breaths per minute b. Patient states he/she has been vomiting for two days c. Patient complains of abdominal pain d. Patient reports smoking tobacco two packs per day Select the appropriate monitoring parameter for the medication? a. b. c. d.
Warfarin-aPTT Enoxaparin-PT Amlodipine-HR Haloperidol-QTc
a. Patient's heart rate is 90 beats per minute and respirations 18 breaths per minute b. Patient states he/she has been vomiting for two days c. Patient needs a chest X-ray to rule out pneumonia d. Patient reports smoking tobacco two packs per day Classify the drug information references listed below: 1. 2. 3. 4.
Trissel's Handbook on Injectable Drugs PubMed DiPiro's Pharmacotherapy: A Pathophysiologic Approach A journal article comparing the efficacy of glipizide to metformin a. Primary reference b. Secondary reference c. Tertiary reference
You are staffing in the inpatient pharmacy. A nurse from the ICU calls down and wants you to verify her settings for the IV pump for a dopamine infusion. Dopamine is ordered at 3 mcg/kg/minute. The dopamine bag contains 800 mg in 500 ml of fluid. The patient weighs 75 kg. What infusion rate, in ml/h, should the nurse program the pump for? Show your work. Select the patient with a significant medication related problem. a. b. c. d.
included questions that were developed to assess students’ comprehension of objectives in line with the Core Domains from Appendix D of the ACPE standards. The Table lists the test questions posed during year one and year two. Students were administered the same IPPE assessment tool as a pre-test prior to the institutional IPPE and as a post-test at the conclusion of their IPPE block. In the first year of the study, the pre-test was administered on the first day of the institutional IPPE before the students began the experience; in the second year, the pre-test was administered at the start of the academic year for third professional year student pharmacists. The post-test was given at the end of each four-week long institutional IPPE during both years. Multiple-choice questions were assessed by faculty at each site, and open-ended questions from year two were scored by the study authors using a standardized rubric. The study authors met to review their independent assessment of open-ended questions to ensure consistency in grading. Students’ responses to demographic questions were analyzed using descriptive statistics and the Wilcoxon signed rank test. Comparison between students’ pre-test and post-test scores was done using paired t-tests. Data analysis was
Lisinopril with a serum potassium of 4.0 Amlodipine with a heart rate of 55 beats per minute Ibuprofen in a patient with chronic kidney disease Memantine in a patient with moderate Alzheimer's
performed using Statistical Package for the Social Sciences (SPSS) version 16.0 (SPSS Inc, Chicago, IL). This study was approved by the Institutional Review Board at Mercer University, and a waiver of informed consent was granted. Results A total of 88 (98.9%) students completed the pre-test and post-test at four faculty-facilitated IPPE institutions in year one and 84 (86.6%) students at three faculty-facilitated IPPE institutions in year two. One institution that was included in year one was not included in data collection for year two because the faculty member who was facilitating IPPE at that site no longer worked at the study institution. Baseline characteristics revealed that 58 (66%) students in year one and 46 (54.8%) students in year two reported a prior visit to a hospital pharmacy. Of those, only 26 (30%) students in year one and 20 (23.8%) students in year two reported volunteering, interning, or working in a hospital pharmacy. For the primary end point, students showed significant improvement in test scores from pre-test to the post-test in year one (71.4% vs. 77.0%; p o 0.01) and year two (68.4% vs. 81.7%; p o 0.001) (Fig. 1). Students
672
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
reported improved confidence in their knowledge of hospital pharmacy practice at the conclusion of IPPE (p o 0.001 for both years), and their likelihood to pursue a career in hospital pharmacy practice did not significantly change by the conclusion of institutional IPPE (p ¼ 0.184 in year one and p ¼ 0.075 in year two). Figures 2 and 3 illustrate the breakdown of student responses regarding their confidence in hospital pharmacy practice for both years. Significantly higher pre-test scores occurred in students with prior hospital experience compared to those without during year one, with mean ⫾ SD scores of 78.5% ⫾ 17.5% vs. 68.4% ⫾ 13.3%, respectively, p o 0.01. However, there was no significant difference in pre-test scores in year two between students with and without prior hospital experience, with mean ⫾ SD scores of 69.8% ⫾ 10.21% vs. 68.0% ⫾ 11.37%, respectively, p = 0.525. At the conclusion of institutional IPPE in each year, no significant difference in post-test scores was observed between students with prior hospital pharmacy experience and those without experience, with mean ⫾ SD scores in year one of 79.2% ⫾ 10.6% vs. 76.1% ⫾ 11.2%, respectively, p = 0.183, and in year two of 82.6% ⫾ 8.84% vs. 81.4% ⫾ 7.49% respectively, p = 0.542. Discussion This study demonstrated improvement in students’ knowledge of hospital pharmacy practice after completion
of a half-day, four-week long institutional IPPE program. Appendix D of the Accreditation Standards and Guidelines put forth by ACPE highlights the IPPE core domains and ability performance statements in which the students must demonstrate their abilities and competency to ensure optimal preparation for APPEs.1 Some of these core domains are patient safety, basic patient assessment, medication information, identification and assessment of drugrelated problems, dose calculation, drug information analysis, and literature search.1 The institutional IPPE program at Mercer University College of Pharmacy was designed to meet objectives in line with these standards, and students’ acquisition of knowledge at the end of this program was evident through their performance on the post-test assessment tool. After our assessment tool was designed, the American Society of Health-System Pharmacists (ASHP) and ACPE joint task force released a report describing competencies necessary for entry-level pharmacy practice in hospitals and health-systems.10 The institutional IPPE program at Mercer University College of Pharmacy is instrumental in ensuring students achieve many of these stated competencies. The institutional IPPE program continues to evolve. Since this study was conducted, the institutional IPPE program has changed to increase the time spent in the institutional setting based on updated ACPE standards. Currently, third-year Doctor of Pharmacy students spend eight hours per day for two weeks in an assigned
100% 90% 81.7%
77.0%
80% 71.4%
68.4%
70% 60% 50%
Pre-Test Post-Test
40% 30% 20% 10% 0% Year 1 Year 2 p < 0.01 for difference in mean pre-test and post-test scores in year 1 (n=88) and year 2 (n= 84) Bars represent standard deviation of the mean
Fig. 1. Overall performance across all institutions: year one and two.
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
673
70% 61% 60%
Percentage of Students
50%
45%
40%
Pre-Test Post-Test
30% 22% 19% 18%
20%
14% 11% 10%
6% 1%
1%
0% Strongly disagree Disagree Neutral Agree Strongly agree p < 0.001 for difference in confidence scores between the pre-test and post-test (n=88)
Fig. 2. Confidence in knowledge of hospital pharmacy practice: year one.
institution during the beginning of their spring semester. The number of sites has also expanded from four to nine to accommodate the increased hours. Despite the changes to the number of hours and the increase in the number of sites, institutional IPPE at Mercer University College of Pharmacy has consistently involved students working with faculty, pharmacists, pharmacy residents, pharmacy interns and technicians, and some APPE students to learn the objectives set forth for the IPPE program.
Institutional IPPE at Mercer University College of Pharmacy, past and present, provides students with experiential training and includes active learning strategies, such as patient case discussions, simulated medication order entry and verification, drug information inquiries, medication therapy management, dose calculations, and intravenous compounding, among others, to enhance students’ learning. During the study period, students who completed the institutional IPPE reported improved confidence in their
90% 80% 80%
Percentage of Students
70% 60%
54% Pre-Test
50%
Post-Test 40% 30%
24% 18%
20% 10%
14% 4%
5%
2%
0% Strongly disagree
Disagree
Agree
Strongly agree
p < 0.001 for difference in confidence scores between the pre-test and post-test (n=84)
Fig. 3. Confidence in knowledge of hospital pharmacy practice: year two.
674
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
knowledge of basic hospital pharmacy practice. Prior hospital pharmacy practice experience did not have a significant impact on the extent of knowledge gained at the end of the institutional IPPE as evidenced by students’ performance on the post-test. There are some limitations to this study. Students’ performance on the assessment tool in both study years could not be combined and jointly analyzed due to the revisions made to the assessment tool in the second year of study. In the first year, the quantitative portion of the assessment tool utilized multiple-choice questions, and in year two, that section was revised to include a combination of short answer and multiple-choice questions. This was done to better assess students’ ability to apply and synthesize knowledge acquired during the IPPE. In addition, we had to exclude one of our sites in year two because the faculty member at that institution relocated, and the data set from their students was incomplete. Students were asked if they had any prior experience interning, working, or volunteering in a hospital pharmacy but were not asked to specify which of these three experiences was most applicable to them. Hence, it is not known if the type of prior hospital experience students had could have affected their baseline knowledge and extent of knowledge gained during the IPPE, and hence their performance on the preand post-tests. However, collectively, no difference was seen in post-test scores between students with prior hospital experience and those without. Additionally, it is difficult to determine if students’ improvement in confidence scores and performance on the post-test was solely due to the institutional IPPE or if additional learning took place in other didactic courses and patient care experiences during the time between the pre-test and post-test.
Conclusions Institutional IPPE plays a key role in preparing student pharmacists to evaluate, interpret, and accurately dispense prescribed medications and to provide optimal patient care in the hospital environment. The institutional IPPE program at Mercer University College of Pharmacy was successful at improving students’ knowledge and confidence in basic hospital pharmacy practice. Colleges of pharmacy should continually strive for ways to enhance institutional IPPE programs to develop confident, competent, and wellrounded pharmacy practitioners.
Acknowledgments We would like to thank Melissa M. Chesson, Bobby C. Jacob, Pamela M. Moye, and Phillip S. Owen for assistance with study instrument design and data collection and Kathryn M. Momary for her aid in statistical analysis. References 1. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. Adopted January 15, 2006. Revised January 23, 2011. Available at: 〈https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf〉; Accessed May 30, 2014. 2. Beck DE, Thomas SG, Janer AL. Introductory practice experiences: a conceptual framework. Am J Pharm Educ. 1996;60(2): 122–131. 3. Ma CS, Holuby RS, Bucci LL. Physician and pharmacist collaboration: the University of Hawaii Hilo College of Pharmacy—JABSOM experience. Hawaii Med J. 2010;69(6 suppl 3):42–44. 4. Chisholm MA, DiPiro JT, Fagan SC. An innovative introductory pharmacy practice experience model. Am J Pharm Educ. 2003;67(1): Article 22. 5. Walker PC, Kinsey KS, Kraft MD, Mason NA, Clark JS. Improving student education and patient care through an innovative introductory pharmacy practice experience. Am J Health Syst Pharm. 2011;68(8):655–660. 6. Kessler TL, Vesta KS, Smith WJ, Dennis VC, Bird ML, Harrison DL. Students’ attitudes and perceptions of a structured longitudinal introductory pharmacy practice experience (IPPE) medicine rotation. Curr Pharm Teach Learn. 2011;3(1):2–8. 7. Smith WJ, Bird ML, Vesta KS, Harrison DL, Dennis VC. Integration of an introductory pharmacy practice experience with an advanced pharmacy practice experience in adult internal medicine. Am J Pharm Educ. 2012;76(3): Article 52. 8. Mort JR, Johnson TJ, Hedge DD. Impact of an introductory pharmacy practice experience on students’ performance in an advanced practice experience. Am J Pharm Educ. 2010;74(1): Article 11. 9. Wuller WR, Luer MS. A sequence of introductory pharmacy practice experiences to address the new standards for experiential learning. Am J Pharm Educ. 2008;72(4): Article 73. 10. American Society of Health-System Pharmacists (ASHP)– American Council for Pharmacy Education (ACPE) Task Force. Entry-level competencies needed for pharmacy practice in hospitals and health-systems. Fall 2010. Available at: 〈http:// www.ashp.org/DocLibrary/MemberCenter/Entry-level-Compe tencies.aspx〉; Accessed May 30, 2014.
Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
Short communication
http://www.pharmacyteaching.com
Effectiveness of institutional Introductory Pharmacy Practice Experiences at achieving instructional objectives Nicole L. Metzger, PharmD, BCPS, Angela O. Shogbon, PharmD, BCPS* Department of Pharmacy Practice, Mercer University College of Pharmacy, Atlanta, GA
Abstract Objective: To assess the effectiveness of four hospital-based, faculty-facilitated, Introductory Pharmacy Practice Experiences (IPPEs) at meeting instructional objectives related to the Accreditation Council for Pharmacy Education (ACPE) standards. Methods: A pre-test and a post-test were administered to students to assess their acquisition of basic institutional pharmacy knowledge and perceptions of their ability and interest in hospital pharmacy practice. The test included demographic, openended, and multiple-choice questions designed to assess objectives in line with ACPE standards. Students’ pre-test and post-test scores were compared using paired t-tests. Differences in test scores and perceptions were assessed based on year and work experience. Results: Analysis of 88 students from year one and 84 students in year two showed significant improvement in test scores from pre-test to the post-test in year one (71.4% vs. 77.0%, p o 0.01) and year two (68.4% vs. 81.7%, p o 0.001). In year one, students with prior hospital experience (30%) scored significantly higher on the pre-test but not the post-test. In year two, no difference in scores was observed for students with prior hospital experience (23.8%) compared to those without. Students reported improved confidence in their knowledge of hospital pharmacy practice at the conclusion of IPPE (p o 0.001 in both years). Students’ likelihood of pursuing a career in hospital pharmacy did not change after institutional IPPE (p ¼ 0.184 in year one, p ¼ 0.075 in year two). Conclusions: The institutional IPPE program was successful at improving students’ knowledge in basic hospital pharmacy practice. Institutional IPPE improved students’ confidence but did not change their perceived career path. r 2014 Elsevier Inc. All rights reserved.
Keywords: Introductory Pharmacy Practice Experiences; Hospitals; Institutions; Instructional objectives; Experiential education
Introduction Pharmacists are widely regarded as medication experts and can provide evidence-based, cost-effective patient care. The Accreditation Council for Pharmacy Education (ACPE) describes their vision of future pharmacy practice as one where pharmacists ensure optimal medication therapy
* Corresponding author: Angela O. Shogbon, PharmD, BCPS, Department of Pharmacy Practice, Mercer University College of Pharmacy, 3001 Mercer University Drive, Atlanta, GA 303414155. E-mail: [email protected] http://dx.doi.org/10.1016/j.cptl.2014.05.008 1877-1297/r 2014 Elsevier Inc. All rights reserved.
outcomes for patients.1 They further recommend that pharmacy education prepares students for the delivery of patient-centered care that optimizes medication use, improves therapeutic outcomes, and promotes public health initiatives.1 However, along with recognition of the pharmacist as a medication expert comes the additional accountability for patient health outcomes, and hopefully, public recognition of the pharmacist’s ability to manage medications, improve cost-effectiveness, and patient outcomes.1 As a result, ACPE has increased emphasis on experiential education, specifically Introductory Pharmacy Practice Experiences (IPPEs). IPPEs have been defined as “practice experiences offered in various practice settings
668
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
during the early sequencing of the curriculum for purposes of providing transitional experiential activities and directed exposure to pharmaceutical care.”2 Guideline 14.4 of the ACPE standards recommends that IPPE include a total of 300 hours, of which, 150 hours be equally divided between community and institutional settings.1 While obtaining community IPPE hours is feasible for many schools, institutional IPPE hours are much more difficult to secure due to the smaller number of hospitals willing to participate. The Mercer University College of Pharmacy developed a unique approach to securing institutional sites for Advanced Pharmacy Practice Experiences (APPE) and IPPEs, which included strategically placing pharmacy practice faculty members at four local hospitals to facilitate these experiences. There are several descriptions of institutional IPPE programs in the literature, each with varying designs and structure, but all with a goal to increase the preparedness of students for APPEs. Some institutional IPPE programs have involved students’ working with interprofessional patient care teams, while others have incorporated direct patient care activities such as patient interviews and provision of medication reconciliation services.3–9 These programs have typically developed IPPE objectives based on competencies derived from their curriculum committee’s interpretation of ACPE standards. Some of these programs have utilized assessment tools to evaluate students’ self-assessment of IPPE and knowledge gained, faculty’s assessment of students’ performances on IPPE, and students’ readiness for experiential rotations.3–9 Current assessment of institutional IPPE in the literature includes mostly subjective data relying on students’ and faculty’s perceptions of competency.4–8 A longitudinal IPPE paired third-year students with faculty and APPE students to interview patients and to evaluate and present patient cases. Students were surveyed before and after IPPE to assess their perceptions of achieving 21 competencies developed by the school’s curriculum committee. Students reported significant improvement following IPPE in the scores for 15 of 21 competencies.4 Another patient carecentered IPPE involved third-year students providing medication reconciliation services at a local health system. Students improved their ability to perform medication reconciliation and reported positive attitudes about the medication reconciliation process and awareness about its importance.5 In a College of Pharmacy, an extensive block IPPE program was developed utilizing the ACPE competencies as a backbone for the IPPE syllabus and learning objectives. The authors reported that student journal entries and student and preceptor perceptions survey data indicated IPPE competencies were being achieved.9 In another report, first-year, second-year, and third-year IPPE students were integrated with APPE students on an internal medicine rotation. Students were surveyed on their attitude and perceptions before and after IPPE and reported improvement in their ability to describe the role of the clinical
pharmacist, describe how a pharmacist interacts with patients, locate information in a medical record, locate the answers to drug information questions, evaluate the medical literature, and counsel patients.6,7 Lastly, in a report evaluating faculty perceptions, faculty assessed students who completed a four-week IPPE course prior to APPE more favorably on a 5-point Likert scale, compared with students who had not taken the IPPE course.8 Though these reports provide valuable information about preceptors, faculty, and students perceptions of IPPE, they are limited in their ability to demonstrate objectively what students have learned. In an effort to more accurately characterize our students’ achievement of institutional IPPE instructional objectives, an assessment tool was designed to evaluate their acquisition and application of basic hospital pharmacy practice knowledge. During the 2009 and 2010 academic years, institutional IPPE at the Mercer University College of Pharmacy was conducted longitudinally during the third professional year. Rationale and objectives Institutional IPPE is critical in the development and application of key skills required to be a pharmacist. Several pedagogical articles on institutional IPPE have reported improvement in students’ perceptions of their abilities,4–7 but there is still a paucity of reports that quantitatively assess students’ knowledge after institutional IPPE. The purpose of this study was to quantitatively assess the effectiveness of institutional IPPE at meeting instructional objectives in line with ACPE standards Appendix D by evaluating students’ performance on a pre-test compared with a post-test knowledge-based assessment. Secondary objectives included assessing students’ baseline hospital experience, determining the impact of IPPE on students’ confidence in basic hospital pharmacy practice and their likelihood of pursuing a career in hospital pharmacy, and to determine if previous hospital experience resulted in higher pre- or post-test scores. Methods Third professional year student pharmacists were sent out in groups of four to six students to one of four local hospitals for a half-day per week, for four consecutive weeks. Students completed a total of 16 hours of institutional IPPE (a four-week block) in either the fall or spring semester of their third professional year. In order to evaluate students’ performance on institutional IPPE competencies derived from the ACPE standards Appendix D, an assessment tool that provided both qualitative and quantitative measurements was designed by IPPE faculty. Students were asked about their baseline hospital pharmacy experience, confidence in knowledge of hospital pharmacy practice, and their likelihood of pursuing a career in hospital pharmacy practice in the qualitative questions. The quantitative section
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
669
Table Test questions Year one
Year two
Have you ever been inside a hospital pharmacy before today?
No change
a. Yes b. No Have you ever volunteered, interned, or worked as a technician in a hospital pharmacy?
No change
a. Yes b. No How long have you volunteered, interned, or worked in a hospital pharmacy? a. b. c. d.
Never Less than one year 1–3 years Greater than three years
Rate your agreement with the statement “I feel confident in my knowledge of hospital pharmacy practice.” a. b. c. d. e.
Strongly disagree Disagree Neutral Agree Strongly agree
Rate your agreement with the statement “I am likely to pursue a career in a hospital pharmacy.” a. b. c. d. e.
How long have you worked in a hospital pharmacy? ____________
Strongly disagree Disagree Neutral Agree Strongly agree
The reason(s) for decentralized pharmacy practice model include which of the following:
Rate your agreement with the statement “I feel confident in my knowledge of basic hospital pharmacy practice.” a. b. c. d.
Strongly disagree Disagree Agree Strongly agree
Rate your agreement with the statement “I am likely to pursue a career in a hospital pharmacy.” a. b. c. d. e.
Strongly disagree Disagree Unsure Agree Strongly agree
List three (3) advantages of a decentralized pharmacy practice model.
a. To provide specialized pharmacy services b. Foster team-based approach to patient care c. Allow for quicker and more efficient clarification of therapeutic orders d. All of the above Which of the following is an example of a satellite pharmacy? a. b. c. d. e.
Operating Room (OR) pharmacy Neonatal Intensive Care Unit (ICU) pharmacy Main inpatient pharmacy A and B None of the above
Select the method of medication distribution from the pharmacy that stores medications on the nursing unit and utilizes electronic perpetual inventory? a. b. c. d.
No change
Automated dispensing cabinets Medication carts Pneumatic tube system Robotic dispensing technology
Describe two (2) ways automated dispensing cabinets, such as Pyxiss, improve medication distribution in the hospital setting?
670
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
Table Continued Year one
Year two
How are expired medications handled?
No change
a. b. c. d.
Incineration Reverse distributor Biohazardous waste Donation to non-profit clinics
Which regulation governs the aseptic standards for compounding sterile products in a hospital? a. b. c. d.
FDA Good Manufacturing Processes NSPG 03.05.01 USP 797 Joint Commission rule 295
Which of the following is an incorrect medication order (either unapproved abbreviation or wrong)? a. b. c. d.
75 ml/minute 64 ml/minute 46 ml/minute 39 ml/minute
Which of the following would be considered a therapeutic substitution? a. b. c. d.
Changing Changing Changing Changing
A A A A
patient patient patient patient
receiving levetiracetam for seizures receiving warfarin for atrial fibrillation receiving prednisone for joint pain taking furosemide for heart failure
Medication reconciliation is performed at the following times EXCEPT? a. b. c. d.
Calculate the creatinine clearance for the patient below using the Cockcroft Gault formula and show all of your work. Patient case: A 50 4″, 80-kg, 82-year-old African American female, with a SCr ¼ 1.2 mg/dl?
No change
metoprolol to amlodipine atorvastatin to simvastatin clopidogrel to aspirin fluticasone nasal spray to fluticasone MDI
Which of the following patients should have therapeutic drug monitoring? a. b. c. d.
Describe the components of a complete medication order
Nystatin 400,000 IU four times daily Nystatin 400,000 mg swish and swallow Q6 Nystatin 400,000 mg/kg swish and swallow QD All are incorrect or inappropriate
What is the creatinine clearance for the following patient: A 50 4″, 65-kg, 80-year-old African American female, with a SCr ¼ 0.8 mg/dl? a. b. c. d.
List two (2) components of an IV room that are compliant with USP 797 Regulations?
Admission Discharge Transfer from ICU to the medical floor Transfer from medical floor A to medical floor B
A physician wants to initiate warfarin 5 mg po daily in a patient. a. What lab value(s) should be monitored for this patient? b. Explain two (2) patient counseling points that you would address with the patient
Identify two (2) errors in this patient’s medication reconciliation? Home medications
Inpatient orders
Metoprolol succinate 50 mg po daily Levothyroxine 100 mcg po daily Lisinopril 10 mg po daily Acetaminophen 500 mg po BID prn pain Glipizide 10 mg po daily
Metoprolol tartrate 50 mg po daily Levothyroxine 100 mg po daily Lisinopril 10 mg po daily Acetaminophen 500 mg IV BID prn pain Glipizide 10 mg daily
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
671
Table Continued Year one
Year two
Which of the following represent(s) “Objective” information obtained for a patient?
Which of the following represents information that would be contained in the “Objective” portion of a patient's SOAP note?
a. Patient's heart rate is 90 beats per minute and respirations 18 breaths per minute b. Patient states he/she has been vomiting for two days c. Patient complains of abdominal pain d. Patient reports smoking tobacco two packs per day Select the appropriate monitoring parameter for the medication? a. b. c. d.
Warfarin-aPTT Enoxaparin-PT Amlodipine-HR Haloperidol-QTc
a. Patient's heart rate is 90 beats per minute and respirations 18 breaths per minute b. Patient states he/she has been vomiting for two days c. Patient needs a chest X-ray to rule out pneumonia d. Patient reports smoking tobacco two packs per day Classify the drug information references listed below: 1. 2. 3. 4.
Trissel's Handbook on Injectable Drugs PubMed DiPiro's Pharmacotherapy: A Pathophysiologic Approach A journal article comparing the efficacy of glipizide to metformin a. Primary reference b. Secondary reference c. Tertiary reference
You are staffing in the inpatient pharmacy. A nurse from the ICU calls down and wants you to verify her settings for the IV pump for a dopamine infusion. Dopamine is ordered at 3 mcg/kg/minute. The dopamine bag contains 800 mg in 500 ml of fluid. The patient weighs 75 kg. What infusion rate, in ml/h, should the nurse program the pump for? Show your work. Select the patient with a significant medication related problem. a. b. c. d.
included questions that were developed to assess students’ comprehension of objectives in line with the Core Domains from Appendix D of the ACPE standards. The Table lists the test questions posed during year one and year two. Students were administered the same IPPE assessment tool as a pre-test prior to the institutional IPPE and as a post-test at the conclusion of their IPPE block. In the first year of the study, the pre-test was administered on the first day of the institutional IPPE before the students began the experience; in the second year, the pre-test was administered at the start of the academic year for third professional year student pharmacists. The post-test was given at the end of each four-week long institutional IPPE during both years. Multiple-choice questions were assessed by faculty at each site, and open-ended questions from year two were scored by the study authors using a standardized rubric. The study authors met to review their independent assessment of open-ended questions to ensure consistency in grading. Students’ responses to demographic questions were analyzed using descriptive statistics and the Wilcoxon signed rank test. Comparison between students’ pre-test and post-test scores was done using paired t-tests. Data analysis was
Lisinopril with a serum potassium of 4.0 Amlodipine with a heart rate of 55 beats per minute Ibuprofen in a patient with chronic kidney disease Memantine in a patient with moderate Alzheimer's
performed using Statistical Package for the Social Sciences (SPSS) version 16.0 (SPSS Inc, Chicago, IL). This study was approved by the Institutional Review Board at Mercer University, and a waiver of informed consent was granted. Results A total of 88 (98.9%) students completed the pre-test and post-test at four faculty-facilitated IPPE institutions in year one and 84 (86.6%) students at three faculty-facilitated IPPE institutions in year two. One institution that was included in year one was not included in data collection for year two because the faculty member who was facilitating IPPE at that site no longer worked at the study institution. Baseline characteristics revealed that 58 (66%) students in year one and 46 (54.8%) students in year two reported a prior visit to a hospital pharmacy. Of those, only 26 (30%) students in year one and 20 (23.8%) students in year two reported volunteering, interning, or working in a hospital pharmacy. For the primary end point, students showed significant improvement in test scores from pre-test to the post-test in year one (71.4% vs. 77.0%; p o 0.01) and year two (68.4% vs. 81.7%; p o 0.001) (Fig. 1). Students
672
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
reported improved confidence in their knowledge of hospital pharmacy practice at the conclusion of IPPE (p o 0.001 for both years), and their likelihood to pursue a career in hospital pharmacy practice did not significantly change by the conclusion of institutional IPPE (p ¼ 0.184 in year one and p ¼ 0.075 in year two). Figures 2 and 3 illustrate the breakdown of student responses regarding their confidence in hospital pharmacy practice for both years. Significantly higher pre-test scores occurred in students with prior hospital experience compared to those without during year one, with mean ⫾ SD scores of 78.5% ⫾ 17.5% vs. 68.4% ⫾ 13.3%, respectively, p o 0.01. However, there was no significant difference in pre-test scores in year two between students with and without prior hospital experience, with mean ⫾ SD scores of 69.8% ⫾ 10.21% vs. 68.0% ⫾ 11.37%, respectively, p = 0.525. At the conclusion of institutional IPPE in each year, no significant difference in post-test scores was observed between students with prior hospital pharmacy experience and those without experience, with mean ⫾ SD scores in year one of 79.2% ⫾ 10.6% vs. 76.1% ⫾ 11.2%, respectively, p = 0.183, and in year two of 82.6% ⫾ 8.84% vs. 81.4% ⫾ 7.49% respectively, p = 0.542. Discussion This study demonstrated improvement in students’ knowledge of hospital pharmacy practice after completion
of a half-day, four-week long institutional IPPE program. Appendix D of the Accreditation Standards and Guidelines put forth by ACPE highlights the IPPE core domains and ability performance statements in which the students must demonstrate their abilities and competency to ensure optimal preparation for APPEs.1 Some of these core domains are patient safety, basic patient assessment, medication information, identification and assessment of drugrelated problems, dose calculation, drug information analysis, and literature search.1 The institutional IPPE program at Mercer University College of Pharmacy was designed to meet objectives in line with these standards, and students’ acquisition of knowledge at the end of this program was evident through their performance on the post-test assessment tool. After our assessment tool was designed, the American Society of Health-System Pharmacists (ASHP) and ACPE joint task force released a report describing competencies necessary for entry-level pharmacy practice in hospitals and health-systems.10 The institutional IPPE program at Mercer University College of Pharmacy is instrumental in ensuring students achieve many of these stated competencies. The institutional IPPE program continues to evolve. Since this study was conducted, the institutional IPPE program has changed to increase the time spent in the institutional setting based on updated ACPE standards. Currently, third-year Doctor of Pharmacy students spend eight hours per day for two weeks in an assigned
100% 90% 81.7%
77.0%
80% 71.4%
68.4%
70% 60% 50%
Pre-Test Post-Test
40% 30% 20% 10% 0% Year 1 Year 2 p < 0.01 for difference in mean pre-test and post-test scores in year 1 (n=88) and year 2 (n= 84) Bars represent standard deviation of the mean
Fig. 1. Overall performance across all institutions: year one and two.
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
673
70% 61% 60%
Percentage of Students
50%
45%
40%
Pre-Test Post-Test
30% 22% 19% 18%
20%
14% 11% 10%
6% 1%
1%
0% Strongly disagree Disagree Neutral Agree Strongly agree p < 0.001 for difference in confidence scores between the pre-test and post-test (n=88)
Fig. 2. Confidence in knowledge of hospital pharmacy practice: year one.
institution during the beginning of their spring semester. The number of sites has also expanded from four to nine to accommodate the increased hours. Despite the changes to the number of hours and the increase in the number of sites, institutional IPPE at Mercer University College of Pharmacy has consistently involved students working with faculty, pharmacists, pharmacy residents, pharmacy interns and technicians, and some APPE students to learn the objectives set forth for the IPPE program.
Institutional IPPE at Mercer University College of Pharmacy, past and present, provides students with experiential training and includes active learning strategies, such as patient case discussions, simulated medication order entry and verification, drug information inquiries, medication therapy management, dose calculations, and intravenous compounding, among others, to enhance students’ learning. During the study period, students who completed the institutional IPPE reported improved confidence in their
90% 80% 80%
Percentage of Students
70% 60%
54% Pre-Test
50%
Post-Test 40% 30%
24% 18%
20% 10%
14% 4%
5%
2%
0% Strongly disagree
Disagree
Agree
Strongly agree
p < 0.001 for difference in confidence scores between the pre-test and post-test (n=84)
Fig. 3. Confidence in knowledge of hospital pharmacy practice: year two.
674
N.L. Metzger, A.O. Shogbon / Currents in Pharmacy Teaching and Learning 6 (2014) 667–674
knowledge of basic hospital pharmacy practice. Prior hospital pharmacy practice experience did not have a significant impact on the extent of knowledge gained at the end of the institutional IPPE as evidenced by students’ performance on the post-test. There are some limitations to this study. Students’ performance on the assessment tool in both study years could not be combined and jointly analyzed due to the revisions made to the assessment tool in the second year of study. In the first year, the quantitative portion of the assessment tool utilized multiple-choice questions, and in year two, that section was revised to include a combination of short answer and multiple-choice questions. This was done to better assess students’ ability to apply and synthesize knowledge acquired during the IPPE. In addition, we had to exclude one of our sites in year two because the faculty member at that institution relocated, and the data set from their students was incomplete. Students were asked if they had any prior experience interning, working, or volunteering in a hospital pharmacy but were not asked to specify which of these three experiences was most applicable to them. Hence, it is not known if the type of prior hospital experience students had could have affected their baseline knowledge and extent of knowledge gained during the IPPE, and hence their performance on the preand post-tests. However, collectively, no difference was seen in post-test scores between students with prior hospital experience and those without. Additionally, it is difficult to determine if students’ improvement in confidence scores and performance on the post-test was solely due to the institutional IPPE or if additional learning took place in other didactic courses and patient care experiences during the time between the pre-test and post-test.
Conclusions Institutional IPPE plays a key role in preparing student pharmacists to evaluate, interpret, and accurately dispense prescribed medications and to provide optimal patient care in the hospital environment. The institutional IPPE program at Mercer University College of Pharmacy was successful at improving students’ knowledge and confidence in basic hospital pharmacy practice. Colleges of pharmacy should continually strive for ways to enhance institutional IPPE programs to develop confident, competent, and wellrounded pharmacy practitioners.
Acknowledgments We would like to thank Melissa M. Chesson, Bobby C. Jacob, Pamela M. Moye, and Phillip S. Owen for assistance with study instrument design and data collection and Kathryn M. Momary for her aid in statistical analysis. References 1. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. Adopted January 15, 2006. Revised January 23, 2011. Available at: 〈https://www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf〉; Accessed May 30, 2014. 2. Beck DE, Thomas SG, Janer AL. Introductory practice experiences: a conceptual framework. Am J Pharm Educ. 1996;60(2): 122–131. 3. Ma CS, Holuby RS, Bucci LL. Physician and pharmacist collaboration: the University of Hawaii Hilo College of Pharmacy—JABSOM experience. Hawaii Med J. 2010;69(6 suppl 3):42–44. 4. Chisholm MA, DiPiro JT, Fagan SC. An innovative introductory pharmacy practice experience model. Am J Pharm Educ. 2003;67(1): Article 22. 5. Walker PC, Kinsey KS, Kraft MD, Mason NA, Clark JS. Improving student education and patient care through an innovative introductory pharmacy practice experience. Am J Health Syst Pharm. 2011;68(8):655–660. 6. Kessler TL, Vesta KS, Smith WJ, Dennis VC, Bird ML, Harrison DL. Students’ attitudes and perceptions of a structured longitudinal introductory pharmacy practice experience (IPPE) medicine rotation. Curr Pharm Teach Learn. 2011;3(1):2–8. 7. Smith WJ, Bird ML, Vesta KS, Harrison DL, Dennis VC. Integration of an introductory pharmacy practice experience with an advanced pharmacy practice experience in adult internal medicine. Am J Pharm Educ. 2012;76(3): Article 52. 8. Mort JR, Johnson TJ, Hedge DD. Impact of an introductory pharmacy practice experience on students’ performance in an advanced practice experience. Am J Pharm Educ. 2010;74(1): Article 11. 9. Wuller WR, Luer MS. A sequence of introductory pharmacy practice experiences to address the new standards for experiential learning. Am J Pharm Educ. 2008;72(4): Article 73. 10. American Society of Health-System Pharmacists (ASHP)– American Council for Pharmacy Education (ACPE) Task Force. Entry-level competencies needed for pharmacy practice in hospitals and health-systems. Fall 2010. Available at: 〈http:// www.ashp.org/DocLibrary/MemberCenter/Entry-level-Compe tencies.aspx〉; Accessed May 30, 2014.