- Email: [email protected]
Integrating Standardized Patients and Objective Structured Clinical Examinations Into a Nurse Practitioner Curriculum
BY FACULTY FOR FACULTY
Integrating Standardized Patients and Objective Structured Clinical Examinations Into a Nurse Practitioner Curriculum Benjamin Miller, PhD, FNP, and Katherine Camacho Carr, PhD, CNM ABSTRACT
Simulation has been used as a teaching pedagogy in health science disciplines for more than a century. Integrating standardized patients into simulation provides a high level of realism and is an excellent teaching and evaluation modality for nurse practitioner and nurse-midwifery programs. A standardized patient (SP) is coached to perform as a patient in a specific clinical scenario, while a number of students engage in a simulated encounter with SPs. Students are observed and evaluated with a checklist to assess performance. SPs are used in a single case teaching experience or in a multistation evaluation experience called an objective standardized clinical examination. Keywords: clinical teaching, objective structured clinical examinations, standardized patients Ó 2016 Elsevier, Inc. All rights reserved.
S
imulation in nursing education is a teaching modality allowing students to practice or demonstrate skills in a safe environment without risk of harm to real patients. The 2014 National Simulation Study by the National Council of State Boards of Nursing suggested high-quality simulation in undergraduate curriculum can be used to substitute up to 50% of the required clinical hours without altering the educational outcomes.1 Clinical placement is a challenge for both undergraduate and graduate nursing programs. Nurse practitioner (NP) and nurse-midwifery (NMW) programs have additional challenges with the scarcity of one student to one clinical preceptor placements. Accrediting agencies for NP and NMW programs mandate a minimum number of clinical hours and currently do not recognize simulation as a substitute for clinical preparation. Barriers to successful preceptor recruitment, including concerns of reduced productivity while precepting students and lack of formal preparation for the clinical preceptor role, have been cited as the leading reasons to not precept NP students.2,3 There are a number of questions that must be answered regarding simulation and advanced practice www.npjournal.org
nursing education outcomes. Integration of standardized patient (SP) encounters and objective structured clinical examinations (OSCEs), as teaching and evaluation tools for NP students may provide evidence of the clinical competence attained during the education program. The purpose of this paper is to describe the use of simulation with SPs as a teaching, learning, and evaluation methodology in graduate nursing education. BACKGROUND
Simulation scenarios with and without SPs have been used by a number of health science disciplines for over 50 years, in line with the need for new models of NP clinical teaching and learning.4 According to Rehmann et al., simulation has 3 distinct dimensions: (1) equipment fidelity; (2) environment fidelity; and (3) psychological fidelity. The term “fidelity” loosely represents the degree of realism of the simulation scenario. Equipment and environmental fidelity refer to the appearance of realism. Psychological fidelity is the degree to which the learner suspends disbelief and perceives the scenario is real. Achieving psychological fidelity is The Journal for Nurse Practitioners - JNP
e201
the most complicated and difficult dimension to achieve.5,6 In health care, a simulation scenario mimics a human experience using a number of methodologic approaches from low-fidelity simulation to complex high-fidelity simulation. Low-fidelity simulation can consist of case study evaluation or task trainers, which focus on skill acquisition. Mid- and high-fidelity simulation extends the learning and evaluation to clinical reasoning and application.6 SPs are prepared to “act” out a specific clinical situation, thus providing a high degree of realism to the learner. The SP simulation experience can be designed as a low- to mid-fidelity experience with the objective focused on skill attainment, or as high-fidelity when the learner is asked to utilize clinical decisionmaking to assess, interpret, and intervene based on the scenario.6 The SP provides a dimension of high environment and psychological fidelity. Integrating simulation with SPs into graduatelevel education programs provides “real-life” situations while offering opportunities for deeper comprehension/application of core competencies, including: (1) physical examination techniques; (2) communication skills; and/or (3) technical/ procedural skills. Depending on the learner’s preparation and simulation objectives, simulation activities can be integrated into the curriculum as a teaching modality, preclinical assessment tool, or summative clinical evaluation.7 OSCEs, utilizing SPs, can create a high-fidelity, high-stakes evaluation environment to assess comprehension, clinical competency, and critical thinking. Depending on the micro-objectives of the OSCE, this can be a potent teaching pedagogy or a dynamic evaluation tool. Throughout the NP educational program, formation of critical thinking in a simulated environment is an important aspect of learning to both think and act like a health care provider.8 Preclinical OSCE evaluations require students to demonstrate basic skill mastery before starting their clinical experience. Academic programs must prepare students to be efficient, productive, and safe in the clinical setting to reduce the burden on the busy preceptor while promoting positive patient outcomes.9
e202
The Journal for Nurse Practitioners - JNP
BENEFITS OF A STANDARDIZED PATIENT/ SIMULATION PROGRAM
One of the major challenges for NP faculty is the evaluation of student clinical performance. Faculty administer written examinations, require studentcreated reports, evaluate oral case presentations, and have students complete case studies to evaluate knowledge and skill acquisition. Classroom-based examinations may lack content validity if not covering all of the necessary content. Standard classroom assessments may lack the ability to evaluate clinical application and essential clinical decisionmaking (CDM), which is needed in clinical practice.10 To evaluate students’ clinical application and essential clinical decisionmaking, faculty perform brief direct observations of students in the clinical setting, while relying on input from the clinical preceptors to fully assess a student’s competency. Direct observation in the clinical setting gives faculty minimal opportunity to observe and evaluate students in real-life clinical encounters. Further challenges include inconsistency in the type of patient visit, the complexity of the visit, the skills required, and the clinical setting. Implementation of a graduate simulation program with standardized patients provides a number of benefits to teaching, learning, and evaluation of NP students. A primary benefit of an SP program is the ability to create a safe, realistic environment for the students to learn. Faculty assess and document a student’s performance in a simulated clinical setting with standardized clinical presentations, which may not be available in all clinical settings. Real-life clinical experiences are unpredictable complicating student evaluations; however, simulated scenarios are developed with standardized specific conditions to demonstrate core competencies. Core competencies are leveled across the curriculm and include the ability to obtain a clinical history, perform an appropriate physical exam, conclude an accurate diagnosis, prescribe/order suitable treatments, provide patient education/counseling, or discuss complex and difficult situations with SPs.11-18 The high-fidelity simulation with SPs can be constructed in a variety of formats. The simulation,
Volume 12, Issue 5, May 2016
which requires comprehensive history collection, conducting a physical exam, integrating diagnostic reasoning, and developing treatments plans, is complex and can take a long time to complete, thus limiting the number of students who can be assessed. Simulations that focus on microtasks of the encounter, such as demonstration of an appropriate physical exam for a predetermined constellation of symptoms, can assess application and skill. Simulations with shorter microtask evaluations can allow a greater number of clinical assessments per student and evaluation of more students over a specific time frame.6,19 Simulation with SPs provides students with the opportunity to be evaluated on their ability to diagnose and develop treatment plans. Simulated patients also reduce risk or adverse consequences to real patients by allowing students to practice newly acquired skills.20 In a simulation learning environment, students receive immediate feedback and remediation techniques to achieve mastery. Formative evaluation of students through simulation assesses the student’s confidence in using the correct technique, whereas summative evaluation assesses the competence level the NP student has achieved with regard to specific clinical objectives.21 Increasing student confidence and competence through preclinical preparation strengthens relationships with clinical partners, preceptors, and agencies. Clinical preceptors are NPs, physician assistants, and physicians who maintain a clinical practice, while volunteering to precept (teach) NP and NMW students. Clinical preceptors expect students to perform core competencies of clinical care and demonstrate a novice level of clinical reasoning. Utilization of preclinical OSCEs with SPs leveled across the curriculum can help to establish a minimum level of performance competency before the student begins clinical rotations. A standard minimum level of performance expectations provides reassurance to clinical agencies and preceptors that students are prepared to maximize their experiences in the rich clinical environment, while decreasing the burden of working with students. The focus of clinical education shifts to the attainment of escalating complex competencies and refinement of required knowledge, skills, and attitudes, as opposed to attainment of a www.npjournal.org
required number of clinical hours and assumed competency upon completion of the clinical rotation.9 DEVELOPING A STANDARDIZED PATIENT PROGRAM
Although the incorporation of an SP program into NP education has many benefits, it can be challenging, costly, and time-consuming for educators and NP education programs. A review of the required competencies for the NP population focus, program outcome objectives, degree requirements, and interdisciplinary and cultural competencies should be considered in developing an SP program. When multiple population foci exist, an identification of the common key competencies is helpful when deciding on the development of case scenarios that may be used by more than one specialty within the NP program of study. For example, each specialty could benefit from an SP case that includes diversity/inclusivity or interdisciplinary collaboration and/or consultation. A blueprint for each population focus can be constructed, detailing both the shared and unique SP scenarios and OSCEs, along with sequencing in the program of study. An SP program should consider the following 4 key aspects: (1) development of the simulation scenario, including the objectives, placement in the curriculum, the scenario template, validation of the checklist, and student evaluation; (2) development of SPs; (3) preparation of the scenario evaluators and faculty; and (4) simulation debriefing. DEVELOPING AN OSCE SIMULATION SCENARIO FOR THE SP
A simulation scenario is a fictitious, but clinically accurate and pertinent, patient presentation that has predefined learning objectives based on the NP population foci. The clinical scenario has a realistic presentation with history of present illness; primary medical, surgical, and social history; and physical exam findings all being relevant data in portraying a realistic scenario. The first step in developing the SP case is to identify clear and measureable learning outcomes for the encounter.22,23 Learning objectives can focus on cognitive, psychomotor, and/or affective goals. The The Journal for Nurse Practitioners - JNP
e203
Table 1. Sample Template for Standardized Patient Case Scenario
Table 1. (continued )
Title and overview
Current life situation and social history (marital status, health habits)
Development date
Personality, culture, language—if pertinent to the case
Developer(s)
Past medical surgical history
Learners and where this fits in the curriculum:
Pertinent family history
Objective(s)
Medications
Logistics:
Allergies
Personnel
Encounter beginning
Forms
Encounter ending (time-limited)
Room/Equipment/Props
Physical exam expected Special instructions, such as: appear to be slow in answering questions; appear agitated; or act as if you are fine despite symptoms
Instructions to the examinee/participant Patient’s name Age Consider marital status, occupation, medications, allergies, and pregnancy history, if pertinent to the case Presenting complaint/reason for encounter Key objectives of the case: history, physical assessment/ evaluation, diagnosis, communication/counseling, patient education, referral, and timing Standardized patient script Case name Name Age Consider marital status, occupation, medications, allergies, medical, surgical, and pregnancy history, if pertinent to the case Components of patient history22,23: History of present illness Mechanism of illness or injury Description of symptoms: Onset, duration, and location and/or radiation of symptoms, including any precipitating events, illnesses, or injury Frequency, intensity, and progression of symptoms Quality, intensity, location, and/or radiation of pain Aggravating and alleviating factors Associated symptoms continued
e204
The Journal for Nurse Practitioners - JNP
Note: Sample variables to be considered in the development of a SP scenario including backstory script for SP.
learning tasks focus on specific competencies, such as attaining a focused health history, performance of accurate physical assessment, communication skills, mental health assessment, cultural sensitivity, and/or spiritual care.24 To optimize student learning and assessment, the scenario should focus on parts of an exam to reinforce content exposure.7 For example, the scenario may focus on a straightforward, focused complaint or a complex presentation necessitating a branching history, or a physical exam to assess the presenting complaint appropriately. Scenarios can be constructed to meet one or more educational objectives, although they should always permit students to incorporate a variety of competencies, allowing for the demonstration of an ability to assess, develop, and implement a plan of care. As students progress through the curriculum, cases should require them to demonstrate the knowledge base and assessment, critical thinking, technical, and interpersonal skills required for beginning level practice.22,23 After development of the learning objectives, a template containing a summary of all the case information should be developed (see Table 1). To provide a realistic scenario, all aspects of the fictitious scenario should be developed in advance to include
Volume 12, Issue 5, May 2016
social history, family history, and employment/ occupational exposure. The details of the scenario are essential for the SP role as the actor prepares for the performance and unanticipated questions. Optimal scenario development for SP cases is a team effort with faculty working together to create a fictional, realistic, and evidence-based patient scenario. The presentation of the illness, injury, or problem, as well as assessment and management, should be based on accepted standards of practice for the population foci. The scenario competencies are appropriate for the student’s level of preparation, with an authentic and typical presentation of signs and symptoms. A clear and consistent initial description of the case will ensure that all students view the case from the same perspective.
Setting a scenario’s time limit contributes to the realistic clinical expectations and should be established for logistical planning. The time limit for each station must consider the level of student preparation and the learning objectives (30 minutes for a first quarter student vs. 10 minutes for a graduating student). Awareness of the time limits is essential for the learner to develop clinical time management skills. Assessing Student Performance
Developing a clinical scenario creates a learning opportunity for the student to experience a standardized, reliable, realistic patient presentation. SPs work from an in-depth script, so students receive the same information during the scenario. Based on the clinical objectives, the microcomponents of the
Table 2. Head- Ears-Eyes- Nose-Throat (HEENT)-focused Physical Exam Evaluation Checklist 1)
Participant washes hands prior to the examination.......... Did not perform ¼ 0, performed ¼ 1
⓪ ①
2)
Introduces self to the patient.....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
3)
Performs a general survey of the client................ Did not perform ¼ 0, performed ¼ 1
4)
Performs an otoscope exam ............................Did not perform ¼ 0 Unilateral ¼ 1 Bilateral ¼ 2
5)
Examines the eyes, looks at conjunctiva and sclera.........Did not perform ¼ 0, performed ¼ 1
⓪ ①
6)
Examines the nasal mucosa with otoscope .............Did not perform ¼ 0, performed ¼ 1
⓪ ①
7)
Examines the oral cavity .......................Did not perform ¼ 0, performed ¼ 1
⓪ ①
8)
Examines the hypopharynx. ............................Did not perform ¼ 0 Performed without use of tongue depressor ¼ 1 Performed with use of tongue depressor ¼ 2
⓪ ① ②
9)
Assess the client’s voice .....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
⓪ ① ⓪ ① ②
10) Examines the neck for adenopathy..... ...................Did not perform ¼ 0 Check tonsillar or cervical (not both) ¼ 1 Check tonsillar and cervical nodes ¼ 2
⓪ ① ②
11) Assess for nuchal rigidity .....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
12) Assess client’s lung sounds ...................Did not perform ¼ 0, performed ¼ 1
⓪ ①
13) Assess clients heart tones .......................Did not perform ¼ 0, performed ¼ 1
⓪ ①
Total score ____________ / 16 points Overall assessment Engages in a therapeutic relationship with the client during the encounter
① ② ③
Had an organized approach during the exam/history
① ② ③
Demonstrated competence and confidence during the exam/history
① ② ③
Note: A sample of a HEENT examination checklist is shown. Each item has a numerical value to indicate completion. The overall assessment is based on the evaluator’s opinion of the entire encounter.
www.npjournal.org
The Journal for Nurse Practitioners - JNP
e205
clinical encounter are identified, and an objective, valid checklist is created to assess the student performance. Checklists are constructed by faculty and/or a small group of experts, based on standards of practice and the scientific evidence relating to the specific case (see Tables 2 and 3). Items on the checklist are usually scored as dichotomous indicators (performed or not performed). Items may also be rank-ordered if the objective has multiple parts, so feedback on performance can be provided to the student for future improvement in interviewing, skill performance, counseling, and clinical decisionmaking.25 Newly developed case-specific checklists should be assessed for intra- and interrater reliability using standard statistical methods. Pilot testing of case scenarios ensures student performance is assessed accurately. Intrarater reliability refers to the consistency of scoring by an evaluator across multiple students. Interrater reliability refers to the consistency of scoring by several evaluators, assessing the same student’s performance. Both intra- and interrater
reliability are dependent on the evaluators consistently interpreting items on the case scenario checklist and consistently scoring student performance, thus providing a detailed explanation of the reliability assessment procedure for performance evaluations.26,27 The reliability scale uses 0 as unreliable and 1 as very reliable. In performance assessments for high-stakes decisions, reliabilities > 0.85 (Cronbach’s alpha) are considered acceptable.10 Depending on the type of simulation (formative or summative), student feedback can be immediate with remediation, delayed, or aggregated. In some situations, the SP may provide “patient” perspective feedback to the student.23 For practice and learning stations, having students function as an evaluator provides a different method of learning, whereas assessment of clinical or course objectives should be evaluated by faculty. Video recording student performance offers a number of positive attributes. Students can watch their own performance after the simulated experience and provide a critical selfappraisal of both the positive aspects and areas for
Table 3. Focused History for a Person With Pneumonia Symptoms 1) Introduces self to the patient...................Did not perform ¼ 0, performed ¼ 1
⓪ ①
2) Asks patient to describe what is going on with open-ended questions...Did not perform ¼ 0 performed ¼ 1
⓪ ①
3) Asks about the onset and duration of symptoms ..................Did not perform ¼ 0 Asked about onset or duration (not both) ¼ 1 Asked about onset and duration ¼ 2
⓪ ① ②
4) Asks about aggravating or alleviating factors ...................Did not perform ¼ 0 Asked about aggravating or alleviating factors (not both) ¼ 1 Asked about aggravating and alleviating factors ¼ 2
⓪ ① ②
5) Asks about hemoptysis....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
6) Asks about medication allergies.................Did not perform ¼ 0, performed ¼ 1
⓪ ①
7) Asks about current medications................Did not perform ¼ 0, performed ¼ 1
⓪ ①
8) Assess for associated symptoms (fevers, chills, rigors).............Did not perform ¼ 0 Asked about 1 associated symptom ¼ 1 Asked about 2 or more associated symptoms ¼ 2
⓪ ① ②
Total score ____________ / 11 points Global assessment Engages in a therapeutic relationship with the client during the encounter
① ② ③
Had an organized approach during the exam/history
① ② ③
Demonstrated competence and confidence during the exam/history
① ② ③
Note: A sample of a history-gathering scenario for a person with pneumonia is shown. Each item has a numerical value to indicate completion. The overall assessment is based on the evaluator’s opinion of the entire encounter.
e206
The Journal for Nurse Practitioners - JNP
Volume 12, Issue 5, May 2016
Table 4. OSCE Time Schedule Session for Session 1a
1300-1315
Station A
Station B
Station C
Station D
Station E
Station F
Chart Station F
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
(Student 007)
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
—
—
—
—
—
Student 007
1315-1317 Rotate 1317-1332
Student 007
1332-1334 Rotate 1334-1349
Student 006
1349-1351 Rotate 1351-1406
Student 005
1406-1408 Rotate 1408-1423
Student 004
1423-1425 Rotate 1425-1440
Student 003
1440-1442 Rotate 1442-1457
Student 002
1457-1459 Rotate 1459-1514
—
1514-1545 Break Note: A sample schedule of an objective structured clinical examination (OSCE) evaluation session (7 testers) is shown. a Session 1 (15-minute stations with 2-minute rotations).
growth. Finally, video recording during high-stakes testing provides documentation of student performance. Faculty review the assessment(s) of performance by sharing the results of the checklist with the student and offer constructive feedback. Faculty can immediately identify strengths and areas of deficiency related to the educational objectives of the case, which can be addressed or remediated. Over time, faculty and students can document professional growth in terms of competency and confidence.28 Standardized Patient Development
Standardized patient scenarios utilize a developed template to meet specific learning objectives for the session. Standardized patients are “actors,” playing a role as a patient. When recruiting SPs, avoid using faculty or health care professionals as the SPs, because there may be a tendency for the health care professional to inadvertently coach or offer critical information. When conducting high-stakes OSCE testing, use of faculty may impose undue stress and, www.npjournal.org
potentially, emphasize a power differential, both of which can affect student performance.14,29 Training “actors” as standardized patients is a critical part of developing an SP program.24,30 Key points in training SPs include detailed and specific signs/symptoms, physical characteristics, facial expressions, and any vocalizations pertinent to the scenario. Consistency in the portrayal of SPs is critical for standardization between students. During training sessions, faculty should encourage SPs to ask questions to obtain a clear understanding. SPs should also be prepared for students to ask questions or perform assessments not pertinent to the main focus of the scenario. The development of the specific case should provide enough data to guide SPs when asked non- essential questions. Instruct SPs to answer the questions and perform to the best of their ability based on the case.30 A standardized training session should be performed 1-2 weeks in advance to introduce the case scenario to SPs. The training should provide enough time to review the scenario and should be followed The Journal for Nurse Practitioners - JNP
e207
by 2 or 3 practice sessions to ensure SP understanding and reliability. SPs must demonstrate they comprehend the detailed history of the present condition, illness, injury, or problem. Scenario details include the mechanism of illness or injury and a comprehensive description of the onset, duration, location, radiation, frequency, intensity, and progression of symptoms. Further details include quality, intensity, location, and radiation of pain symptoms. Actors must be informed of the typical aggravating and alleviating factors and any associated symptoms. The current life situation and social history, such as marital status or health habits, should be available and rehearsed with SPs to portray a realistic presentation. If pertinent to the case objectives, SPs should also be informed of the personality type, culture, and language to be portrayed. Past family, medical, surgical, and obstetric history, as well as medications and allergies, should also be included in the script. It is helpful to inform SPs how long the encounter is to last, what physical exam and counseling may be expected, and whether or not they should cue the student or not. Special instructions, such as appearing agitated, acting slow in response time, or being in denial about a condition, can also be communicated in the script. A complete and clearly written SP script will help the actor portray a reproducible and realistic situation. It is critical that SPs respond the same way with every student encounter to maintain intrarater reliability and scenario standardization. Evaluator Development
Faculty and qualified members of the profession, and SPs in some cases, may be asked to perform as evaluators during the simulation or OSCE sessions. The evaluation checklists are developed according to evidence-based practice guidelines and standards of care to establish content validity and evaluated prior to the simulation or OSCE to establish interrater reliability. The reliability of the checklist is dependent on the evaluator. To maintain reliability, the evaluators should receive an in-depth orientation to the scenario and maintain a clear and consistent evaluation of every student. It is important to establish both intra- and interrater reliability of the case-specific checklists during a pilot test to ensure that student
e208
The Journal for Nurse Practitioners - JNP
performance is assessed accurately, especially in highstakes situations. Meeting with the evaluators before each SP simulation or OSCE encounter is valuable to practice the case-specific checklist and identify any items that are problematic. During the scenario, many events happen simultaneously, and being able to observe, listen, and record information can be challenging. Practicing these skills during the evaluator orientation through a mock scenario will help ensure intra- and interrater reliability. If the simulation or OSCE is video recorded, multiple evaluators can re-evaluate the same scenario, providing ongoing interrater reliability as well as validating student performance. A timely review is essential to provide feedback to students, SPs, and evaluators. Faculty Facilitator Role
The faculty facilitator plays a pivotal role in guiding the simulated clinical learning experience. The facilitator should understand the prerequisite knowledge, level, and abilities of the student participants and develop the expected outcomes of the case scenario or OSCE based on this information. The faculty facilitator should be clinically proficient and have knowledge of evidence-based practices to identify knowledge or performance gaps. The facilitator must clearly communicate to the participants the learning objectives and expected outcomes of the experience. The overall goal is to create a safe experiential learning environment that encourages active learning, repetitive practice, and reflection. In addition, the facilitator must promote fidelity and provide clear direction for formative or summative evaluation of student performance, using a valid and reliable instrument. Facilitators provides support to students, SPs, and evaluators throughout the simulated experience. They also provide constructive feedback and debriefing with the participants, encouraging self-reflection on performance and identification of meaningful feedback from peers and faculty that will assist participants to improve their practice. Evaluation of the SP or OSCE experience by the faculty facilitator and participants is used to make adjustments to the simulated experience.24
Volume 12, Issue 5, May 2016
Debriefing
Debriefing is an essential aspect of all simulation experiences where students engage in reflective learning. Students are exposed to a number of sensory, tactile, and emotional stimuli during a scenario, and the debriefing process allows for a safe interpretation of these experiences.31,32 SP and OSCE simulated experience debriefing sessions should be based on a structured framework to promote reflective discussion.24 It is helpful if the faculty facilitator establishes rules for participation in debriefing that encourage constructive, respectful, and honest feedback, thus creating a supportive learning environment. Participants should express their emotional reaction to the experience because realistic simulation activities can cause student stress and anxiety about their performance. It is helpful to invite participation debriefing from all participants, including SPs and evaluators, to stimulate collaborative learning. The facilitator should allow progression through several phases of debriefing, including reaction to the experience, analysis and critique of actual versus desired performance, and summary, including recommended activities to address performance or knowledge gaps identified.31,33 Student suggestions to improve the simulation experience are also encouraged. SP AND OSCE LOGISTICS
Scheduling an SP scenario or OSCE-style evaluation requires a significant number of people to be successful, especially when a large number of student participants are involved. For each scenario, there needs to be a minimum of one SP and an evaluator. There should be 1 or 2 people who do not perform direct evaluation of student performance, but rather maintain the flow of events. A dedicated time-keeper is essential, as well as a mechanism, such as an intercom or a 2-way radio, to alert the OSCE participants of the time to rotate to the next station. SPs need to have a rest period every 2 hours for hydration, meals, and attendance to personal issues. Depending on the number of students and the length of the scenarios, there may need to be a rotation of SPs to prevent fatigue (see Table 4 for OSCE sample timeline when multiple stations are included).
www.npjournal.org
Conducting SP and OSCE evaluation of student performance requires a number of evaluators and other participants, depending on the number of students. It can take a full day or more to go through a large number of participants, making this demanding of faculty time. It is essential that SPs and evaluators be compensated for their time, which will encourage them to return for future simulation activities. CONCLUSION
Use of SPs provides a realistic human simulation experience for NP and NMW students and can be an excellent format for evaluating and documenting student outcomes in an academic program. The application of SPs in the curriculum can be formative with active teaching and learning or summative in an OSCE-style testing situation. The case-specific patient scenarios provide a safe, controlled environment for teaching and evaluating students. Assessments conducted during SP scenarios and OSCE evaluations can be used to evaluate interpersonal skills, communication, cultural sensitivity, as well as skill attainment for procedures such physical exam techniques, critical thinking, and diagnosis and management of a number of conditions. There are a number of benefits to integrating an NP program with SPs and using OSCE-style testing to document clinical outcomes. More research is needed to evaluate the integration of SP scenarios into NP programs to determine whether simulated patient encounters may substitute for some of the direct clinical hours in the education program. References 1. Hayden J, Smiley R, Alexander M, Kardong-Edgren S, Jeffries P. The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. J Nurs Regul. 2014;5(2) Supp:1-40. 2. Forsberg I, Swartwout K, Murphy M, Danko K, Delaney K. Nurse practitioner education: Greater demand, reduced training opportunities. J Am Assoc Nurse Pract. 2015;27(2):66-71. 3. Logan B, Kovacs K, Barry T. Precepting nurse practitioner students: one medical center’s efforts to improve the precepting process. J Am Assoc Nurse Pract. 2015;27(5):676-682. 4. May W, Park J, Lee J. A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996-2005. Med Teacher. 2009;31:487-492. 5. Rehmann A, Mitmann R, Reynolds M. A handbook of flight simulation fidelity requirements for human factors research. Technical Report No. DOT/FAA/CTTN95/46. Wright-Patterson AFB; 1995. 6. Beaubien J, Baker D. The use of simulation for training teamwork skills in health care: how low can you go? Qual Safety Health Care. 2004; 13(Suppl 1):i51-i56.
The Journal for Nurse Practitioners - JNP
e209
7. Munshi F, Lababidi H, Alyousef S. Low-versus high-fidelity simulations in teaching and assessing clinical skills. J Taibah Univ Med Sci. 2015;10(1):12-15. 8. Benner P, Sutphen M, Leonard V, Day L. Educating Nurses—A Call for Radical Transformation. San Francisco: Jossey-Bass; 2010. 9. Giddens JF, Lauzon-Clabo L, Morton PG, Jeffries P, McQuade-Jones B, Ryan S. Re-envisioning clinical education for nurse practitioner programs: themes from a national leaders’ dialogue. J Prof Nurs. 2014;30(3):273-278. 10. Bashook PG. Best practices for assessing competence and performance of the behavioral health workforce. Admin Policy Mental Health. 2005;32(5):563-592. 11. Cooper S, Cant R, Porter J, et al. Simulation based learning in midwifery education: a systematic review. Women Birth. 2011;186:1-15. 12. Gallow S. Simulation techniques to bridge the gap between novice and competent healthcare professionals. J Issues Nurs. 2009;14(2):1-9. 13. Long T. Overview of teaching strategies for cultural competence in nursing students. J Cultural Divers. 2012;19(3):102-108. 14. McWilliam P, Botwinski C. Developing a successful nursing objective structured clinical examination. J Nurs Educ. 2010;49(1):36-41. 15. Rutledge CM, Garzon L, Scott M, Karlowicz K. Using standardized patients to teach and evaluate nurse practitioner students on cultural competency. Int J Nurs Educ Scholarship. 2004;1(1). 16. Shawler C. Palliative and end-of-life care: using standardized patient family for gerontological nurse practitioner students. Nurs Educ Perspect. 2011;32(3):168-172. 17. Shawler C. Standardized patients: a creative teaching strategy for psychiatric mental health nurse practitioner students. J Nurs Educ. 2008;47(11):528-531. 18. Theroux R, Pearce C. Graduate students experience with standardized patients as adjuncts for teaching pelvic examinations. J Am Acad Nurse Pract. 2006;18:429-535. 19. Brydges R, Carnahan H, Rose D, Rose L, Dubrowski A. Coordinating progressive levels of simulation fidelity to maximize educational benefit. Acad Med. 2010;85(5):806-812. 20. Decker S, Sportsman S, Puetz L, Billings L. The evolution of simulation and its contribution to competency. J Contin Nurs Educ. 2008;39(2):74-80. 21. Farahat E, Rice G, Dager N, Heine N, Scheider L, Connell B. Objective Structured Clinical Examination (OSCE) improves perceived readiness for clinical placement in nutrition and dietetic students. J Allied Health. 2015;44(4):208-214. 22. Armstrong KJ, Walker S. Standardized patients, part 2: Developing a case. Int J Athlet Ther Train. 2011;16(3):24-29. 23. Armstrong KJ, Walker S. Standardized patients, part 3: Assessing student performance. J Athlet Ther Train. 2011;16(4):40-44. 24. International Nursing Association for Clinical Simulation and Learning. INACLS standards for of best practice: simulation. Clin Simul Nurs. 2013; 9(6, Suppl):S2-S32.
e210
The Journal for Nurse Practitioners - JNP
25. Whelan GP, Boulet JR, McKinley DW, et al. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG clinical skills assessment. Med Teach. 2005;27(3):200-206. 26. Lunz ME. Statistical methods to improve decision reproducability. In: Mancall EL, Bashook PG, eds. Assessing Clinical Reasoning: The Oral Examination and Alternative Methods. Evanston, Ill: American Board of Medical Specialties; 1995:97-106. 27. Swanson DB, Norman GR, Linn RI. Performance-based assessment: lessons from the health professions. Educ Res. 1995;24(5):5-11. 28. Walker SE, Weidner TG. Standardized patients provide realistic and worthwhile experiences for athletic training students. Athlet Train Educ J. 2010;5(2):24-32. 29. Langan WH, Hansen D, Fein F, Parkhurst D. The evaluation of physician assistant students’ history taking ability using actors as standardized patients. J Phys Assist Educ. 2011;22(4):34-37. 30. Walker S, Armstrong KJ, Jarriel AJ. Standardized patients, part 4: training. Int J Athlet Ther Train. 2011;16(9):29-33. 31. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2:115-125. 32. Cheng A, Grant V, Dieckmann P, Arora S, Robinson T, Eppich W. Faculty development for simulation programs: five issues for the future of debriefing training. Simul Healthc. 2015;10(4):217-222. 33. Dreifuerst KT, Decker S. Debriefing: an essential component for learning in simulation pedagogy. In: Jeffries P, ed. Simulation in Nursing Education From Conceptualization to Evaluation. 2nd ed. New York: NLN; 2012:105-130.
Both authors are affiliated with the College of Nursing at Seattle University in Seattle, WA. Benjamin Miller, PhD, FNP, ACNP, is an assistant professor, He can be reached at [email protected]. Katherine Camacho Carr, PhD, CNM, FACNM, is a professor. In compliance with national ethical guidelines, the authors report no relationships with business or industry that would pose a conflict of interest. 1555-4155/16/$ see front matter © 2016 Elsevier, Inc. All rights reserved. http://dx.doi.org/10.1016/j.nurpra.2016.01.017
Volume 12, Issue 5, May 2016
Integrating Standardized Patients and Objective Structured Clinical Examinations Into a Nurse Practitioner Curriculum Benjamin Miller, PhD, FNP, and Katherine Camacho Carr, PhD, CNM ABSTRACT
Simulation has been used as a teaching pedagogy in health science disciplines for more than a century. Integrating standardized patients into simulation provides a high level of realism and is an excellent teaching and evaluation modality for nurse practitioner and nurse-midwifery programs. A standardized patient (SP) is coached to perform as a patient in a specific clinical scenario, while a number of students engage in a simulated encounter with SPs. Students are observed and evaluated with a checklist to assess performance. SPs are used in a single case teaching experience or in a multistation evaluation experience called an objective standardized clinical examination. Keywords: clinical teaching, objective structured clinical examinations, standardized patients Ó 2016 Elsevier, Inc. All rights reserved.
S
imulation in nursing education is a teaching modality allowing students to practice or demonstrate skills in a safe environment without risk of harm to real patients. The 2014 National Simulation Study by the National Council of State Boards of Nursing suggested high-quality simulation in undergraduate curriculum can be used to substitute up to 50% of the required clinical hours without altering the educational outcomes.1 Clinical placement is a challenge for both undergraduate and graduate nursing programs. Nurse practitioner (NP) and nurse-midwifery (NMW) programs have additional challenges with the scarcity of one student to one clinical preceptor placements. Accrediting agencies for NP and NMW programs mandate a minimum number of clinical hours and currently do not recognize simulation as a substitute for clinical preparation. Barriers to successful preceptor recruitment, including concerns of reduced productivity while precepting students and lack of formal preparation for the clinical preceptor role, have been cited as the leading reasons to not precept NP students.2,3 There are a number of questions that must be answered regarding simulation and advanced practice www.npjournal.org
nursing education outcomes. Integration of standardized patient (SP) encounters and objective structured clinical examinations (OSCEs), as teaching and evaluation tools for NP students may provide evidence of the clinical competence attained during the education program. The purpose of this paper is to describe the use of simulation with SPs as a teaching, learning, and evaluation methodology in graduate nursing education. BACKGROUND
Simulation scenarios with and without SPs have been used by a number of health science disciplines for over 50 years, in line with the need for new models of NP clinical teaching and learning.4 According to Rehmann et al., simulation has 3 distinct dimensions: (1) equipment fidelity; (2) environment fidelity; and (3) psychological fidelity. The term “fidelity” loosely represents the degree of realism of the simulation scenario. Equipment and environmental fidelity refer to the appearance of realism. Psychological fidelity is the degree to which the learner suspends disbelief and perceives the scenario is real. Achieving psychological fidelity is The Journal for Nurse Practitioners - JNP
e201
the most complicated and difficult dimension to achieve.5,6 In health care, a simulation scenario mimics a human experience using a number of methodologic approaches from low-fidelity simulation to complex high-fidelity simulation. Low-fidelity simulation can consist of case study evaluation or task trainers, which focus on skill acquisition. Mid- and high-fidelity simulation extends the learning and evaluation to clinical reasoning and application.6 SPs are prepared to “act” out a specific clinical situation, thus providing a high degree of realism to the learner. The SP simulation experience can be designed as a low- to mid-fidelity experience with the objective focused on skill attainment, or as high-fidelity when the learner is asked to utilize clinical decisionmaking to assess, interpret, and intervene based on the scenario.6 The SP provides a dimension of high environment and psychological fidelity. Integrating simulation with SPs into graduatelevel education programs provides “real-life” situations while offering opportunities for deeper comprehension/application of core competencies, including: (1) physical examination techniques; (2) communication skills; and/or (3) technical/ procedural skills. Depending on the learner’s preparation and simulation objectives, simulation activities can be integrated into the curriculum as a teaching modality, preclinical assessment tool, or summative clinical evaluation.7 OSCEs, utilizing SPs, can create a high-fidelity, high-stakes evaluation environment to assess comprehension, clinical competency, and critical thinking. Depending on the micro-objectives of the OSCE, this can be a potent teaching pedagogy or a dynamic evaluation tool. Throughout the NP educational program, formation of critical thinking in a simulated environment is an important aspect of learning to both think and act like a health care provider.8 Preclinical OSCE evaluations require students to demonstrate basic skill mastery before starting their clinical experience. Academic programs must prepare students to be efficient, productive, and safe in the clinical setting to reduce the burden on the busy preceptor while promoting positive patient outcomes.9
e202
The Journal for Nurse Practitioners - JNP
BENEFITS OF A STANDARDIZED PATIENT/ SIMULATION PROGRAM
One of the major challenges for NP faculty is the evaluation of student clinical performance. Faculty administer written examinations, require studentcreated reports, evaluate oral case presentations, and have students complete case studies to evaluate knowledge and skill acquisition. Classroom-based examinations may lack content validity if not covering all of the necessary content. Standard classroom assessments may lack the ability to evaluate clinical application and essential clinical decisionmaking (CDM), which is needed in clinical practice.10 To evaluate students’ clinical application and essential clinical decisionmaking, faculty perform brief direct observations of students in the clinical setting, while relying on input from the clinical preceptors to fully assess a student’s competency. Direct observation in the clinical setting gives faculty minimal opportunity to observe and evaluate students in real-life clinical encounters. Further challenges include inconsistency in the type of patient visit, the complexity of the visit, the skills required, and the clinical setting. Implementation of a graduate simulation program with standardized patients provides a number of benefits to teaching, learning, and evaluation of NP students. A primary benefit of an SP program is the ability to create a safe, realistic environment for the students to learn. Faculty assess and document a student’s performance in a simulated clinical setting with standardized clinical presentations, which may not be available in all clinical settings. Real-life clinical experiences are unpredictable complicating student evaluations; however, simulated scenarios are developed with standardized specific conditions to demonstrate core competencies. Core competencies are leveled across the curriculm and include the ability to obtain a clinical history, perform an appropriate physical exam, conclude an accurate diagnosis, prescribe/order suitable treatments, provide patient education/counseling, or discuss complex and difficult situations with SPs.11-18 The high-fidelity simulation with SPs can be constructed in a variety of formats. The simulation,
Volume 12, Issue 5, May 2016
which requires comprehensive history collection, conducting a physical exam, integrating diagnostic reasoning, and developing treatments plans, is complex and can take a long time to complete, thus limiting the number of students who can be assessed. Simulations that focus on microtasks of the encounter, such as demonstration of an appropriate physical exam for a predetermined constellation of symptoms, can assess application and skill. Simulations with shorter microtask evaluations can allow a greater number of clinical assessments per student and evaluation of more students over a specific time frame.6,19 Simulation with SPs provides students with the opportunity to be evaluated on their ability to diagnose and develop treatment plans. Simulated patients also reduce risk or adverse consequences to real patients by allowing students to practice newly acquired skills.20 In a simulation learning environment, students receive immediate feedback and remediation techniques to achieve mastery. Formative evaluation of students through simulation assesses the student’s confidence in using the correct technique, whereas summative evaluation assesses the competence level the NP student has achieved with regard to specific clinical objectives.21 Increasing student confidence and competence through preclinical preparation strengthens relationships with clinical partners, preceptors, and agencies. Clinical preceptors are NPs, physician assistants, and physicians who maintain a clinical practice, while volunteering to precept (teach) NP and NMW students. Clinical preceptors expect students to perform core competencies of clinical care and demonstrate a novice level of clinical reasoning. Utilization of preclinical OSCEs with SPs leveled across the curriculum can help to establish a minimum level of performance competency before the student begins clinical rotations. A standard minimum level of performance expectations provides reassurance to clinical agencies and preceptors that students are prepared to maximize their experiences in the rich clinical environment, while decreasing the burden of working with students. The focus of clinical education shifts to the attainment of escalating complex competencies and refinement of required knowledge, skills, and attitudes, as opposed to attainment of a www.npjournal.org
required number of clinical hours and assumed competency upon completion of the clinical rotation.9 DEVELOPING A STANDARDIZED PATIENT PROGRAM
Although the incorporation of an SP program into NP education has many benefits, it can be challenging, costly, and time-consuming for educators and NP education programs. A review of the required competencies for the NP population focus, program outcome objectives, degree requirements, and interdisciplinary and cultural competencies should be considered in developing an SP program. When multiple population foci exist, an identification of the common key competencies is helpful when deciding on the development of case scenarios that may be used by more than one specialty within the NP program of study. For example, each specialty could benefit from an SP case that includes diversity/inclusivity or interdisciplinary collaboration and/or consultation. A blueprint for each population focus can be constructed, detailing both the shared and unique SP scenarios and OSCEs, along with sequencing in the program of study. An SP program should consider the following 4 key aspects: (1) development of the simulation scenario, including the objectives, placement in the curriculum, the scenario template, validation of the checklist, and student evaluation; (2) development of SPs; (3) preparation of the scenario evaluators and faculty; and (4) simulation debriefing. DEVELOPING AN OSCE SIMULATION SCENARIO FOR THE SP
A simulation scenario is a fictitious, but clinically accurate and pertinent, patient presentation that has predefined learning objectives based on the NP population foci. The clinical scenario has a realistic presentation with history of present illness; primary medical, surgical, and social history; and physical exam findings all being relevant data in portraying a realistic scenario. The first step in developing the SP case is to identify clear and measureable learning outcomes for the encounter.22,23 Learning objectives can focus on cognitive, psychomotor, and/or affective goals. The The Journal for Nurse Practitioners - JNP
e203
Table 1. Sample Template for Standardized Patient Case Scenario
Table 1. (continued )
Title and overview
Current life situation and social history (marital status, health habits)
Development date
Personality, culture, language—if pertinent to the case
Developer(s)
Past medical surgical history
Learners and where this fits in the curriculum:
Pertinent family history
Objective(s)
Medications
Logistics:
Allergies
Personnel
Encounter beginning
Forms
Encounter ending (time-limited)
Room/Equipment/Props
Physical exam expected Special instructions, such as: appear to be slow in answering questions; appear agitated; or act as if you are fine despite symptoms
Instructions to the examinee/participant Patient’s name Age Consider marital status, occupation, medications, allergies, and pregnancy history, if pertinent to the case Presenting complaint/reason for encounter Key objectives of the case: history, physical assessment/ evaluation, diagnosis, communication/counseling, patient education, referral, and timing Standardized patient script Case name Name Age Consider marital status, occupation, medications, allergies, medical, surgical, and pregnancy history, if pertinent to the case Components of patient history22,23: History of present illness Mechanism of illness or injury Description of symptoms: Onset, duration, and location and/or radiation of symptoms, including any precipitating events, illnesses, or injury Frequency, intensity, and progression of symptoms Quality, intensity, location, and/or radiation of pain Aggravating and alleviating factors Associated symptoms continued
e204
The Journal for Nurse Practitioners - JNP
Note: Sample variables to be considered in the development of a SP scenario including backstory script for SP.
learning tasks focus on specific competencies, such as attaining a focused health history, performance of accurate physical assessment, communication skills, mental health assessment, cultural sensitivity, and/or spiritual care.24 To optimize student learning and assessment, the scenario should focus on parts of an exam to reinforce content exposure.7 For example, the scenario may focus on a straightforward, focused complaint or a complex presentation necessitating a branching history, or a physical exam to assess the presenting complaint appropriately. Scenarios can be constructed to meet one or more educational objectives, although they should always permit students to incorporate a variety of competencies, allowing for the demonstration of an ability to assess, develop, and implement a plan of care. As students progress through the curriculum, cases should require them to demonstrate the knowledge base and assessment, critical thinking, technical, and interpersonal skills required for beginning level practice.22,23 After development of the learning objectives, a template containing a summary of all the case information should be developed (see Table 1). To provide a realistic scenario, all aspects of the fictitious scenario should be developed in advance to include
Volume 12, Issue 5, May 2016
social history, family history, and employment/ occupational exposure. The details of the scenario are essential for the SP role as the actor prepares for the performance and unanticipated questions. Optimal scenario development for SP cases is a team effort with faculty working together to create a fictional, realistic, and evidence-based patient scenario. The presentation of the illness, injury, or problem, as well as assessment and management, should be based on accepted standards of practice for the population foci. The scenario competencies are appropriate for the student’s level of preparation, with an authentic and typical presentation of signs and symptoms. A clear and consistent initial description of the case will ensure that all students view the case from the same perspective.
Setting a scenario’s time limit contributes to the realistic clinical expectations and should be established for logistical planning. The time limit for each station must consider the level of student preparation and the learning objectives (30 minutes for a first quarter student vs. 10 minutes for a graduating student). Awareness of the time limits is essential for the learner to develop clinical time management skills. Assessing Student Performance
Developing a clinical scenario creates a learning opportunity for the student to experience a standardized, reliable, realistic patient presentation. SPs work from an in-depth script, so students receive the same information during the scenario. Based on the clinical objectives, the microcomponents of the
Table 2. Head- Ears-Eyes- Nose-Throat (HEENT)-focused Physical Exam Evaluation Checklist 1)
Participant washes hands prior to the examination.......... Did not perform ¼ 0, performed ¼ 1
⓪ ①
2)
Introduces self to the patient.....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
3)
Performs a general survey of the client................ Did not perform ¼ 0, performed ¼ 1
4)
Performs an otoscope exam ............................Did not perform ¼ 0 Unilateral ¼ 1 Bilateral ¼ 2
5)
Examines the eyes, looks at conjunctiva and sclera.........Did not perform ¼ 0, performed ¼ 1
⓪ ①
6)
Examines the nasal mucosa with otoscope .............Did not perform ¼ 0, performed ¼ 1
⓪ ①
7)
Examines the oral cavity .......................Did not perform ¼ 0, performed ¼ 1
⓪ ①
8)
Examines the hypopharynx. ............................Did not perform ¼ 0 Performed without use of tongue depressor ¼ 1 Performed with use of tongue depressor ¼ 2
⓪ ① ②
9)
Assess the client’s voice .....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
⓪ ① ⓪ ① ②
10) Examines the neck for adenopathy..... ...................Did not perform ¼ 0 Check tonsillar or cervical (not both) ¼ 1 Check tonsillar and cervical nodes ¼ 2
⓪ ① ②
11) Assess for nuchal rigidity .....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
12) Assess client’s lung sounds ...................Did not perform ¼ 0, performed ¼ 1
⓪ ①
13) Assess clients heart tones .......................Did not perform ¼ 0, performed ¼ 1
⓪ ①
Total score ____________ / 16 points Overall assessment Engages in a therapeutic relationship with the client during the encounter
① ② ③
Had an organized approach during the exam/history
① ② ③
Demonstrated competence and confidence during the exam/history
① ② ③
Note: A sample of a HEENT examination checklist is shown. Each item has a numerical value to indicate completion. The overall assessment is based on the evaluator’s opinion of the entire encounter.
www.npjournal.org
The Journal for Nurse Practitioners - JNP
e205
clinical encounter are identified, and an objective, valid checklist is created to assess the student performance. Checklists are constructed by faculty and/or a small group of experts, based on standards of practice and the scientific evidence relating to the specific case (see Tables 2 and 3). Items on the checklist are usually scored as dichotomous indicators (performed or not performed). Items may also be rank-ordered if the objective has multiple parts, so feedback on performance can be provided to the student for future improvement in interviewing, skill performance, counseling, and clinical decisionmaking.25 Newly developed case-specific checklists should be assessed for intra- and interrater reliability using standard statistical methods. Pilot testing of case scenarios ensures student performance is assessed accurately. Intrarater reliability refers to the consistency of scoring by an evaluator across multiple students. Interrater reliability refers to the consistency of scoring by several evaluators, assessing the same student’s performance. Both intra- and interrater
reliability are dependent on the evaluators consistently interpreting items on the case scenario checklist and consistently scoring student performance, thus providing a detailed explanation of the reliability assessment procedure for performance evaluations.26,27 The reliability scale uses 0 as unreliable and 1 as very reliable. In performance assessments for high-stakes decisions, reliabilities > 0.85 (Cronbach’s alpha) are considered acceptable.10 Depending on the type of simulation (formative or summative), student feedback can be immediate with remediation, delayed, or aggregated. In some situations, the SP may provide “patient” perspective feedback to the student.23 For practice and learning stations, having students function as an evaluator provides a different method of learning, whereas assessment of clinical or course objectives should be evaluated by faculty. Video recording student performance offers a number of positive attributes. Students can watch their own performance after the simulated experience and provide a critical selfappraisal of both the positive aspects and areas for
Table 3. Focused History for a Person With Pneumonia Symptoms 1) Introduces self to the patient...................Did not perform ¼ 0, performed ¼ 1
⓪ ①
2) Asks patient to describe what is going on with open-ended questions...Did not perform ¼ 0 performed ¼ 1
⓪ ①
3) Asks about the onset and duration of symptoms ..................Did not perform ¼ 0 Asked about onset or duration (not both) ¼ 1 Asked about onset and duration ¼ 2
⓪ ① ②
4) Asks about aggravating or alleviating factors ...................Did not perform ¼ 0 Asked about aggravating or alleviating factors (not both) ¼ 1 Asked about aggravating and alleviating factors ¼ 2
⓪ ① ②
5) Asks about hemoptysis....................Did not perform ¼ 0, performed ¼ 1
⓪ ①
6) Asks about medication allergies.................Did not perform ¼ 0, performed ¼ 1
⓪ ①
7) Asks about current medications................Did not perform ¼ 0, performed ¼ 1
⓪ ①
8) Assess for associated symptoms (fevers, chills, rigors).............Did not perform ¼ 0 Asked about 1 associated symptom ¼ 1 Asked about 2 or more associated symptoms ¼ 2
⓪ ① ②
Total score ____________ / 11 points Global assessment Engages in a therapeutic relationship with the client during the encounter
① ② ③
Had an organized approach during the exam/history
① ② ③
Demonstrated competence and confidence during the exam/history
① ② ③
Note: A sample of a history-gathering scenario for a person with pneumonia is shown. Each item has a numerical value to indicate completion. The overall assessment is based on the evaluator’s opinion of the entire encounter.
e206
The Journal for Nurse Practitioners - JNP
Volume 12, Issue 5, May 2016
Table 4. OSCE Time Schedule Session for Session 1a
1300-1315
Station A
Station B
Station C
Station D
Station E
Station F
Chart Station F
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
(Student 007)
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
Student 002
Student 003
Student 004
Student 005
Student 006
Student 007
Student 001
—
—
—
—
—
Student 007
1315-1317 Rotate 1317-1332
Student 007
1332-1334 Rotate 1334-1349
Student 006
1349-1351 Rotate 1351-1406
Student 005
1406-1408 Rotate 1408-1423
Student 004
1423-1425 Rotate 1425-1440
Student 003
1440-1442 Rotate 1442-1457
Student 002
1457-1459 Rotate 1459-1514
—
1514-1545 Break Note: A sample schedule of an objective structured clinical examination (OSCE) evaluation session (7 testers) is shown. a Session 1 (15-minute stations with 2-minute rotations).
growth. Finally, video recording during high-stakes testing provides documentation of student performance. Faculty review the assessment(s) of performance by sharing the results of the checklist with the student and offer constructive feedback. Faculty can immediately identify strengths and areas of deficiency related to the educational objectives of the case, which can be addressed or remediated. Over time, faculty and students can document professional growth in terms of competency and confidence.28 Standardized Patient Development
Standardized patient scenarios utilize a developed template to meet specific learning objectives for the session. Standardized patients are “actors,” playing a role as a patient. When recruiting SPs, avoid using faculty or health care professionals as the SPs, because there may be a tendency for the health care professional to inadvertently coach or offer critical information. When conducting high-stakes OSCE testing, use of faculty may impose undue stress and, www.npjournal.org
potentially, emphasize a power differential, both of which can affect student performance.14,29 Training “actors” as standardized patients is a critical part of developing an SP program.24,30 Key points in training SPs include detailed and specific signs/symptoms, physical characteristics, facial expressions, and any vocalizations pertinent to the scenario. Consistency in the portrayal of SPs is critical for standardization between students. During training sessions, faculty should encourage SPs to ask questions to obtain a clear understanding. SPs should also be prepared for students to ask questions or perform assessments not pertinent to the main focus of the scenario. The development of the specific case should provide enough data to guide SPs when asked non- essential questions. Instruct SPs to answer the questions and perform to the best of their ability based on the case.30 A standardized training session should be performed 1-2 weeks in advance to introduce the case scenario to SPs. The training should provide enough time to review the scenario and should be followed The Journal for Nurse Practitioners - JNP
e207
by 2 or 3 practice sessions to ensure SP understanding and reliability. SPs must demonstrate they comprehend the detailed history of the present condition, illness, injury, or problem. Scenario details include the mechanism of illness or injury and a comprehensive description of the onset, duration, location, radiation, frequency, intensity, and progression of symptoms. Further details include quality, intensity, location, and radiation of pain symptoms. Actors must be informed of the typical aggravating and alleviating factors and any associated symptoms. The current life situation and social history, such as marital status or health habits, should be available and rehearsed with SPs to portray a realistic presentation. If pertinent to the case objectives, SPs should also be informed of the personality type, culture, and language to be portrayed. Past family, medical, surgical, and obstetric history, as well as medications and allergies, should also be included in the script. It is helpful to inform SPs how long the encounter is to last, what physical exam and counseling may be expected, and whether or not they should cue the student or not. Special instructions, such as appearing agitated, acting slow in response time, or being in denial about a condition, can also be communicated in the script. A complete and clearly written SP script will help the actor portray a reproducible and realistic situation. It is critical that SPs respond the same way with every student encounter to maintain intrarater reliability and scenario standardization. Evaluator Development
Faculty and qualified members of the profession, and SPs in some cases, may be asked to perform as evaluators during the simulation or OSCE sessions. The evaluation checklists are developed according to evidence-based practice guidelines and standards of care to establish content validity and evaluated prior to the simulation or OSCE to establish interrater reliability. The reliability of the checklist is dependent on the evaluator. To maintain reliability, the evaluators should receive an in-depth orientation to the scenario and maintain a clear and consistent evaluation of every student. It is important to establish both intra- and interrater reliability of the case-specific checklists during a pilot test to ensure that student
e208
The Journal for Nurse Practitioners - JNP
performance is assessed accurately, especially in highstakes situations. Meeting with the evaluators before each SP simulation or OSCE encounter is valuable to practice the case-specific checklist and identify any items that are problematic. During the scenario, many events happen simultaneously, and being able to observe, listen, and record information can be challenging. Practicing these skills during the evaluator orientation through a mock scenario will help ensure intra- and interrater reliability. If the simulation or OSCE is video recorded, multiple evaluators can re-evaluate the same scenario, providing ongoing interrater reliability as well as validating student performance. A timely review is essential to provide feedback to students, SPs, and evaluators. Faculty Facilitator Role
The faculty facilitator plays a pivotal role in guiding the simulated clinical learning experience. The facilitator should understand the prerequisite knowledge, level, and abilities of the student participants and develop the expected outcomes of the case scenario or OSCE based on this information. The faculty facilitator should be clinically proficient and have knowledge of evidence-based practices to identify knowledge or performance gaps. The facilitator must clearly communicate to the participants the learning objectives and expected outcomes of the experience. The overall goal is to create a safe experiential learning environment that encourages active learning, repetitive practice, and reflection. In addition, the facilitator must promote fidelity and provide clear direction for formative or summative evaluation of student performance, using a valid and reliable instrument. Facilitators provides support to students, SPs, and evaluators throughout the simulated experience. They also provide constructive feedback and debriefing with the participants, encouraging self-reflection on performance and identification of meaningful feedback from peers and faculty that will assist participants to improve their practice. Evaluation of the SP or OSCE experience by the faculty facilitator and participants is used to make adjustments to the simulated experience.24
Volume 12, Issue 5, May 2016
Debriefing
Debriefing is an essential aspect of all simulation experiences where students engage in reflective learning. Students are exposed to a number of sensory, tactile, and emotional stimuli during a scenario, and the debriefing process allows for a safe interpretation of these experiences.31,32 SP and OSCE simulated experience debriefing sessions should be based on a structured framework to promote reflective discussion.24 It is helpful if the faculty facilitator establishes rules for participation in debriefing that encourage constructive, respectful, and honest feedback, thus creating a supportive learning environment. Participants should express their emotional reaction to the experience because realistic simulation activities can cause student stress and anxiety about their performance. It is helpful to invite participation debriefing from all participants, including SPs and evaluators, to stimulate collaborative learning. The facilitator should allow progression through several phases of debriefing, including reaction to the experience, analysis and critique of actual versus desired performance, and summary, including recommended activities to address performance or knowledge gaps identified.31,33 Student suggestions to improve the simulation experience are also encouraged. SP AND OSCE LOGISTICS
Scheduling an SP scenario or OSCE-style evaluation requires a significant number of people to be successful, especially when a large number of student participants are involved. For each scenario, there needs to be a minimum of one SP and an evaluator. There should be 1 or 2 people who do not perform direct evaluation of student performance, but rather maintain the flow of events. A dedicated time-keeper is essential, as well as a mechanism, such as an intercom or a 2-way radio, to alert the OSCE participants of the time to rotate to the next station. SPs need to have a rest period every 2 hours for hydration, meals, and attendance to personal issues. Depending on the number of students and the length of the scenarios, there may need to be a rotation of SPs to prevent fatigue (see Table 4 for OSCE sample timeline when multiple stations are included).
www.npjournal.org
Conducting SP and OSCE evaluation of student performance requires a number of evaluators and other participants, depending on the number of students. It can take a full day or more to go through a large number of participants, making this demanding of faculty time. It is essential that SPs and evaluators be compensated for their time, which will encourage them to return for future simulation activities. CONCLUSION
Use of SPs provides a realistic human simulation experience for NP and NMW students and can be an excellent format for evaluating and documenting student outcomes in an academic program. The application of SPs in the curriculum can be formative with active teaching and learning or summative in an OSCE-style testing situation. The case-specific patient scenarios provide a safe, controlled environment for teaching and evaluating students. Assessments conducted during SP scenarios and OSCE evaluations can be used to evaluate interpersonal skills, communication, cultural sensitivity, as well as skill attainment for procedures such physical exam techniques, critical thinking, and diagnosis and management of a number of conditions. There are a number of benefits to integrating an NP program with SPs and using OSCE-style testing to document clinical outcomes. More research is needed to evaluate the integration of SP scenarios into NP programs to determine whether simulated patient encounters may substitute for some of the direct clinical hours in the education program. References 1. Hayden J, Smiley R, Alexander M, Kardong-Edgren S, Jeffries P. The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. J Nurs Regul. 2014;5(2) Supp:1-40. 2. Forsberg I, Swartwout K, Murphy M, Danko K, Delaney K. Nurse practitioner education: Greater demand, reduced training opportunities. J Am Assoc Nurse Pract. 2015;27(2):66-71. 3. Logan B, Kovacs K, Barry T. Precepting nurse practitioner students: one medical center’s efforts to improve the precepting process. J Am Assoc Nurse Pract. 2015;27(5):676-682. 4. May W, Park J, Lee J. A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996-2005. Med Teacher. 2009;31:487-492. 5. Rehmann A, Mitmann R, Reynolds M. A handbook of flight simulation fidelity requirements for human factors research. Technical Report No. DOT/FAA/CTTN95/46. Wright-Patterson AFB; 1995. 6. Beaubien J, Baker D. The use of simulation for training teamwork skills in health care: how low can you go? Qual Safety Health Care. 2004; 13(Suppl 1):i51-i56.
The Journal for Nurse Practitioners - JNP
e209
7. Munshi F, Lababidi H, Alyousef S. Low-versus high-fidelity simulations in teaching and assessing clinical skills. J Taibah Univ Med Sci. 2015;10(1):12-15. 8. Benner P, Sutphen M, Leonard V, Day L. Educating Nurses—A Call for Radical Transformation. San Francisco: Jossey-Bass; 2010. 9. Giddens JF, Lauzon-Clabo L, Morton PG, Jeffries P, McQuade-Jones B, Ryan S. Re-envisioning clinical education for nurse practitioner programs: themes from a national leaders’ dialogue. J Prof Nurs. 2014;30(3):273-278. 10. Bashook PG. Best practices for assessing competence and performance of the behavioral health workforce. Admin Policy Mental Health. 2005;32(5):563-592. 11. Cooper S, Cant R, Porter J, et al. Simulation based learning in midwifery education: a systematic review. Women Birth. 2011;186:1-15. 12. Gallow S. Simulation techniques to bridge the gap between novice and competent healthcare professionals. J Issues Nurs. 2009;14(2):1-9. 13. Long T. Overview of teaching strategies for cultural competence in nursing students. J Cultural Divers. 2012;19(3):102-108. 14. McWilliam P, Botwinski C. Developing a successful nursing objective structured clinical examination. J Nurs Educ. 2010;49(1):36-41. 15. Rutledge CM, Garzon L, Scott M, Karlowicz K. Using standardized patients to teach and evaluate nurse practitioner students on cultural competency. Int J Nurs Educ Scholarship. 2004;1(1). 16. Shawler C. Palliative and end-of-life care: using standardized patient family for gerontological nurse practitioner students. Nurs Educ Perspect. 2011;32(3):168-172. 17. Shawler C. Standardized patients: a creative teaching strategy for psychiatric mental health nurse practitioner students. J Nurs Educ. 2008;47(11):528-531. 18. Theroux R, Pearce C. Graduate students experience with standardized patients as adjuncts for teaching pelvic examinations. J Am Acad Nurse Pract. 2006;18:429-535. 19. Brydges R, Carnahan H, Rose D, Rose L, Dubrowski A. Coordinating progressive levels of simulation fidelity to maximize educational benefit. Acad Med. 2010;85(5):806-812. 20. Decker S, Sportsman S, Puetz L, Billings L. The evolution of simulation and its contribution to competency. J Contin Nurs Educ. 2008;39(2):74-80. 21. Farahat E, Rice G, Dager N, Heine N, Scheider L, Connell B. Objective Structured Clinical Examination (OSCE) improves perceived readiness for clinical placement in nutrition and dietetic students. J Allied Health. 2015;44(4):208-214. 22. Armstrong KJ, Walker S. Standardized patients, part 2: Developing a case. Int J Athlet Ther Train. 2011;16(3):24-29. 23. Armstrong KJ, Walker S. Standardized patients, part 3: Assessing student performance. J Athlet Ther Train. 2011;16(4):40-44. 24. International Nursing Association for Clinical Simulation and Learning. INACLS standards for of best practice: simulation. Clin Simul Nurs. 2013; 9(6, Suppl):S2-S32.
e210
The Journal for Nurse Practitioners - JNP
25. Whelan GP, Boulet JR, McKinley DW, et al. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG clinical skills assessment. Med Teach. 2005;27(3):200-206. 26. Lunz ME. Statistical methods to improve decision reproducability. In: Mancall EL, Bashook PG, eds. Assessing Clinical Reasoning: The Oral Examination and Alternative Methods. Evanston, Ill: American Board of Medical Specialties; 1995:97-106. 27. Swanson DB, Norman GR, Linn RI. Performance-based assessment: lessons from the health professions. Educ Res. 1995;24(5):5-11. 28. Walker SE, Weidner TG. Standardized patients provide realistic and worthwhile experiences for athletic training students. Athlet Train Educ J. 2010;5(2):24-32. 29. Langan WH, Hansen D, Fein F, Parkhurst D. The evaluation of physician assistant students’ history taking ability using actors as standardized patients. J Phys Assist Educ. 2011;22(4):34-37. 30. Walker S, Armstrong KJ, Jarriel AJ. Standardized patients, part 4: training. Int J Athlet Ther Train. 2011;16(9):29-33. 31. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2:115-125. 32. Cheng A, Grant V, Dieckmann P, Arora S, Robinson T, Eppich W. Faculty development for simulation programs: five issues for the future of debriefing training. Simul Healthc. 2015;10(4):217-222. 33. Dreifuerst KT, Decker S. Debriefing: an essential component for learning in simulation pedagogy. In: Jeffries P, ed. Simulation in Nursing Education From Conceptualization to Evaluation. 2nd ed. New York: NLN; 2012:105-130.
Both authors are affiliated with the College of Nursing at Seattle University in Seattle, WA. Benjamin Miller, PhD, FNP, ACNP, is an assistant professor, He can be reached at [email protected]. Katherine Camacho Carr, PhD, CNM, FACNM, is a professor. In compliance with national ethical guidelines, the authors report no relationships with business or industry that would pose a conflict of interest. 1555-4155/16/$ see front matter © 2016 Elsevier, Inc. All rights reserved. http://dx.doi.org/10.1016/j.nurpra.2016.01.017
Volume 12, Issue 5, May 2016