- Email: [email protected]
The use of human patient simulation to improve academic test scores in nursing students
Teaching and Learning in Nursing (2014) 9, 23–26
www.jtln.org
The use of human patient simulation to improve academic test scores in nursing students1,2 Laura Glidewell MSN, RN⁎, Cheryl Conley MSN, RN Polk State College, Winter Haven, FL 33881, USA KEYWORDS: Academic test scores; Simulation; Cognitive domain
Abstract Many technologies that were not yet invented when today's educators went to nursing school are now staples in nursing colleges and schools worldwide. Although there are numerous studies that examine the relationship between simulation and student success, most focus on the affective and psychomotor domains of learning. This article focuses on the cognitive domain of learning and discusses the correlation between the use of high-fidelity human patient simulation and improved academic test scores. © 2014 National Organization for Associate Degree Nursing. Published by Elsevier Inc. All rights reserved.
1. Introduction Many technologies that were not yet invented when today's educators went to nursing school are now staples in nursing colleges and schools worldwide. For example, human patient simulation (HPS) for most of the educators consisted of a Resusci-Annie, which is now replaced by high-fidelity HPS that is able to produce life-like physiologic responses to students' actions. The National Council of State Boards of Nursing found that over 1,060 schools nationwide are using simulation in the preparation of prelicensure nursing students (Hayden, 2010). Because more schools compete for limited clinical resources, it is expected that this number will increase. Although there are numerous studies that examine the relationship between simulation and student
success, most focus on the affective and psychomotor domains of learning. Using HPS in nursing education promotes clinical competency, patient safety, and student confidence (Bremner, Aduddell, & Adamson, 2008; Jenson & Forsyth, 2012; Richards, Simpson, Aaltonene, Krebs, & Davis, 2010). A review of the literature related to the cognitive domain in nursing students using HPS reveals no studies specific to this topic. The purpose of this study was to examine the relationship between academic test scores and the use of HPS. National accrediting bodies expect to see measurable learning outcomes with the use of technology. These issues in nursing education prompted the writers to ask the following questions: Is there a correlation between the use of simulation and improved academic outcomes? If so, what is that correlation and how can it best be measured?
1
The authors wish to thank the National Organization of Associate Degree Nursing Foundation for their generous grant support of this research. 2 The authors have no conflict of interest to disclose. ⁎ Corresponding author. Laura Glidewell or Cheryl Conley, Polk State College School of Nursing, 999 Avenue H NE, Winter Haven, FL 33881, USA. E-mail address: [email protected]
2. Background Faculty of an advanced medical–surgical course identified student weaknesses in certain areas based on high-miss
1557-3087/$ – see front matter © 2014 National Organization for Associate Degree Nursing. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.teln.2013.08.001
24
L. Glidewell, C. Conley
test concepts. The areas included pneumothorax management, ventilator settings, chest pain management, complex electrolyte imbalances, and acute stroke management. With this information, the faculty developed four HPS learning activity modules to be incorporated into the course. The four modules were (a) respiratory, which included the pneumothorax and ventilator settings; (b) cardiac; (c) renal, to cover complex electrolyte issues; and (d) neuro. The modules focused on both theory content and areas of clinical concern or skills that were not easily obtained by a large number of students, such as chest tube and ventilator management. Once content and skills were selected and the modules written, piloting of the modules began as an optional learning activity. The four modules were fully incorporated into the course as a mandatory activity the following semester. Using test analysis data on individual test items kept by the department, we analyzed results of three unit examinations covering the topics above. The analysis of the unit examinations from the semester before any simulation was initiated, and the semester where participation was optional was compared against the results of the semester when simulation became a mandatory activity (Table 1). The targeted high-miss items were the same on all three sets of unit examinations. For test security, other test items were equal in difficulty and content but not identical. Group 1 represents the students that took the unit examination without any simulation activities in the semester prior to writing the modules. Group 2 represents the piloting of simulation as an optional activity for students that wanted to participate, and Group 3 represents the students where the modules were mandatory. In two out of three unit examinations, the students with mandatory simulation (Group 3) had a mean score higher than Group 1 and Group 2. One unexpected finding when doing the initial comparison was that Group 3 did not perform as high on the unit Examination 3 as the previous group. The team discovered that Group 3 struggled with medication dosage calculation in Table 1
Student academic test scores (Groups 1–3)
Module 3 that might have taken away from the learning as it was intended. A modification of Module 3 was completed and implemented in the next semester. Table 2 represents the test scores of the students enrolled at the time of the modified Module 3 (Group 4), showing that the new test scores were significantly higher than those reported in Table 1 after the modification. Researcher bias was identified as a potential limitation. Knowing that the area of difficulty was the starting point for writing the modules, emphasis was placed on the review of those concepts and the additional time allotted for the mastery of the material. In addition, the unit examination covered material not specific to the module but rather an entire unit of study; therefore, the student grade was not a reliable indication of only the learning activity. After reviewing this early data, the need for an empirical research study was clear. Preliminary data were shared with administration, and institutional approval was received to conduct a well-designed study. The researchers then sought and received grant assistance from the National Organization for Associate Degree Nursing (N-OADN) to assist in funding the study.
3. Study design Initially, 50 students were invited to participate in the study; however, the study was expanded to a final count of n = 184 and covered three total semesters. The students were considered a homogenous group because each student had taken the same general education courses and prerequisite courses and completed all their nursing education within the program. The students had completed the research lecture in the capstone course prior to being asked to participate. Students signed consent to participate, and each was advised of the right to withdraw at any time. None withdrew. Table 2 Student academic test scores for unit Examination 3 only (Groups 1–4)
90
90
88
88
86
86
84
84
82
82
80
80
78
78 Exam1
Exam 2 Group 1
Group 2
Exam 3 Group 3
Exam1
Exam 2 Group 1
Group 2
Exam 3 Group 3
HPS and academic test scores
25
Two content areas out of the four modules were selected for further study: cardiac and renal. The course syllabus, unit objectives, learning outcomes, and simulation module were forwarded to the development team at Health Education Systems, Inc. (HESI) ©/Elsevier, who developed a custom examination for each module. The examination was returned for review. Recognizing bias as a limitation to the background data, only the program director, who had previous experience teaching the content, reviewed the examinations to determine suitability for the course and objective. Course faculty were not permitted to preview the examination, thus minimizing testing bias. The decision to utilize HESI also resulted from the use of the HESI Exit Examination in the course, thus giving students familiarity with the testing procedure, log on, and administration. Once the examination was selected, a cost proposal was sent to administration for review. The protocol for the study was designed by taking the roster of students that agreed to participate in the study and assigning them randomly into presimulation (test Group A) and postsimulation (test Group B) groups. Funding did not permit for all students to test twice, but consistency was achieved by grouping them this way. With funding support from both the college and NOADN, test fees were covered for the students to take both a cardiac and a renal examination. All students attended the usual course lecture. After all material was covered, test Group A completed the HESI examination prior to their mandatory simulation experience. Then, test Group B took the same HESI examination immediately after their simulation experience. Scores for each group (n = 50) were then evaluated. The department of institutional effectiveness reported that a larger sample would be required to determine the generalization to the nursing student population as a whole; therefore, testing continued for two additional semesters bringing the total sample to n = 184.
4. Results All data have remained confidential, and the department of institutional effectiveness analyzed only raw scores without student identifiers. On the cardiac examination of the first group that tested (Group A), there was a higher score variance among students testing before the simulation with a maximum score of 1,284 (out of 1,500) and an average score of 840, and
the maximum score for the postsimulation group (Group B) was 1,271, but the average score was higher at 892. This represents a differential score average of 52, which is less than three points higher than the threshold required to arrive at a statistical significance. In evaluation of the renal examinations, the score differential is 85 points. The students testing before simulation (Group A) scored an average 951, whereas the students testing after simulation (Group B) scored an average 1,036. These results were what prompted the researchers to continue the testing for two additional semesters. The evaluation of the results of the students that tested after having simulation (Group B) was statistically significant. When combining both the renal and cardiac subsets together and looking at n = 368, there was a score difference of 66 points. This reflects a difference at the P b .01 level, showing the greatest statistical significance. The analysis of variance summarizes the results of the raw data in the study (Table 3).
5. Discussion Review of the results led the researchers to conclude that the use of HPS improves academic test scores. This illustrates a model by which the learning activity and learning outcomes can be measured. There were some limitations to the study. Because of class schedule, students in the postsimulation test groups had additional time between lecture and the examination; this may have contributed to the higher test results when compared with the presimulation group. Second, because the groups tested at different or staggered intervals based on the simulation schedule, students may have divulged test material.
6. Recommendations The researchers believe that this pilot study showed positive results and is of value to the nursing education community. Further research into both academic performance and the cognitive domain of learning using HPS needs to be conducted. Further discussion on the impact of HPS on improving student academic performance and its implication on retention and potential performance on National Council Licensure Examination should also be explored. Based on the successful implementation of HPS in the advanced medical– surgical course, the team recommended expanded use of HPS
Table 3
Analysis of variance
Group
n
t-Value (regression)
F Value (GLM)
Probability N |t| or F
Critical threshold
Actual difference
Combined Renal Cardiac
368 184 184
64.73 2.98 2.08
63.02 8.90 4.32
b0.0001 0.0032 0.0390
38.54 55.95 49.64
66.72 84.60 52.32
For model: postsimulation test = 1 (true) versus 0 (false). GLM, general linear model.
26 within the course, and additional modules were developed. An increased use of HPS within other program courses was recommended to the nursing program director. This study has opened doors for not only the nursing program but also for other allied health programs at the college and will continue to advance new ideas about the use of HPS.
References Bremner, M. N., Aduddell, K. F., & Adamson, J. S. (2008). Evidence-based practices related to the human patient simulator and first year
L. Glidewell, C. Conley baccalaureate nursing students' anxiety. Online Journal of Nursing Informatics, 12(1), 10p. Hayden, J. (2010). Use of simulation in nursing education: National survey results. Journal of Nursing Regulation, 1(3), 52−57. Jenson, C., & Forsyth, D. M. (2012). Virtual reality simulation: Using threedimensional technology to teach nursing students. CIN: Computers, Informatics, Nursing, 30(6), 312−318, http://dx.doi.org/10.1097/ NXN.0b013e1824af6ae. Richards, E., Simpson, V., Aaltonene, P., Krebs, L., & Davis, L. (2010). Public health nursing student home visit preparation: The role of simulation in increasing confidence. Home Healthcare Nurse: The Journal for Home Care and Hospice Professional, 28(10), 631−638, http://dx.doi.org/10.1097/NHH.0b013e3181f85e10.
www.jtln.org
The use of human patient simulation to improve academic test scores in nursing students1,2 Laura Glidewell MSN, RN⁎, Cheryl Conley MSN, RN Polk State College, Winter Haven, FL 33881, USA KEYWORDS: Academic test scores; Simulation; Cognitive domain
Abstract Many technologies that were not yet invented when today's educators went to nursing school are now staples in nursing colleges and schools worldwide. Although there are numerous studies that examine the relationship between simulation and student success, most focus on the affective and psychomotor domains of learning. This article focuses on the cognitive domain of learning and discusses the correlation between the use of high-fidelity human patient simulation and improved academic test scores. © 2014 National Organization for Associate Degree Nursing. Published by Elsevier Inc. All rights reserved.
1. Introduction Many technologies that were not yet invented when today's educators went to nursing school are now staples in nursing colleges and schools worldwide. For example, human patient simulation (HPS) for most of the educators consisted of a Resusci-Annie, which is now replaced by high-fidelity HPS that is able to produce life-like physiologic responses to students' actions. The National Council of State Boards of Nursing found that over 1,060 schools nationwide are using simulation in the preparation of prelicensure nursing students (Hayden, 2010). Because more schools compete for limited clinical resources, it is expected that this number will increase. Although there are numerous studies that examine the relationship between simulation and student
success, most focus on the affective and psychomotor domains of learning. Using HPS in nursing education promotes clinical competency, patient safety, and student confidence (Bremner, Aduddell, & Adamson, 2008; Jenson & Forsyth, 2012; Richards, Simpson, Aaltonene, Krebs, & Davis, 2010). A review of the literature related to the cognitive domain in nursing students using HPS reveals no studies specific to this topic. The purpose of this study was to examine the relationship between academic test scores and the use of HPS. National accrediting bodies expect to see measurable learning outcomes with the use of technology. These issues in nursing education prompted the writers to ask the following questions: Is there a correlation between the use of simulation and improved academic outcomes? If so, what is that correlation and how can it best be measured?
1
The authors wish to thank the National Organization of Associate Degree Nursing Foundation for their generous grant support of this research. 2 The authors have no conflict of interest to disclose. ⁎ Corresponding author. Laura Glidewell or Cheryl Conley, Polk State College School of Nursing, 999 Avenue H NE, Winter Haven, FL 33881, USA. E-mail address: [email protected]
2. Background Faculty of an advanced medical–surgical course identified student weaknesses in certain areas based on high-miss
1557-3087/$ – see front matter © 2014 National Organization for Associate Degree Nursing. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.teln.2013.08.001
24
L. Glidewell, C. Conley
test concepts. The areas included pneumothorax management, ventilator settings, chest pain management, complex electrolyte imbalances, and acute stroke management. With this information, the faculty developed four HPS learning activity modules to be incorporated into the course. The four modules were (a) respiratory, which included the pneumothorax and ventilator settings; (b) cardiac; (c) renal, to cover complex electrolyte issues; and (d) neuro. The modules focused on both theory content and areas of clinical concern or skills that were not easily obtained by a large number of students, such as chest tube and ventilator management. Once content and skills were selected and the modules written, piloting of the modules began as an optional learning activity. The four modules were fully incorporated into the course as a mandatory activity the following semester. Using test analysis data on individual test items kept by the department, we analyzed results of three unit examinations covering the topics above. The analysis of the unit examinations from the semester before any simulation was initiated, and the semester where participation was optional was compared against the results of the semester when simulation became a mandatory activity (Table 1). The targeted high-miss items were the same on all three sets of unit examinations. For test security, other test items were equal in difficulty and content but not identical. Group 1 represents the students that took the unit examination without any simulation activities in the semester prior to writing the modules. Group 2 represents the piloting of simulation as an optional activity for students that wanted to participate, and Group 3 represents the students where the modules were mandatory. In two out of three unit examinations, the students with mandatory simulation (Group 3) had a mean score higher than Group 1 and Group 2. One unexpected finding when doing the initial comparison was that Group 3 did not perform as high on the unit Examination 3 as the previous group. The team discovered that Group 3 struggled with medication dosage calculation in Table 1
Student academic test scores (Groups 1–3)
Module 3 that might have taken away from the learning as it was intended. A modification of Module 3 was completed and implemented in the next semester. Table 2 represents the test scores of the students enrolled at the time of the modified Module 3 (Group 4), showing that the new test scores were significantly higher than those reported in Table 1 after the modification. Researcher bias was identified as a potential limitation. Knowing that the area of difficulty was the starting point for writing the modules, emphasis was placed on the review of those concepts and the additional time allotted for the mastery of the material. In addition, the unit examination covered material not specific to the module but rather an entire unit of study; therefore, the student grade was not a reliable indication of only the learning activity. After reviewing this early data, the need for an empirical research study was clear. Preliminary data were shared with administration, and institutional approval was received to conduct a well-designed study. The researchers then sought and received grant assistance from the National Organization for Associate Degree Nursing (N-OADN) to assist in funding the study.
3. Study design Initially, 50 students were invited to participate in the study; however, the study was expanded to a final count of n = 184 and covered three total semesters. The students were considered a homogenous group because each student had taken the same general education courses and prerequisite courses and completed all their nursing education within the program. The students had completed the research lecture in the capstone course prior to being asked to participate. Students signed consent to participate, and each was advised of the right to withdraw at any time. None withdrew. Table 2 Student academic test scores for unit Examination 3 only (Groups 1–4)
90
90
88
88
86
86
84
84
82
82
80
80
78
78 Exam1
Exam 2 Group 1
Group 2
Exam 3 Group 3
Exam1
Exam 2 Group 1
Group 2
Exam 3 Group 3
HPS and academic test scores
25
Two content areas out of the four modules were selected for further study: cardiac and renal. The course syllabus, unit objectives, learning outcomes, and simulation module were forwarded to the development team at Health Education Systems, Inc. (HESI) ©/Elsevier, who developed a custom examination for each module. The examination was returned for review. Recognizing bias as a limitation to the background data, only the program director, who had previous experience teaching the content, reviewed the examinations to determine suitability for the course and objective. Course faculty were not permitted to preview the examination, thus minimizing testing bias. The decision to utilize HESI also resulted from the use of the HESI Exit Examination in the course, thus giving students familiarity with the testing procedure, log on, and administration. Once the examination was selected, a cost proposal was sent to administration for review. The protocol for the study was designed by taking the roster of students that agreed to participate in the study and assigning them randomly into presimulation (test Group A) and postsimulation (test Group B) groups. Funding did not permit for all students to test twice, but consistency was achieved by grouping them this way. With funding support from both the college and NOADN, test fees were covered for the students to take both a cardiac and a renal examination. All students attended the usual course lecture. After all material was covered, test Group A completed the HESI examination prior to their mandatory simulation experience. Then, test Group B took the same HESI examination immediately after their simulation experience. Scores for each group (n = 50) were then evaluated. The department of institutional effectiveness reported that a larger sample would be required to determine the generalization to the nursing student population as a whole; therefore, testing continued for two additional semesters bringing the total sample to n = 184.
4. Results All data have remained confidential, and the department of institutional effectiveness analyzed only raw scores without student identifiers. On the cardiac examination of the first group that tested (Group A), there was a higher score variance among students testing before the simulation with a maximum score of 1,284 (out of 1,500) and an average score of 840, and
the maximum score for the postsimulation group (Group B) was 1,271, but the average score was higher at 892. This represents a differential score average of 52, which is less than three points higher than the threshold required to arrive at a statistical significance. In evaluation of the renal examinations, the score differential is 85 points. The students testing before simulation (Group A) scored an average 951, whereas the students testing after simulation (Group B) scored an average 1,036. These results were what prompted the researchers to continue the testing for two additional semesters. The evaluation of the results of the students that tested after having simulation (Group B) was statistically significant. When combining both the renal and cardiac subsets together and looking at n = 368, there was a score difference of 66 points. This reflects a difference at the P b .01 level, showing the greatest statistical significance. The analysis of variance summarizes the results of the raw data in the study (Table 3).
5. Discussion Review of the results led the researchers to conclude that the use of HPS improves academic test scores. This illustrates a model by which the learning activity and learning outcomes can be measured. There were some limitations to the study. Because of class schedule, students in the postsimulation test groups had additional time between lecture and the examination; this may have contributed to the higher test results when compared with the presimulation group. Second, because the groups tested at different or staggered intervals based on the simulation schedule, students may have divulged test material.
6. Recommendations The researchers believe that this pilot study showed positive results and is of value to the nursing education community. Further research into both academic performance and the cognitive domain of learning using HPS needs to be conducted. Further discussion on the impact of HPS on improving student academic performance and its implication on retention and potential performance on National Council Licensure Examination should also be explored. Based on the successful implementation of HPS in the advanced medical– surgical course, the team recommended expanded use of HPS
Table 3
Analysis of variance
Group
n
t-Value (regression)
F Value (GLM)
Probability N |t| or F
Critical threshold
Actual difference
Combined Renal Cardiac
368 184 184
64.73 2.98 2.08
63.02 8.90 4.32
b0.0001 0.0032 0.0390
38.54 55.95 49.64
66.72 84.60 52.32
For model: postsimulation test = 1 (true) versus 0 (false). GLM, general linear model.
26 within the course, and additional modules were developed. An increased use of HPS within other program courses was recommended to the nursing program director. This study has opened doors for not only the nursing program but also for other allied health programs at the college and will continue to advance new ideas about the use of HPS.
References Bremner, M. N., Aduddell, K. F., & Adamson, J. S. (2008). Evidence-based practices related to the human patient simulator and first year
L. Glidewell, C. Conley baccalaureate nursing students' anxiety. Online Journal of Nursing Informatics, 12(1), 10p. Hayden, J. (2010). Use of simulation in nursing education: National survey results. Journal of Nursing Regulation, 1(3), 52−57. Jenson, C., & Forsyth, D. M. (2012). Virtual reality simulation: Using threedimensional technology to teach nursing students. CIN: Computers, Informatics, Nursing, 30(6), 312−318, http://dx.doi.org/10.1097/ NXN.0b013e1824af6ae. Richards, E., Simpson, V., Aaltonene, P., Krebs, L., & Davis, L. (2010). Public health nursing student home visit preparation: The role of simulation in increasing confidence. Home Healthcare Nurse: The Journal for Home Care and Hospice Professional, 28(10), 631−638, http://dx.doi.org/10.1097/NHH.0b013e3181f85e10.