- Email: [email protected]
‘eSimulation’ Part 2: Evaluation of an interactive multimedia mental health education program for generalist nurses
Collegian (2014) 21, 3—9
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/coll
‘eSimulation’ Part 2: Evaluation of an interactive multimedia mental health education program for generalist nurses Scott Lamont, RN, RMN, MN (Hons) ∗, Scott Brunero, RN, DipApSc, BAHsc, MA (nurs prac) Mental Health Liaison Nursing, Prince of Wales Hospital, Sydney, NSW, Australia Received 14 September 2012; received in revised form 5 November 2012; accepted 6 November 2012
KEYWORDS Simulation; Elearning; Scenario-based learning; Situated learning; Health education
Summary This paper reports on an evaluation of an eSimulation mental health education program for generalist nurses; developed using the following five key attributes of simulation: (1) creating a hypothetical opportunity; (2) authentic representation; (3) active participation; (4) integration; (5) repetition, evaluation and reflection. Four themes emerged from a qualitative thematic analysis of semi-structured interview data involving fourteen generalist nurses. The following four themes: (1) authenticity; (2) participation; (3) clinical reasoning; (4) control of learning provide supporting evidence that these attributes are positioned within the learning resource. Participants found the scenarios within the resource realistic, engaging and relevant to their scope of practice. This type of learning resource may help in developing the knowledge, skills and confidence of generalist nurses in delivering safe and competent mental health care in the generalist setting, when access to specialist services and appropriate means of training are unavailable. © 2012 Australian College of Nursing Ltd. Published by Elsevier Ltd.
Introduction The development of an ‘eSimulation’ mental health education program (MHEP) was described in Part 1. The ‘eSimulation’ MHEP was broadly influenced in the adult learning theory but more specifically in the work of Lave and Wenger (1991). Lave and Wenger (1991) in their situated learning theory argue that learning should not
∗
Corresponding author. Tel.: +61 2 9382 2793. E-mail address: [email protected] (S. Lamont).
be viewed as simply the transmission of information and knowledge from one individual to another, but a process of ‘socialisation’, ‘visualisation’, and ‘imitation’. This process lends itself to the five key attributes of simulation as described by Bland, Topping, and Wood (2011) and articulated in Part 1 of the paper: (1) creating a hypothetical opportunity; (2) authentic representation; (3) active participation; (4) integration; (5) repetition, evaluation and reflection. Whilst improvements in technology have seen an increase in the use of simulation as a means of integrating theory and knowledge within contemporary nursing education, evaluation of the effectiveness of these
1322-7696/$ — see front matter © 2012 Australian College of Nursing Ltd. Published by Elsevier Ltd.
http://dx.doi.org/10.1016/j.colegn.2012.11.002
4 simulation techniques are still lacking (Guise, Chamber, & Valimaki, 2012; Kardong-Edgren, Adamson, & Fitzgerald, 2010). Areas where the use of eSimulation technology have been previously evaluated positively include: critical care nursing competencies (Brady, Molzen, Graham, & O’Neill, 2006), blood transfusion practices (Hogg, Pirie, & Ker, 2006), undergraduate surgery training (Corrigan, Reardon, Shields, & Redmond, 2008), pharmacology training (Collins, Graves, Gullette, & Edwards, 2010), difficult nurse—patient relationships (Brunero & Lamont, 2010), and student nurse decision-making skills (McCallum, Ness, & Price, 2011). Iskander and Curtis (2005), in a review of studies comparing interactivity versus text only instruction, suggest an increase in amount learnt and less study time required to learn new material. An eSimulation versus didactic lecture comparative study was conducted by Cason et al. (2010), measuring knowledge acquisition, transfer and retention relating to airway management in nursing students. Cason et al. (2010) concluded that the eSimulation method was as effective as a lecture in acquiring knowledge and concepts of airway management. Other authors have argued for eSimulations cost effectiveness, but suggest that more empirical data is required regarding its use and cost benefits (Hope, Garside, & Prescott, 2011; Murray, Grant, Horvath, & Leigh, 2007); particularly within the mental health specialty (Guise et al., 2012).
Aim The aim of this paper is to report on the evaluation of an interactive multimedia eSimulation mental health program, developed for learners within the generalist health setting.
S. Lamont, S. Brunero
Data collection Field notes were taken during the follow-up interviews, which ranged from 15 to 30 min in length. Semi-structured interviews were based around the following questions: (1) Can you tell me about your experience of using the resource? (2) How realistic was the scenario? (3) Can you describe your experience of navigating through the resource? (4) How does the learning style compare to other learning methods? The semi-structured interviews were influenced by the five key attributes of simulation by Bland et al. (2011).
Data analysis A six phase theoretical thematic analysis was undertaken of the field notes, which allowed for the development of themes. This process involved: (1) familiarisation with the data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes; (5) defining and naming themes; (6) producing the report (Braun & Clarke, 2006). The analysis within this six stage process is recursive as opposed to linear, where the researchers move back and forth within the data set and throughout these phases, generating codes and subsequent themes. The data set was independently read and re-read to immerse the researchers in the data, with initial notes and ideas being documented to identify information of interest and patterns of meaning. Initial codes were then developed from this data which was then categorised into potential themes, following consensus between both researchers. The themes were then reviewed by each researcher and defined by checking with the coded extracts and entire data set. The final analysis relates the selected themes back to the theoretical underpinnings of the eSimulation resource (Braun & Clarke, 2006).
Ethical considerations
Method A qualitative approach was undertaken to explore the learning experiences of nurses who had used the eSimulation resource. The qualitative approach was informed by the six phases of thematic analysis as described by Braun and Clarke (2006).
Sample An email invitation was sent to generalist nursing staff at a metropolitan tertiary referral hospital in Sydney to identify interest in evaluating the learning resource and a purposive sample of fourteen were subsequently recruited. Participants were sent a web link to the resource and were asked to select one of the three case scenarios. Participants were then interviewed to explore their experiences and perceptions of their learning experience. Twelve of the participants were female and two were male. The majority of the participants had 10 years or more of clinical experience. Various clinical specialties were represented in the group: aged care; neurology; surgical; critical care; oncology; education; respiratory; cardiology.
Ethical approval for the study was sought and granted by the local human research ethics committee. Participants were informed in an information sheet that their participation was voluntary and they could withdraw from the study at any time point. They were advised that they would not be individually identified in the data set and that any reports, publications or presentations from the study would be de identified.
Results Four key themes emerged from the thematic analysis of the interview data. Fig. 1 shows the thematic analysis highlighting the key themes: (1) authenticity; (2) participation; (3) clinical reasoning; (4) control of learning.
Authenticity Participants commented positively on the degree of simulation experienced and to what extent they perceived the relevant resource reflected an authentic clinical situation. Participants expressed the idea of developing a relationship with the clinical scenario, expressing the view that it
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses
5 Life Like
Realistic and believable
Relevance to role Instructional
Navigational difficulty
Positive & negative control
Authenticity
Past experience
Control of Learning
eSimulation
User friendly
Feedback
Clinical reasoning
Visual
Critical thinking
Knowledge gain
Participation Multimedia Interface Reflection Stimulating Engaging
Imagination
Figure 1
Thematic analysis.
was believable, life like and plausible in their own world view. Participants suggested that the scenario they were completing triggered memories of previous clinical experiences, evoking a strong sense of a reflection of their previous practice. Where participants had not reflected on previous experience they suggested the scenarios were still representative of real practice; describing how the scenarios revealed the unknown and investigative nature of their daily clinical work. The authenticity of the scenarios was captured in the following participant statements: ‘‘The scenario was realistic and plausible. . .It triggered my own memories of an incident I was involved in’’ (Participant 9) ‘‘Really engaging so it must have been realistic. . .didn’t know what was going to happen next which is like real practice’’ (Participant 5) The authenticity of the scenarios was expressed by some of the participants through their emotional and behavioural responses when completing the scenario. The tension and emotion in the participants created by aspects of the scenario was apparent, which participants felt reflected contemporary nursing. These emotions and behaviours were seen through and evoked by the pictures, video and audio material in the scenario. The emotional content portrayed by the patients in the scenario was expressed through their verbal and non-verbal behaviour. This behaviour was seen as the trigger for the emotions evoked in the participant
which led to a greater sense of authenticity. The following participants reported: ‘‘Very realistic, patient looked real, facial expressions were real. . .. It drew me in, I felt I was there and I was anxious as it was really busy’’ (Participant 1) ‘‘I could relate to the in-charge as I’ve been there before; where the weight of responsibility is with you. . .It made me feel isolated’’(Participant 2) The ability to create a life-like, realistic representation, which a learner can subsequently engage and learn from, was highlighted. One participant experienced in simulation based education with Hi-Fidelity mannequins compared the resource favourably with existing simulation methods. This participant suggested that whilst the use of high fidelity mannequins was useful, the ability for the trainer to create a life like situation was still limited as it was obvious to participants that the patient was not real: ‘‘More realistic than live simulation we do with HiFidelity dummies. . .. . .staff are uncomfortable acting so it’s not life-like’’ (Participant 1)
Participation Participants overwhelmingly commented on their interaction with the materials within the eSimulation. Participants discussed the importance of adult learning principles and the need to be actively engaged by and within the learning
6
S. Lamont, S. Brunero
process; as opposed to more traditional didactive learning styles, where a sense of detachment is created via passive learning. Several participants specifically commented on the ability of the eSimulation to immerse them in the scenario, almost as if they were part of it: ‘Very realistic, especially the conversation being played out with the mother and in the corridor. . .I thought the scenario brought the cases to life and allowed me to become part of it’’(Participant 3) ‘‘It was realistic and interactive. . .I felt myself involved in the scenario. . .great use of multimedia’’ (Participant 11) The multimedia interface using a mix of videos, photos, audio, and various interactive question and feedback platforms were highlighted as variables which provide greater interactivity with, and interest in the learning. It is this use of multimedia which participants believed separated the eSimulation from more traditional online elearning platforms or indeed classroom based learning: ‘‘Easier to learn with this method of learning than just reading text and answering quizzes like mandatory elearning’’ (Participant 5) ‘‘More user friendly than existing elearning resources. . .. . .is less of a game like quality and more in line with a learning platform’’ (Participant 10) ‘‘Much more interactive, captures your interest. . .. . .traditional learning is less engaging and boring’’ (Participant 6) However, feedback from one participant suggested that this experiential learning style involving active participation may not suit all: ‘‘It would be good to see an entire gold standard management of the situation incorporating the desired staff behaviour. . .with a summary of key points at the end’’ (Participant 12)
Clinical reasoning The processes by which participants arrived at decisions within the eSimulations were a key feature of the evaluations; whether those decisions related to clinical assessment, diagnostic information, interventions, or management issues. The processing and synthesising of clinical and contextual information, the application of critical thinking to cues, questions or circumstances, and the ability to reflect upon different responses to different situations; were all perceived to have been enabled by the eSimulation, and ultimately contributed to the learning and processing of new knowledge. Participants captured this theme by stating: ‘‘Being able to watch examples of patient’s behaviours, seeing how to take the time and effort to identify what is going on and then reacting in a planned way was valuable’’ (Participant 10) ‘‘I liked the way reflective feedback provided learning when a wrong response was chosen. . .as opposed to other online learning systems which simply tell you that your response is incorrect’’ (Participant 7)
‘‘Integration was interesting. . .the learn more icon and additional resources helped contextualise the learning’’ (Participant 3) There was also a sense from participants that the level of interest and engagement created by the eSimulation may in fact enable critical thinking: ‘‘I felt more compelled to think about my responses to questions when asked because I found it interesting’’ (Participant 11) Participants suggested the resource was interesting and thought provoking and conveyed that new knowledge was acquired as a result of engagement with the eSimulation; indicating that the knowledge related objectives of each scenario, as agreed by the authors during the development phase of the resource, had in some way been met. The following participant captured knowledge gain in the following way: ‘‘One of the key messages for me was that aggression can be a maladaptive means of expressing a need. . .and it contextualises a situation I was involved in with an aggressive patient years ago which I don’t think we managed well. . .looking back, he was expressing that needs were not being met’’ (Participant 9) Participants reflected upon the knowledge gained from constructive feedback and the resources within the eSimulation as somewhere to return to and retrieve information as and when required. The resource contains various links to further reading, a dynamic glossary and short one page downloadable resource documents. Whilst the learning is a discrete experience, there is ability to return to the resources and retrieve and revisit the knowledge gained: ‘‘I found the links to additional resources both fascinating and valuable. They helped me to understand the legalities and appropriate terminology’’ (Participant 10) ‘‘There’s lots of Information (legalities) that can be re-accessed if situations arise on the ward’’ (Participant 6)
Control of learning Participants discussed the problems within contemporary nursing in leaving clinical areas or creating protected time in order to access traditional face to face or classroom learning. Participants believed that learning was not prioritised amongst the many competing interests in day to day nursing. ‘‘Attending in-services is difficult because they take place away from the clinical area and the times are often inconvenient. . .although I’m not sure any time is convenient these days, given our workloads’’ (Participant 8) The ability to control or self-regulate the pace of the learning was conveyed by participants. It was perceived by participants that eSimulation allows them to access learning at a time of convenience and control the pace of the learning. Not only was this seen as a benefit which is commensurate with dynamic schedules, but also beneficial in allowing for simulated clinical reasoning:
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses ‘‘Pausing the scenario for questions and feedback allows opportunity to practice decision-making during the learning. . .unlike other learning where you need to wait for an opportunity after the learning has taken place’’ (Participant 2) Participants also commented on a software programming aspect of the eSimulation, which participants perceived compromised an element of control. Learners can navigate back and forth between the different parts of an eSimulation, but cannot do so between different components within parts: ‘‘The inability to navigate back and forth within parts of the scenario was frustrating at times as some parts are lengthy and I completed the package in several sittings’’ (Participant 4)
Discussion The study sought to develop and evaluate an eSimulation MHEP for generalist nurses based on the concept analysis described by Bland et al. (2011). The four themes identified in the thematic evaluation give supporting evidence to suggest that the attributes identified by Bland et al. (2011) in their concept analysis are positioned within the learning resource. Participants perceived that the hypothetical scenarios authentically represented aspects of everyday clinical practice, were plausible, and stated that this realism was a conduit to actively engaging or participating in the learning resource. The benefits of this participation are well described in so far as creating a safe learning environment which affords learners an opportunity to critically think and make decisions in real time, based upon unfolding dynamic events (Medley & Horne, 2005; Lamb, 2007; Rochester et al., 2012). Learners are encouraged and enabled to integrate theory with practice, where any mistakes are free of clinical consequences or personal discomfort (Guise et al., 2012). The authors are encouraged by the engagement of the learners in the resource as outlined earlier in this paper. Engagement and interaction with the learning resource was perceived by participants to be favourably enhanced by the use of multimedia. Educationalists in the interactive multimedia sphere are moving from the didactic based learning efforts typically described as ‘elearning’, to efforts to provide real time simulation to the learner through ‘eSimulation’ (Holt, Segrave, & Cybulski, 2012). Most participants believed this interface and integration with the learning method increased their interest and ultimately the learning outcomes, and also made favourable comparisons to traditional didactic and elearning methods. One participant, however, described a desire for a more didactic learning experience within the eSimulation, highlighting that not all learners have a desire for the participatory learning method involved in eSimulation (Davies, Nathan, & Clarke, 2012; Guise et al., 2012). The opportunity to assess relevant information, critically think and make decisions, and subsequently receive feedback and reflect upon the decisions, is integral to the development of clinical reasoning within healthcare. Clinical reasoning is a process of using information and cues about patient diagnostic or management issues, and
7
subsequently using these to make relevant decisions about patient care (Jensen, in press). Participants commented on the integration of pdf and web resources within the relevant scenarios, and the reflective feedback offered throughout, as conduits to the development of their critical thinking and decision-making skills. The ability to engage the learning at a time of convenience was viewed favourably by participants, whom described difficulties in accessing existing training opportunities or support mechanisms within their dynamic and busy schedules. Regulating the pace of the learning and being able to return to it was also perceived as a strength of the eSimulation, which ultimately enables accessibility to large numbers of nursing staff (Goldsworth, 2012). Using this highly interactive and engaging approach, the learning resource has begun to bridge the gap in engaging clinicians in topics that they may not have ordinarily had a primary interest in. eSimulation allows the learner into situations that they may not be familiar with, engaging in a clinical decision pathway and making decisions in an environment where the fear of making mistakes is mitigated by the simulation. Benefits and limitations are, however, described in the literature (Holt et al., 2012). Benefits of eSimulation educational methodology include its wide availability and accessibility. For example, the software can be made available on health service intranet systems, Internet, CD and DVD; thus learners can use individually at any time when convenient (Goldsworth, 2012). This methodology can therefore facilitate learning in a large volume of people that educators are simply unable to reach, due to time, accessibility and location constraints. Educational eSimulation programs are also novel in that they take the learning to the learner, are innovative and interactive; and when designed in a multimedia format, can be entertaining and fun, thus engaging the learner more than traditional methods (Holt et al., 2012). Limitations of eSimulation as an educational method could be argued to be its prescriptive nature which does not allow for the nuances and contingents of face to face education between ‘novice’ and ‘expert’. Thus questions cannot be asked and context or supporting information cannot be provided, if not already within the eSimulation education resource. However, this can be offset by the provision of relevant further reading within the eSimulation resource. It is also suggested that the use of simulation scenarios may place a considerable burden on the educators whose responsibility it is to write the scenario, in respect of the complexities and time involved in doing so (Wiseman & Horton, 2011). The development of effective simulation scenarios may require educators to combine narrative case study writing and gaming principles; areas which arguably, they are often not familiar with (Begg, 2010; Kirk & MacPhail, 2002). It is further posited that the development and implementation of these learning resources can be costly and time consuming (Guise et al., 2012), although such costs are arguably offset once complete.
Implications for eSimulation MHEPs for generalist healthcare professionals The knowledge, skills and confidence of generalist nurses has been identified as a specific need within contemporary
8 healthcare (World Health Organisation, 2007). It is unclear to what extent these needs have been met at undergraduate level, or indeed whether liaison mental health services are effectively resourced to fulfil this requirement (Gordon & Wolf, 2010). Although this type of learning is primarily positioned within undergraduate nursing curricula; the use of eSimulation MHEPs for health care professionals in the general hospital setting is one way of addressing the mental health literacy needs of this professional group, when access to specialist services and appropriate means of training are unavailable. Mental health or psychiatry liaison services traditionally have on call or low coverage beyond business hours, therefore are unable to provide specialist education programs ‘out of hours’ and spend long periods of time consumed with patient diagnostics and management within hours. Mental health liaison nurses are therefore positioned well to develop eSimulations which can be accessed by large numbers of generalist nurses, at times of convenience. Not only can these eSimulations be used as a means of engaging safely in tailored knowledge and skill acquisition processes; the policy, web link, management plan, bibliography, and pdf resources which are integrated with the learning process, can be used as a resource bank when access to specialist consultation or support is absent.
Study limits The study primarily involved nurses with lengthy clinical experience. Undergraduates or new graduates may have produced different results. Despite the qualitative nature of the interviews generating rich in depth data, the outcome of analysis cannot be generalised by the authors beyond the small sample provided. Rather, it is for the reader to consider and determine the implications and potential application of this type of learning resource within their own community of practice. Also, the fidelity of the results may have been compromised by the quality of the field notes taken by the two different researchers, who approached the evaluation from a theoretical perspective.
Conclusion The interactive multimedia eSimulation resource created from this project is a novel way of engaging learners to meet the requirements of safe and effective patient care. Using this methodology, learners can safely make clinical or managerial decisions and reflect upon them, within a simulated environment which reflects ‘real life’ critical incidents and clinical specialties.
References Begg, M. (2010). Virtual patients: Practical advice for clinical authors using Labyrinth. The Clinical Teacher, 7, 202—205. Bland, A., Topping, A., & Wood, B. (2011). A concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Education Today, 31, 664—670. Brady, D., Molzen, S., Graham, S., & O’Neill, V. (2006). Using the synergy of online education and simulation to inspire a new
S. Lamont, S. Brunero model for a community critical care course. Critical Care Nursing Quarterly, 29(3), 231—236. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77—101. Brunero, S., & Lamont, S. (2010). The ‘‘difficult’’ nurse patient relationship; modelling nursing care through scenario based elearning. Contemporary Nurse, 35(2), 136—146. Cason, C. L., Cazzell, M. A., Nelson, K. A., Hartman, V., Roye, J., & Mancini, M. E. (2010). Improving learning of airway management with case-based computer microsimulations. Clinical Simulation in Nursing, 6(1), 15—23. Collins, A. S., Graves, B. A., Gullette, D., & Edwards, R. (2010). Developing an interactive microsimulation method in pharmacology. Journal of Nursing Education, 49(7), 410—413. Corrigan, M., Reardon, M., Shields, C., & Redmond, H. (2008). ‘‘SURGENT’’ — student e-learning for reality. The application of interactive visual images to problem-based learning in undergraduate surgery. Journal of Surgical Education, 65(2), 120—125. Davies, J., Nathan, M., & Clarke, D. (2012). An evaluation of a complex simulated scenario with final year undergraduate children’s nursing student. Collegian, 19, 131—138. Goldsworth, S. (2012). High fidelity simulation in critical care: A Canadian perspective. Collegian, 19, 139—143. Gordon, L., & Wolf, S. (2010). Liaison psychiatry services in south England. The Psychiatrist, 34, 270—273. Guise, V., Chamber, M., & Valimaki, M. (2012). What can virtual patient simulation offer mental health nursing education? Journal of Psychiatric and Mental Health Nursing, 19, 410—418. Hogg, G., Pirie, E. S., & Ker, J. (2006). The use of simulated learning to promote safe blood transfusion practice. Nurse Education in Practice, 6(4), 214—223. Holt, D., Segrave, S., & Cybulski, J. (2012). E-Simulations for educating the professions in blended learning environments. In D. Holt, S. Segrave, & J. Cybulski (Eds.), Professional education using e-simulations: Benefits of blended learning design. Hershey, PA, USA: IGI Global. Hope, A., Garside, J., & Prescott, S. (2011). Rethinking theory and practice: Pre-registration student nurses experiences of simulation teaching and learning in the acquisition of clinical skills in preparation for practice. Nurse Education Today, 31(7), 711—715. Iskander, W., & Curtis, S. (2005). Use of colour and interactive animation in learning 3D vectors. Journal of Computers in Mathematics and Science Teaching, 24(2), 149—156. Jensen, R. (in press). Clinical reasoning during simulation: Comparison of student and faculty ratings. Nurse Education in Practice. Kardong-Edgren, S., Adamson, K. A., & Fitzgerald, C. (2010). A review of currently published instruments for human patient simulation. Clinical Simulation in Nursing, 6, 25—35. Kirk, D., & MacPhail, A. (2002). Teaching games for understanding and situated learning: Rethinking the Bunker—Thorpe model. Journal of Teaching in Physical Education, 21(2), 177—192. Lamb, D. (2007). Could simulated emergency procedures practised in a static environment improve the clinical performance of a Critical Care Air Support Team (CCAST)? A literature review. Intensive and Critical Care Nursing, 23(1), 33—42. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University press. McCallum, J., Ness, V., & Price, T. (2011). Exploring nursing students’ decision-making skills whilst in a Second Life clinical simulation laboratory. Nurse Education Today, 31(7), 699—704.
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses Medley, C. F., & Horne, C. (2005). Using simulation technology for undergraduate nursing education. Journal of Nursing Education, 44(1), 31—34. Murray, C., Grant, M. J., Horvath, M. L., & Leigh, J. (2007). The use of simulation as a teaching and learning approach to support practice learning. Nurse Education in Practice, 8(1), 5—8. Rochester, S., Kelly, M., Disler, R., White, H., Forber, J., & Matiuk, S. (2012). Providing simulation experiences for large cohorts of 1st year nursing students: Evaluating quality and impact. Collegian, 19, 117—124.
9
Wiseman, A., & Horton, K. (2011). Developing clinical scenarios from a European perspective: Successes and challenges. Nurse Education Today, 31(7), 677—681. World Health Organisation. (2007). Atlas: Nurses in mental health 2007. Available from:. http://www.who.int/mental health/evidence/nursing atlas 2007.pdf
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/coll
‘eSimulation’ Part 2: Evaluation of an interactive multimedia mental health education program for generalist nurses Scott Lamont, RN, RMN, MN (Hons) ∗, Scott Brunero, RN, DipApSc, BAHsc, MA (nurs prac) Mental Health Liaison Nursing, Prince of Wales Hospital, Sydney, NSW, Australia Received 14 September 2012; received in revised form 5 November 2012; accepted 6 November 2012
KEYWORDS Simulation; Elearning; Scenario-based learning; Situated learning; Health education
Summary This paper reports on an evaluation of an eSimulation mental health education program for generalist nurses; developed using the following five key attributes of simulation: (1) creating a hypothetical opportunity; (2) authentic representation; (3) active participation; (4) integration; (5) repetition, evaluation and reflection. Four themes emerged from a qualitative thematic analysis of semi-structured interview data involving fourteen generalist nurses. The following four themes: (1) authenticity; (2) participation; (3) clinical reasoning; (4) control of learning provide supporting evidence that these attributes are positioned within the learning resource. Participants found the scenarios within the resource realistic, engaging and relevant to their scope of practice. This type of learning resource may help in developing the knowledge, skills and confidence of generalist nurses in delivering safe and competent mental health care in the generalist setting, when access to specialist services and appropriate means of training are unavailable. © 2012 Australian College of Nursing Ltd. Published by Elsevier Ltd.
Introduction The development of an ‘eSimulation’ mental health education program (MHEP) was described in Part 1. The ‘eSimulation’ MHEP was broadly influenced in the adult learning theory but more specifically in the work of Lave and Wenger (1991). Lave and Wenger (1991) in their situated learning theory argue that learning should not
∗
Corresponding author. Tel.: +61 2 9382 2793. E-mail address: [email protected] (S. Lamont).
be viewed as simply the transmission of information and knowledge from one individual to another, but a process of ‘socialisation’, ‘visualisation’, and ‘imitation’. This process lends itself to the five key attributes of simulation as described by Bland, Topping, and Wood (2011) and articulated in Part 1 of the paper: (1) creating a hypothetical opportunity; (2) authentic representation; (3) active participation; (4) integration; (5) repetition, evaluation and reflection. Whilst improvements in technology have seen an increase in the use of simulation as a means of integrating theory and knowledge within contemporary nursing education, evaluation of the effectiveness of these
1322-7696/$ — see front matter © 2012 Australian College of Nursing Ltd. Published by Elsevier Ltd.
http://dx.doi.org/10.1016/j.colegn.2012.11.002
4 simulation techniques are still lacking (Guise, Chamber, & Valimaki, 2012; Kardong-Edgren, Adamson, & Fitzgerald, 2010). Areas where the use of eSimulation technology have been previously evaluated positively include: critical care nursing competencies (Brady, Molzen, Graham, & O’Neill, 2006), blood transfusion practices (Hogg, Pirie, & Ker, 2006), undergraduate surgery training (Corrigan, Reardon, Shields, & Redmond, 2008), pharmacology training (Collins, Graves, Gullette, & Edwards, 2010), difficult nurse—patient relationships (Brunero & Lamont, 2010), and student nurse decision-making skills (McCallum, Ness, & Price, 2011). Iskander and Curtis (2005), in a review of studies comparing interactivity versus text only instruction, suggest an increase in amount learnt and less study time required to learn new material. An eSimulation versus didactic lecture comparative study was conducted by Cason et al. (2010), measuring knowledge acquisition, transfer and retention relating to airway management in nursing students. Cason et al. (2010) concluded that the eSimulation method was as effective as a lecture in acquiring knowledge and concepts of airway management. Other authors have argued for eSimulations cost effectiveness, but suggest that more empirical data is required regarding its use and cost benefits (Hope, Garside, & Prescott, 2011; Murray, Grant, Horvath, & Leigh, 2007); particularly within the mental health specialty (Guise et al., 2012).
Aim The aim of this paper is to report on the evaluation of an interactive multimedia eSimulation mental health program, developed for learners within the generalist health setting.
S. Lamont, S. Brunero
Data collection Field notes were taken during the follow-up interviews, which ranged from 15 to 30 min in length. Semi-structured interviews were based around the following questions: (1) Can you tell me about your experience of using the resource? (2) How realistic was the scenario? (3) Can you describe your experience of navigating through the resource? (4) How does the learning style compare to other learning methods? The semi-structured interviews were influenced by the five key attributes of simulation by Bland et al. (2011).
Data analysis A six phase theoretical thematic analysis was undertaken of the field notes, which allowed for the development of themes. This process involved: (1) familiarisation with the data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes; (5) defining and naming themes; (6) producing the report (Braun & Clarke, 2006). The analysis within this six stage process is recursive as opposed to linear, where the researchers move back and forth within the data set and throughout these phases, generating codes and subsequent themes. The data set was independently read and re-read to immerse the researchers in the data, with initial notes and ideas being documented to identify information of interest and patterns of meaning. Initial codes were then developed from this data which was then categorised into potential themes, following consensus between both researchers. The themes were then reviewed by each researcher and defined by checking with the coded extracts and entire data set. The final analysis relates the selected themes back to the theoretical underpinnings of the eSimulation resource (Braun & Clarke, 2006).
Ethical considerations
Method A qualitative approach was undertaken to explore the learning experiences of nurses who had used the eSimulation resource. The qualitative approach was informed by the six phases of thematic analysis as described by Braun and Clarke (2006).
Sample An email invitation was sent to generalist nursing staff at a metropolitan tertiary referral hospital in Sydney to identify interest in evaluating the learning resource and a purposive sample of fourteen were subsequently recruited. Participants were sent a web link to the resource and were asked to select one of the three case scenarios. Participants were then interviewed to explore their experiences and perceptions of their learning experience. Twelve of the participants were female and two were male. The majority of the participants had 10 years or more of clinical experience. Various clinical specialties were represented in the group: aged care; neurology; surgical; critical care; oncology; education; respiratory; cardiology.
Ethical approval for the study was sought and granted by the local human research ethics committee. Participants were informed in an information sheet that their participation was voluntary and they could withdraw from the study at any time point. They were advised that they would not be individually identified in the data set and that any reports, publications or presentations from the study would be de identified.
Results Four key themes emerged from the thematic analysis of the interview data. Fig. 1 shows the thematic analysis highlighting the key themes: (1) authenticity; (2) participation; (3) clinical reasoning; (4) control of learning.
Authenticity Participants commented positively on the degree of simulation experienced and to what extent they perceived the relevant resource reflected an authentic clinical situation. Participants expressed the idea of developing a relationship with the clinical scenario, expressing the view that it
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses
5 Life Like
Realistic and believable
Relevance to role Instructional
Navigational difficulty
Positive & negative control
Authenticity
Past experience
Control of Learning
eSimulation
User friendly
Feedback
Clinical reasoning
Visual
Critical thinking
Knowledge gain
Participation Multimedia Interface Reflection Stimulating Engaging
Imagination
Figure 1
Thematic analysis.
was believable, life like and plausible in their own world view. Participants suggested that the scenario they were completing triggered memories of previous clinical experiences, evoking a strong sense of a reflection of their previous practice. Where participants had not reflected on previous experience they suggested the scenarios were still representative of real practice; describing how the scenarios revealed the unknown and investigative nature of their daily clinical work. The authenticity of the scenarios was captured in the following participant statements: ‘‘The scenario was realistic and plausible. . .It triggered my own memories of an incident I was involved in’’ (Participant 9) ‘‘Really engaging so it must have been realistic. . .didn’t know what was going to happen next which is like real practice’’ (Participant 5) The authenticity of the scenarios was expressed by some of the participants through their emotional and behavioural responses when completing the scenario. The tension and emotion in the participants created by aspects of the scenario was apparent, which participants felt reflected contemporary nursing. These emotions and behaviours were seen through and evoked by the pictures, video and audio material in the scenario. The emotional content portrayed by the patients in the scenario was expressed through their verbal and non-verbal behaviour. This behaviour was seen as the trigger for the emotions evoked in the participant
which led to a greater sense of authenticity. The following participants reported: ‘‘Very realistic, patient looked real, facial expressions were real. . .. It drew me in, I felt I was there and I was anxious as it was really busy’’ (Participant 1) ‘‘I could relate to the in-charge as I’ve been there before; where the weight of responsibility is with you. . .It made me feel isolated’’(Participant 2) The ability to create a life-like, realistic representation, which a learner can subsequently engage and learn from, was highlighted. One participant experienced in simulation based education with Hi-Fidelity mannequins compared the resource favourably with existing simulation methods. This participant suggested that whilst the use of high fidelity mannequins was useful, the ability for the trainer to create a life like situation was still limited as it was obvious to participants that the patient was not real: ‘‘More realistic than live simulation we do with HiFidelity dummies. . .. . .staff are uncomfortable acting so it’s not life-like’’ (Participant 1)
Participation Participants overwhelmingly commented on their interaction with the materials within the eSimulation. Participants discussed the importance of adult learning principles and the need to be actively engaged by and within the learning
6
S. Lamont, S. Brunero
process; as opposed to more traditional didactive learning styles, where a sense of detachment is created via passive learning. Several participants specifically commented on the ability of the eSimulation to immerse them in the scenario, almost as if they were part of it: ‘Very realistic, especially the conversation being played out with the mother and in the corridor. . .I thought the scenario brought the cases to life and allowed me to become part of it’’(Participant 3) ‘‘It was realistic and interactive. . .I felt myself involved in the scenario. . .great use of multimedia’’ (Participant 11) The multimedia interface using a mix of videos, photos, audio, and various interactive question and feedback platforms were highlighted as variables which provide greater interactivity with, and interest in the learning. It is this use of multimedia which participants believed separated the eSimulation from more traditional online elearning platforms or indeed classroom based learning: ‘‘Easier to learn with this method of learning than just reading text and answering quizzes like mandatory elearning’’ (Participant 5) ‘‘More user friendly than existing elearning resources. . .. . .is less of a game like quality and more in line with a learning platform’’ (Participant 10) ‘‘Much more interactive, captures your interest. . .. . .traditional learning is less engaging and boring’’ (Participant 6) However, feedback from one participant suggested that this experiential learning style involving active participation may not suit all: ‘‘It would be good to see an entire gold standard management of the situation incorporating the desired staff behaviour. . .with a summary of key points at the end’’ (Participant 12)
Clinical reasoning The processes by which participants arrived at decisions within the eSimulations were a key feature of the evaluations; whether those decisions related to clinical assessment, diagnostic information, interventions, or management issues. The processing and synthesising of clinical and contextual information, the application of critical thinking to cues, questions or circumstances, and the ability to reflect upon different responses to different situations; were all perceived to have been enabled by the eSimulation, and ultimately contributed to the learning and processing of new knowledge. Participants captured this theme by stating: ‘‘Being able to watch examples of patient’s behaviours, seeing how to take the time and effort to identify what is going on and then reacting in a planned way was valuable’’ (Participant 10) ‘‘I liked the way reflective feedback provided learning when a wrong response was chosen. . .as opposed to other online learning systems which simply tell you that your response is incorrect’’ (Participant 7)
‘‘Integration was interesting. . .the learn more icon and additional resources helped contextualise the learning’’ (Participant 3) There was also a sense from participants that the level of interest and engagement created by the eSimulation may in fact enable critical thinking: ‘‘I felt more compelled to think about my responses to questions when asked because I found it interesting’’ (Participant 11) Participants suggested the resource was interesting and thought provoking and conveyed that new knowledge was acquired as a result of engagement with the eSimulation; indicating that the knowledge related objectives of each scenario, as agreed by the authors during the development phase of the resource, had in some way been met. The following participant captured knowledge gain in the following way: ‘‘One of the key messages for me was that aggression can be a maladaptive means of expressing a need. . .and it contextualises a situation I was involved in with an aggressive patient years ago which I don’t think we managed well. . .looking back, he was expressing that needs were not being met’’ (Participant 9) Participants reflected upon the knowledge gained from constructive feedback and the resources within the eSimulation as somewhere to return to and retrieve information as and when required. The resource contains various links to further reading, a dynamic glossary and short one page downloadable resource documents. Whilst the learning is a discrete experience, there is ability to return to the resources and retrieve and revisit the knowledge gained: ‘‘I found the links to additional resources both fascinating and valuable. They helped me to understand the legalities and appropriate terminology’’ (Participant 10) ‘‘There’s lots of Information (legalities) that can be re-accessed if situations arise on the ward’’ (Participant 6)
Control of learning Participants discussed the problems within contemporary nursing in leaving clinical areas or creating protected time in order to access traditional face to face or classroom learning. Participants believed that learning was not prioritised amongst the many competing interests in day to day nursing. ‘‘Attending in-services is difficult because they take place away from the clinical area and the times are often inconvenient. . .although I’m not sure any time is convenient these days, given our workloads’’ (Participant 8) The ability to control or self-regulate the pace of the learning was conveyed by participants. It was perceived by participants that eSimulation allows them to access learning at a time of convenience and control the pace of the learning. Not only was this seen as a benefit which is commensurate with dynamic schedules, but also beneficial in allowing for simulated clinical reasoning:
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses ‘‘Pausing the scenario for questions and feedback allows opportunity to practice decision-making during the learning. . .unlike other learning where you need to wait for an opportunity after the learning has taken place’’ (Participant 2) Participants also commented on a software programming aspect of the eSimulation, which participants perceived compromised an element of control. Learners can navigate back and forth between the different parts of an eSimulation, but cannot do so between different components within parts: ‘‘The inability to navigate back and forth within parts of the scenario was frustrating at times as some parts are lengthy and I completed the package in several sittings’’ (Participant 4)
Discussion The study sought to develop and evaluate an eSimulation MHEP for generalist nurses based on the concept analysis described by Bland et al. (2011). The four themes identified in the thematic evaluation give supporting evidence to suggest that the attributes identified by Bland et al. (2011) in their concept analysis are positioned within the learning resource. Participants perceived that the hypothetical scenarios authentically represented aspects of everyday clinical practice, were plausible, and stated that this realism was a conduit to actively engaging or participating in the learning resource. The benefits of this participation are well described in so far as creating a safe learning environment which affords learners an opportunity to critically think and make decisions in real time, based upon unfolding dynamic events (Medley & Horne, 2005; Lamb, 2007; Rochester et al., 2012). Learners are encouraged and enabled to integrate theory with practice, where any mistakes are free of clinical consequences or personal discomfort (Guise et al., 2012). The authors are encouraged by the engagement of the learners in the resource as outlined earlier in this paper. Engagement and interaction with the learning resource was perceived by participants to be favourably enhanced by the use of multimedia. Educationalists in the interactive multimedia sphere are moving from the didactic based learning efforts typically described as ‘elearning’, to efforts to provide real time simulation to the learner through ‘eSimulation’ (Holt, Segrave, & Cybulski, 2012). Most participants believed this interface and integration with the learning method increased their interest and ultimately the learning outcomes, and also made favourable comparisons to traditional didactic and elearning methods. One participant, however, described a desire for a more didactic learning experience within the eSimulation, highlighting that not all learners have a desire for the participatory learning method involved in eSimulation (Davies, Nathan, & Clarke, 2012; Guise et al., 2012). The opportunity to assess relevant information, critically think and make decisions, and subsequently receive feedback and reflect upon the decisions, is integral to the development of clinical reasoning within healthcare. Clinical reasoning is a process of using information and cues about patient diagnostic or management issues, and
7
subsequently using these to make relevant decisions about patient care (Jensen, in press). Participants commented on the integration of pdf and web resources within the relevant scenarios, and the reflective feedback offered throughout, as conduits to the development of their critical thinking and decision-making skills. The ability to engage the learning at a time of convenience was viewed favourably by participants, whom described difficulties in accessing existing training opportunities or support mechanisms within their dynamic and busy schedules. Regulating the pace of the learning and being able to return to it was also perceived as a strength of the eSimulation, which ultimately enables accessibility to large numbers of nursing staff (Goldsworth, 2012). Using this highly interactive and engaging approach, the learning resource has begun to bridge the gap in engaging clinicians in topics that they may not have ordinarily had a primary interest in. eSimulation allows the learner into situations that they may not be familiar with, engaging in a clinical decision pathway and making decisions in an environment where the fear of making mistakes is mitigated by the simulation. Benefits and limitations are, however, described in the literature (Holt et al., 2012). Benefits of eSimulation educational methodology include its wide availability and accessibility. For example, the software can be made available on health service intranet systems, Internet, CD and DVD; thus learners can use individually at any time when convenient (Goldsworth, 2012). This methodology can therefore facilitate learning in a large volume of people that educators are simply unable to reach, due to time, accessibility and location constraints. Educational eSimulation programs are also novel in that they take the learning to the learner, are innovative and interactive; and when designed in a multimedia format, can be entertaining and fun, thus engaging the learner more than traditional methods (Holt et al., 2012). Limitations of eSimulation as an educational method could be argued to be its prescriptive nature which does not allow for the nuances and contingents of face to face education between ‘novice’ and ‘expert’. Thus questions cannot be asked and context or supporting information cannot be provided, if not already within the eSimulation education resource. However, this can be offset by the provision of relevant further reading within the eSimulation resource. It is also suggested that the use of simulation scenarios may place a considerable burden on the educators whose responsibility it is to write the scenario, in respect of the complexities and time involved in doing so (Wiseman & Horton, 2011). The development of effective simulation scenarios may require educators to combine narrative case study writing and gaming principles; areas which arguably, they are often not familiar with (Begg, 2010; Kirk & MacPhail, 2002). It is further posited that the development and implementation of these learning resources can be costly and time consuming (Guise et al., 2012), although such costs are arguably offset once complete.
Implications for eSimulation MHEPs for generalist healthcare professionals The knowledge, skills and confidence of generalist nurses has been identified as a specific need within contemporary
8 healthcare (World Health Organisation, 2007). It is unclear to what extent these needs have been met at undergraduate level, or indeed whether liaison mental health services are effectively resourced to fulfil this requirement (Gordon & Wolf, 2010). Although this type of learning is primarily positioned within undergraduate nursing curricula; the use of eSimulation MHEPs for health care professionals in the general hospital setting is one way of addressing the mental health literacy needs of this professional group, when access to specialist services and appropriate means of training are unavailable. Mental health or psychiatry liaison services traditionally have on call or low coverage beyond business hours, therefore are unable to provide specialist education programs ‘out of hours’ and spend long periods of time consumed with patient diagnostics and management within hours. Mental health liaison nurses are therefore positioned well to develop eSimulations which can be accessed by large numbers of generalist nurses, at times of convenience. Not only can these eSimulations be used as a means of engaging safely in tailored knowledge and skill acquisition processes; the policy, web link, management plan, bibliography, and pdf resources which are integrated with the learning process, can be used as a resource bank when access to specialist consultation or support is absent.
Study limits The study primarily involved nurses with lengthy clinical experience. Undergraduates or new graduates may have produced different results. Despite the qualitative nature of the interviews generating rich in depth data, the outcome of analysis cannot be generalised by the authors beyond the small sample provided. Rather, it is for the reader to consider and determine the implications and potential application of this type of learning resource within their own community of practice. Also, the fidelity of the results may have been compromised by the quality of the field notes taken by the two different researchers, who approached the evaluation from a theoretical perspective.
Conclusion The interactive multimedia eSimulation resource created from this project is a novel way of engaging learners to meet the requirements of safe and effective patient care. Using this methodology, learners can safely make clinical or managerial decisions and reflect upon them, within a simulated environment which reflects ‘real life’ critical incidents and clinical specialties.
References Begg, M. (2010). Virtual patients: Practical advice for clinical authors using Labyrinth. The Clinical Teacher, 7, 202—205. Bland, A., Topping, A., & Wood, B. (2011). A concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Education Today, 31, 664—670. Brady, D., Molzen, S., Graham, S., & O’Neill, V. (2006). Using the synergy of online education and simulation to inspire a new
S. Lamont, S. Brunero model for a community critical care course. Critical Care Nursing Quarterly, 29(3), 231—236. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77—101. Brunero, S., & Lamont, S. (2010). The ‘‘difficult’’ nurse patient relationship; modelling nursing care through scenario based elearning. Contemporary Nurse, 35(2), 136—146. Cason, C. L., Cazzell, M. A., Nelson, K. A., Hartman, V., Roye, J., & Mancini, M. E. (2010). Improving learning of airway management with case-based computer microsimulations. Clinical Simulation in Nursing, 6(1), 15—23. Collins, A. S., Graves, B. A., Gullette, D., & Edwards, R. (2010). Developing an interactive microsimulation method in pharmacology. Journal of Nursing Education, 49(7), 410—413. Corrigan, M., Reardon, M., Shields, C., & Redmond, H. (2008). ‘‘SURGENT’’ — student e-learning for reality. The application of interactive visual images to problem-based learning in undergraduate surgery. Journal of Surgical Education, 65(2), 120—125. Davies, J., Nathan, M., & Clarke, D. (2012). An evaluation of a complex simulated scenario with final year undergraduate children’s nursing student. Collegian, 19, 131—138. Goldsworth, S. (2012). High fidelity simulation in critical care: A Canadian perspective. Collegian, 19, 139—143. Gordon, L., & Wolf, S. (2010). Liaison psychiatry services in south England. The Psychiatrist, 34, 270—273. Guise, V., Chamber, M., & Valimaki, M. (2012). What can virtual patient simulation offer mental health nursing education? Journal of Psychiatric and Mental Health Nursing, 19, 410—418. Hogg, G., Pirie, E. S., & Ker, J. (2006). The use of simulated learning to promote safe blood transfusion practice. Nurse Education in Practice, 6(4), 214—223. Holt, D., Segrave, S., & Cybulski, J. (2012). E-Simulations for educating the professions in blended learning environments. In D. Holt, S. Segrave, & J. Cybulski (Eds.), Professional education using e-simulations: Benefits of blended learning design. Hershey, PA, USA: IGI Global. Hope, A., Garside, J., & Prescott, S. (2011). Rethinking theory and practice: Pre-registration student nurses experiences of simulation teaching and learning in the acquisition of clinical skills in preparation for practice. Nurse Education Today, 31(7), 711—715. Iskander, W., & Curtis, S. (2005). Use of colour and interactive animation in learning 3D vectors. Journal of Computers in Mathematics and Science Teaching, 24(2), 149—156. Jensen, R. (in press). Clinical reasoning during simulation: Comparison of student and faculty ratings. Nurse Education in Practice. Kardong-Edgren, S., Adamson, K. A., & Fitzgerald, C. (2010). A review of currently published instruments for human patient simulation. Clinical Simulation in Nursing, 6, 25—35. Kirk, D., & MacPhail, A. (2002). Teaching games for understanding and situated learning: Rethinking the Bunker—Thorpe model. Journal of Teaching in Physical Education, 21(2), 177—192. Lamb, D. (2007). Could simulated emergency procedures practised in a static environment improve the clinical performance of a Critical Care Air Support Team (CCAST)? A literature review. Intensive and Critical Care Nursing, 23(1), 33—42. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University press. McCallum, J., Ness, V., & Price, T. (2011). Exploring nursing students’ decision-making skills whilst in a Second Life clinical simulation laboratory. Nurse Education Today, 31(7), 699—704.
‘eSimulation’ Part 2: Evaluation of a mental health education program for generalist nurses Medley, C. F., & Horne, C. (2005). Using simulation technology for undergraduate nursing education. Journal of Nursing Education, 44(1), 31—34. Murray, C., Grant, M. J., Horvath, M. L., & Leigh, J. (2007). The use of simulation as a teaching and learning approach to support practice learning. Nurse Education in Practice, 8(1), 5—8. Rochester, S., Kelly, M., Disler, R., White, H., Forber, J., & Matiuk, S. (2012). Providing simulation experiences for large cohorts of 1st year nursing students: Evaluating quality and impact. Collegian, 19, 117—124.
9
Wiseman, A., & Horton, K. (2011). Developing clinical scenarios from a European perspective: Successes and challenges. Nurse Education Today, 31(7), 677—681. World Health Organisation. (2007). Atlas: Nurses in mental health 2007. Available from:. http://www.who.int/mental health/evidence/nursing atlas 2007.pdf