- Email: [email protected]
Development and Implementation of an Electronic Learning Module for Volumetric Image-Guided Radiation Therapy
Journal of Medical Imaging and Radiation Sciences
Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Journal de l’imagerie médicale et des sciences de la radiation
www.elsevier.com/locate/jmir
Research Article
Development and Implementation of an Electronic Learning Module for Volumetric Image-Guided Radiation Therapy Winnie Li, MSc, MRT(T)ab*, Angela Cashell, MSc, MRT(T)ab, David A. Jaffray, PhDab and Douglas Moseley, PhDab a
Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario b Department of Radiation Oncology, University of Toronto, Toronto, Ontario
ABSTRACT Background: Image-guided radiation therapy (IGRT) through daily cone-beam computed tomography (CBCT) has significantly impacted the role of the radiation therapist at our institution; continuing education is crucial to ensure safe practice of technology in the clinical environment. The purpose of this work was to develop and implement an electronic learning (eLearning) module as a yearly refresher for CBCT-practicing radiation therapists. Methods: To provide an innovative interface to engage the learner’s interest, a module themed ‘‘Myths in Cone Beam Computed Tomography Practice’’ was developed by content experts at our institution. The eLearning tool focused on the technical aspects and fundamental theory of CBCT acquisition, with an aim to refresh the user’s knowledge and confidence in image fusion and assessment. Ten myths were identified in the module and evidence-based content was referenced within each myth to support theory. Evaluation of the learner was performed through a multiple choice quiz at the end of the module. The tool and 12 multiple choice questions were reviewed and validated by subject matter and non–subject matter experts in CBCT–IGRT before departmental implementation. Results: The CBCT eLearning module has been clinically implemented and used over the last 3 years by radiation therapists in our department. Completion of the tool is an annual mandatory requirement for CBCT-practicing therapists; over 100 participants completed the module per year. The median time for module completion decreased over the 3-year interval, from 42 minutes 25 seconds during the first year of implementation to 20 minutes and 48 seconds in the third year. Conclusions: An electronic online training tool for CBCT refresher training has been developed and implemented at our institution, with an aim to equip staff with the critical thinking skills and clinical judgment required to operate in a CBCT–IGRT environment. The
module’s format ensures delivery of consistent information as a component of yearly continuing education for radiation therapists.
RESUM E But : La radiotherapie guidee par l’image (RTGI) par tomodensitometrie a faisceau conique quotidienne (TDM FC) a eu des repercussions importantes sur le r^ole des radiotherapeutes dans notre institution; la formation continue est essentielle pour assurer la pratique securitaire de la technologie Dans l’environnement clinique. Le but de ce travail etait de developper et de mettre en place un module d’apprentissage electronique pour la formation de rappel annuelle des radiotherapeutes œuvrant en RTGI. Methodologie : Afin d’offrir une interface stimulante et de susciter l’inter^et de l’apprenant, un module a ete developpe par des experts en la matiere de notre etablissement, sous le theme « Mythes de la pratique en radiotherapie guidee par l’image ». L’outil d’apprentissage electronique met l’accent sur les aspects techniques et les fondements theoriques de l’acquisition d’images en TDM FC, dans le but de rafra^ıchir les connaissances et la confiance de l’utilisateur en matiere de fusion et d’evaluation des images. Dix mythes ont ete recenses dans le module, et un contenu fonde sur des donnees probantes a ete mis en reference pour chacun des mythes afin d’appuyer la theorie. L’evaluation de l’apprenant se fait par un questionnaire de 12 questions a choix multiples a la fin du module. L’outil et les questions a choix multiple ont ete examines et valides par des experts et des non-experts en RTGI/TDM FC avant la mise en œuvre dans le service. Resultats : Le module d’apprentissage electronique en RTGI a ete mis en place dans le milieu clinique et est utilise depuis trois ans par les radiotherapeutes de notre service. Son utilisation est une exigence annuelle obligatoire pour les therapeutes en RTGI; plus de 100 participants utilisent le module chaque annee. La duree
This work was presented in part at the 9th Annual Radiation Therapy Conference, February 28th–March 2nd 2013, Toronto, ON. The author(s) has no financial disclosures or conflicts of interest to declare. * Corresponding author: Winnie Li, MSc, MRT(T), Radiation Medicine Program, Princess Margaret Cancer Centre, 610 University Avenue, Level 2B–Cobalt Lounge, Toronto, ON, Canada. E-mail address: [email protected] (W. Li). 1939-8654/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jmir.2015.12.001
mediane de l’exercice a diminue au fil des trois annees d’utilisation, passant de 42 minutes 25 secondes durant la premiere annee a 20 minutes et 48 seconds durant la troisieme. Conclusions : Un outil de formation electronique enligne pour la formation de rappel en RTGI a ete developpe et mis en place dans notre institution, dans le but de fournir aux membres du personnel
les outils de reflexion critique et de jugement clinique necessaires pour fonctionner dans un environnement RTGI/TDM FC. Le format du module permet d’assurer la presentation d’une information uniforme dans le cadre de la formation continue annuelle des radiotherapeutes.
Keywords: Image-guided radiation therapy (IGRT); cone-beam computed tomography (CBCT); electronic learning (eLearning); refresher training
Introduction Advances in image-guided radiation therapy (IGRT) have been facilitated by the availability of 3-dimensional conebeam computed tomography (CBCT) guidance systems [1, 2]. CBCT allows online volumetric visualization of patient anatomy and has the capability to correct for daily setup variations while monitoring patient changes and deformations in real-time. As more information is immediately available to front line users of the technology, it has been identified that IGRT through CBCT has significantly impacted the role of the radiation therapist [3]. In the changing environment of volumetric IGRT technology, skill development and integration of knowledge is essential to ensure clinical competence of radiation therapists [4]. At the onset of clinical implementation of CBCT technology at our institution, to ensure therapists were readily equipped to manage daily decision-making regarding patient setups, a two-level training program was designed and implemented [5]. The first level of training focused on the basic hardware and software components of the CBCT system, arranged within the first week of work on a CBCT unit. The second level of training focused on consolidation of knowledge, where therapists are led through a series of cases for image matching and registration and are given an opportunity to observe and ask questions. Other centres have also implemented training programs at the onset of CBCT implementation to increase staff confidence for the clinical introduction of volumetric IGRT technology [6]. Aided by education, well-defined processes, and critical analysis skills, radiation therapists are readily making online decisions about patient setups through volumetric IGRT. Therefore, to ensure consistency in practice, education and training for CBCT–IGRT should adapt to the needs of the radiation therapists. The training model previously developed was only performed and received by therapists during their initial placement on a CBCT machine [5]. There was a lack of refresher training provided to therapists at our institution on a continuous basis to reinforce theory and concepts to guide clinical image guidance decision making. Thus, education strategies for front-line users of CBCT technology have to evolve to ensure that daily use of CBCT is effective and efficient on a longitudinal spectrum [7]. The successful implementation of electronic learning (eLearning) education tools has been identified in the nursing 44
profession [8–16]. E-learning was used to facilitate undergraduate nursing education on topics such as pain management [15] and infection control [9]. At the graduate level, advanced practice nurses responded positively to online learning to facilitate their education [13]. In another graduate level study, evidence-based online case studies were found to be an effective strategy for teaching nursing students about complementary medical therapies [16]. Various health care professions have used an electronic module to assess acute asthma severity on pediatric patients in the clinical setting with positive results [17]. The need for continuing education through an electronic platform has been previously recognized in radiation therapy [18]. We hypothesize that an eLearning refresher module is relevant and effective for practicing radiation therapists in the CBCT–IGRT era. The aim of this study was to develop, validate, and implement an electronic module for volumetric imaging as an essential component of continuing education for radiation therapists.
Methods Development Format As the focus on developing yearly refresher training sessions was to ensure IGRT concepts are understood and enforced, various platforms and clinical requirements for educational delivery were considered. It was decided that ideally, refresher training would require low yearly maintenance, and the format should require minimal requirements of oversight and facilitator resources once developed. An electronic online platform was deemed most feasible to deliver consistent information in a timely manner. The benefits associated with an electronic learning tool include: delivery of consistent information; reduction in instruction time; convenience and increased accessibility for the learner; improved tracking; and no time constraints on the learner as staff can complete the module at their own pace [8, 10, 11]. As such, based on the literature and current continuing education models, an electronic learning module developed through the institution’s consolidated platform (SumTotal Systems, LLC, Skillsoft Company) was deemed the most appropriate media for education dissemination.
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Content The electronic module focused on the technical aspects and fundamental theory of CBCT acquisition, with an aim to refresh the user’s knowledge and confidence in image fusion and assessment. Considerations for development included ensuring the information provided in the module is current and relevant. Strategically, to ensure sustainability and longitudinal use, only fundamental concepts and theory of image guidance were included. Defining effective learning objectives is an essential step to ensure success of the training module. Specific module objectives were to ensure accurate use of current IGRT software; to understand the limitations of IGRT software; to appreciate matching strategies and surrogate use; and to support continuing education for staff in the CBCT–IGRT environment. Information was collated and vetted by the expert CBCT– IGRT research radiation therapist (W.L.) at our institution. Information was initially adapted from previously developed in-house training of image guidance concepts [5], with additional data curated and accumulated from other in-house resources. This information was supplemented by a search of published literature, with an aim to support theory with practice to ensure the information presented was evidence-based and inclusive of widely accepted principles in the community. Data were refined by the content expert to ensure the breadth and depth of CBCT imaging practice within the gathered information was cohesive and applicable to practicing IGRT radiation therapists. To provide an innovative interface to engage the learner’s interest, the module was themed ‘‘Myths in Cone Beam Computed Tomography Practice’’ to outline common misconceptions about image guidance practice. From the collated content material focused on fundamental concepts and theory of image guidance, 10 myths were identified for development in the module, with evidence-based content referenced within each myth to support theory. Graphics to support the content were collected and developed accordingly. As a vital part of the module, an evaluation component is essential to confirm that learners are achieving their learning objectives and processing key messages. As such, the electronic module was designed with a section aimed at assessing the learner’s outcome. A summative assessment and evaluation of the learner was performed through a postcourse 12 multiple choice question (MCQ) evaluation quiz developed by the content experts. Validation Validation of the eLearning module occurred in two stages as described in the following sections.
subject matter experts in CBCT–IGRT: two research radiation therapists and one medical physicist. The experts were asked to focus their critique on: (1) General review of image-guidance concepts and information, with a focus on content: is there too much content or essential content missing? (2) Appropriateness of module length as a yearly refresher. (3) Assess quiz questions for level of difficulty. (4) Determine appropriate quiz pass rate. The content and evaluation questions were revised based on feedback provided from the subject matter expert reviews. On complete revision, the draft module content was sent to the institution’s eLearning centre for development, copying editing, and design. After the eLearning module design was completed, the initial version of the eLearning module was presented back to the content experts. The graphics, content, and flow of the module on the electronic platform was tested and revised as required. Various links within the module were tested for functionality, and graphics re-evaluated for clarity. Revisions and redrafts of the eLearning module were performed over six cycles, leading to a near-finalized beta product on the eLearning platform. Module Pilot Evaluation A pilot evaluation on the eLearning module was performed to obtain further critical analysis, with a focus on the evaluation of the content, design, and layout. The near-finalized beta version of the module was reviewed by a combination of subject matter and non–subject matter experts. The pilot evaluators included a radiation therapy clinical educator, radiation therapy treatment delivery practice leader, radiation therapy manager, and four in-house radiation therapy imaging experts. The pilot group was asked to complete the entire module and MCQs and provide critical feedback on the content, interface, and format. After the pilot, modification to select graphics within the module was performed to increase clarity and reduce ambiguity. Text within subsections was also revised to increase readability, whereas explanations in other areas were increased to facilitate further understanding of the more challenging concepts. The pilot also confirmed the module completion time of 60 minutes, and the pass rate of the module was modified from 100% to 75% because of the level of difficulty of some of the MCQs. The identified changes from the pilot group were corrected in the eLearning module before clinical implementation. Clinical Implementation
Content Evaluation The module content and quiz questions were reviewed and validated before development by the eLearning centre. Once the draft module was designed and developed by the content expert, evaluation was performed by three independent
After the module was launched on the institution’s eLearning site, a communication was sent to all the radiation therapists in our department to introduce the module. Staff was subsequently asked to complete the module on a yearly basis as a requirement of ongoing continuing education. The
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
45
completion of the module was mandatory for all CBCT practicing radiation therapists at our institution, specifically those on the treatment units, as well as those who participate in after hours on-call. To facilitate evaluation of the content, design, relevance, and effectiveness of the eLearning module from a participant perspective, an evaluation of the time required to complete the module and quiz results was performed. The time required for module completion informs whether the time outlined to complete the module is realistic and to assess the appropriateness of module length. The quiz scores for therapists who completed the module will also inform future modifications of the tool, potentially identifying confusing or inappropriate questions. Participants’ data was retrospectively collected under approval from the institution research ethics board.
Month 1
Initiation Draft proposal for CBCT eLearning Module
Month 2
Approved for development
Month 3
Module Content Design and Development
Month 7
Initial meeting with institution eLearning Team
Month 9
Module content validation by Subject Matter Experts
Month 10
Module content forwarded to eLearning Team Development and design
Month 14
First draft of eLearning module complete
Month 15
Module reviews and redrafts cycles complete
Month 15
Module Pilot Evaluation
Month 16
Revisions performed based on pilot assessment
Month 17
Clinical Implementation Final version posted on eLearning site
Results The module was developed, validated, and clinically implemented at our institution for radiation therapists in 2012. The timeline for module development and implementation occurred over a 17-month time frame from initiation to clinical dissemination (see Figure 1). As noted in the timeline, the longest part of the process was content curation and development (ie, 6 months) and initial copy editing and design by the eLearning team (ie, 4 months). The final product and module interface is shown in Figure 2. Reports of participant activity on the module were performed through the tracking system within the eLearning platform. Results were collected over a 3-year period (2012–2014), including number of staff who completed the module, module pass rates, and time for completion (see Table 1). The level of staff use remained consistent as module completion is a yearly mandatory requirement of continuing education. The median time for module completion decreased from 42 minutes 25 seconds during the first year of implementation to 20 minutes and 48 seconds in the third year. The minimum time for module completion represents the cohort of staff who clicked through the module to get to the quiz and completed the quiz quickly. The maximum time for module completion may represent the cohort of staff who chose to open the eLearning module and complete each myth throughout their work day. The module quiz scores of 100% were achieved by a larger proportion of participants in 2014 (60.4%) than in 2012 (33.8%). Discussion As the culture of radiotherapy evolves with increasing technological advances, these innovations and evolving paradigms impact the role, knowledge, and skill set requirements of the radiation therapist [7]. An electronic refresher module for CBCT-practicing radiation therapists was developed at our institution, with an aim to equip staff with the critical thinking skills and clinical judgment required to operate in 46
Figure 1. CBCT eLearning module timeline. CBCT, cone-beam computed tomography.
a CBCT–IGRT environment. The online module promotes standardization by delivering consistent information and fundamental theory and background to support clinical practice and protocol on a yearly basis. As the module is self-led and widely available, its adoption among staff has been high over the first 3 years of implementation. This continuing education tool has successfully served the purpose of a yearly refresher of fundamental IGRT theory for practicing therapists, as well as for therapists who are returning to a treatment delivery unit from other areas (ie, treatment planning and maternity leave). Volumetric image guidance has greatly impacted the roles and responsibilities of the radiation therapist [3, 5]. It has also been previously identified that with an increasing scope of practice driven by technological advances, there is inconsistency in perceptions, practices, and role assignment when interpreting CBCT images [4]. As such, continuing professional development activities, such as the CBCT module presented in this study, is a step toward providing appropriate information through an appropriate medium at the right time. In addition to this module, it is the intention
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Figure 2. ‘‘Myths in cone-beam computed tomography’’ eLearning module interface.
that this theory-based module be coupled with an interactive, instructor-led component. Four additional sessions based on anatomic regions supplement the eLearning module at our institution, aiding to translate theory into everyday practice. Table 1 Participant Results for Module Completion Over 3 Years
Module completion rate Number of participants Percentage of all staff (%) Time for completion (h:min:s) Median Minimum Maximum Module quiz scores (%) 75 83 92 100
2012
2013
2014
130 84
108 72
106 71
0:42:25 0:05:56 4:08:17
0:32:36 0:02:02 6:50:03
0:20:48 0:02:36 4:36:15
22.3 24.6 19.2 33.8
14.3 20.4 15.3 50.0
16.7 13.5 9.4 60.4
Results from the module participants over the last 3 years indicate that the time required for completion is decreasing, and percentage of staff receiving perfect scores on the evaluation quiz is increasing. As such, future modifications to the module may involve increasing the number of postmodule evaluation MCQs and changing the order in which the questions are asked to decrease memory bias from previous years. Additional evaluation of the MCQs will be performed to ensure the questions asked are effective in fulfilling learning objectives [19]. It can also be observed from the range of time required for module completion that a learner has flexibility with the module, taking as little or as much time required to complete the eLearning objectives. As CBCT implementation spreads from a local scale to a standard of practice in rapidly changing paradigms [7], it is logical to consider the module’s relevance and applicability to centres outside our profession and institution. It has been outlined by Dzau et al that two common obstacles to the implementation of innovation is the initial effective translation of discovery into clinical practice and the expansion beyond
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
47
niche expert acceptance to broader adoption in the community [20]. As this module was built within an institution that performed extensive research and development on CBCT technology, the validated module may be a good resource of standardized image-guidance knowledge for radiation therapists outside our institution. Future plans include converting the module to an app to increase mobility within our institution and accessibility outside our institution. In an environment of technological change, the roles and responsibilities of the professionals within the radiation medicine team have been impacted [21]. Entry to practice curriculum overview and continuing education is under redevelopment in radiation oncology to solidify a new framework to keep pace with technology and practice nationally and internationally [22, 23]. As image guidance technology impacts all disciplines in radiation medicine, the module’s applicability to radiation oncology and radiation physics is currently being explored. Dissemination of evidence-based information through the module ensures safe use and practice of CBCT that is becoming mainstream practice in the interprofessional model. Validation of the module may allow its dissemination to other centres as a continuing education tool to enhance the practice of CBCT–IGRT. A challenge of the online tool is that it needs to be reviewed yearly to ensure that the information provided is current and relevant. This presents a good opportunity to incorporate feedback from staff by modifying sections of the module that were unclear. It also allows the content development team to assess whether some pertinent sections of image guidance practice were missing. In addition, it should be evaluated whether other resources, such as videos or interactive components, may be more effective in delivering more complex concepts. One of the main issues already identified was the small font size on some of the screens; this was an uncontrolled factor, limited by requirements of the institution’s eLearning team to develop the module on the lowest screen limitations across the department.
Conclusion An electronic online training tool for CBCT refresher training has been developed and implemented at our institution. The module’s format ensures delivery of consistent information as a component of yearly continuing education for radiation therapists. The module’s applicability to radiation therapists at other centres and professional disciplines is under evaluation. References [1] Jaffray, D. A., Siewerdsen, J. H., & Wong, J. W., et al. (2002). Flatpanel cone-beam computed tomography for image-guided radiation therapy. Int J Radiat Oncol Biol Phys 53, 1337–1349. [2] Jaffray, D. A. (2005). Emergent technologies for 3-dimensional imageguided radiation delivery. Semin Radiat Oncol 15, 208–216.
48
[3] White, E., & Kane, G. (2007). Radiation medicine practice in the image-guided radiation therapy era: new roles and new opportunities. Semin Radiat Oncol 17, 298–305. [4] Gillan, C., Li, W., & Harnett, N. (2013). Radiation therapist perspectives on cone-beam computed tomography practices and response to information. J Radiother Pract 12, 237–244. [5] Li, W., Harnett, N., Moseley, D. J., Higgins, J., Chan, K., & Jaffray, D. A. (2010). Investigating user perspective on training and clinical implementation of volumetric imaging. J Med Imaging Radiat Sci 41, 57–65. [6] Liszewski, B., DiProspero, L., Bagley, R., Osmar, K., & D’Alimonte, L. (2012). A preliminary evaluation of a clinical training program for volumetric imaging. J Med Imaging Radiat Sci 43, S31. [7] Jaffray, D. A. (2012). Image-guided radiotherapy: from current concept to future perspectives. Nat Rev Clin Oncol 9, 688–699. [8] Welsh, E., Wanberg, C., Brown, K., & Simmering, M. (2003). E-learning: emerging uses, empirical results and future directions. International Journal of Training and Development 7, 245–258. [9] Reime, M., Harris, A., Aksnees, J., & Mikkelsen, J. (2008). The most successful method in teaching nursing students infection controld e-learning or lecture? Nurse Educ Today 28, 798–806. [10] Ellaway, R., & Masters, K. (2008). Amee guide 32: e-learning in medical education part 1: learning, teaching and assessment. Med Teach 30, 455–473. [11] Masters, K., & Ellaway, R. (2008). E-learning in medical education guide 32 part 2: technology, management and design. Med Teach 30, 474–489. [12] Bloomfied, J., White, A., & Roberts, J. (2008). Using computer assisted learning for clinical skills education in nursing: integrative review. J Adv Nurs 63, 222–235. [13] Huckstadt, A., & Hayes, K. (2005). Evaluation of interactive online courses for advanced practice nurses. J Am Acad Nurse Pract 17, 85–89. [14] Kala, S., Isramalai, S., & Pohthong, A. (2010). Electronic learning and constructivism: a model for nursing education. Nurse Educ Today 30, 61–66. [15] Keefe, G., & Wharrad, H. (2012). Using e-learning to enhance nursing students’ pain management educati. Nurse Educ Today 32, e66–e72. [16] Swanson, B., Zeller, J., & Keithley, J., et al. (2012). Case-based online modules to teach graduate-level nursing students about complementary and alternative medical therapies. J Prof Nurs 28, 125–129. [17] Lehr, A. R., McKinney, M. L., & Gouin, S., et al. (2013). Development and pretesting of an electronic learning module to train health care professionals on the use of the pediatric respiratory assessment measure to assess acute asthma severity. Can Respir J 20, 435–441. [18] Foroudi, F., Wong, J., & Kron, T., et al. (2010). Development and evaluation of a training program for therapeutic radiographers as a basis for online adaptive radiation therapy for bladder carcinoma. Radiography 16, 14–20. [19] Collins, J. (2006). Writing multiple-choice questions for continuing medical education activities and self-assessment modules. Radiographics 26, 543–551. [20] Dzau, V. J., Ackerly, D. C., & Sutton-Wallace, P., et al. (2010). The role of academic health science systems in the transformation of medicine. Lancet 375, 949–953. [21] Gillan, C., Wiljer, D., & Harnett, N., et al. (2010). Changing stress while stressing change: the role of interprofessional education in mediating stress in the introduction of a transformative technology. J Interprof Care 24, 710–721. [22] Giuliani, M. E., Gillan, C., & Milne, R. A., et al. (2014). Determining an imaging literacy curriculum for radiation oncologists: an international delphi study. Int J Radiat Oncol Biol Phys 88, 961–966. [23] Turner, S., Eriksen, J. G., & Trotter, T., et al. (2015). Establishing a global radiation oncology collaboration in education (grace): objectives and priorities. Radiother Oncol 117, 188–192.
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Journal de l’imagerie médicale et des sciences de la radiation
www.elsevier.com/locate/jmir
Research Article
Development and Implementation of an Electronic Learning Module for Volumetric Image-Guided Radiation Therapy Winnie Li, MSc, MRT(T)ab*, Angela Cashell, MSc, MRT(T)ab, David A. Jaffray, PhDab and Douglas Moseley, PhDab a
Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario b Department of Radiation Oncology, University of Toronto, Toronto, Ontario
ABSTRACT Background: Image-guided radiation therapy (IGRT) through daily cone-beam computed tomography (CBCT) has significantly impacted the role of the radiation therapist at our institution; continuing education is crucial to ensure safe practice of technology in the clinical environment. The purpose of this work was to develop and implement an electronic learning (eLearning) module as a yearly refresher for CBCT-practicing radiation therapists. Methods: To provide an innovative interface to engage the learner’s interest, a module themed ‘‘Myths in Cone Beam Computed Tomography Practice’’ was developed by content experts at our institution. The eLearning tool focused on the technical aspects and fundamental theory of CBCT acquisition, with an aim to refresh the user’s knowledge and confidence in image fusion and assessment. Ten myths were identified in the module and evidence-based content was referenced within each myth to support theory. Evaluation of the learner was performed through a multiple choice quiz at the end of the module. The tool and 12 multiple choice questions were reviewed and validated by subject matter and non–subject matter experts in CBCT–IGRT before departmental implementation. Results: The CBCT eLearning module has been clinically implemented and used over the last 3 years by radiation therapists in our department. Completion of the tool is an annual mandatory requirement for CBCT-practicing therapists; over 100 participants completed the module per year. The median time for module completion decreased over the 3-year interval, from 42 minutes 25 seconds during the first year of implementation to 20 minutes and 48 seconds in the third year. Conclusions: An electronic online training tool for CBCT refresher training has been developed and implemented at our institution, with an aim to equip staff with the critical thinking skills and clinical judgment required to operate in a CBCT–IGRT environment. The
module’s format ensures delivery of consistent information as a component of yearly continuing education for radiation therapists.
RESUM E But : La radiotherapie guidee par l’image (RTGI) par tomodensitometrie a faisceau conique quotidienne (TDM FC) a eu des repercussions importantes sur le r^ole des radiotherapeutes dans notre institution; la formation continue est essentielle pour assurer la pratique securitaire de la technologie Dans l’environnement clinique. Le but de ce travail etait de developper et de mettre en place un module d’apprentissage electronique pour la formation de rappel annuelle des radiotherapeutes œuvrant en RTGI. Methodologie : Afin d’offrir une interface stimulante et de susciter l’inter^et de l’apprenant, un module a ete developpe par des experts en la matiere de notre etablissement, sous le theme « Mythes de la pratique en radiotherapie guidee par l’image ». L’outil d’apprentissage electronique met l’accent sur les aspects techniques et les fondements theoriques de l’acquisition d’images en TDM FC, dans le but de rafra^ıchir les connaissances et la confiance de l’utilisateur en matiere de fusion et d’evaluation des images. Dix mythes ont ete recenses dans le module, et un contenu fonde sur des donnees probantes a ete mis en reference pour chacun des mythes afin d’appuyer la theorie. L’evaluation de l’apprenant se fait par un questionnaire de 12 questions a choix multiples a la fin du module. L’outil et les questions a choix multiple ont ete examines et valides par des experts et des non-experts en RTGI/TDM FC avant la mise en œuvre dans le service. Resultats : Le module d’apprentissage electronique en RTGI a ete mis en place dans le milieu clinique et est utilise depuis trois ans par les radiotherapeutes de notre service. Son utilisation est une exigence annuelle obligatoire pour les therapeutes en RTGI; plus de 100 participants utilisent le module chaque annee. La duree
This work was presented in part at the 9th Annual Radiation Therapy Conference, February 28th–March 2nd 2013, Toronto, ON. The author(s) has no financial disclosures or conflicts of interest to declare. * Corresponding author: Winnie Li, MSc, MRT(T), Radiation Medicine Program, Princess Margaret Cancer Centre, 610 University Avenue, Level 2B–Cobalt Lounge, Toronto, ON, Canada. E-mail address: [email protected] (W. Li). 1939-8654/$ - see front matter Ó 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jmir.2015.12.001
mediane de l’exercice a diminue au fil des trois annees d’utilisation, passant de 42 minutes 25 secondes durant la premiere annee a 20 minutes et 48 seconds durant la troisieme. Conclusions : Un outil de formation electronique enligne pour la formation de rappel en RTGI a ete developpe et mis en place dans notre institution, dans le but de fournir aux membres du personnel
les outils de reflexion critique et de jugement clinique necessaires pour fonctionner dans un environnement RTGI/TDM FC. Le format du module permet d’assurer la presentation d’une information uniforme dans le cadre de la formation continue annuelle des radiotherapeutes.
Keywords: Image-guided radiation therapy (IGRT); cone-beam computed tomography (CBCT); electronic learning (eLearning); refresher training
Introduction Advances in image-guided radiation therapy (IGRT) have been facilitated by the availability of 3-dimensional conebeam computed tomography (CBCT) guidance systems [1, 2]. CBCT allows online volumetric visualization of patient anatomy and has the capability to correct for daily setup variations while monitoring patient changes and deformations in real-time. As more information is immediately available to front line users of the technology, it has been identified that IGRT through CBCT has significantly impacted the role of the radiation therapist [3]. In the changing environment of volumetric IGRT technology, skill development and integration of knowledge is essential to ensure clinical competence of radiation therapists [4]. At the onset of clinical implementation of CBCT technology at our institution, to ensure therapists were readily equipped to manage daily decision-making regarding patient setups, a two-level training program was designed and implemented [5]. The first level of training focused on the basic hardware and software components of the CBCT system, arranged within the first week of work on a CBCT unit. The second level of training focused on consolidation of knowledge, where therapists are led through a series of cases for image matching and registration and are given an opportunity to observe and ask questions. Other centres have also implemented training programs at the onset of CBCT implementation to increase staff confidence for the clinical introduction of volumetric IGRT technology [6]. Aided by education, well-defined processes, and critical analysis skills, radiation therapists are readily making online decisions about patient setups through volumetric IGRT. Therefore, to ensure consistency in practice, education and training for CBCT–IGRT should adapt to the needs of the radiation therapists. The training model previously developed was only performed and received by therapists during their initial placement on a CBCT machine [5]. There was a lack of refresher training provided to therapists at our institution on a continuous basis to reinforce theory and concepts to guide clinical image guidance decision making. Thus, education strategies for front-line users of CBCT technology have to evolve to ensure that daily use of CBCT is effective and efficient on a longitudinal spectrum [7]. The successful implementation of electronic learning (eLearning) education tools has been identified in the nursing 44
profession [8–16]. E-learning was used to facilitate undergraduate nursing education on topics such as pain management [15] and infection control [9]. At the graduate level, advanced practice nurses responded positively to online learning to facilitate their education [13]. In another graduate level study, evidence-based online case studies were found to be an effective strategy for teaching nursing students about complementary medical therapies [16]. Various health care professions have used an electronic module to assess acute asthma severity on pediatric patients in the clinical setting with positive results [17]. The need for continuing education through an electronic platform has been previously recognized in radiation therapy [18]. We hypothesize that an eLearning refresher module is relevant and effective for practicing radiation therapists in the CBCT–IGRT era. The aim of this study was to develop, validate, and implement an electronic module for volumetric imaging as an essential component of continuing education for radiation therapists.
Methods Development Format As the focus on developing yearly refresher training sessions was to ensure IGRT concepts are understood and enforced, various platforms and clinical requirements for educational delivery were considered. It was decided that ideally, refresher training would require low yearly maintenance, and the format should require minimal requirements of oversight and facilitator resources once developed. An electronic online platform was deemed most feasible to deliver consistent information in a timely manner. The benefits associated with an electronic learning tool include: delivery of consistent information; reduction in instruction time; convenience and increased accessibility for the learner; improved tracking; and no time constraints on the learner as staff can complete the module at their own pace [8, 10, 11]. As such, based on the literature and current continuing education models, an electronic learning module developed through the institution’s consolidated platform (SumTotal Systems, LLC, Skillsoft Company) was deemed the most appropriate media for education dissemination.
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Content The electronic module focused on the technical aspects and fundamental theory of CBCT acquisition, with an aim to refresh the user’s knowledge and confidence in image fusion and assessment. Considerations for development included ensuring the information provided in the module is current and relevant. Strategically, to ensure sustainability and longitudinal use, only fundamental concepts and theory of image guidance were included. Defining effective learning objectives is an essential step to ensure success of the training module. Specific module objectives were to ensure accurate use of current IGRT software; to understand the limitations of IGRT software; to appreciate matching strategies and surrogate use; and to support continuing education for staff in the CBCT–IGRT environment. Information was collated and vetted by the expert CBCT– IGRT research radiation therapist (W.L.) at our institution. Information was initially adapted from previously developed in-house training of image guidance concepts [5], with additional data curated and accumulated from other in-house resources. This information was supplemented by a search of published literature, with an aim to support theory with practice to ensure the information presented was evidence-based and inclusive of widely accepted principles in the community. Data were refined by the content expert to ensure the breadth and depth of CBCT imaging practice within the gathered information was cohesive and applicable to practicing IGRT radiation therapists. To provide an innovative interface to engage the learner’s interest, the module was themed ‘‘Myths in Cone Beam Computed Tomography Practice’’ to outline common misconceptions about image guidance practice. From the collated content material focused on fundamental concepts and theory of image guidance, 10 myths were identified for development in the module, with evidence-based content referenced within each myth to support theory. Graphics to support the content were collected and developed accordingly. As a vital part of the module, an evaluation component is essential to confirm that learners are achieving their learning objectives and processing key messages. As such, the electronic module was designed with a section aimed at assessing the learner’s outcome. A summative assessment and evaluation of the learner was performed through a postcourse 12 multiple choice question (MCQ) evaluation quiz developed by the content experts. Validation Validation of the eLearning module occurred in two stages as described in the following sections.
subject matter experts in CBCT–IGRT: two research radiation therapists and one medical physicist. The experts were asked to focus their critique on: (1) General review of image-guidance concepts and information, with a focus on content: is there too much content or essential content missing? (2) Appropriateness of module length as a yearly refresher. (3) Assess quiz questions for level of difficulty. (4) Determine appropriate quiz pass rate. The content and evaluation questions were revised based on feedback provided from the subject matter expert reviews. On complete revision, the draft module content was sent to the institution’s eLearning centre for development, copying editing, and design. After the eLearning module design was completed, the initial version of the eLearning module was presented back to the content experts. The graphics, content, and flow of the module on the electronic platform was tested and revised as required. Various links within the module were tested for functionality, and graphics re-evaluated for clarity. Revisions and redrafts of the eLearning module were performed over six cycles, leading to a near-finalized beta product on the eLearning platform. Module Pilot Evaluation A pilot evaluation on the eLearning module was performed to obtain further critical analysis, with a focus on the evaluation of the content, design, and layout. The near-finalized beta version of the module was reviewed by a combination of subject matter and non–subject matter experts. The pilot evaluators included a radiation therapy clinical educator, radiation therapy treatment delivery practice leader, radiation therapy manager, and four in-house radiation therapy imaging experts. The pilot group was asked to complete the entire module and MCQs and provide critical feedback on the content, interface, and format. After the pilot, modification to select graphics within the module was performed to increase clarity and reduce ambiguity. Text within subsections was also revised to increase readability, whereas explanations in other areas were increased to facilitate further understanding of the more challenging concepts. The pilot also confirmed the module completion time of 60 minutes, and the pass rate of the module was modified from 100% to 75% because of the level of difficulty of some of the MCQs. The identified changes from the pilot group were corrected in the eLearning module before clinical implementation. Clinical Implementation
Content Evaluation The module content and quiz questions were reviewed and validated before development by the eLearning centre. Once the draft module was designed and developed by the content expert, evaluation was performed by three independent
After the module was launched on the institution’s eLearning site, a communication was sent to all the radiation therapists in our department to introduce the module. Staff was subsequently asked to complete the module on a yearly basis as a requirement of ongoing continuing education. The
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
45
completion of the module was mandatory for all CBCT practicing radiation therapists at our institution, specifically those on the treatment units, as well as those who participate in after hours on-call. To facilitate evaluation of the content, design, relevance, and effectiveness of the eLearning module from a participant perspective, an evaluation of the time required to complete the module and quiz results was performed. The time required for module completion informs whether the time outlined to complete the module is realistic and to assess the appropriateness of module length. The quiz scores for therapists who completed the module will also inform future modifications of the tool, potentially identifying confusing or inappropriate questions. Participants’ data was retrospectively collected under approval from the institution research ethics board.
Month 1
Initiation Draft proposal for CBCT eLearning Module
Month 2
Approved for development
Month 3
Module Content Design and Development
Month 7
Initial meeting with institution eLearning Team
Month 9
Module content validation by Subject Matter Experts
Month 10
Module content forwarded to eLearning Team Development and design
Month 14
First draft of eLearning module complete
Month 15
Module reviews and redrafts cycles complete
Month 15
Module Pilot Evaluation
Month 16
Revisions performed based on pilot assessment
Month 17
Clinical Implementation Final version posted on eLearning site
Results The module was developed, validated, and clinically implemented at our institution for radiation therapists in 2012. The timeline for module development and implementation occurred over a 17-month time frame from initiation to clinical dissemination (see Figure 1). As noted in the timeline, the longest part of the process was content curation and development (ie, 6 months) and initial copy editing and design by the eLearning team (ie, 4 months). The final product and module interface is shown in Figure 2. Reports of participant activity on the module were performed through the tracking system within the eLearning platform. Results were collected over a 3-year period (2012–2014), including number of staff who completed the module, module pass rates, and time for completion (see Table 1). The level of staff use remained consistent as module completion is a yearly mandatory requirement of continuing education. The median time for module completion decreased from 42 minutes 25 seconds during the first year of implementation to 20 minutes and 48 seconds in the third year. The minimum time for module completion represents the cohort of staff who clicked through the module to get to the quiz and completed the quiz quickly. The maximum time for module completion may represent the cohort of staff who chose to open the eLearning module and complete each myth throughout their work day. The module quiz scores of 100% were achieved by a larger proportion of participants in 2014 (60.4%) than in 2012 (33.8%). Discussion As the culture of radiotherapy evolves with increasing technological advances, these innovations and evolving paradigms impact the role, knowledge, and skill set requirements of the radiation therapist [7]. An electronic refresher module for CBCT-practicing radiation therapists was developed at our institution, with an aim to equip staff with the critical thinking skills and clinical judgment required to operate in 46
Figure 1. CBCT eLearning module timeline. CBCT, cone-beam computed tomography.
a CBCT–IGRT environment. The online module promotes standardization by delivering consistent information and fundamental theory and background to support clinical practice and protocol on a yearly basis. As the module is self-led and widely available, its adoption among staff has been high over the first 3 years of implementation. This continuing education tool has successfully served the purpose of a yearly refresher of fundamental IGRT theory for practicing therapists, as well as for therapists who are returning to a treatment delivery unit from other areas (ie, treatment planning and maternity leave). Volumetric image guidance has greatly impacted the roles and responsibilities of the radiation therapist [3, 5]. It has also been previously identified that with an increasing scope of practice driven by technological advances, there is inconsistency in perceptions, practices, and role assignment when interpreting CBCT images [4]. As such, continuing professional development activities, such as the CBCT module presented in this study, is a step toward providing appropriate information through an appropriate medium at the right time. In addition to this module, it is the intention
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
Figure 2. ‘‘Myths in cone-beam computed tomography’’ eLearning module interface.
that this theory-based module be coupled with an interactive, instructor-led component. Four additional sessions based on anatomic regions supplement the eLearning module at our institution, aiding to translate theory into everyday practice. Table 1 Participant Results for Module Completion Over 3 Years
Module completion rate Number of participants Percentage of all staff (%) Time for completion (h:min:s) Median Minimum Maximum Module quiz scores (%) 75 83 92 100
2012
2013
2014
130 84
108 72
106 71
0:42:25 0:05:56 4:08:17
0:32:36 0:02:02 6:50:03
0:20:48 0:02:36 4:36:15
22.3 24.6 19.2 33.8
14.3 20.4 15.3 50.0
16.7 13.5 9.4 60.4
Results from the module participants over the last 3 years indicate that the time required for completion is decreasing, and percentage of staff receiving perfect scores on the evaluation quiz is increasing. As such, future modifications to the module may involve increasing the number of postmodule evaluation MCQs and changing the order in which the questions are asked to decrease memory bias from previous years. Additional evaluation of the MCQs will be performed to ensure the questions asked are effective in fulfilling learning objectives [19]. It can also be observed from the range of time required for module completion that a learner has flexibility with the module, taking as little or as much time required to complete the eLearning objectives. As CBCT implementation spreads from a local scale to a standard of practice in rapidly changing paradigms [7], it is logical to consider the module’s relevance and applicability to centres outside our profession and institution. It has been outlined by Dzau et al that two common obstacles to the implementation of innovation is the initial effective translation of discovery into clinical practice and the expansion beyond
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48
47
niche expert acceptance to broader adoption in the community [20]. As this module was built within an institution that performed extensive research and development on CBCT technology, the validated module may be a good resource of standardized image-guidance knowledge for radiation therapists outside our institution. Future plans include converting the module to an app to increase mobility within our institution and accessibility outside our institution. In an environment of technological change, the roles and responsibilities of the professionals within the radiation medicine team have been impacted [21]. Entry to practice curriculum overview and continuing education is under redevelopment in radiation oncology to solidify a new framework to keep pace with technology and practice nationally and internationally [22, 23]. As image guidance technology impacts all disciplines in radiation medicine, the module’s applicability to radiation oncology and radiation physics is currently being explored. Dissemination of evidence-based information through the module ensures safe use and practice of CBCT that is becoming mainstream practice in the interprofessional model. Validation of the module may allow its dissemination to other centres as a continuing education tool to enhance the practice of CBCT–IGRT. A challenge of the online tool is that it needs to be reviewed yearly to ensure that the information provided is current and relevant. This presents a good opportunity to incorporate feedback from staff by modifying sections of the module that were unclear. It also allows the content development team to assess whether some pertinent sections of image guidance practice were missing. In addition, it should be evaluated whether other resources, such as videos or interactive components, may be more effective in delivering more complex concepts. One of the main issues already identified was the small font size on some of the screens; this was an uncontrolled factor, limited by requirements of the institution’s eLearning team to develop the module on the lowest screen limitations across the department.
Conclusion An electronic online training tool for CBCT refresher training has been developed and implemented at our institution. The module’s format ensures delivery of consistent information as a component of yearly continuing education for radiation therapists. The module’s applicability to radiation therapists at other centres and professional disciplines is under evaluation. References [1] Jaffray, D. A., Siewerdsen, J. H., & Wong, J. W., et al. (2002). Flatpanel cone-beam computed tomography for image-guided radiation therapy. Int J Radiat Oncol Biol Phys 53, 1337–1349. [2] Jaffray, D. A. (2005). Emergent technologies for 3-dimensional imageguided radiation delivery. Semin Radiat Oncol 15, 208–216.
48
[3] White, E., & Kane, G. (2007). Radiation medicine practice in the image-guided radiation therapy era: new roles and new opportunities. Semin Radiat Oncol 17, 298–305. [4] Gillan, C., Li, W., & Harnett, N. (2013). Radiation therapist perspectives on cone-beam computed tomography practices and response to information. J Radiother Pract 12, 237–244. [5] Li, W., Harnett, N., Moseley, D. J., Higgins, J., Chan, K., & Jaffray, D. A. (2010). Investigating user perspective on training and clinical implementation of volumetric imaging. J Med Imaging Radiat Sci 41, 57–65. [6] Liszewski, B., DiProspero, L., Bagley, R., Osmar, K., & D’Alimonte, L. (2012). A preliminary evaluation of a clinical training program for volumetric imaging. J Med Imaging Radiat Sci 43, S31. [7] Jaffray, D. A. (2012). Image-guided radiotherapy: from current concept to future perspectives. Nat Rev Clin Oncol 9, 688–699. [8] Welsh, E., Wanberg, C., Brown, K., & Simmering, M. (2003). E-learning: emerging uses, empirical results and future directions. International Journal of Training and Development 7, 245–258. [9] Reime, M., Harris, A., Aksnees, J., & Mikkelsen, J. (2008). The most successful method in teaching nursing students infection controld e-learning or lecture? Nurse Educ Today 28, 798–806. [10] Ellaway, R., & Masters, K. (2008). Amee guide 32: e-learning in medical education part 1: learning, teaching and assessment. Med Teach 30, 455–473. [11] Masters, K., & Ellaway, R. (2008). E-learning in medical education guide 32 part 2: technology, management and design. Med Teach 30, 474–489. [12] Bloomfied, J., White, A., & Roberts, J. (2008). Using computer assisted learning for clinical skills education in nursing: integrative review. J Adv Nurs 63, 222–235. [13] Huckstadt, A., & Hayes, K. (2005). Evaluation of interactive online courses for advanced practice nurses. J Am Acad Nurse Pract 17, 85–89. [14] Kala, S., Isramalai, S., & Pohthong, A. (2010). Electronic learning and constructivism: a model for nursing education. Nurse Educ Today 30, 61–66. [15] Keefe, G., & Wharrad, H. (2012). Using e-learning to enhance nursing students’ pain management educati. Nurse Educ Today 32, e66–e72. [16] Swanson, B., Zeller, J., & Keithley, J., et al. (2012). Case-based online modules to teach graduate-level nursing students about complementary and alternative medical therapies. J Prof Nurs 28, 125–129. [17] Lehr, A. R., McKinney, M. L., & Gouin, S., et al. (2013). Development and pretesting of an electronic learning module to train health care professionals on the use of the pediatric respiratory assessment measure to assess acute asthma severity. Can Respir J 20, 435–441. [18] Foroudi, F., Wong, J., & Kron, T., et al. (2010). Development and evaluation of a training program for therapeutic radiographers as a basis for online adaptive radiation therapy for bladder carcinoma. Radiography 16, 14–20. [19] Collins, J. (2006). Writing multiple-choice questions for continuing medical education activities and self-assessment modules. Radiographics 26, 543–551. [20] Dzau, V. J., Ackerly, D. C., & Sutton-Wallace, P., et al. (2010). The role of academic health science systems in the transformation of medicine. Lancet 375, 949–953. [21] Gillan, C., Wiljer, D., & Harnett, N., et al. (2010). Changing stress while stressing change: the role of interprofessional education in mediating stress in the introduction of a transformative technology. J Interprof Care 24, 710–721. [22] Giuliani, M. E., Gillan, C., & Milne, R. A., et al. (2014). Determining an imaging literacy curriculum for radiation oncologists: an international delphi study. Int J Radiat Oncol Biol Phys 88, 961–966. [23] Turner, S., Eriksen, J. G., & Trotter, T., et al. (2015). Establishing a global radiation oncology collaboration in education (grace): objectives and priorities. Radiother Oncol 117, 188–192.
W. Li et al./Journal of Medical Imaging and Radiation Sciences 47 (2016) 43-48