- Email: [email protected]
Blending online techniques with traditional face to face teaching methods to deliver final year undergraduate radiology learning content
European Journal of Radiology 78 (2011) 334–341
Contents lists available at ScienceDirect
European Journal of Radiology journal homepage: www.elsevier.com/locate/ejrad
Blending online techniques with traditional face to face teaching methods to deliver final year undergraduate radiology learning content David Howlett a,∗ , Tim Vincent b , Gillian Watson a , Emma Owens a , Richard Webb c , Nicola Gainsborough c , Jil Fairclough b , Nick Taylor d , Ken Miles e , Jon Cohen f , Richard Vincent g a
Department of Radiology, Eastbourne District General Hospital, Kings Drive, Eastbourne, East Sussex BN21 2UD, United Kingdom Department of IT, Brighton and Sussex Medical School (BSMS), United Kingdom c Department of Medicine, Royal Sussex County Hospital, Brighton, United Kingdom d Department of Medical Illustration, Eastbourne District General Hospital, United Kingdom e Department of Imaging, BSMS, United Kingdom f Department of Infectious Diseases, BSMS, United Kingdom g Department of Cardiology, BSMS, United Kingdom b
a r t i c l e
i n f o
Article history: Received 9 June 2009 Accepted 21 July 2009 Keywords: Online education Radiology Undergraduate Blended learning
a b s t r a c t Aim: To review the initial experience of blending a variety of online educational techniques with traditional face to face or contact-based teaching methods to deliver final year undergraduate radiology content at a UK Medical School. Materials and methods: The Brighton and Sussex Medical School opened in 2003 and offers a 5-year undergraduate programme, with the final 5 spent in several regional centres. Year 5 involves several core clinical specialities with onsite radiology teaching provided at regional centres in the form of small-group tutorials, imaging seminars and also a one-day course. An online educational module was introduced in 2007 to facilitate equitable delivery of the year 5 curriculum between the regional centres and to support students on placement. This module had a strong radiological emphasis, with a combination of imaging integrated into clinical cases to reflect everyday practice and also dedicated radiology cases. For the second cohort of year 5 students in 2008 two additional online media-rich initiatives were introduced, to complement the online module, comprising imaging tutorials and an online case discussion room. Results: In the first year for the 2007/2008 cohort, 490 cases were written, edited and delivered via the Medical School managed learning environment as part of the online module. 253 cases contained a form of image media, of which 195 cases had a radiological component with a total of 325 radiology images. Important aspects of radiology practice (e.g. consent, patient safety, contrast toxicity, ionising radiation) were also covered. There were 274,000 student hits on cases the first year, with students completing a mean of 169 cases each. High levels of student satisfaction were recorded in relation to the online module and also additional online radiology teaching initiatives. Conclusion: Online educational techniques can be effectively blended with other forms of teaching to allow successful undergraduate delivery of radiology. Efficient IT links and good image quality are essential ingredients for successful student/clinician engagement. © 2009 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Recent advances in learning technology and computer-assisted learning (CAL) systems have combined with wide availability of the internet to give online learning an increasingly important role in medical education, at both undergraduate [1–4] and postgraduate [5–7] level. Radiology as a speciality is well-suited to online delivery as it is largely image-based, particularly utilising
∗ Corresponding author. Tel.: +44 01323 417400; fax: +44 01323 414933. E-mail address: [email protected] (D. Howlett). 0720-048X/$ – see front matter © 2009 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ejrad.2009.07.028
a case-based format and a move towards web-based teaching is a necessity in the face of increasing workload and time demands faced by radiologists in the UK. A large number of studies have been published describing radiological teaching using a variety of electronic resources, including CD-ROMs, departmental or web-based digital teaching file databases, hand-held computers (PDAs) and in addition dedicated radiology websites [1,8–14]. Studies have also demonstrated that CAL can be associated with improvements in radiology problem-solving at undergraduate level [15,16]. Electronic learning as a concept is associated with consistently higher levels of student satisfaction but it is generally accepted that online learning works best when blended with
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
more traditional learning techniques, rather than trying to replace them [17]. In this paper we describe the use of several innovative online techniques blended with more traditional means of educational delivery. To help deliver the radiological component of a medical school curriculum we also examine student engagement in terms of case utilisation and satisfaction and the ingredients needed to make this form of education a success. 2. Methods Brighton and Sussex Medical School (BSMS) opened in 2003 offering a 5-year undergraduate programme with a vertically integrated course with students exposed to theory and practice together from the outset. The first 4 years are spent mainly at the central teaching institution, Royal Sussex County Hospital in Brighton and in the final year students rotate through several regional centres in addition to their base hospital. The final year curriculum is constructed around speciality placements case-based learning and an online (clinical and professional studies) and casebased module was proposed for year 5 to help overcome the educational and logistical difficulties of equitable curriculum delivery at multiple different centres. The online module was designed to support the core specialities encountered in year 5 and would be broadly mapped to a list of index cases per speciality drawn up by discipline leads to provide a baseline if expected competence. Cases were to be used as a formative learning exercise and scores from cases would not be recorded or used in summative assessment. Speciality areas included: • • • • • • • • • •
Medicine Elderly care Surgery Primary care Mental health Therapeutics Clinical Investigations (including radiology) Obstetrics and gynaecology Paediatrics Professional studies-covering aspects of “professionalism” including medical ethics, patient safety, consent, error, data protection, complaints.
Years 1–4 involve exposure to radiology from the outset, with an emphasis on teaching anatomy via cross-sectional imaging, with dedicated time spent in the radiology department in year 4. For the final year a programme of traditional “face to face” radiological teaching was also arranged and the online module would support this to cover both relevant areas involving radiology in the core year 5 specialities and also a specific set of year 5 radiology learning outcomes and index cases. The structured programme of face to face regional centre radiological teaching was included: • Three group radiology seminars as part of the weekly seminar programme in regional centres. • Regular small-group teaching in the regional centres. • A dedicated one-day course on radiology at which attendance was voluntary. The year 5 radiology curriculum was prepared with particular reference to a recent Royal College of Radiologists document relating to the role of radiology in undergraduate medical education [18] and emphasis was placed on imaging principles and modalities considered to be of most relevance to undergraduates to prepare them for the early postgraduate years, namely:
335
Table 1 List of index cases for year 5, chest radiographic interpretation. An appreciation of normality Normal positioning of lines (especially nasogastric tubes and CVP lines) and complications of insertion Cardiomegaly and cardiac failure COPD Pneumonia and signs of consolidation Patterns of lobar collapse Pleural effusion and pleural mass Pneumothorax, pneumomediastinum and surgical emphysema Lung mass, or masses and cavitation Mediastinal lymphadenopathy Interstitial lung disease Subdiaphragmatic free air or fluid collection Fractures to the thoracic skeleton
• Patient safety, consent and the safe use of ionising radiation (including an awareness of current ionising radiation regulations) particularly in pregnancy and children. • To appreciate the role of imaging in the patient clinical management pathway and the role of the radiologist in the multidisciplinary team. • To understand the indications, contraindications and hazards associated with the key imaging modalities. • To revise relevant radiological anatomy as encountered in clinical cases. • To be able to interpret radiological investigations relevant to year 5 and the foundation years, with emphasis on interpretation of chest, abdominal and musculoskeletal plain film and to have an understanding of more complex investigations, mainly cranial/body computed tomography, likely to be encountered in the emergency scenario. A list of radiological index cases for chest radiographic interpretation is included in Table 1, as an example. Work began on the online module in 2006 to be ready for the first intake of year 5 students in August 2007 and radiology was covered in two ways—predominantly by integrating radiological images, or learning points, into clinical cases to emphasise the role of radiology in patient management, but also with a dedicated radiology section in the Clinical Investigations component of the module. An editorial board was established including a radiology lead and editor, tasked with writing, commissioning and editing cases to ensure appropriate level of difficulty and relevance to the overall year 5 curriculum. The project was delivered by Studentcentral, the pre-existing Medical School virtual learning environment (VLE) based on Blackboard® software (Blackboard Academic Suite TM version 6, 1997–2004 Blackboard Inc., USA). This web-based system provides a test building and management tool supporting a variety of formats and multimedia. As part of the year 5 development a generalized computer upgrade was also undertaken in all regional centres to provide adequate computer access for the students. Initially a draft case was prepared in 2006 and loaded onto the system—this case involved several types of image media including radiological images and clinical photographs, to ensure that the style of case presentation was acceptable and that images were accessible and of sufficient resolution to allow accurate diagnosis. The case was written in what would become a standard format, based on an initial clinical scenario with a case history followed by a series of questions, most commonly single best answer or multiple answers. As students progressed through the case they were not allowed to return to previous answers but additional information was included in subsequent question stems to allow them to continue even if unsure of the diagnosis initially. Once a case was completed the students were provided with comprehensive feed-
336
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
Fig. 1. Screenshot image from a dedicated radiology cases as seen by the student.
back provided from the case author and cases also had selected online reading links attached to provide additional learning opportunities. Throughout cases students could access drop down menus to review the case history, lists of normal laboratory indices or to review images. A dedicated word template was prepared and sent to case writers in spring 2006 to commence case collection and cases and image files were submitted via email or CD. The initial draft case gave important information as to the types of image media that the VLE would support. Most radiology images (and clinical images) were edited in Adobe® Photoshop, adjusting contrast and brightness levels, by a Medical Photographer. Annotations and arrows were then added before resizing images to a maximum 900 pixel linear dimension—this allowed image quality to be maintained and also fast transfer across the web. Hard copy plain films were photographed using a digital camera and images were uploaded for editing with Adobe® Photoshop. Other radiology images were sourced from local PACS systems and downloaded using web browser software giving compressed (jpeg) images, maximum 1024 pixel × 1024 pixel dimension. All images supplied by contributors were assessed for size and quality and anonymised and unsuitable images were replaced. An example of a screen shot web page as seen by the students is included in Fig. 1. Consultant radiologists and radiology trainees wrote cases containing integrated radiology and dedicated radiology cases and also a large number of cases not directly involving imaging. Two further online initiatives were introduced into year 5 to enhance the online module for the second cohort of year 5 students commencing in September 2008. 2.1. Online imaging tutorials Three self-contained and web-based tutorials were prepared with links to the main online module: • Chest Radiograph Interpretation—this tutorial was designed in MS Word and converted to a web-based package and used a more conventional “textbook” approach to the chest radiograph. Images were prepared as previously, as MS Word files and
uploaded onto the VLE and the web pages formatted to allow images to open into a new window to allow the user to expand the image as necessary. Legends were provided as a separate pop-up window to allow students to attempt their own image interpretation before accessing author guidance. • Musculoskeletal Plain Film Interpretation—prepared in a similar manner to the chest tutorial but utilising multiple answers with feedback rather than textbook style and concentrating on trauma. • Abdominal Radiograph Interpretation—this was delivered using a video-based method. The audio of the tutorial (based on a live seminar session) was recorded in a studio and then a corresponding series of images were complied and matched to the audio in a non-linear editing system (Adobe Premier Pro® ). The tutorial was divided into 9 sections and delivered to the students via an existing video streaming service at the University.
2.2. Online case discussion—the “Case Discussion Room” These were introduced to provide the students with the opportunity to discuss case scenarios with a moderator—the forum needed to be easy to use, available to all students, to support images and video and to be secure from non-module user access. The solution was provided by an open-source social networking plugin to the VLE powered by Elgg (Elgg.org® ) where a specific case discussion room community was created. A clinical case scenario was entered by the module team as a new block entry (including embedded radiological images and video files to simulate scrolling through PACS) and the students were invited to add their comments using open-ended questions. Once sufficient interactive discussion had occurred between moderator and students, an answer page with text, online references and annotated images as appropriate was posted. An example screen shot image in included in Fig. 2. This forum was used to support all curriculum specialities, not only Clinical Investigations. Students from the second cohort attending the radiology course in December 2008 were asked to complete a paper-based questionnaire assessing their satisfaction with the varying modes of radiology curriculum delivery available in year 5 and how these had affected their understanding of radiology and its role in patient
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
337
Fig. 2. Screenshot image as seen by the students of an interactive case, with video sequence opened by clicking on the image.
management (see Table 2). This survey supplemented 3 online student surveys undertaken during the 2007/2008 academic year relating to the main online module. 3. Results One hundred and one students commenced year 5 in August 2007 with a second cohort of 110 students in August 2008. By the end of the 2007/2008 academic year in May 2008 the main online module contained: • 490 cases written, edited and stored on the database with 363 of the cases released to the students by academic year end. • 2462 questions within cases with an average of 5 questions per case and an average of 5 stems per question. • 195 cases contained radiology image media, mainly plain films or CT/MR. There were a total of 405 images in cases of which 325 were radiological. Fifty cases were dedicated imaging cases, with 145 clinical cases containing integrated radiological images or learning points. Outside the Clinical Investigations section, the majority of radiology featured in medicine/elderly care, surgery or paediatrics. Important aspects of radiology not involving an image (e.g. consent for interventional procedures, treatment of contrast reaction, patient safety in the department and hazards of ionising radiation) were also covered in the Professional Studies section.
• The total number of student hits on the content areas (including drop down menus of normal values for laboratory investigations, images, and learning resources) was 274,000 over the academic year. • An estimated 2450 h of time were involved in setting-up the online module, with an average of 5 h per case (medical case writing 2 h, photographer ½ h, librarian ½ h, IT 1½ h, medical editing ½ h) at an estimated cost of 135 euros per case. The online student surveys undertaken in year 1 had revealed high levels of student satisfaction with the online module with positive freetext feedback relating to: • The ability to choose when and where to work and to control the pace of their interaction. • The image quality of radiographs and the radiology teaching within the module. • Cases were felt to be a good preparation for clinical practice and an excellent revision tool for examinations. • Case writer feedback, learning resources and the case-based format.
Student utilisation of the online module was examined throughout the first year and student access to the online cases was recorded by the system in terms of “hits” (this gave a record of each time a user entered the initial case introduction). In Fig. 3, student hits on the online cases are documented throughout the year, with huge surges in activity identified before finals parts 1 (January 2008) and 2 (April 2008). • The mean number of hits on cases per student was 1766 (range 111–7700) with students completing a mean of 169 cases (range 18–346).
Fig. 3. Graph showing student activity throughout academic year 2007/2008 in terms of “hits”. Note huge peaks in activity (>6000 hits/day) before finals parts 1 and 2.
338
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
• The ability to cover large areas of knowledge in a short time frame and to repeat questions following revision. In December 2008, at the radiology course, the second cohort of year 5 students were asked to complete a further survey to focus specifically on radiology and to include the online innovations introduced in this second year and the results of this are included in Table 2. The results of this survey will be discussed further in the discussion but a number of points can be highlighted: • The majority of students found both online and traditional techniques good or excellent for their radiology learning although 25% preferred online alone.
• A number of students have not made use of the online tutorials and from the feedback section in the questionnaire were apparently unaware of them. Most students who have accessed the online tutorials have found them useful and likewise a majority of students also like the case discussion room, with between 6 and 8 students posting responses—it is likely that many more observe the teaching interaction, but do not post. Again a number of students were apparently unaware of the existence of this facility. • Overall the radiology teaching techniques employed in year 5 appear to be delivering successfully from the student point of view in terms of content, relevance, image quality and level of difficulty.
Table 2 Results of radiology questionnaire given to 2nd cohort year 5 students 4 months into academic year beginning 2008. Number of respondents = 36, attendees 70.
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
339
Table 2 (Continued )
Student feedback was generally highly positive in the paper questionnaire freetext feedback section and mirrored to a large extent the positive comments from the earlier online surveys. Students specifically liked the use of video sequences in the case discussion forum simulating PACS and used to demonstrate pathologies on CT such as aortic dissection, aortic aneurysm leak and pulmonary embolism. Negative feedback was limited (5 comments) but it was clear that despite apparently positive responses in the survey that discrepancy exists in the perceived quality of face to face radiology delivery in the regional centres.
4. Discussion Online learning offers significant potential advantages over more traditional techniques as it is able to disseminate large amounts of information to increased numbers of students and at a potentially reduced cost while providing a valuable educational resource [7]. These principles are applicable to radiology in particular when coupled with the recent technological advances in radiology departments and more widespread availability of PACS systems [19]. Online learning has become the cultural norm for current undergraduate and postgraduate students and they will
340
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
readily engage with this form of teaching. A majority of online radiological education is currently aimed at postgraduate level and is often relatively specialised—the recent development of the radiology integrated training initiative (R-ITI) by the Royal College of Radiologists [20] represents a significant step forward in UK postgraduate radiology training. However, despite these online initiatives digital technology remains frequently underutilised for radiological teaching, particularly at undergraduate level [11]. Radiology, as a speciality, is well-suited to the electronic mode of delivery. Digital images can be collated as “reusable learning objects” which can then be compiled into a digital learning programme [21]. Digital image files can be easily accessed, updated and also reviewed and edited, the process of periodic re-approval. Building a collection of learning objects is part of the initial phase of creation of an online module, namely development of content [22]. The initial preparation and subsequent delivery of the three online radiological initiatives for the BSMS year 5 curriculum (as part of the main online module, the case discussion room and the imaging tutorials) has already been described. This required forward planning with careful mapping of radiology topics and cases both to the year 5 general speciality index cases, but also to specific radiology outcomes and index cases for that undergraduate year. The radiology content was developed according to models proposed by the Royal College of Radiologists and also other institutions [18,23]. At the outset it was decided that the majority of imaging should be integrated into clinical cases to mirror actual clinical practice and to allow students to appreciate the clinical role of radiology. Radiologists wrote both purely radiological cases and also the clinical cases including radiology and many clinical cases without imaging content and all the cases were scrutinised by the editorial team for relevance to the undergraduate curriculum and also to confirm appropriate level of difficulty. Radiology trainees in particular engaged in this process, seeing case writing as a valuable learning opportunity and also to enhance their portfolio of teaching and educational skills. Standard medical textbooks, medical school learning outcomes and relevant national publications [18,24] were used to help guide case writers. The second component of online module development revolves around management of content and involves storing, indexing and filing of material. For this dedicated IT support is essential and we were able to utilise the pre-existing VLE at the Medical School which was able to support all necessary forms of image media planned within the module. The third and final phase of development involves delivery of the written content. We chose to use a number of different styles of online radiological teaching, to support the varying learning requirements of the students and to supplement the more traditional learning strategies running concurrently. It was not our intention to try and replace traditional face to face teaching methods, but to enhance them and it is clear from both the literature and from our student surveys that most students prefer to learn using a blended combination of written and online delivery. Our survey did show however that a significant proportion of students (25%) preferred online teaching alone as a means of self-directed learning. An asynchronous approach was chosen for the online cases and delivery and receipt of information were not simultaneous [22]. This was felt necessary to reduce the demands on teachers due to the large amounts of teaching material within the module. The use of online case discussion room in undergraduate radiology teaching is original and not well described in the literature. Studies on radiology e-learning have incorporated this form of teaching and have confirmed its potential [25,26]. We used this forum to simulate scrolling through PACS using CT video files and this was well received by the students. The third means of online delivery comprised the three imaging tutorials—two delivery formats were used for these (see earlier) namely web page and video sequences. The
piloting of a video-based imaging tutorial worked well on the system and again the use of this format for radiological teaching is again not well described at undergraduate level and there is scope for wider application. How is the success of online teaching tool evaluated? This is complex and often hard to assess. There are three main areas to be addressed: 4.1. Do users like the product? There have been high levels of engagement by both students and clinician teachers with the online radiology components of the year 5 curriculum. The student surveys have confirmed the known benefits of using online techniques, namely the ability to work at their own pace and at a time of their choosing and to be able to cover large areas of knowledge in a short space of time, particularly before examinations. The data in Table 2 indicate high levels of student satisfaction with both the online radiology teaching and also traditional teaching methods. For the majority of students their year 5 radiology teaching: • Had helped in their understanding of the role of imaging in patient care. • Had helped in improving their confidence in interpretation of imaging studies. • Had been relevant and of appropriate level of difficulty to year 5. For clinicians and students to engage with radiology online learning there are important factors to be considered and addressed at the outset: • Resistance to change and poor IT skills—less of a problem amongst current generations of IT literate students and (most) clinicians. • Prohibitive set-up costs (see (b) below). • Poor image quality. • Poor IT support and links making system inaccessible. • Lack of set-up time, both for writing/editing cases and IT. Time constraints are a major issue for radiologists, with increasing clinical demands. • We have also experienced reduced utilisation by students in some areas of the online package due to apparent lack of awareness of their existence—advertising the product and encouraging participation is part of the packaging. We also engaged students in case utilisation by making them aware that cases would reflect the style and level of difficulty of their final examinations as students will not readily utilise learning resources which they do not think will contribute to their success in examination. 4.2. What is the financial cost? Studies have shown that e-learning can result in significant cost savings, up to 50%, when compared to onsite teaching methods [27]. These financial savings will only accrue following initial setup and these start-up costs can be significant in terms of staffing and equipment [28]. We did estimate a cost per case of 135 euros, based on 5 h per case of clinician/IT/medical photographer/librarian time. Maintenance costs following initial set-up will reduce significantly and online packages are appealing to managers due to reduced travelling, training and staffing costs. However this cost does not include the cost of the University VLE online resources purchased by the Medical School, an important component of the case feedback. The University VLE is a commercial product and the high purchase and support overheads of these types of systems have led some educational institutions to develop open-source, enterprise-wide, VLEs [1]. These systems do not have
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
licensing costs and are customisable, allow ongoing development and also implementation of improvements as suggested by users. 4.3. Is the product useful educationally? Outcome evaluations of e-learning packages are complex and often difficult to undertake. Future research will concentrate on outcome evaluation which may relate to: user satisfaction (see earlier), evidence of improved learning, or changes in learner/organisational behaviour or patient outcome [22]. Digital images are objective and can be used for examination and assessment and have been used to demonstrate improvement in undergraduate radiological interpretation although this area has not been evaluated in our paper. Student learning is multifactorial however and online learning reflects only one style of learning choice, which will complement other techniques, but this can make it difficult to evaluate the utility of online learning in isolation. 5. Conclusion Increased utilisation of e-learning in undergraduate and postgraduate radiological teaching is a necessity in the face of the increasing clinical/managerial workloads of radiologists with constant pressure on available teaching time. Online techniques can be effectively blended with more “traditional” face to face forms of radiological content of a final year undergraduate teaching programme. Students and radiologists will both engage readily with online teaching but successful delivery is highly dependent on effective IT and medical photography support and can have high set-up costs. References [1] Sparacia G, Cannizzaro F, D’Alessandro D, D’Alessandro M, Caruso G, Lagalla R. Initial experiences in radiology e-learning. Radiographics 2007;27:573– 81. [2] Harden RM, Hart IR. An international virtual medical school (IVIMEDS): the future for medical education? Med Teach 2002;24(3):261–7. [3] Petrusa ER, Barry Issenberg S, Mayer JW, Felner JM, Brown DD, Waugh RA, Kondos GT, Gessner IH, McGaghie WC. Implementation of a four year multimedia computer curriculum in cardiology at six medical schools. Acad Med 1999;74(2):123–9. [4] Lau F, Bates J. A review of e-learning practices for undergraduate medical education. J Med Syst 2004;28(1):71–87. [5] Lacey Bryant S, Ringrose T. Evaluating the Doctors.net.uk model of electronic continuing medical education. Work Based Learn Primary Care 2005;3(2):129–42.
341
[6] Wilson AS, Goodall JE, Ambrosini G, Carruthers DM, Chan H, Ong SG, Gordon C, Young SP. Development of an interactive learning tool for teaching rheumatology—a simulated clinical case studies programme. Rheumatology 2006;45:1158–61. [7] Choules AP. The use of e-learning in medical education: a review of the current situation. Postgrad Med J 2007;83:212–6. [8] Yang G-L, Lim T. Singapore national medical image resource centre (SN. MIRC): a world wide web resource for radiology education. Ann Acad Med 2006;35:558–63. [9] Grunewald M, Heckemann RA, Gebhard H, Lell M, Bautz WA. Compare radiology; creating an interactive web-based training programme for radiology with multimedia authoring software. Acad Radiol 2003;10:543–53. [10] Shaffer K. Radiology education in the digital era. Radiology 2005;235:359–60. [11] Durfree Fee SM, Jain S, Shaffer K. Incorporating electronic media into medical student education: a survey of AMSER members on computer and web-base use in radiology courses. Acad Radiol 2003:524–30. [12] Scarsbrook AF, Foley PT, Perriss RW, et al. Radiological digital teaching fite development: an overview. Clin Radiol 2005;60:831–7. [13] Flanders AE, Wiggins RH, Gozum ME. Handheld computers in radiology. Radiographics 2003;33:1035–47. [14] Scarsbrook AF, Graham RNJ, Perriss RW. Radiology education: a glimpse into the future. Clin Radiol 2006;61:640–8. [15] Makeck M, Fischer MR, Kammer B, et al. Do computers teach better? A media comparison study for case-based teaching in radiology. Radiographics 2001;21:1025–32. [16] Davison BD, Tello R, Blickman JG. World wide web program for optimising and assessing medical student performance during the radiology clerkship. Acad Radiol 2000;7:260–3. [17] Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med 2002;77:86–93. [18] Royal College of Radiologists, Radiology For Medical Students, Papers 1–4, 2008 [at: www.rcr.ac.uk]. [19] Miles K. Diagnostic imaging in undergraduate medical education: an expanding role. Clin Radiol 2005;60:742–5. [20] Radiology Integrated Training Initiative—R-ITI; http://riti.org.uk. [21] Greenhalgh T. Computer assisted learning in undergraduate education. BMJ 2001;332:40–4. [22] Ruiz JG, Mintzner MJ, Leipzig RM. The impact of e-learning in medical evaluation. Acad Med 2006;81(3):207–12. [23] Lee JL, Aldrich JE, Eftekhari A, Nicolaou S, Muller NL. Implementation of a new undergraduate radiology curriculum. experience at the University of British Columbia. Can Assoc Radiol 2007;58(5):272–8. [24] General Medical Council. Tomorrows doctors. London: General Medical Council; 1993. [25] Gotthardt M, Siegert MJ, Schlieck A, et al. How to successfully implement elearning for both students and teachers. Acad Radiol 2006;13:379–90. [26] Filippucci E, Meenagh G, Ciapetti A, et al. E-learning in ultrasonography: a webbased approach. Ann Rheum Dis 2007;66:962–5. [27] Gibbons A, Fairweather P. Computer-based instruction. In: Tobias S, Fletcher J, editors. Training and retraining. A handbook for business, industry, government and the military. New York: Mac Millan Reference USA; 2000. p. 410–42. [28] Fall LH, Berman NB, Smith S, White CB, Woodhead JC, Olson AL. Multiinstitutional development and utilisation of a computer-assisted learning programme for the paediatrics clerkship: the CLIPP project. Acad Med 2005;80(9):847–55.
Contents lists available at ScienceDirect
European Journal of Radiology journal homepage: www.elsevier.com/locate/ejrad
Blending online techniques with traditional face to face teaching methods to deliver final year undergraduate radiology learning content David Howlett a,∗ , Tim Vincent b , Gillian Watson a , Emma Owens a , Richard Webb c , Nicola Gainsborough c , Jil Fairclough b , Nick Taylor d , Ken Miles e , Jon Cohen f , Richard Vincent g a
Department of Radiology, Eastbourne District General Hospital, Kings Drive, Eastbourne, East Sussex BN21 2UD, United Kingdom Department of IT, Brighton and Sussex Medical School (BSMS), United Kingdom c Department of Medicine, Royal Sussex County Hospital, Brighton, United Kingdom d Department of Medical Illustration, Eastbourne District General Hospital, United Kingdom e Department of Imaging, BSMS, United Kingdom f Department of Infectious Diseases, BSMS, United Kingdom g Department of Cardiology, BSMS, United Kingdom b
a r t i c l e
i n f o
Article history: Received 9 June 2009 Accepted 21 July 2009 Keywords: Online education Radiology Undergraduate Blended learning
a b s t r a c t Aim: To review the initial experience of blending a variety of online educational techniques with traditional face to face or contact-based teaching methods to deliver final year undergraduate radiology content at a UK Medical School. Materials and methods: The Brighton and Sussex Medical School opened in 2003 and offers a 5-year undergraduate programme, with the final 5 spent in several regional centres. Year 5 involves several core clinical specialities with onsite radiology teaching provided at regional centres in the form of small-group tutorials, imaging seminars and also a one-day course. An online educational module was introduced in 2007 to facilitate equitable delivery of the year 5 curriculum between the regional centres and to support students on placement. This module had a strong radiological emphasis, with a combination of imaging integrated into clinical cases to reflect everyday practice and also dedicated radiology cases. For the second cohort of year 5 students in 2008 two additional online media-rich initiatives were introduced, to complement the online module, comprising imaging tutorials and an online case discussion room. Results: In the first year for the 2007/2008 cohort, 490 cases were written, edited and delivered via the Medical School managed learning environment as part of the online module. 253 cases contained a form of image media, of which 195 cases had a radiological component with a total of 325 radiology images. Important aspects of radiology practice (e.g. consent, patient safety, contrast toxicity, ionising radiation) were also covered. There were 274,000 student hits on cases the first year, with students completing a mean of 169 cases each. High levels of student satisfaction were recorded in relation to the online module and also additional online radiology teaching initiatives. Conclusion: Online educational techniques can be effectively blended with other forms of teaching to allow successful undergraduate delivery of radiology. Efficient IT links and good image quality are essential ingredients for successful student/clinician engagement. © 2009 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Recent advances in learning technology and computer-assisted learning (CAL) systems have combined with wide availability of the internet to give online learning an increasingly important role in medical education, at both undergraduate [1–4] and postgraduate [5–7] level. Radiology as a speciality is well-suited to online delivery as it is largely image-based, particularly utilising
∗ Corresponding author. Tel.: +44 01323 417400; fax: +44 01323 414933. E-mail address: [email protected] (D. Howlett). 0720-048X/$ – see front matter © 2009 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ejrad.2009.07.028
a case-based format and a move towards web-based teaching is a necessity in the face of increasing workload and time demands faced by radiologists in the UK. A large number of studies have been published describing radiological teaching using a variety of electronic resources, including CD-ROMs, departmental or web-based digital teaching file databases, hand-held computers (PDAs) and in addition dedicated radiology websites [1,8–14]. Studies have also demonstrated that CAL can be associated with improvements in radiology problem-solving at undergraduate level [15,16]. Electronic learning as a concept is associated with consistently higher levels of student satisfaction but it is generally accepted that online learning works best when blended with
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
more traditional learning techniques, rather than trying to replace them [17]. In this paper we describe the use of several innovative online techniques blended with more traditional means of educational delivery. To help deliver the radiological component of a medical school curriculum we also examine student engagement in terms of case utilisation and satisfaction and the ingredients needed to make this form of education a success. 2. Methods Brighton and Sussex Medical School (BSMS) opened in 2003 offering a 5-year undergraduate programme with a vertically integrated course with students exposed to theory and practice together from the outset. The first 4 years are spent mainly at the central teaching institution, Royal Sussex County Hospital in Brighton and in the final year students rotate through several regional centres in addition to their base hospital. The final year curriculum is constructed around speciality placements case-based learning and an online (clinical and professional studies) and casebased module was proposed for year 5 to help overcome the educational and logistical difficulties of equitable curriculum delivery at multiple different centres. The online module was designed to support the core specialities encountered in year 5 and would be broadly mapped to a list of index cases per speciality drawn up by discipline leads to provide a baseline if expected competence. Cases were to be used as a formative learning exercise and scores from cases would not be recorded or used in summative assessment. Speciality areas included: • • • • • • • • • •
Medicine Elderly care Surgery Primary care Mental health Therapeutics Clinical Investigations (including radiology) Obstetrics and gynaecology Paediatrics Professional studies-covering aspects of “professionalism” including medical ethics, patient safety, consent, error, data protection, complaints.
Years 1–4 involve exposure to radiology from the outset, with an emphasis on teaching anatomy via cross-sectional imaging, with dedicated time spent in the radiology department in year 4. For the final year a programme of traditional “face to face” radiological teaching was also arranged and the online module would support this to cover both relevant areas involving radiology in the core year 5 specialities and also a specific set of year 5 radiology learning outcomes and index cases. The structured programme of face to face regional centre radiological teaching was included: • Three group radiology seminars as part of the weekly seminar programme in regional centres. • Regular small-group teaching in the regional centres. • A dedicated one-day course on radiology at which attendance was voluntary. The year 5 radiology curriculum was prepared with particular reference to a recent Royal College of Radiologists document relating to the role of radiology in undergraduate medical education [18] and emphasis was placed on imaging principles and modalities considered to be of most relevance to undergraduates to prepare them for the early postgraduate years, namely:
335
Table 1 List of index cases for year 5, chest radiographic interpretation. An appreciation of normality Normal positioning of lines (especially nasogastric tubes and CVP lines) and complications of insertion Cardiomegaly and cardiac failure COPD Pneumonia and signs of consolidation Patterns of lobar collapse Pleural effusion and pleural mass Pneumothorax, pneumomediastinum and surgical emphysema Lung mass, or masses and cavitation Mediastinal lymphadenopathy Interstitial lung disease Subdiaphragmatic free air or fluid collection Fractures to the thoracic skeleton
• Patient safety, consent and the safe use of ionising radiation (including an awareness of current ionising radiation regulations) particularly in pregnancy and children. • To appreciate the role of imaging in the patient clinical management pathway and the role of the radiologist in the multidisciplinary team. • To understand the indications, contraindications and hazards associated with the key imaging modalities. • To revise relevant radiological anatomy as encountered in clinical cases. • To be able to interpret radiological investigations relevant to year 5 and the foundation years, with emphasis on interpretation of chest, abdominal and musculoskeletal plain film and to have an understanding of more complex investigations, mainly cranial/body computed tomography, likely to be encountered in the emergency scenario. A list of radiological index cases for chest radiographic interpretation is included in Table 1, as an example. Work began on the online module in 2006 to be ready for the first intake of year 5 students in August 2007 and radiology was covered in two ways—predominantly by integrating radiological images, or learning points, into clinical cases to emphasise the role of radiology in patient management, but also with a dedicated radiology section in the Clinical Investigations component of the module. An editorial board was established including a radiology lead and editor, tasked with writing, commissioning and editing cases to ensure appropriate level of difficulty and relevance to the overall year 5 curriculum. The project was delivered by Studentcentral, the pre-existing Medical School virtual learning environment (VLE) based on Blackboard® software (Blackboard Academic Suite TM version 6, 1997–2004 Blackboard Inc., USA). This web-based system provides a test building and management tool supporting a variety of formats and multimedia. As part of the year 5 development a generalized computer upgrade was also undertaken in all regional centres to provide adequate computer access for the students. Initially a draft case was prepared in 2006 and loaded onto the system—this case involved several types of image media including radiological images and clinical photographs, to ensure that the style of case presentation was acceptable and that images were accessible and of sufficient resolution to allow accurate diagnosis. The case was written in what would become a standard format, based on an initial clinical scenario with a case history followed by a series of questions, most commonly single best answer or multiple answers. As students progressed through the case they were not allowed to return to previous answers but additional information was included in subsequent question stems to allow them to continue even if unsure of the diagnosis initially. Once a case was completed the students were provided with comprehensive feed-
336
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
Fig. 1. Screenshot image from a dedicated radiology cases as seen by the student.
back provided from the case author and cases also had selected online reading links attached to provide additional learning opportunities. Throughout cases students could access drop down menus to review the case history, lists of normal laboratory indices or to review images. A dedicated word template was prepared and sent to case writers in spring 2006 to commence case collection and cases and image files were submitted via email or CD. The initial draft case gave important information as to the types of image media that the VLE would support. Most radiology images (and clinical images) were edited in Adobe® Photoshop, adjusting contrast and brightness levels, by a Medical Photographer. Annotations and arrows were then added before resizing images to a maximum 900 pixel linear dimension—this allowed image quality to be maintained and also fast transfer across the web. Hard copy plain films were photographed using a digital camera and images were uploaded for editing with Adobe® Photoshop. Other radiology images were sourced from local PACS systems and downloaded using web browser software giving compressed (jpeg) images, maximum 1024 pixel × 1024 pixel dimension. All images supplied by contributors were assessed for size and quality and anonymised and unsuitable images were replaced. An example of a screen shot web page as seen by the students is included in Fig. 1. Consultant radiologists and radiology trainees wrote cases containing integrated radiology and dedicated radiology cases and also a large number of cases not directly involving imaging. Two further online initiatives were introduced into year 5 to enhance the online module for the second cohort of year 5 students commencing in September 2008. 2.1. Online imaging tutorials Three self-contained and web-based tutorials were prepared with links to the main online module: • Chest Radiograph Interpretation—this tutorial was designed in MS Word and converted to a web-based package and used a more conventional “textbook” approach to the chest radiograph. Images were prepared as previously, as MS Word files and
uploaded onto the VLE and the web pages formatted to allow images to open into a new window to allow the user to expand the image as necessary. Legends were provided as a separate pop-up window to allow students to attempt their own image interpretation before accessing author guidance. • Musculoskeletal Plain Film Interpretation—prepared in a similar manner to the chest tutorial but utilising multiple answers with feedback rather than textbook style and concentrating on trauma. • Abdominal Radiograph Interpretation—this was delivered using a video-based method. The audio of the tutorial (based on a live seminar session) was recorded in a studio and then a corresponding series of images were complied and matched to the audio in a non-linear editing system (Adobe Premier Pro® ). The tutorial was divided into 9 sections and delivered to the students via an existing video streaming service at the University.
2.2. Online case discussion—the “Case Discussion Room” These were introduced to provide the students with the opportunity to discuss case scenarios with a moderator—the forum needed to be easy to use, available to all students, to support images and video and to be secure from non-module user access. The solution was provided by an open-source social networking plugin to the VLE powered by Elgg (Elgg.org® ) where a specific case discussion room community was created. A clinical case scenario was entered by the module team as a new block entry (including embedded radiological images and video files to simulate scrolling through PACS) and the students were invited to add their comments using open-ended questions. Once sufficient interactive discussion had occurred between moderator and students, an answer page with text, online references and annotated images as appropriate was posted. An example screen shot image in included in Fig. 2. This forum was used to support all curriculum specialities, not only Clinical Investigations. Students from the second cohort attending the radiology course in December 2008 were asked to complete a paper-based questionnaire assessing their satisfaction with the varying modes of radiology curriculum delivery available in year 5 and how these had affected their understanding of radiology and its role in patient
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
337
Fig. 2. Screenshot image as seen by the students of an interactive case, with video sequence opened by clicking on the image.
management (see Table 2). This survey supplemented 3 online student surveys undertaken during the 2007/2008 academic year relating to the main online module. 3. Results One hundred and one students commenced year 5 in August 2007 with a second cohort of 110 students in August 2008. By the end of the 2007/2008 academic year in May 2008 the main online module contained: • 490 cases written, edited and stored on the database with 363 of the cases released to the students by academic year end. • 2462 questions within cases with an average of 5 questions per case and an average of 5 stems per question. • 195 cases contained radiology image media, mainly plain films or CT/MR. There were a total of 405 images in cases of which 325 were radiological. Fifty cases were dedicated imaging cases, with 145 clinical cases containing integrated radiological images or learning points. Outside the Clinical Investigations section, the majority of radiology featured in medicine/elderly care, surgery or paediatrics. Important aspects of radiology not involving an image (e.g. consent for interventional procedures, treatment of contrast reaction, patient safety in the department and hazards of ionising radiation) were also covered in the Professional Studies section.
• The total number of student hits on the content areas (including drop down menus of normal values for laboratory investigations, images, and learning resources) was 274,000 over the academic year. • An estimated 2450 h of time were involved in setting-up the online module, with an average of 5 h per case (medical case writing 2 h, photographer ½ h, librarian ½ h, IT 1½ h, medical editing ½ h) at an estimated cost of 135 euros per case. The online student surveys undertaken in year 1 had revealed high levels of student satisfaction with the online module with positive freetext feedback relating to: • The ability to choose when and where to work and to control the pace of their interaction. • The image quality of radiographs and the radiology teaching within the module. • Cases were felt to be a good preparation for clinical practice and an excellent revision tool for examinations. • Case writer feedback, learning resources and the case-based format.
Student utilisation of the online module was examined throughout the first year and student access to the online cases was recorded by the system in terms of “hits” (this gave a record of each time a user entered the initial case introduction). In Fig. 3, student hits on the online cases are documented throughout the year, with huge surges in activity identified before finals parts 1 (January 2008) and 2 (April 2008). • The mean number of hits on cases per student was 1766 (range 111–7700) with students completing a mean of 169 cases (range 18–346).
Fig. 3. Graph showing student activity throughout academic year 2007/2008 in terms of “hits”. Note huge peaks in activity (>6000 hits/day) before finals parts 1 and 2.
338
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
• The ability to cover large areas of knowledge in a short time frame and to repeat questions following revision. In December 2008, at the radiology course, the second cohort of year 5 students were asked to complete a further survey to focus specifically on radiology and to include the online innovations introduced in this second year and the results of this are included in Table 2. The results of this survey will be discussed further in the discussion but a number of points can be highlighted: • The majority of students found both online and traditional techniques good or excellent for their radiology learning although 25% preferred online alone.
• A number of students have not made use of the online tutorials and from the feedback section in the questionnaire were apparently unaware of them. Most students who have accessed the online tutorials have found them useful and likewise a majority of students also like the case discussion room, with between 6 and 8 students posting responses—it is likely that many more observe the teaching interaction, but do not post. Again a number of students were apparently unaware of the existence of this facility. • Overall the radiology teaching techniques employed in year 5 appear to be delivering successfully from the student point of view in terms of content, relevance, image quality and level of difficulty.
Table 2 Results of radiology questionnaire given to 2nd cohort year 5 students 4 months into academic year beginning 2008. Number of respondents = 36, attendees 70.
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
339
Table 2 (Continued )
Student feedback was generally highly positive in the paper questionnaire freetext feedback section and mirrored to a large extent the positive comments from the earlier online surveys. Students specifically liked the use of video sequences in the case discussion forum simulating PACS and used to demonstrate pathologies on CT such as aortic dissection, aortic aneurysm leak and pulmonary embolism. Negative feedback was limited (5 comments) but it was clear that despite apparently positive responses in the survey that discrepancy exists in the perceived quality of face to face radiology delivery in the regional centres.
4. Discussion Online learning offers significant potential advantages over more traditional techniques as it is able to disseminate large amounts of information to increased numbers of students and at a potentially reduced cost while providing a valuable educational resource [7]. These principles are applicable to radiology in particular when coupled with the recent technological advances in radiology departments and more widespread availability of PACS systems [19]. Online learning has become the cultural norm for current undergraduate and postgraduate students and they will
340
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
readily engage with this form of teaching. A majority of online radiological education is currently aimed at postgraduate level and is often relatively specialised—the recent development of the radiology integrated training initiative (R-ITI) by the Royal College of Radiologists [20] represents a significant step forward in UK postgraduate radiology training. However, despite these online initiatives digital technology remains frequently underutilised for radiological teaching, particularly at undergraduate level [11]. Radiology, as a speciality, is well-suited to the electronic mode of delivery. Digital images can be collated as “reusable learning objects” which can then be compiled into a digital learning programme [21]. Digital image files can be easily accessed, updated and also reviewed and edited, the process of periodic re-approval. Building a collection of learning objects is part of the initial phase of creation of an online module, namely development of content [22]. The initial preparation and subsequent delivery of the three online radiological initiatives for the BSMS year 5 curriculum (as part of the main online module, the case discussion room and the imaging tutorials) has already been described. This required forward planning with careful mapping of radiology topics and cases both to the year 5 general speciality index cases, but also to specific radiology outcomes and index cases for that undergraduate year. The radiology content was developed according to models proposed by the Royal College of Radiologists and also other institutions [18,23]. At the outset it was decided that the majority of imaging should be integrated into clinical cases to mirror actual clinical practice and to allow students to appreciate the clinical role of radiology. Radiologists wrote both purely radiological cases and also the clinical cases including radiology and many clinical cases without imaging content and all the cases were scrutinised by the editorial team for relevance to the undergraduate curriculum and also to confirm appropriate level of difficulty. Radiology trainees in particular engaged in this process, seeing case writing as a valuable learning opportunity and also to enhance their portfolio of teaching and educational skills. Standard medical textbooks, medical school learning outcomes and relevant national publications [18,24] were used to help guide case writers. The second component of online module development revolves around management of content and involves storing, indexing and filing of material. For this dedicated IT support is essential and we were able to utilise the pre-existing VLE at the Medical School which was able to support all necessary forms of image media planned within the module. The third and final phase of development involves delivery of the written content. We chose to use a number of different styles of online radiological teaching, to support the varying learning requirements of the students and to supplement the more traditional learning strategies running concurrently. It was not our intention to try and replace traditional face to face teaching methods, but to enhance them and it is clear from both the literature and from our student surveys that most students prefer to learn using a blended combination of written and online delivery. Our survey did show however that a significant proportion of students (25%) preferred online teaching alone as a means of self-directed learning. An asynchronous approach was chosen for the online cases and delivery and receipt of information were not simultaneous [22]. This was felt necessary to reduce the demands on teachers due to the large amounts of teaching material within the module. The use of online case discussion room in undergraduate radiology teaching is original and not well described in the literature. Studies on radiology e-learning have incorporated this form of teaching and have confirmed its potential [25,26]. We used this forum to simulate scrolling through PACS using CT video files and this was well received by the students. The third means of online delivery comprised the three imaging tutorials—two delivery formats were used for these (see earlier) namely web page and video sequences. The
piloting of a video-based imaging tutorial worked well on the system and again the use of this format for radiological teaching is again not well described at undergraduate level and there is scope for wider application. How is the success of online teaching tool evaluated? This is complex and often hard to assess. There are three main areas to be addressed: 4.1. Do users like the product? There have been high levels of engagement by both students and clinician teachers with the online radiology components of the year 5 curriculum. The student surveys have confirmed the known benefits of using online techniques, namely the ability to work at their own pace and at a time of their choosing and to be able to cover large areas of knowledge in a short space of time, particularly before examinations. The data in Table 2 indicate high levels of student satisfaction with both the online radiology teaching and also traditional teaching methods. For the majority of students their year 5 radiology teaching: • Had helped in their understanding of the role of imaging in patient care. • Had helped in improving their confidence in interpretation of imaging studies. • Had been relevant and of appropriate level of difficulty to year 5. For clinicians and students to engage with radiology online learning there are important factors to be considered and addressed at the outset: • Resistance to change and poor IT skills—less of a problem amongst current generations of IT literate students and (most) clinicians. • Prohibitive set-up costs (see (b) below). • Poor image quality. • Poor IT support and links making system inaccessible. • Lack of set-up time, both for writing/editing cases and IT. Time constraints are a major issue for radiologists, with increasing clinical demands. • We have also experienced reduced utilisation by students in some areas of the online package due to apparent lack of awareness of their existence—advertising the product and encouraging participation is part of the packaging. We also engaged students in case utilisation by making them aware that cases would reflect the style and level of difficulty of their final examinations as students will not readily utilise learning resources which they do not think will contribute to their success in examination. 4.2. What is the financial cost? Studies have shown that e-learning can result in significant cost savings, up to 50%, when compared to onsite teaching methods [27]. These financial savings will only accrue following initial setup and these start-up costs can be significant in terms of staffing and equipment [28]. We did estimate a cost per case of 135 euros, based on 5 h per case of clinician/IT/medical photographer/librarian time. Maintenance costs following initial set-up will reduce significantly and online packages are appealing to managers due to reduced travelling, training and staffing costs. However this cost does not include the cost of the University VLE online resources purchased by the Medical School, an important component of the case feedback. The University VLE is a commercial product and the high purchase and support overheads of these types of systems have led some educational institutions to develop open-source, enterprise-wide, VLEs [1]. These systems do not have
D. Howlett et al. / European Journal of Radiology 78 (2011) 334–341
licensing costs and are customisable, allow ongoing development and also implementation of improvements as suggested by users. 4.3. Is the product useful educationally? Outcome evaluations of e-learning packages are complex and often difficult to undertake. Future research will concentrate on outcome evaluation which may relate to: user satisfaction (see earlier), evidence of improved learning, or changes in learner/organisational behaviour or patient outcome [22]. Digital images are objective and can be used for examination and assessment and have been used to demonstrate improvement in undergraduate radiological interpretation although this area has not been evaluated in our paper. Student learning is multifactorial however and online learning reflects only one style of learning choice, which will complement other techniques, but this can make it difficult to evaluate the utility of online learning in isolation. 5. Conclusion Increased utilisation of e-learning in undergraduate and postgraduate radiological teaching is a necessity in the face of the increasing clinical/managerial workloads of radiologists with constant pressure on available teaching time. Online techniques can be effectively blended with more “traditional” face to face forms of radiological content of a final year undergraduate teaching programme. Students and radiologists will both engage readily with online teaching but successful delivery is highly dependent on effective IT and medical photography support and can have high set-up costs. References [1] Sparacia G, Cannizzaro F, D’Alessandro D, D’Alessandro M, Caruso G, Lagalla R. Initial experiences in radiology e-learning. Radiographics 2007;27:573– 81. [2] Harden RM, Hart IR. An international virtual medical school (IVIMEDS): the future for medical education? Med Teach 2002;24(3):261–7. [3] Petrusa ER, Barry Issenberg S, Mayer JW, Felner JM, Brown DD, Waugh RA, Kondos GT, Gessner IH, McGaghie WC. Implementation of a four year multimedia computer curriculum in cardiology at six medical schools. Acad Med 1999;74(2):123–9. [4] Lau F, Bates J. A review of e-learning practices for undergraduate medical education. J Med Syst 2004;28(1):71–87. [5] Lacey Bryant S, Ringrose T. Evaluating the Doctors.net.uk model of electronic continuing medical education. Work Based Learn Primary Care 2005;3(2):129–42.
341
[6] Wilson AS, Goodall JE, Ambrosini G, Carruthers DM, Chan H, Ong SG, Gordon C, Young SP. Development of an interactive learning tool for teaching rheumatology—a simulated clinical case studies programme. Rheumatology 2006;45:1158–61. [7] Choules AP. The use of e-learning in medical education: a review of the current situation. Postgrad Med J 2007;83:212–6. [8] Yang G-L, Lim T. Singapore national medical image resource centre (SN. MIRC): a world wide web resource for radiology education. Ann Acad Med 2006;35:558–63. [9] Grunewald M, Heckemann RA, Gebhard H, Lell M, Bautz WA. Compare radiology; creating an interactive web-based training programme for radiology with multimedia authoring software. Acad Radiol 2003;10:543–53. [10] Shaffer K. Radiology education in the digital era. Radiology 2005;235:359–60. [11] Durfree Fee SM, Jain S, Shaffer K. Incorporating electronic media into medical student education: a survey of AMSER members on computer and web-base use in radiology courses. Acad Radiol 2003:524–30. [12] Scarsbrook AF, Foley PT, Perriss RW, et al. Radiological digital teaching fite development: an overview. Clin Radiol 2005;60:831–7. [13] Flanders AE, Wiggins RH, Gozum ME. Handheld computers in radiology. Radiographics 2003;33:1035–47. [14] Scarsbrook AF, Graham RNJ, Perriss RW. Radiology education: a glimpse into the future. Clin Radiol 2006;61:640–8. [15] Makeck M, Fischer MR, Kammer B, et al. Do computers teach better? A media comparison study for case-based teaching in radiology. Radiographics 2001;21:1025–32. [16] Davison BD, Tello R, Blickman JG. World wide web program for optimising and assessing medical student performance during the radiology clerkship. Acad Radiol 2000;7:260–3. [17] Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med 2002;77:86–93. [18] Royal College of Radiologists, Radiology For Medical Students, Papers 1–4, 2008 [at: www.rcr.ac.uk]. [19] Miles K. Diagnostic imaging in undergraduate medical education: an expanding role. Clin Radiol 2005;60:742–5. [20] Radiology Integrated Training Initiative—R-ITI; http://riti.org.uk. [21] Greenhalgh T. Computer assisted learning in undergraduate education. BMJ 2001;332:40–4. [22] Ruiz JG, Mintzner MJ, Leipzig RM. The impact of e-learning in medical evaluation. Acad Med 2006;81(3):207–12. [23] Lee JL, Aldrich JE, Eftekhari A, Nicolaou S, Muller NL. Implementation of a new undergraduate radiology curriculum. experience at the University of British Columbia. Can Assoc Radiol 2007;58(5):272–8. [24] General Medical Council. Tomorrows doctors. London: General Medical Council; 1993. [25] Gotthardt M, Siegert MJ, Schlieck A, et al. How to successfully implement elearning for both students and teachers. Acad Radiol 2006;13:379–90. [26] Filippucci E, Meenagh G, Ciapetti A, et al. E-learning in ultrasonography: a webbased approach. Ann Rheum Dis 2007;66:962–5. [27] Gibbons A, Fairweather P. Computer-based instruction. In: Tobias S, Fletcher J, editors. Training and retraining. A handbook for business, industry, government and the military. New York: Mac Millan Reference USA; 2000. p. 410–42. [28] Fall LH, Berman NB, Smith S, White CB, Woodhead JC, Olson AL. Multiinstitutional development and utilisation of a computer-assisted learning programme for the paediatrics clerkship: the CLIPP project. Acad Med 2005;80(9):847–55.