- Email: [email protected]
Cytopathology whole slide images and adaptive tutorials for postgraduate pathology trainees: a randomized crossover trial
Human Pathology (2015) 46, 1297–1305
www.elsevier.com/locate/humpath
Original contribution
Cytopathology whole slide images and adaptive tutorials for postgraduate pathology trainees: a randomized crossover trial☆,☆☆ Simone L. Van Es MBBS, Grad Dip Med, FRCPA, Cert FPA a,⁎, Rakesh K. Kumar MBBS, PhD, MD, FRCPA (Hon), FFSc (RCPA) a , Wendy M. Pryor MBBS, PhD, FRCPA b , Elizabeth L. Salisbury MBBS (Hons 1), FRCPA, FIAC, FFOP c , Gary M. Velan MBBS, DipHEd, PhD a a
Department of Pathology, School of Medical Sciences, The University of New South Wales, Sydney NSW 2052, Australia Royal College of Pathologists of Australasia, Surry Hills 2010, Australia c Department of Anatomical Pathology, Prince of Wales Hospital, Randwick 2031, Australia b
Received 2 April 2015; revised 29 April 2015; accepted 7 May 2015
Keywords: Virtual slides; Cytopathology; Virtual microscopy adaptive tutorials; Digital microscopy; Whole slide imaging; WSI; Pathology education
Summary To determine whether cytopathology whole slide images and virtual microscopy adaptive tutorials aid learning by postgraduate trainees, we designed a randomized crossover trial to evaluate the quantitative and qualitative impact of whole slide images and virtual microscopy adaptive tutorials compared with traditional glass slide and textbook methods of learning cytopathology. Forty-three anatomical pathology registrars were recruited from Australia, New Zealand, and Malaysia. Online assessments were used to determine efficacy, whereas user experience and perceptions of efficiency were evaluated using online Likert scales and open-ended questions. Outcomes of online assessments indicated that, with respect to performance, learning with whole slide images and virtual microscopy adaptive tutorials was equivalent to using traditional methods. High-impact learning, efficiency, and equity of learning from virtual microscopy adaptive tutorials were strong themes identified in open-ended responses. Participants raised concern about the lack of z-axis capability in the cytopathology whole slide images, suggesting that delivery of z-stacked whole slide images online may be important for future educational development. In this trial, learning cytopathology with whole slide images and virtual microscopy adaptive tutorials was found to be as effective as and perceived as more efficient than learning from glass slides and textbooks. The use of whole slide images and virtual microscopy adaptive tutorials has the potential to provide equitable access to effective learning from teaching material of consistently high quality. It also has broader implications for continuing professional development and maintenance of competence and quality assurance in specialist practice. © 2015 Elsevier Inc. All rights reserved.
☆
Competing interests: Nil. Funding/Support: This project was supported in part by the Royal College of Pathologists of Australasia, Surry Hills, Australia, which has received Australian Government funding under the Specialist Training Program (no. 247). ⁎ Corresponding author. Department of Pathology, School of Medical Sciences, The University of New South Wales, Sydney NSW 2052, Australia. E-mail addresses: [email protected] (S. L. Van Es), [email protected] (R. K. Kumar), [email protected] (W. M. Pryor), [email protected] (E. L. Salisbury), [email protected] (G. M. Velan). ☆☆
http://dx.doi.org/10.1016/j.humpath.2015.05.009 0046-8177/© 2015 Elsevier Inc. All rights reserved.
1298
1. Introduction There are extensive descriptions in the literature of the use of digital imaging technology in pathology. “Digital” or “virtual” slides (vslides) or “whole slide images” (WSI) with x- and y-axis capability are widely used [1,2]; these are scanned images of tissue sections or cytology smears, typically stored in a multiresolution format. Scanning devices for creating digital images are well described and illustrated by Khalbuss et al [1]. Other technologies include telepathology [3,4], transmitted still images [5], WSI with z-axis capability (so-called z-stacking) [6-9], WSI with extended-focus imaging [10] or so-called focus fusion [11], video image capture [12], and virtual pathology tracking software for training and assessment purposes [13-15]. Numerous groups have examined the use of virtual histopathology for teaching [16-22] and the use of virtual histopathology and cytopathology for diagnosis [6,23,24]. The value and reliability of virtual microscopy compared with traditional microscopy have been extensively assessed [6,16-20,22] with virtual methods scored as equal or even superior [6,17,18,22]. Most studies show strong acceptance of digital pathology as a learning tool, although there is some reluctance in using it for assessment purposes [23], especially in cytopathology [6], due to the perceived slower speed of analyzing cytology WSI [6,9,25]. Cytopathology is z-axis dependent. Microscopic examination of a traditional slide is not limited by focal depth, whereas focusing through different planes cannot be achieved with a wholly or partially 2-dimensional WSI. Nevertheless, studies comparing diagnostic accuracy between virtual versus traditional cytopathology have concluded either that both methods were similar [6,25] or that the degree of inaccuracy was small [26-28]. Moreover, the number of included z-stacks did not seem to make a great deal of difference to reliability of the diagnosis [9]. However, in several studies, participants felt more comfortable if vertical focus was available [6,9,29]. It has therefore been established in non-Australian settings that WSI are an acceptable and accurate way to learn both histopathology and cytopathology. However, the use of cytopathology WSI for education and proficiency has never been assessed in the Australasian pathology setting. Although virtual microscopy has been shown to be a very useful learning tool, provision of WSI alone for teaching may also not be sufficient to convey salient teaching features [30]. Additional support in an interactive e-learning environment is readily achievable and may be valuable with image-based subjects such as microscopic pathology. Interactive virtual teaching modules have been shown to improve general pattern recognition [31] and pathology examination performance [32] and are an effective and acceptable way for medical students to learn pathology [33]. We have previously reported the successful use of virtual microscopy adaptive tutorials (VMATs) to effectively engage medical students in learning basic histopathology from WSI [33]. VMATs are built on a platform that readily allows an expert to create an interactive educational module around a selection of WSI. The platform enables real-time
S. L. Van Es et al. tracking and monitoring of participant responses, allowing feedback to be adapted as appropriate to help remediate common misconceptions. No published evidence of efficacy and efficiency of cytology VMATs currently exists. Consequently, in the present study, we aimed to compare the efficacy and efficiency of both cytology WSI and VMATs to traditional glass slides and textbooks for learning by postgraduate anatomical pathology trainees.
2. Materials and methods 2.1. Virtual slides An Aperio Scanscope XT (Aperio Technologies Inc., Vista, CA) was used for image acquisition at original magnification ×40. Additional focal points were manually inserted to maximize the in-focus areas on the final WSI. Files were cropped to reduce final size to less than 2 GB. The WSI were stored in a biomedical image database known as Slice (https:// www.best.edu.au/slice/featured) and delivered using a custom-built whole slide viewer. Accompanying VMATs were developed to augment the majority of WSI, whereas a subset was used for online assessments. An example of our WSI and the Slice viewer interface can be seen via the following link: https://www.best.edu.au/s/ft8pw2tc/l8iw37sx.
2.2. Development of VMATS VMATs were created using the Adaptive e-Learning Platform, an intelligent tutoring system developed by Smart Sparrow (https://www.smartsparrow.com/). Feedback on 3 online pilot cytopathology VMATs was sought from pathology trainees and specialist pathologists. This was taken into account when developing a further 22 VMATs for the trial. Adobe Captivate was used to create short instructional videos to augment some VMATs. The VMAT interface can be seen in Figs. 1 and 2. These VMATs can be accessed via the following links: https:// aelp.smartsparrow.com/bronte/viewer/open/9r43hp68 and https://aelp.smartsparrow.com/bronte/viewer/open/nrreaien.
2.3. Trial design and analysis The trial focused on the main themes of specialist cytopathology training, including gynecology, fine needle aspiration (FNA), and exfoliative/effusion fluid cytopathology. The diagnostic categories of the 22 VMAT-supported cases are listed in Table 1. The trial was approved by the UNSW Medicine Human Research Ethics Committee (HREC 1311). Participants were volunteer anatomical pathology trainees from all states in Australia as well as from New Zealand and Malaysia. They were recruited by advertisement and broadcast email through the college. Informed consent was obtained from all participants, who
Cytopathology WSI and VMATs
1299
Fig. 1
Example of an interactive question within a VMAT.
had the option of withdrawing at any time. The trial was divided into 3 phases, as seen in Table 2, with a crossover between phases and an online assessment at the end of each phase. A crossover design was used, seeking to account for differences between groups with respect to knowledge, experience with cytology, or pretrial familiarity with WSI. Diagnostic categories in the first online assessment included negative smears (including changes associated with microorganisms) and atypical smears (low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial
Fig. 2
lesion, squamous cell carcinoma, and endocervical adenocarcinoma in situ/adenocarcinoma). The second online assessment on FNA cytology included specimens from the thyroid (papillary carcinoma), lymph nodes (metastatic melanoma, non-Hodgkin lymphoma), breast (ductal adenocarcinoma), and lung (adenocarcinoma, squamous cell carcinoma, and small cell carcinoma). The final online assessment on fluid/exfoliative cytology included pleural fluid specimens (reactive, metastatic adenocarcinoma), sputum specimens (normal, small cell carcinoma), bronchial
Example of immediate feedback within a cytopathology VMAT.
1300 Table 1
S. L. Van Es et al. Diagnostic categories of the 22 cases (VMATs and WSI) studied during the trial
Nature of specimen
Diagnostic category/specific diagnosis
Gynecological cytopathology CS/TP
Negative With endocervical cells, mature/immature squamous cells With HSV With Candida albicans Atypical LGSIL (CINI/HPV) HSIL (CINIII) SCC Endometrial adenocarcinoma Endocervical adenocarcinoma
FNA Lung
Granulomatous inflammation Adenocarcinoma Papillary carcinoma Metastatic melanoma NHL Fibroadenoma Ductal adenocarcinoma
Thyroid Lymph node Breast Fluid cytopathology Pleural
Reactive Mesothelioma Metastatic gastric adenocarcinoma Ovarian serous carcinoma Normal Urothelial carcinoma Benign/reactive
Peritoneal Urine Respiratory/BB
Abbreviations: CS, cervical smear; TP, thin prep; HSV, herpes simplex virus; LGSIL, low-grade squamous intraepithelial lesion; CINI, cervical intraepithelial neoplasia I; HPV, human papillomavirus; HSIL, high-grade squamous intraepithelial lesion; CINIII, cervical intraepithelial neoplasia III; SCC, squamous cell carcinoma; FNA, fine needle aspiration; NHL, non-Hodgkin lymphoma; BB, bronchial brushing.
brush specimens (benign), and a urine specimen (urothelial carcinoma). A preliminary power analysis had indicated that, to demonstrate a 30% difference between groups with greater than 90% probability, 9 to 10 participants per group were required for the study. Participants were randomized into 2 groups such that there were equal numbers of junior (years 1 and 2) and senior (years 3, 4, and 5) trainees. Participants were then clustered geographically, each group having an approximate equal number from major metropolitan teaching hospitals and more peripherally located departments. At the commencement of each 2-week phase of the trial, all members of a group received either links to WSI and Table 2
VMATs or a list of diagnoses/disorders to be studied in the traditional way using departmental teaching slide sets and textbooks. Participants were asked to record the amount of time they spent studying cytopathology during each phase. They were also asked to complete online questionnaires on WSI and VMATs as cytopathology educational tools (using a scale of 1 [low] to 10 [high]) and other aspects of their experience (using Likert 5-point scales). The questionnaire also invited open-ended responses. Performance at the end of each phase was measured by a timed (60 minutes) online virtual cytopathology assessment containing links to WSI (which did not always contain the screener's diagnostic marks) using Questionmark Perception
Timeline and format of virtual cytopathology trial for Royal College of Pathologists of Australasia trainees
Topic
Group 1
Gynecological cytology
VMATs and WSI VMATs and WSI Online assessment—gynecological cytology VMATs and WSI Traditional Online assessment—FNA cytology Traditional VMATs and WSI Online assessment—fluid cytology
FNA cytology Fluid/exfoliative cytology
Group 2
Timeline (d) 0-14 15-16 17-31 32-33 34-48 49-50
Abbreviations: RCPA, Royal College of Pathologists of Australasia; FNA, fine needle aspiration; WSI, whole slide images; VMATs, virtual microscopy adaptive tutorials.
Cytopathology WSI and VMATs (Questionmark Computing Ltd, London, UK). A link to an example assessment can be provided upon request. Each assessment covered the theme that had been studied during that phase of the trial by both groups. The assessment following each module was subject matter specific, with the purpose of isolating any potential carry-over effect in this crossover trial design. The assessment could only be attempted once, and participants agreed to adhere to an honor code, without using any outside aids or assistance (eg, consultation with colleagues or access to textbooks or Internet sources). If participants exceeded the time limit, answers were automatically submitted. On average, each assessment comprised 7 WSI with 1 to 3 associated multiple-choice questions. After submission of answers, participants received immediate automated feedback. WSI were used in the assessments rather than glass slides, as participants were distributed widely across Australia, New Zealand, and Malaysia, and given the nature of cytology specimens, it is not possible to distribute identical glass cytopathology slides to all participants. Using whole slide images guaranteed that all participants received identical cytological material to assess, and any difference in assessment scores could not be attributed to any differences in the assessment material. Assessment scores and hours of study were compared using an unpaired Student t test. Data shown are mean ± SEM unless otherwise stated. For Likert scale and rating items in the questionnaire, median ratings were compared using Mann-Whitney U tests. Responses to open-ended questionnaire items were exported into a spreadsheet to facilitate thematic analysis. Two authors (S. L. V. E. and G. M. V.) independently identified common themes. Responses were coded, and emergent themes were subsequently identified.
1301 In phase 3 of the trial, emphasizing fluid cytopathology, only group 2 received digital resources. Again, there was no significant difference between the mean assessment scores of the groups (group 1, 77.3 ± 3.2, n = 12; group 2, 72.5 ± 2.8, n = 17; P = .28) (Fig. 3C). Furthermore, there was no significant difference between groups in their self-reported time spent studying the material (P = .11). Comparing WSI and VMATS for learning, there was a nonsignificant trend toward preferring VMATs (median rating 8 of 10, n = 26) over WSI (7 of 10, n = 23) (P = .06). Responses to Likert scale questions on efficiency and usability of WSI and VMATs are presented in Fig. 4. Qualitative analysis of open-ended questionnaire responses resulted in the emergence of a number of themes. A total of 25 participants commented on the cytology VMATs, with 7 emerging themes in the responses. The dominant positive themes were as follows: the value of VMATs as a high-impact learning tool, followed by convenience and equity of training (Table 3). VMATs were considered by trainees as likely to be “one of the main sources of teaching for the (pathology) trainee
3. Results Forty-three anatomical pathology trainees volunteered for the trial. Of these, 21 were randomized to group 1 (11 from year 1 or 2 and 10 from year 3, 4, or 5), and 22, to group 2 (10 from year 1 or 2 and 12 from year 3, 4, or 5). There was no significant difference between the 2 groups in years of training (P = .4). In phase 1 of the trial, both groups were provided with the WSI and VMATs relating to gynecological cytopathology. There was no difference between the 2 groups in the mean scores of those who completed the first online assessment (group 1, 55.6 ± 5.6, n = 19; group 2, 51.1 ± 6.0, n = 19; P = .5) (Fig. 3A). In phase 2 of the trial, which focused on FNA cytology, only group 1 received the educational material in digital form, that is, WSI and VMATs. The difference between the mean assessment scores of the 2 groups just failed to reach statistical significance (group 1, 91.8 ± 2.3, n = 12; group 2, 82.6 ± 4.3, n = 15; P = .09) (Fig. 3B).
Fig. 3 A to C, Group 1 and group 2 assessment scores after studying: gynecological cytology (A), FNA cytology (B), and fluid/effusion cytology (C).
1302
Fig. 4
S. L. Van Es et al.
Perceived benefit of WSI and VMATs as a cytopathology learning tool, where 5, strongly agree, and 1, strongly disagree.
in the near future.” They made a “huge difference to those of us who are junior” and made it “feel like you are getting a personalized experience compared to traditional teaching.” Trainees also commented that “the interactive nature made it easier to understand and retain the information” and that learning with VMATs was “much more appealing than the WSI alone and reading from a textbook.” The theme of equity emerged with comments such as “even if you didn't have access to teaching material or teachers you can learn via the VMATs.” For the WSI, dominant positive themes were the following: convenience, followed by equity of training and valuable/ high-impact learning (Table 4). WSI were seen by trainees to be very useful for “learning on the go,” with touch screen devices providing pleasing resolution and easy panning. Comments included that equity was ensured by “access to great cases” in every laboratory and “easy access to study from home.” A representative comment on the theme of equity was “I guess the availability of resources in each department is variable. The WSI can make it an even playing field for trainees,” and on the theme of valuable learning, it was “a comprehensive repository of cytology WSI would be invaluable.” An emerging negative theme was that analyzing a cytopathology WSI could be cumbersome and slow in comparison to a traditional glass slide, emerging with comments such as “I found screening (the whole slide image) in the exam really time-consuming.” The lack of 3-dimensional (3-D) focus, although mentioned, was not a strong theme. Table 3
Selection of stronger emerging themes for VMATs a
Themes
Response rate, n (%)
Valuable/high-impact learning Equity of training Convenient/easy access (at home, in-transit) Time efficient Needs to be tailored to level of knowledge/training
45 8 6 5 5
(63) (11) (9) (7) (7)
Abbreviations: VMATs, virtual microscopy adaptive tutorials. a Responses containing multiple themes were given by many participants.
4. Discussion The aim of our trial was to study the use of cytopathology WSI, VMATs, and online assessments incorporating WSI in a postgraduate pathology setting in Australasia. Our trial and the material being tested are novel and cutting edge in a number of ways. Although we are among a handful of groups who have evaluated cytopathology WSI for accuracy and efficiency [6,9,10,25,26,29,34], we are the first group to evaluate virtual cytopathology in the Australasian setting and the only group that has additionally evaluated an online “adaptive” tutoring system to accompany the cytopathology WSI in any setting. The VMATs are novel in that they are specifically designed to “adapt” to the user's decision-making process through remediation of misconceptions. Furthermore, analytics provided by the Adaptive e-Learning Platform provides teachers with evidence of the effectiveness of the adaptive feedback used in the tutorial. This provides scope to constantly monitor, improve, and streamline the feedback provided in each VMAT, mimicking real-life practice more than other currently available online formats. Although there are a small number of other online surgical biopsy and cytopathology tutorials available from institutions worldwide, it is arguable that without detailed analytics and subsequent adaption of
Table 4
Selection of stronger emerging themes for WSI a
Themes
Response rate, n (%)
Convenient (at home, in-transit on iPad) Equity of training Resolutions not as good as traditional microscopy Valuable/high-impact learning Screening the slide tedious and slow Lack of 3-D focus Archiving advantage—no fading, reservoir
23 (19) 19 (16) 19 (15) 16 (13) 10 (8) 9 (7) 7 (6)
Abbreviations: WSI, whole slide images. a Responses containing multiple themes were given by many participants.
Cytopathology WSI and VMATs feedback, the desired dialog between teachers and students in the online environment is diminished. An additional benefit of VMATs is the intuitive user interface. Trial participants were not given prior instruction and did not have prior experience with using the software. However, there were no negative comments about the interface in the open feedback. When specifically asked whether the WSI and VMAT interface was intuitive and easy to use, the majority agreed (Fig. 4). In this study, we found that the diagnostic reliability and accuracy of cytopathology WSI and VMATs for pathology trainees in Australasia were similar to those of traditional methods with glass teaching slides and textbooks. This is in accordance with other studies, which have also found diagnostic accuracy using digital methods to be equal to traditional methods [6,9,17,18,22,28,29]. The equivalence of WSI and VMATs with traditional methods for learning cytopathology is important, particularly because WSI and VMATs were perceived more positively as a way of learning by trainees in comparison to learning with glass slides and textbooks. Users of WSI and VMATS described major advantages: time effectiveness and flexibility, high learning impact, and equity in learning and training. These were perceived benefits in remote and rural pathology departments, in which the number of trainees is increasing. In accordance with previous studies [6,9,29], participants in our trial expressed frustration and uncertainty about the lack of 3-D focus in WSI. However, also consistent with previous reports, we found that proficiency in cytopathology using only 2-dimensional virtual technology compares quite satisfactorily with traditional techniques [6]. Z-stacking an entire cytopathology WSI can produce file sizes in excess of 7 GB [9], resulting in long response times because of network congestion [20]. Therefore, providing z-axis capability on WSI and delivering it efficiently remains a task for the future. Extended-focus imaging may represent an alternative to z-stacking. However, loss of crisp cellular definition can be an issue with this technique [10]. The experience of our participants concurred with previous observations that providing immediate visual feedback enhances the learning experience [32]. Without guidance from a VMAT to analyze and learn from cytopathology WSI, many found it “frustratingly stop-start.” Numerous participants commented that having different diagnostic cells or patterns on cytology WSI highlighted in different colors within the VMAT adaptive feedback helped quite considerably to reinforce the concepts they needed to learn. Some participants in our trial also found that the immediate feedback, not just in the VMATs but also in the in the online assessments, was also very helpful: “the instant feedback at the end of the test was great along with the explanation of the relevant features.” We acknowledge several potential limitations of our trial. Of note, this study had a small sample size, and it is possible that our online assessments may have had insufficient items to reliably demonstrate differences between groups. We did seek to deal with some other potential confounders, such as
1303 ensuring a balance of year training levels in each group as well as assessing for differences in the prior level of knowledge and familiarity with digital slide technology via the “pretest” provided by the online assessment at the end of the first phase. It is possible that carry-over effects might have resulted in reduced observed impact of the WSI and VMATs on assessment scores in the final 2 phases of the trial. However, we believe that distinct differences between the materials covered in each phase of the trial helped to minimize such carry-over effects. Another potential confounder is selection bias: all participants volunteered for the trial, which may reflect an interest in online learning technology that could, in turn, contribute to the positive feedback on the WSI and VMATs. An additional argument is that each postmodule assessment should incorporate glass slides, as this is the ultimate goal of pathology specialist training at present. However, the focus of this study was on the effectiveness of learning using this method of delivery and the materials provided. Moreover, using WSI in the assessments guaranteed that all participants received identical cytological material to assess, so that any difference in assessment scores could not be a result of differences in the assessment material. Lastly, this trial was designed to be a short-term intermediate study. Further studies to look at the long-term effect of this learning methodology, after a significant washout period, may be useful. Nevertheless, with technology constantly and rapidly advancing, it is inevitable that pathology trainees will have increasingly higher expectations of the learning resources supplied in their work environment and by specialist certification bodies responsible for their training. In particular, these expectations include that, in postgraduate education, learning should be available online [1] and programs/resources should be mobile to fit in with the demands of routine work. Additionally, the positive findings associated with the use of VMATs in this study may have broader implications for specialist practice as well, including continuing professional development and maintenance of competence and quality assurance. Adequate training in cytological diagnosis will remain essential for anatomical pathology trainees and specialists alike. Thus, access to large sets of instructive cytopathology WSI and associated VMATs has the potential to provide equitable access to teaching material of a consistent high quality, attempting to mirror real-life settings as much as possible. Additionally, networking and collaborative study for all trainees is promoted, which is of particular importance for those trainees in rural or remote locations.
Distribution of author contributions Trial design, S. L. V. E., G. M. V., R. K. K., W. M. P., and E. L. S.; WSI selection, scanning, and editing, S. L. V. E.; VMAT author, S. L. V. E.; VMAT editing, S. L. V. E. and G. M. V.; online assessments, S. L. V. E. and G. M. V.; statistics, S. L. V. E.
1304 and G. M. V. and W. M. P.; authoring manuscript, S. L. V. E.; editing manuscript, S. L. V. E., G. M. V., R. K. K., W. M. P., and E. L. S.; participant recruitment, W. M. P., S. L. V. E., and G. M. V.
Ethics Before the commencement of this project, ethics approval had been obtained (UNSW HREC 11311 and the Board of Censors Royal College of Pathologists of Australasia).
Acknowledgments Image acquisition: Dr Maria Sarris (Histology and Microscopy Unit, UNSW); technical support: Mr Jake Surman (UNSW), Mr Peter Zarzour (UNSW), Smart Sparrow, the BEST Network; statistics: Dr Michael Bennett (Prince of Wales Hospital, Sydney), Dr Roy Wilson (School of Mathematics & Statistics, UNSW), A/Prof Boaz Shulruf (Medical Education, UNSW); participant recruitment: Ms Vanessa White (Royal College of Pathologists of Australasia); glass cytopathology slides: Ms Joanne La Malfa (Anatomical Pathology Department, Prince of Wales Hospital, Sydney). The authors thank Ms Joanne La Malfa for assistance with cytopathology slides interpretation.
References [1] Khalbuss WE, Pantanowitz L, Parwani AV. Digital imaging in cytopathology. Pathol Res Int 2011;2011:264683. http://dx.doi.org/10. 4061/2011/264683. [2] Isaacs M, Lennerz JK, Yates S, Clermont W, Rossi J, Pfeifer JD. Implementation of whole slide imaging in surgical pathology: a value added approach. J Pathol Inf 2011;2:39. http://dx.doi.org/10.4103/ 2153-3539.84232. [3] Weinstein RS, Graham AR, Richter LC, et al. Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. HUM PATHOL 2009;40:1057-69. [4] Hitchcock CL. The future of telepathology for the developing world. Arch Pathol Lab Med 2011;135:211-4. [5] van den Tweel JG, Bosman FT. The use of virtual slides in the EUROPALS examination. Diagn Pathol 2011;6:S23. http://dx.doi.org/ 10.1186/1746-1596-6-s1-s23. [6] Dee FR, Donnelly A, Radio S, Leaven T, Zaleski MS, Kreiter C. Utility of 2-D and 3-D virtual microscopy in cervical cytology education and testing. Acta Cytol 2007;51:523-9. [7] Kalinski T, Zwonitzer R, Sel S, et al. Virtual 3D microscopy using multiplane whole slide images in diagnostic pathology. Am J Clin Pathol 2008;130:259-64. [8] Zwonitzer R, Hofmann H, Roessner A, Kalinski T. Virtual 3D microscopy in pathology education. HUM PATHOL 2010;41:457-8. [9] Evered A, Dudding N. Accuracy and perceptions of virtual microscopy compared with glass slide microscopy in cervical cytology. Cytopathology 2011;22:82-7. [10] Lee RE, McClintock DS, Laver NM, Yagi Y. Evaluation and optimization for liquid-based preparation cytology in whole slide imaging. J Pathol Inf 2011;2:46. http://dx.doi.org/10.4103/2153-3539.86285.
S. L. Van Es et al. [11] Mori I, Ozaki T, Taniguchi E, Kakudo K. Study of parameters in focus simulation functions of virtual slide. Diagn Pathol 2011;6:S24. http:// dx.doi.org/10.1186/1746-1596-6-s1-s24. [12] Melin-Aldana H, Gasilionis V, Kapur U. Use of digital video for documentation of microscopic features of tissue samples. Arch Pathol Lab Med 2008;132:820-2. [13] Treanor D, Lim CH, Magee D, Bulpitt A, Quirke P. Tracking with virtual slides: a tool to study diagnostic error in histopathology. Histopathology 2009;55:37-45. [14] Mello-Thoms C, Mello CAB, Medvedeva O, et al. Perceptual analysis of the reading of dermatopathology virtual slides by pathology residents. Arch Pathol Lab Med 2012;136:551-62. [15] Krupinski EA, Graham AR, Weinstein RS. Characterizing the development of visual search expertise in pathology residents viewing whole slide images. HUM PATHOL 2013;44:357-64. [16] Blake CA, Lavoie HA, Millette CF. Teaching medical histology at the University of South Carolina School of Medicine: transition to virtual slides and virtual microscopes. Anat Rec B 2003;275B: 196-206. [17] Kumar RK, Velan GM, Korell SO, Kandara M, Dee FR, Wakefield D. Virtual microscopy for learning and assessment in pathology. J Pathol 2004;204:613-8. [18] Kumar RK, Freeman B, Velan GM, De Permentier PJ. Integrating histology and histopathology teaching in practical classes using virtual slides. Anat Rec B 2006;289:128-33. [19] Dee FR, Meyerholz DK. Teaching medical pathology in the twentyfirst century: virtual microscopy applications. J Vet Med Educ 2007; 34:431-6. [20] Dee FR. Virtual microscopy in pathology education. HUM PATHOL 2009;40:1112-21. [21] Fonyad L, Gerely L, Cserneky M, Molnar B, Matolcsy A. Shifting gears higher–digital slides in graduate education—4 years experience at Semmelweis University. Diagn Pathol 2010;5:73. http://dx.doi.org/ 10.1186/1746-1596-5-73. [22] Triola MM, Holloway WJ. Enhanced virtual microscopy for collaborative education. BMC Med Educ 2011;11:4. http://dx.doi. org/10.1186/1472-6920-11-4. [23] Koch LH, Lampros JN, Delong LK, Chen SC, Woosley JT, Hood AF. Randomized comparison of virtual microscopy and traditional glass microscopy in diagnostic accuracy among dermatology and pathology residents. HUM PATHOL 2009;40:662-7. [24] Nielsen PS, Lindebjerg J, Rasmussen J, Starklint H, Waldstrom M, Nielsen B. Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. HUM PATHOL 2010;41:1770-6. [25] Stewart III J, Miyazaki K, Bevans-Wilkins K, Ye C, Kurtycz DF, Selvaggi SM. Virtual microscopy for cytology proficiency testing: are we there yet? Cancer 2007;111:203-9. [26] Gagnon M, Inhorn S, Hancock J, et al. Comparison of cytology proficiency testing: glass slides vs. virtual slides. Acta Cytol 2004;48: 788-94. [27] Wilbur DC, Madi K, Colvin RB, et al. Whole-slide imaging digital pathology as a platform for teleconsultation: a pilot study using paired subspecialist correlations. Arch Pathol Lab Med 2009; 133:1949-53. [28] Marchevsky AM, Wan Y, Thomas P, Krishnan L, Evans-Simon H, Haber H. Virtual microscopy as a tool for proficiency testing in cytopathology: a model using multiple digital images of Papanicolaou tests. Arch Pathol Lab Med 2003;127:1320-4. [29] Steinberg DM, Ali SZ. Application of virtual microscopy in clinical cytopathology. Diagn Cytopathol 2001;25:389-96. [30] Weaker FJ, Herbert DC. Transition of a dental histology course from light to virtual microscopy. J Dent Educ 2009;73:1213-21. [31] Bejjanki VR, Zhang R, Li R, et al. Action video game play facilitates the development of better perceptual templates. PNAS 2014;11: 16961-6.
Cytopathology WSI and VMATs [32] Kanthan R. Digital games designed for pathology education. Lab Invest 2009;89:113A-4A. [33] Velan GM, Ben-Naim D, Kumar RK, Bain M, Kan B, Marcus N. Adaptive tutorials using virtual slides to enhance learning of microscopic morphology. In: Richards G, editor. World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009.
1305 Vancouver, Canada: Chesapeake, VA: AACE; 2009. p. 759-63 [Accessed 22 April 2015, from http://www.editlib.org/p/32546]. [34] Stewart III J, Bevans-Wilkins K, Bhattacharya A, Ye C, Miyazaki K, Kurtycz DF. Virtual microscopy: an educator's tool for the enhancement of cytotechnology students' locator skills. Diagn Cytopathol 2008;36:363-8.
www.elsevier.com/locate/humpath
Original contribution
Cytopathology whole slide images and adaptive tutorials for postgraduate pathology trainees: a randomized crossover trial☆,☆☆ Simone L. Van Es MBBS, Grad Dip Med, FRCPA, Cert FPA a,⁎, Rakesh K. Kumar MBBS, PhD, MD, FRCPA (Hon), FFSc (RCPA) a , Wendy M. Pryor MBBS, PhD, FRCPA b , Elizabeth L. Salisbury MBBS (Hons 1), FRCPA, FIAC, FFOP c , Gary M. Velan MBBS, DipHEd, PhD a a
Department of Pathology, School of Medical Sciences, The University of New South Wales, Sydney NSW 2052, Australia Royal College of Pathologists of Australasia, Surry Hills 2010, Australia c Department of Anatomical Pathology, Prince of Wales Hospital, Randwick 2031, Australia b
Received 2 April 2015; revised 29 April 2015; accepted 7 May 2015
Keywords: Virtual slides; Cytopathology; Virtual microscopy adaptive tutorials; Digital microscopy; Whole slide imaging; WSI; Pathology education
Summary To determine whether cytopathology whole slide images and virtual microscopy adaptive tutorials aid learning by postgraduate trainees, we designed a randomized crossover trial to evaluate the quantitative and qualitative impact of whole slide images and virtual microscopy adaptive tutorials compared with traditional glass slide and textbook methods of learning cytopathology. Forty-three anatomical pathology registrars were recruited from Australia, New Zealand, and Malaysia. Online assessments were used to determine efficacy, whereas user experience and perceptions of efficiency were evaluated using online Likert scales and open-ended questions. Outcomes of online assessments indicated that, with respect to performance, learning with whole slide images and virtual microscopy adaptive tutorials was equivalent to using traditional methods. High-impact learning, efficiency, and equity of learning from virtual microscopy adaptive tutorials were strong themes identified in open-ended responses. Participants raised concern about the lack of z-axis capability in the cytopathology whole slide images, suggesting that delivery of z-stacked whole slide images online may be important for future educational development. In this trial, learning cytopathology with whole slide images and virtual microscopy adaptive tutorials was found to be as effective as and perceived as more efficient than learning from glass slides and textbooks. The use of whole slide images and virtual microscopy adaptive tutorials has the potential to provide equitable access to effective learning from teaching material of consistently high quality. It also has broader implications for continuing professional development and maintenance of competence and quality assurance in specialist practice. © 2015 Elsevier Inc. All rights reserved.
☆
Competing interests: Nil. Funding/Support: This project was supported in part by the Royal College of Pathologists of Australasia, Surry Hills, Australia, which has received Australian Government funding under the Specialist Training Program (no. 247). ⁎ Corresponding author. Department of Pathology, School of Medical Sciences, The University of New South Wales, Sydney NSW 2052, Australia. E-mail addresses: [email protected] (S. L. Van Es), [email protected] (R. K. Kumar), [email protected] (W. M. Pryor), [email protected] (E. L. Salisbury), [email protected] (G. M. Velan). ☆☆
http://dx.doi.org/10.1016/j.humpath.2015.05.009 0046-8177/© 2015 Elsevier Inc. All rights reserved.
1298
1. Introduction There are extensive descriptions in the literature of the use of digital imaging technology in pathology. “Digital” or “virtual” slides (vslides) or “whole slide images” (WSI) with x- and y-axis capability are widely used [1,2]; these are scanned images of tissue sections or cytology smears, typically stored in a multiresolution format. Scanning devices for creating digital images are well described and illustrated by Khalbuss et al [1]. Other technologies include telepathology [3,4], transmitted still images [5], WSI with z-axis capability (so-called z-stacking) [6-9], WSI with extended-focus imaging [10] or so-called focus fusion [11], video image capture [12], and virtual pathology tracking software for training and assessment purposes [13-15]. Numerous groups have examined the use of virtual histopathology for teaching [16-22] and the use of virtual histopathology and cytopathology for diagnosis [6,23,24]. The value and reliability of virtual microscopy compared with traditional microscopy have been extensively assessed [6,16-20,22] with virtual methods scored as equal or even superior [6,17,18,22]. Most studies show strong acceptance of digital pathology as a learning tool, although there is some reluctance in using it for assessment purposes [23], especially in cytopathology [6], due to the perceived slower speed of analyzing cytology WSI [6,9,25]. Cytopathology is z-axis dependent. Microscopic examination of a traditional slide is not limited by focal depth, whereas focusing through different planes cannot be achieved with a wholly or partially 2-dimensional WSI. Nevertheless, studies comparing diagnostic accuracy between virtual versus traditional cytopathology have concluded either that both methods were similar [6,25] or that the degree of inaccuracy was small [26-28]. Moreover, the number of included z-stacks did not seem to make a great deal of difference to reliability of the diagnosis [9]. However, in several studies, participants felt more comfortable if vertical focus was available [6,9,29]. It has therefore been established in non-Australian settings that WSI are an acceptable and accurate way to learn both histopathology and cytopathology. However, the use of cytopathology WSI for education and proficiency has never been assessed in the Australasian pathology setting. Although virtual microscopy has been shown to be a very useful learning tool, provision of WSI alone for teaching may also not be sufficient to convey salient teaching features [30]. Additional support in an interactive e-learning environment is readily achievable and may be valuable with image-based subjects such as microscopic pathology. Interactive virtual teaching modules have been shown to improve general pattern recognition [31] and pathology examination performance [32] and are an effective and acceptable way for medical students to learn pathology [33]. We have previously reported the successful use of virtual microscopy adaptive tutorials (VMATs) to effectively engage medical students in learning basic histopathology from WSI [33]. VMATs are built on a platform that readily allows an expert to create an interactive educational module around a selection of WSI. The platform enables real-time
S. L. Van Es et al. tracking and monitoring of participant responses, allowing feedback to be adapted as appropriate to help remediate common misconceptions. No published evidence of efficacy and efficiency of cytology VMATs currently exists. Consequently, in the present study, we aimed to compare the efficacy and efficiency of both cytology WSI and VMATs to traditional glass slides and textbooks for learning by postgraduate anatomical pathology trainees.
2. Materials and methods 2.1. Virtual slides An Aperio Scanscope XT (Aperio Technologies Inc., Vista, CA) was used for image acquisition at original magnification ×40. Additional focal points were manually inserted to maximize the in-focus areas on the final WSI. Files were cropped to reduce final size to less than 2 GB. The WSI were stored in a biomedical image database known as Slice (https:// www.best.edu.au/slice/featured) and delivered using a custom-built whole slide viewer. Accompanying VMATs were developed to augment the majority of WSI, whereas a subset was used for online assessments. An example of our WSI and the Slice viewer interface can be seen via the following link: https://www.best.edu.au/s/ft8pw2tc/l8iw37sx.
2.2. Development of VMATS VMATs were created using the Adaptive e-Learning Platform, an intelligent tutoring system developed by Smart Sparrow (https://www.smartsparrow.com/). Feedback on 3 online pilot cytopathology VMATs was sought from pathology trainees and specialist pathologists. This was taken into account when developing a further 22 VMATs for the trial. Adobe Captivate was used to create short instructional videos to augment some VMATs. The VMAT interface can be seen in Figs. 1 and 2. These VMATs can be accessed via the following links: https:// aelp.smartsparrow.com/bronte/viewer/open/9r43hp68 and https://aelp.smartsparrow.com/bronte/viewer/open/nrreaien.
2.3. Trial design and analysis The trial focused on the main themes of specialist cytopathology training, including gynecology, fine needle aspiration (FNA), and exfoliative/effusion fluid cytopathology. The diagnostic categories of the 22 VMAT-supported cases are listed in Table 1. The trial was approved by the UNSW Medicine Human Research Ethics Committee (HREC 1311). Participants were volunteer anatomical pathology trainees from all states in Australia as well as from New Zealand and Malaysia. They were recruited by advertisement and broadcast email through the college. Informed consent was obtained from all participants, who
Cytopathology WSI and VMATs
1299
Fig. 1
Example of an interactive question within a VMAT.
had the option of withdrawing at any time. The trial was divided into 3 phases, as seen in Table 2, with a crossover between phases and an online assessment at the end of each phase. A crossover design was used, seeking to account for differences between groups with respect to knowledge, experience with cytology, or pretrial familiarity with WSI. Diagnostic categories in the first online assessment included negative smears (including changes associated with microorganisms) and atypical smears (low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial
Fig. 2
lesion, squamous cell carcinoma, and endocervical adenocarcinoma in situ/adenocarcinoma). The second online assessment on FNA cytology included specimens from the thyroid (papillary carcinoma), lymph nodes (metastatic melanoma, non-Hodgkin lymphoma), breast (ductal adenocarcinoma), and lung (adenocarcinoma, squamous cell carcinoma, and small cell carcinoma). The final online assessment on fluid/exfoliative cytology included pleural fluid specimens (reactive, metastatic adenocarcinoma), sputum specimens (normal, small cell carcinoma), bronchial
Example of immediate feedback within a cytopathology VMAT.
1300 Table 1
S. L. Van Es et al. Diagnostic categories of the 22 cases (VMATs and WSI) studied during the trial
Nature of specimen
Diagnostic category/specific diagnosis
Gynecological cytopathology CS/TP
Negative With endocervical cells, mature/immature squamous cells With HSV With Candida albicans Atypical LGSIL (CINI/HPV) HSIL (CINIII) SCC Endometrial adenocarcinoma Endocervical adenocarcinoma
FNA Lung
Granulomatous inflammation Adenocarcinoma Papillary carcinoma Metastatic melanoma NHL Fibroadenoma Ductal adenocarcinoma
Thyroid Lymph node Breast Fluid cytopathology Pleural
Reactive Mesothelioma Metastatic gastric adenocarcinoma Ovarian serous carcinoma Normal Urothelial carcinoma Benign/reactive
Peritoneal Urine Respiratory/BB
Abbreviations: CS, cervical smear; TP, thin prep; HSV, herpes simplex virus; LGSIL, low-grade squamous intraepithelial lesion; CINI, cervical intraepithelial neoplasia I; HPV, human papillomavirus; HSIL, high-grade squamous intraepithelial lesion; CINIII, cervical intraepithelial neoplasia III; SCC, squamous cell carcinoma; FNA, fine needle aspiration; NHL, non-Hodgkin lymphoma; BB, bronchial brushing.
brush specimens (benign), and a urine specimen (urothelial carcinoma). A preliminary power analysis had indicated that, to demonstrate a 30% difference between groups with greater than 90% probability, 9 to 10 participants per group were required for the study. Participants were randomized into 2 groups such that there were equal numbers of junior (years 1 and 2) and senior (years 3, 4, and 5) trainees. Participants were then clustered geographically, each group having an approximate equal number from major metropolitan teaching hospitals and more peripherally located departments. At the commencement of each 2-week phase of the trial, all members of a group received either links to WSI and Table 2
VMATs or a list of diagnoses/disorders to be studied in the traditional way using departmental teaching slide sets and textbooks. Participants were asked to record the amount of time they spent studying cytopathology during each phase. They were also asked to complete online questionnaires on WSI and VMATs as cytopathology educational tools (using a scale of 1 [low] to 10 [high]) and other aspects of their experience (using Likert 5-point scales). The questionnaire also invited open-ended responses. Performance at the end of each phase was measured by a timed (60 minutes) online virtual cytopathology assessment containing links to WSI (which did not always contain the screener's diagnostic marks) using Questionmark Perception
Timeline and format of virtual cytopathology trial for Royal College of Pathologists of Australasia trainees
Topic
Group 1
Gynecological cytology
VMATs and WSI VMATs and WSI Online assessment—gynecological cytology VMATs and WSI Traditional Online assessment—FNA cytology Traditional VMATs and WSI Online assessment—fluid cytology
FNA cytology Fluid/exfoliative cytology
Group 2
Timeline (d) 0-14 15-16 17-31 32-33 34-48 49-50
Abbreviations: RCPA, Royal College of Pathologists of Australasia; FNA, fine needle aspiration; WSI, whole slide images; VMATs, virtual microscopy adaptive tutorials.
Cytopathology WSI and VMATs (Questionmark Computing Ltd, London, UK). A link to an example assessment can be provided upon request. Each assessment covered the theme that had been studied during that phase of the trial by both groups. The assessment following each module was subject matter specific, with the purpose of isolating any potential carry-over effect in this crossover trial design. The assessment could only be attempted once, and participants agreed to adhere to an honor code, without using any outside aids or assistance (eg, consultation with colleagues or access to textbooks or Internet sources). If participants exceeded the time limit, answers were automatically submitted. On average, each assessment comprised 7 WSI with 1 to 3 associated multiple-choice questions. After submission of answers, participants received immediate automated feedback. WSI were used in the assessments rather than glass slides, as participants were distributed widely across Australia, New Zealand, and Malaysia, and given the nature of cytology specimens, it is not possible to distribute identical glass cytopathology slides to all participants. Using whole slide images guaranteed that all participants received identical cytological material to assess, and any difference in assessment scores could not be attributed to any differences in the assessment material. Assessment scores and hours of study were compared using an unpaired Student t test. Data shown are mean ± SEM unless otherwise stated. For Likert scale and rating items in the questionnaire, median ratings were compared using Mann-Whitney U tests. Responses to open-ended questionnaire items were exported into a spreadsheet to facilitate thematic analysis. Two authors (S. L. V. E. and G. M. V.) independently identified common themes. Responses were coded, and emergent themes were subsequently identified.
1301 In phase 3 of the trial, emphasizing fluid cytopathology, only group 2 received digital resources. Again, there was no significant difference between the mean assessment scores of the groups (group 1, 77.3 ± 3.2, n = 12; group 2, 72.5 ± 2.8, n = 17; P = .28) (Fig. 3C). Furthermore, there was no significant difference between groups in their self-reported time spent studying the material (P = .11). Comparing WSI and VMATS for learning, there was a nonsignificant trend toward preferring VMATs (median rating 8 of 10, n = 26) over WSI (7 of 10, n = 23) (P = .06). Responses to Likert scale questions on efficiency and usability of WSI and VMATs are presented in Fig. 4. Qualitative analysis of open-ended questionnaire responses resulted in the emergence of a number of themes. A total of 25 participants commented on the cytology VMATs, with 7 emerging themes in the responses. The dominant positive themes were as follows: the value of VMATs as a high-impact learning tool, followed by convenience and equity of training (Table 3). VMATs were considered by trainees as likely to be “one of the main sources of teaching for the (pathology) trainee
3. Results Forty-three anatomical pathology trainees volunteered for the trial. Of these, 21 were randomized to group 1 (11 from year 1 or 2 and 10 from year 3, 4, or 5), and 22, to group 2 (10 from year 1 or 2 and 12 from year 3, 4, or 5). There was no significant difference between the 2 groups in years of training (P = .4). In phase 1 of the trial, both groups were provided with the WSI and VMATs relating to gynecological cytopathology. There was no difference between the 2 groups in the mean scores of those who completed the first online assessment (group 1, 55.6 ± 5.6, n = 19; group 2, 51.1 ± 6.0, n = 19; P = .5) (Fig. 3A). In phase 2 of the trial, which focused on FNA cytology, only group 1 received the educational material in digital form, that is, WSI and VMATs. The difference between the mean assessment scores of the 2 groups just failed to reach statistical significance (group 1, 91.8 ± 2.3, n = 12; group 2, 82.6 ± 4.3, n = 15; P = .09) (Fig. 3B).
Fig. 3 A to C, Group 1 and group 2 assessment scores after studying: gynecological cytology (A), FNA cytology (B), and fluid/effusion cytology (C).
1302
Fig. 4
S. L. Van Es et al.
Perceived benefit of WSI and VMATs as a cytopathology learning tool, where 5, strongly agree, and 1, strongly disagree.
in the near future.” They made a “huge difference to those of us who are junior” and made it “feel like you are getting a personalized experience compared to traditional teaching.” Trainees also commented that “the interactive nature made it easier to understand and retain the information” and that learning with VMATs was “much more appealing than the WSI alone and reading from a textbook.” The theme of equity emerged with comments such as “even if you didn't have access to teaching material or teachers you can learn via the VMATs.” For the WSI, dominant positive themes were the following: convenience, followed by equity of training and valuable/ high-impact learning (Table 4). WSI were seen by trainees to be very useful for “learning on the go,” with touch screen devices providing pleasing resolution and easy panning. Comments included that equity was ensured by “access to great cases” in every laboratory and “easy access to study from home.” A representative comment on the theme of equity was “I guess the availability of resources in each department is variable. The WSI can make it an even playing field for trainees,” and on the theme of valuable learning, it was “a comprehensive repository of cytology WSI would be invaluable.” An emerging negative theme was that analyzing a cytopathology WSI could be cumbersome and slow in comparison to a traditional glass slide, emerging with comments such as “I found screening (the whole slide image) in the exam really time-consuming.” The lack of 3-dimensional (3-D) focus, although mentioned, was not a strong theme. Table 3
Selection of stronger emerging themes for VMATs a
Themes
Response rate, n (%)
Valuable/high-impact learning Equity of training Convenient/easy access (at home, in-transit) Time efficient Needs to be tailored to level of knowledge/training
45 8 6 5 5
(63) (11) (9) (7) (7)
Abbreviations: VMATs, virtual microscopy adaptive tutorials. a Responses containing multiple themes were given by many participants.
4. Discussion The aim of our trial was to study the use of cytopathology WSI, VMATs, and online assessments incorporating WSI in a postgraduate pathology setting in Australasia. Our trial and the material being tested are novel and cutting edge in a number of ways. Although we are among a handful of groups who have evaluated cytopathology WSI for accuracy and efficiency [6,9,10,25,26,29,34], we are the first group to evaluate virtual cytopathology in the Australasian setting and the only group that has additionally evaluated an online “adaptive” tutoring system to accompany the cytopathology WSI in any setting. The VMATs are novel in that they are specifically designed to “adapt” to the user's decision-making process through remediation of misconceptions. Furthermore, analytics provided by the Adaptive e-Learning Platform provides teachers with evidence of the effectiveness of the adaptive feedback used in the tutorial. This provides scope to constantly monitor, improve, and streamline the feedback provided in each VMAT, mimicking real-life practice more than other currently available online formats. Although there are a small number of other online surgical biopsy and cytopathology tutorials available from institutions worldwide, it is arguable that without detailed analytics and subsequent adaption of
Table 4
Selection of stronger emerging themes for WSI a
Themes
Response rate, n (%)
Convenient (at home, in-transit on iPad) Equity of training Resolutions not as good as traditional microscopy Valuable/high-impact learning Screening the slide tedious and slow Lack of 3-D focus Archiving advantage—no fading, reservoir
23 (19) 19 (16) 19 (15) 16 (13) 10 (8) 9 (7) 7 (6)
Abbreviations: WSI, whole slide images. a Responses containing multiple themes were given by many participants.
Cytopathology WSI and VMATs feedback, the desired dialog between teachers and students in the online environment is diminished. An additional benefit of VMATs is the intuitive user interface. Trial participants were not given prior instruction and did not have prior experience with using the software. However, there were no negative comments about the interface in the open feedback. When specifically asked whether the WSI and VMAT interface was intuitive and easy to use, the majority agreed (Fig. 4). In this study, we found that the diagnostic reliability and accuracy of cytopathology WSI and VMATs for pathology trainees in Australasia were similar to those of traditional methods with glass teaching slides and textbooks. This is in accordance with other studies, which have also found diagnostic accuracy using digital methods to be equal to traditional methods [6,9,17,18,22,28,29]. The equivalence of WSI and VMATs with traditional methods for learning cytopathology is important, particularly because WSI and VMATs were perceived more positively as a way of learning by trainees in comparison to learning with glass slides and textbooks. Users of WSI and VMATS described major advantages: time effectiveness and flexibility, high learning impact, and equity in learning and training. These were perceived benefits in remote and rural pathology departments, in which the number of trainees is increasing. In accordance with previous studies [6,9,29], participants in our trial expressed frustration and uncertainty about the lack of 3-D focus in WSI. However, also consistent with previous reports, we found that proficiency in cytopathology using only 2-dimensional virtual technology compares quite satisfactorily with traditional techniques [6]. Z-stacking an entire cytopathology WSI can produce file sizes in excess of 7 GB [9], resulting in long response times because of network congestion [20]. Therefore, providing z-axis capability on WSI and delivering it efficiently remains a task for the future. Extended-focus imaging may represent an alternative to z-stacking. However, loss of crisp cellular definition can be an issue with this technique [10]. The experience of our participants concurred with previous observations that providing immediate visual feedback enhances the learning experience [32]. Without guidance from a VMAT to analyze and learn from cytopathology WSI, many found it “frustratingly stop-start.” Numerous participants commented that having different diagnostic cells or patterns on cytology WSI highlighted in different colors within the VMAT adaptive feedback helped quite considerably to reinforce the concepts they needed to learn. Some participants in our trial also found that the immediate feedback, not just in the VMATs but also in the in the online assessments, was also very helpful: “the instant feedback at the end of the test was great along with the explanation of the relevant features.” We acknowledge several potential limitations of our trial. Of note, this study had a small sample size, and it is possible that our online assessments may have had insufficient items to reliably demonstrate differences between groups. We did seek to deal with some other potential confounders, such as
1303 ensuring a balance of year training levels in each group as well as assessing for differences in the prior level of knowledge and familiarity with digital slide technology via the “pretest” provided by the online assessment at the end of the first phase. It is possible that carry-over effects might have resulted in reduced observed impact of the WSI and VMATs on assessment scores in the final 2 phases of the trial. However, we believe that distinct differences between the materials covered in each phase of the trial helped to minimize such carry-over effects. Another potential confounder is selection bias: all participants volunteered for the trial, which may reflect an interest in online learning technology that could, in turn, contribute to the positive feedback on the WSI and VMATs. An additional argument is that each postmodule assessment should incorporate glass slides, as this is the ultimate goal of pathology specialist training at present. However, the focus of this study was on the effectiveness of learning using this method of delivery and the materials provided. Moreover, using WSI in the assessments guaranteed that all participants received identical cytological material to assess, so that any difference in assessment scores could not be a result of differences in the assessment material. Lastly, this trial was designed to be a short-term intermediate study. Further studies to look at the long-term effect of this learning methodology, after a significant washout period, may be useful. Nevertheless, with technology constantly and rapidly advancing, it is inevitable that pathology trainees will have increasingly higher expectations of the learning resources supplied in their work environment and by specialist certification bodies responsible for their training. In particular, these expectations include that, in postgraduate education, learning should be available online [1] and programs/resources should be mobile to fit in with the demands of routine work. Additionally, the positive findings associated with the use of VMATs in this study may have broader implications for specialist practice as well, including continuing professional development and maintenance of competence and quality assurance. Adequate training in cytological diagnosis will remain essential for anatomical pathology trainees and specialists alike. Thus, access to large sets of instructive cytopathology WSI and associated VMATs has the potential to provide equitable access to teaching material of a consistent high quality, attempting to mirror real-life settings as much as possible. Additionally, networking and collaborative study for all trainees is promoted, which is of particular importance for those trainees in rural or remote locations.
Distribution of author contributions Trial design, S. L. V. E., G. M. V., R. K. K., W. M. P., and E. L. S.; WSI selection, scanning, and editing, S. L. V. E.; VMAT author, S. L. V. E.; VMAT editing, S. L. V. E. and G. M. V.; online assessments, S. L. V. E. and G. M. V.; statistics, S. L. V. E.
1304 and G. M. V. and W. M. P.; authoring manuscript, S. L. V. E.; editing manuscript, S. L. V. E., G. M. V., R. K. K., W. M. P., and E. L. S.; participant recruitment, W. M. P., S. L. V. E., and G. M. V.
Ethics Before the commencement of this project, ethics approval had been obtained (UNSW HREC 11311 and the Board of Censors Royal College of Pathologists of Australasia).
Acknowledgments Image acquisition: Dr Maria Sarris (Histology and Microscopy Unit, UNSW); technical support: Mr Jake Surman (UNSW), Mr Peter Zarzour (UNSW), Smart Sparrow, the BEST Network; statistics: Dr Michael Bennett (Prince of Wales Hospital, Sydney), Dr Roy Wilson (School of Mathematics & Statistics, UNSW), A/Prof Boaz Shulruf (Medical Education, UNSW); participant recruitment: Ms Vanessa White (Royal College of Pathologists of Australasia); glass cytopathology slides: Ms Joanne La Malfa (Anatomical Pathology Department, Prince of Wales Hospital, Sydney). The authors thank Ms Joanne La Malfa for assistance with cytopathology slides interpretation.
References [1] Khalbuss WE, Pantanowitz L, Parwani AV. Digital imaging in cytopathology. Pathol Res Int 2011;2011:264683. http://dx.doi.org/10. 4061/2011/264683. [2] Isaacs M, Lennerz JK, Yates S, Clermont W, Rossi J, Pfeifer JD. Implementation of whole slide imaging in surgical pathology: a value added approach. J Pathol Inf 2011;2:39. http://dx.doi.org/10.4103/ 2153-3539.84232. [3] Weinstein RS, Graham AR, Richter LC, et al. Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. HUM PATHOL 2009;40:1057-69. [4] Hitchcock CL. The future of telepathology for the developing world. Arch Pathol Lab Med 2011;135:211-4. [5] van den Tweel JG, Bosman FT. The use of virtual slides in the EUROPALS examination. Diagn Pathol 2011;6:S23. http://dx.doi.org/ 10.1186/1746-1596-6-s1-s23. [6] Dee FR, Donnelly A, Radio S, Leaven T, Zaleski MS, Kreiter C. Utility of 2-D and 3-D virtual microscopy in cervical cytology education and testing. Acta Cytol 2007;51:523-9. [7] Kalinski T, Zwonitzer R, Sel S, et al. Virtual 3D microscopy using multiplane whole slide images in diagnostic pathology. Am J Clin Pathol 2008;130:259-64. [8] Zwonitzer R, Hofmann H, Roessner A, Kalinski T. Virtual 3D microscopy in pathology education. HUM PATHOL 2010;41:457-8. [9] Evered A, Dudding N. Accuracy and perceptions of virtual microscopy compared with glass slide microscopy in cervical cytology. Cytopathology 2011;22:82-7. [10] Lee RE, McClintock DS, Laver NM, Yagi Y. Evaluation and optimization for liquid-based preparation cytology in whole slide imaging. J Pathol Inf 2011;2:46. http://dx.doi.org/10.4103/2153-3539.86285.
S. L. Van Es et al. [11] Mori I, Ozaki T, Taniguchi E, Kakudo K. Study of parameters in focus simulation functions of virtual slide. Diagn Pathol 2011;6:S24. http:// dx.doi.org/10.1186/1746-1596-6-s1-s24. [12] Melin-Aldana H, Gasilionis V, Kapur U. Use of digital video for documentation of microscopic features of tissue samples. Arch Pathol Lab Med 2008;132:820-2. [13] Treanor D, Lim CH, Magee D, Bulpitt A, Quirke P. Tracking with virtual slides: a tool to study diagnostic error in histopathology. Histopathology 2009;55:37-45. [14] Mello-Thoms C, Mello CAB, Medvedeva O, et al. Perceptual analysis of the reading of dermatopathology virtual slides by pathology residents. Arch Pathol Lab Med 2012;136:551-62. [15] Krupinski EA, Graham AR, Weinstein RS. Characterizing the development of visual search expertise in pathology residents viewing whole slide images. HUM PATHOL 2013;44:357-64. [16] Blake CA, Lavoie HA, Millette CF. Teaching medical histology at the University of South Carolina School of Medicine: transition to virtual slides and virtual microscopes. Anat Rec B 2003;275B: 196-206. [17] Kumar RK, Velan GM, Korell SO, Kandara M, Dee FR, Wakefield D. Virtual microscopy for learning and assessment in pathology. J Pathol 2004;204:613-8. [18] Kumar RK, Freeman B, Velan GM, De Permentier PJ. Integrating histology and histopathology teaching in practical classes using virtual slides. Anat Rec B 2006;289:128-33. [19] Dee FR, Meyerholz DK. Teaching medical pathology in the twentyfirst century: virtual microscopy applications. J Vet Med Educ 2007; 34:431-6. [20] Dee FR. Virtual microscopy in pathology education. HUM PATHOL 2009;40:1112-21. [21] Fonyad L, Gerely L, Cserneky M, Molnar B, Matolcsy A. Shifting gears higher–digital slides in graduate education—4 years experience at Semmelweis University. Diagn Pathol 2010;5:73. http://dx.doi.org/ 10.1186/1746-1596-5-73. [22] Triola MM, Holloway WJ. Enhanced virtual microscopy for collaborative education. BMC Med Educ 2011;11:4. http://dx.doi. org/10.1186/1472-6920-11-4. [23] Koch LH, Lampros JN, Delong LK, Chen SC, Woosley JT, Hood AF. Randomized comparison of virtual microscopy and traditional glass microscopy in diagnostic accuracy among dermatology and pathology residents. HUM PATHOL 2009;40:662-7. [24] Nielsen PS, Lindebjerg J, Rasmussen J, Starklint H, Waldstrom M, Nielsen B. Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. HUM PATHOL 2010;41:1770-6. [25] Stewart III J, Miyazaki K, Bevans-Wilkins K, Ye C, Kurtycz DF, Selvaggi SM. Virtual microscopy for cytology proficiency testing: are we there yet? Cancer 2007;111:203-9. [26] Gagnon M, Inhorn S, Hancock J, et al. Comparison of cytology proficiency testing: glass slides vs. virtual slides. Acta Cytol 2004;48: 788-94. [27] Wilbur DC, Madi K, Colvin RB, et al. Whole-slide imaging digital pathology as a platform for teleconsultation: a pilot study using paired subspecialist correlations. Arch Pathol Lab Med 2009; 133:1949-53. [28] Marchevsky AM, Wan Y, Thomas P, Krishnan L, Evans-Simon H, Haber H. Virtual microscopy as a tool for proficiency testing in cytopathology: a model using multiple digital images of Papanicolaou tests. Arch Pathol Lab Med 2003;127:1320-4. [29] Steinberg DM, Ali SZ. Application of virtual microscopy in clinical cytopathology. Diagn Cytopathol 2001;25:389-96. [30] Weaker FJ, Herbert DC. Transition of a dental histology course from light to virtual microscopy. J Dent Educ 2009;73:1213-21. [31] Bejjanki VR, Zhang R, Li R, et al. Action video game play facilitates the development of better perceptual templates. PNAS 2014;11: 16961-6.
Cytopathology WSI and VMATs [32] Kanthan R. Digital games designed for pathology education. Lab Invest 2009;89:113A-4A. [33] Velan GM, Ben-Naim D, Kumar RK, Bain M, Kan B, Marcus N. Adaptive tutorials using virtual slides to enhance learning of microscopic morphology. In: Richards G, editor. World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009.
1305 Vancouver, Canada: Chesapeake, VA: AACE; 2009. p. 759-63 [Accessed 22 April 2015, from http://www.editlib.org/p/32546]. [34] Stewart III J, Bevans-Wilkins K, Bhattacharya A, Ye C, Miyazaki K, Kurtycz DF. Virtual microscopy: an educator's tool for the enhancement of cytotechnology students' locator skills. Diagn Cytopathol 2008;36:363-8.