- Email: [email protected]
Does experience in colposcopy improve identification of high grade abnormalities?
European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78
Contents lists available at ScienceDirect
European Journal of Obstetrics & Gynecology and Reproductive Biology journal homepage: www.elsevier.com/locate/ejogrb
Does experience in colposcopy improve identification of high grade abnormalities? Ruud L. Bekkers a,*, Hedwig P. van de Nieuwenhof a, Deborah E. Neesham c,d, Jan H. Hendriks b, Jeff Tan c,d, Michael A. Quinn c,d a
Department of Gynecology/Obstetrics, Radboud University Nijmegen Medical Centre, The Netherlands Department of Epidemiology and Biostatistics, Radboud University Nijmegen Medical Centre, The Netherlands Department of Gynecological Oncology and Dysplasia, Royal Women’s Hospital, Carlton, Victoria, Australia d Department of Obstetrics and Gynaecology, University of Melbourne, Melbourne, Australia b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 20 December 2007 Received in revised form 22 May 2008 Accepted 5 July 2008
Objective: This study investigates whether experience in colposcopy improves identification of high grade abnormalities. The sensitivity and positive predictive value (PPV) of colposcopy in identifying high grade intra-epithelial lesions (HSIL) performed by relatively inexperienced as compared to experienced colposcopists are evaluated. Study design: Of 18,421 colposcopies performed at the Royal Women’s Hospital, Melbourne, Australia, between 1999 and 2004 by 5 senior and 11 junior colposcopists, the colposcopic impression was correlated with the histopathology result of the biopsy taken at 6020 colposcopies, with respect to the experience of the colposcopist. Results: Colposcopy had a 60% sensitivity and 60% PPV in identifying HSIL in this study. In case of a highgrade referral smear the sensitivity and PPV in identifying HSIL were, respectively 76% and 73%, compared with 26% and 48% in case of a low-grade referral smear, no difference in overall colposcopic performance between experienced and inexperienced colposcopists was observed. However, the sensitivity of identifying HSIL was significantly higher with inexperienced colposcopists, and the PPV was significantly higher with experienced colposcopists. Conclusion: In this study experience did not improve colposcopic performance, but differences in colposcopic strategy between the two groups were noted. The rather low overall sensitivity and PPV of colposcopy in identifying HSIL, especially in case of a low-grade referral smear, indicate that the role of colposcopy in the detection and treatment of cervical abnormalities is to assess size, site, and extent of an abnormality, rather than to assess the severity of this abnormality. Histology must remain the gold standard for treatment. ß 2008 Published by Elsevier Ireland Ltd.
Keywords: Experience Colposcopy Identification HSIL
1. Introduction Cancer of the uterine cervix is the second most frequent invasive cancer and the major cause of cancer deaths in women worldwide. With the introduction of population-based screening programs, both the incidence and the mortality of cervical cancer have been reduced. This reduction is mainly the result of diagnosis and treatment of pre-malignant lesions of the uterine cervix [1].
* Corresponding author at: Department of Gynecology/Obstetrics 791, Radboud University Nijmegen Medical Centre, PO Box 9101, 6500 HB Nijmegen, The Netherlands. Tel.: +31 24 3614725; fax: +31 24 3668597. E-mail address: [email protected] (R.L. Bekkers). 0301-2115/$ – see front matter ß 2008 Published by Elsevier Ireland Ltd. doi:10.1016/j.ejogrb.2008.07.007
Women with an abnormal smear result in screening programs are generally referred for colposcopy [1,2]. The role of colposcopy is to assess size, site and extent of a lesion, and select the most severe part of the abnormality to take a biopsy [3,4]. Alternatively, colposcopy is often used to assess the severity of the abnormality as part of a ‘see and treat’ policy [5,6]. Interpreting colposcopic epithelial patterns and subsequently selecting the site for biopsy is likely to be a subjective procedure that is strongly correlated to the colposcopist’s skill and experience [7–9]. A meta-analysis has shown that colposcopy has a 57% positive predictive value (PPV) in detecting high grade intraepithelial lesions (HSIL), however the PPV in these studies varied between 20% and 84% [10]. For low-grade squamous intraepithelial lesions (LSIL) the PPV is lower [3,10].
76
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78
Many professionals and professional bodies claim that experience in colposcopy is the main deciding factor in correctly identifying the severity of a lesion [7,10]. Guidelines indicate that a minimum number of colposcopies is needed to acquire and maintain colposcopy skills. In some countries, specific training and examination programs are compulsory before practitioners can be certified as a colposcopist [2,11,12]. In other countries no specific certification with local professional bodies is required [13,14]. With this study we wanted to investigate whether experience in colposcopy improves identification of high grade abnormalities. For this reason, the sensitivity and PPV of colposcopy in identifying HSIL performed by relatively inexperienced colposcopists is compared to that of experienced colposcopists. 2. Methods
Table 1 The performance of colposcopy of the total group in identifying HSIL compared with the biopsy result Colposcopic assessment
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
1036 399 275
671 2234 1405
1707 2633 1680
Total
1710
4310
6020
Sensitivity of colposcopy to identify HSIL = 60.7% (1036 of 1707). PPV of colposcopy suggestive of HSIL = 60.6% (1036 of 1710).
Table 2 The performance of colposcopy in identifying HSIL for junior and senior colposcopists compared with the biopsy result Colposcopic assessment
From the database of the Royal Women’s Hospital (RWH), Carlton, Victoria, Australia, all 18,421 satisfactory colposcopies performed between 1999 and 2004 by 5 senior and 11 junior colposcopists, were analysed. These colposcopies were performed in both newly referred women with an abnormal Papanicolaou (Pap) smear, as well as women who came for follow up. Colposcopists in Australia are trained in colposcopy as a compulsory part of their obstetrics and gynaecology training, without specific examination and certification. Senior colposcopists were defined as colposcopists with >5 years of experience in colposcopy, performing >100 colposcopies/ year. Junior colposcopists all had 0–2 years of experience in colposcopy. During the study period none of the junior colposcopists worked longer than 3 years in the Dysplasia Department of the RWH, so no junior became a senior during the study period. At the start of the study, junior colposcopists had seen between approximately 50 and 150 new patients with cervical abnormalities, compared to approximately 100–300 at the end of their participation in the study. After exclusion of unsatisfactory colposcopies, 18,421 colposcopies were analysed of which 7000 were performed by junior colposcopists and 11,421 by senior colposcopists. Punch biopsies were taken during 6020 colposcopies and these data were further analysed. The histopathological results of the punch biopsies were used as the gold standard to calculate the PPV, as well as the sensitivity in detecting HSIL. The biopsy result was used because, if further treatment was deemed necessary, many women underwent laser treatment without a new histopathological specimen. At the end of each colposcopy, the colposcopists had to enter their colposcopic assessment in the database indicating the severity of the lesion, without having seen the histopathological outcome of the biopsy. It was impossible for colposcopists to alter data entered in the database at a later stage. The correlation between the colposcopic assessment and the histopathological outcome of the biopsy was studied for all colposcopists and comparisons were made between juniors and seniors, as well as between the first and last year for each colposcopist. Additionally, a comparison was made between referrals with a low grade abnormal smear, and referrals with a high grade abnormal smear. Statistical analyses was performed, using 95% confidence intervals (95% CI), x2-, McNemar-, and Kappa-tests. A value of p < 0.05 was considered significant. 3. Results Histopathological examination showed HSIL in 1707 (28%) of the 6020 biopsies, while LSIL was detected in 2633 (44%) biopsies, and no dysplasia in 1680 (28%).
Juniors HSIL
Seniors LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
408 184 136
208 855 582
616 1039 718
628 215 139
463 1379 823
1091 1594 962
Total
728
1645
2373
982
2665
3647
Sensitivity of colposcopy to identify HSIL is 66.7% (408 of 616) for juniors, and 57.5% (628 of 1091) for seniors (p < 0.001); PPV of colposcopy suggestive of HSIL is 56% (408 of 728) for juniors and 64% (628 of 982) for seniors (p < 0.001); the difference in sensitivity = 9.2% (95%CI 4.3–14.1%); the difference in PPV = 8.0% (95%CI 3.2– 12.8%).
The comparison of the colposcopic assessment and the histopathology of the biopsy is presented in Table 1. Of the total group of 1707 women with a histopathological diagnosis of HSIL, HSIL was predicted at colposcopy in 1036 women, indicating a sensitivity for the total group of 60.7%. In 1710 women the colposcopist interpreted the lesion as a HSIL lesion, which was correct in 1036, indicating a PPV of 60.6% in identifying HSIL at colposcopy. Table 2 shows the comparison between junior and senior colposcopists. Colposcopy by junior colposcopists had a significantly higher sensitivity (p < 0.001) for identifying HSIL (66.7% vs. 57.5%), but a significantly lower PPV (56% vs. 64%) (p < 0.001), compared with senior colposcopists (see Table 2). In order to assess whether a learning curve was present, the results of both juniors and seniors in their first and last year registered in the database were compared (see Tables 3 and 4). The juniors in their first and last year had, respectively a sensitivity of 68.8%, and 63.8%, and a PPV of 58.8%, and 53.8%. These differences were not significant. The seniors in their first and last year had, respectively a sensitivity of 56.4% and 57.3%, and a PPV of 64.8% and 66.6%. These differences were also not significant. Table 3 The performance of junior colposcopists in their first and last year Colposcopic assessment
First year
Last year
HSIL
LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
163 70 44
74 330 207
237 400 251
134 58 57
76 280 264
210 338 321
Total
277
611
888
249
620
869
Sensitivity of colposcopy to identify HSIL is 68.8% (163 of 237) for juniors in the first year, and 63.8% (134 of 210) for juniors in the last year (not significant, 95%CI of this difference incorporates 0); PPV of colposcopy suggestive of HSIL is 58.8% (163 of 277) for juniors in the first year, and 53.8% 134 of 249) for juniors in the last year (not significant, 95% CI of this difference incorporates 0).
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78 Table 4 The performance of senior colposcopists in their first and last year Colposcopic assessment
First year
Last year
HSIL
LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy Total
166 59 31 256
128 299 165 592
294 358 196 848
82 23 18 123
61 155 136 352
143 178 154 475
Sensitivity of colposcopy to identify HSIL is 56.4% (166 of 294) for seniors in the first year, and 57.3% (82 of 143) for seniors in the last year (not significant, 95%CI of this difference incorporates 0); PPV of colposcopy suggestive of HSIL is 64.8% (166 of 256) for seniors in the first year, and 66.6% (82 of 123) for seniors in the last year (not significant, 95%CI of this difference incorporates 0).
Table 5 Kappa values for colposcopy compared with biopsy for all 18,421 colposcopies First year
Juniors Seniors All
Last year
All colposcopies
Kappa
95%CI
Kappa
95%CI
Kappa
95%CI
0.58a 0.53b
0.53–0.64 0.47–0.58
0.53a 0.57b
0.47–0.59 0.49–0.64
0.55a 0.55b 0.55c
0.52–0.59 0.52–0.58 0.53–0.57
a McNemar test, p < 0.01 (chances of HSIL with colposcopy > chances of HSIL with biopsy). b McNemar test, p = 0.01 (chances of HSIL with colposcopy < chances of HSIL with biopsy). c McNemar test, p = 0.93.
Table 5 shows that there are no significant differences in Kappa values between junior and senior colposcopists. However, the chances of correctly identifying HSIL by colposcopy by juniors were significantly higher than having a HSIL biopsy, while for seniors the opposite was true. Referral smears were registered in the database in 3510 cases. In patients referred with a high-grade abnormality in the cervical smear, the sensitivity of colposcopy was 76%, and the PPV was 73% for the detection of HSIL. In patients referred with a low grade abnormality in the referral smear, the sensitivity and PPV were, respectively 26% and 48%. Junior colposcopists had a significantly higher sensitivity, and a similar PPV than senior colposcopists. Furthermore, in patients referred with a high grade smear, the sensitivity and PPV for identifying LSIL correctly were, respectively 46%, and 34%. 4. Discussion and conclusions Meta-analysis suggests a high average weighted sensitivity of colposcopy in identifying HSIL of 85%, however sensitivities in studies varied from 30% to 99% [10]. In this study an overall sensitivity of only 60% was found, based on over 6000 colposcopies with colposcopic guided biopsies. This difference in sensitivity may be explained by the relatively low prevalence of HSIL (28%) in this group of 6020 patients. Additionally, this study investigated retrospectively the colposcopy practice in a real-life situation. The colposcopists were not taking part in a study, a factor that may have positively influenced other study results with higher sensitivities [10]. Recent studies also found a low sensitivity for colposcopy in correctly identifying HSIL [9,15,16]. In fact, the 60% sensitivity found in this study may even be an overestimation of the true sensitivity, because additional cases of HSIL may have remained undetected in those patients in whom no biopsy was taken. Finally, the use of colposcopic guided biopsies as the gold standard may have resulted in this low sensitivity. Several studies have reported sensitivities of punch biopsies in correctly identifying HSIL in only 69–82% [5,6,15]. However, biopsies are
77
ideally taken from the most severely visually abnormal part of the lesion. If a biopsy under-represents the severity of the lesion, it has obviously been taken from the wrong site, indicating a poor colposcopic performance [5,15,17]. The PPV of colposcopy in identifying HSIL was 60% in this study. This is lower than reported by some [3,5,16], while others described a similar PPV of 57% in their meta-analysis [10]. Several authors have described a correlation between the PPV and the severity of the lesion being assessed. The highest PPVs were found in cases with the most severe SIL lesions, but not in those with micro-invasive disease [3,10]. The PPV is also related to the prevalence of HSIL in the study group. The low number of HSIL patients in this study may thus have influenced the PPV negatively. The overall colposcopy performance of both junior and senior colposcopists did not differ as illustrated by a Kappa value of 0.55 in both groups. This Kappa value is comparable to the Kappa values of other studies [4,8], which compared different colposcopists who assessed the same set of colposcopic images. Others also found no differences between more and less experienced colposcopists [4,15,16]. Despite a comparable overall performance between junior and senior colposcopists, significant differences in sensitivity and PPV were detected. Junior colposcopists had a significantly higher sensitivity, but a significantly lower PPV than senior colposcopists. This indicates that although the overall performance does not differ, there is a difference in colposcopic strategy. Juniors, feeling less sure may tend to more often call a lesion HSIL, and to more often take a biopsy when in doubt. This results in a higher sensitivity, but at the cost of a lower PPV [10]. In seniors the opposite may be true, a higher confidence in colposcopic assessment leads to a higher PPV at the cost of a lower sensitivity. No learning curve in colposcopic performance was observed in these groups. This lack of learning curve may be caused by the fact that juniors had sufficient training in colposcopy before starting colposcopies in the RWH. On the other hand, the fact that cytology results were known at time of the colposcopy, may have influenced the colposcopic assessment. Some studies have shown a high correlation between cytology and histopathology results of cervical abnormalities [7,17]. So, a bias in colposcopic assessment towards the cytology result may have masked any learning effect in colposcopic performance, as well as differences between junior and senior colposcopists. Indeed, knowledge about the referral smear had a high impact on the sensitivity and PPV of colposcopy in this study. The subanalysis showed that the sensitivity and PPV of colposcopy in identifying HSIL are higher in patients with a high grade abnormality in the referral smear, while both are very poor (26% and 48%, respectively) in patients with a low grade abnormality in the referral smear. In general, this study, as well as other studies [3,15–17] shows that colposcopy is a rather poor instrument in the assessment of the severity of cervical lesions. On the other hand, colposcopy remains very useful in assessing the size, site, and extent of cervical lesions, which are important factors in deciding on the best method of treatment in each individual patient. 5. Conclusion No difference in overall colposcopic performance between experienced and inexperienced colposcopists was observed in this study, but the sensitivity to identify HSIL was higher for inexperienced, and the PPV was higher for experienced colposcopists. This indicates a difference in colposcopic strategy between these groups. The rather low overall sensitivity (60%) and PPV (60%) of colposcopy in identifying HSIL, even worse in case of a low-grade referral smear, indicates that the role of colposcopy in
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78
78
detection and treatment of cervical abnormalities is to assess size, site, and extent of the lesion, rather than to assess the severity of an abnormality. Acknowledgements None. Conflict of interest None. References [1] Bekkers RL, Massuger LF, Bulten J, Melchers WJ. Epidemiological and clinical aspects of human papillomavirus detection in the prevention of cervical cancer. Rev Med Virol 2004;14(2):95–105. [2] Wright Jr TC, Cox JT, Massad LS, Twiggs LB, Wilkinson EJ. Consensus guidelines for the management of women with cervical cytological abnormalities. JAMA 2002;287(April (16)):2120–9. [3] Hopman EH, Kenemans P, Helmerhorst TJ. Positive predictive rate of colposcopic examination of the cervix uteri: an overview of literature. Obstet Gynecol Surv 1998;53(2):97–106. [4] Sheshadri V, O’Connor D. The agreement of colposcopic grading as compared to directed biopsy results. Journal of Lower Genital Tract Disease 1999;3(3):150–4 (ref type: Journal (full)).
[5] Byrom J, Douce G, Jones PW, et al. Should punch biopsies be used when highgrade disease is suspected at initial colposcopic assessment? A prospective study. Int J Gynecol Cancer 2006;16(1):253–6. [6] Denny LA, Soeters R, Dehaeck K, Bloch B. Does colposcopically directed punch biopsy reduce the incidence of negative LLETZ? Br J Obstet Gynaecol 1995;102(7):545–8. [7] Benedet JL, Matisic JP, Bertrand MA. The quality of community colposcopic practice. Obstet Gynecol 2004;103(1):92–100. [8] Hopman EH, Voorhorst FJ, Kenemans P, Meyer CJ, Helmerhorst TJ. Observer agreement on interpreting colposcopic images of CIN. Gynecol Oncol 1995;58(2):206–9. [9] Sideri M, Spolti N, Spinaci L, et al. Interobserver variability of colposcopic interpretations and consistency with final histologic results. J Low Genit Tract Dis 2004;8(3):212–6. [10] Mitchell MF, Schottenfeld D, Tortolero-Luna G, Cantor SB, Richards-Kortum R. Colposcopy for the diagnosis of squamous intraepithelial lesions: a metaanalysis. Obstet Gynecol 1998;91(4):626–31. [11] American Society for colposcopy and cervical pathology. Available from www.asccp.org. [12] The British Society of colposcopy and cervical pathology. Available from www.bsccp.org.uk. [13] Dutch oncology guidelines. Available from www.oncoline.nl. [14] Australian Society for colposcopy and cervical pathology. Available from www.asccp.com.au. [15] Gage JC, Hanson VW, Abbey K, et al. Number of cervical biopsies and sensitivity of colposcopy. Obstet Gynecol 2006;108(2):264–72. [16] Massad LS, Collins YC. Strength of correlations between colposcopic impression and biopsy histology. Gynecol Oncol 2003;89(3):424–8. [17] Massad LS. More is more: improving the sensitivity of colposcopy. Obstet Gynecol 2006;108(2):246–7.
Contents lists available at ScienceDirect
European Journal of Obstetrics & Gynecology and Reproductive Biology journal homepage: www.elsevier.com/locate/ejogrb
Does experience in colposcopy improve identification of high grade abnormalities? Ruud L. Bekkers a,*, Hedwig P. van de Nieuwenhof a, Deborah E. Neesham c,d, Jan H. Hendriks b, Jeff Tan c,d, Michael A. Quinn c,d a
Department of Gynecology/Obstetrics, Radboud University Nijmegen Medical Centre, The Netherlands Department of Epidemiology and Biostatistics, Radboud University Nijmegen Medical Centre, The Netherlands Department of Gynecological Oncology and Dysplasia, Royal Women’s Hospital, Carlton, Victoria, Australia d Department of Obstetrics and Gynaecology, University of Melbourne, Melbourne, Australia b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 20 December 2007 Received in revised form 22 May 2008 Accepted 5 July 2008
Objective: This study investigates whether experience in colposcopy improves identification of high grade abnormalities. The sensitivity and positive predictive value (PPV) of colposcopy in identifying high grade intra-epithelial lesions (HSIL) performed by relatively inexperienced as compared to experienced colposcopists are evaluated. Study design: Of 18,421 colposcopies performed at the Royal Women’s Hospital, Melbourne, Australia, between 1999 and 2004 by 5 senior and 11 junior colposcopists, the colposcopic impression was correlated with the histopathology result of the biopsy taken at 6020 colposcopies, with respect to the experience of the colposcopist. Results: Colposcopy had a 60% sensitivity and 60% PPV in identifying HSIL in this study. In case of a highgrade referral smear the sensitivity and PPV in identifying HSIL were, respectively 76% and 73%, compared with 26% and 48% in case of a low-grade referral smear, no difference in overall colposcopic performance between experienced and inexperienced colposcopists was observed. However, the sensitivity of identifying HSIL was significantly higher with inexperienced colposcopists, and the PPV was significantly higher with experienced colposcopists. Conclusion: In this study experience did not improve colposcopic performance, but differences in colposcopic strategy between the two groups were noted. The rather low overall sensitivity and PPV of colposcopy in identifying HSIL, especially in case of a low-grade referral smear, indicate that the role of colposcopy in the detection and treatment of cervical abnormalities is to assess size, site, and extent of an abnormality, rather than to assess the severity of this abnormality. Histology must remain the gold standard for treatment. ß 2008 Published by Elsevier Ireland Ltd.
Keywords: Experience Colposcopy Identification HSIL
1. Introduction Cancer of the uterine cervix is the second most frequent invasive cancer and the major cause of cancer deaths in women worldwide. With the introduction of population-based screening programs, both the incidence and the mortality of cervical cancer have been reduced. This reduction is mainly the result of diagnosis and treatment of pre-malignant lesions of the uterine cervix [1].
* Corresponding author at: Department of Gynecology/Obstetrics 791, Radboud University Nijmegen Medical Centre, PO Box 9101, 6500 HB Nijmegen, The Netherlands. Tel.: +31 24 3614725; fax: +31 24 3668597. E-mail address: [email protected] (R.L. Bekkers). 0301-2115/$ – see front matter ß 2008 Published by Elsevier Ireland Ltd. doi:10.1016/j.ejogrb.2008.07.007
Women with an abnormal smear result in screening programs are generally referred for colposcopy [1,2]. The role of colposcopy is to assess size, site and extent of a lesion, and select the most severe part of the abnormality to take a biopsy [3,4]. Alternatively, colposcopy is often used to assess the severity of the abnormality as part of a ‘see and treat’ policy [5,6]. Interpreting colposcopic epithelial patterns and subsequently selecting the site for biopsy is likely to be a subjective procedure that is strongly correlated to the colposcopist’s skill and experience [7–9]. A meta-analysis has shown that colposcopy has a 57% positive predictive value (PPV) in detecting high grade intraepithelial lesions (HSIL), however the PPV in these studies varied between 20% and 84% [10]. For low-grade squamous intraepithelial lesions (LSIL) the PPV is lower [3,10].
76
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78
Many professionals and professional bodies claim that experience in colposcopy is the main deciding factor in correctly identifying the severity of a lesion [7,10]. Guidelines indicate that a minimum number of colposcopies is needed to acquire and maintain colposcopy skills. In some countries, specific training and examination programs are compulsory before practitioners can be certified as a colposcopist [2,11,12]. In other countries no specific certification with local professional bodies is required [13,14]. With this study we wanted to investigate whether experience in colposcopy improves identification of high grade abnormalities. For this reason, the sensitivity and PPV of colposcopy in identifying HSIL performed by relatively inexperienced colposcopists is compared to that of experienced colposcopists. 2. Methods
Table 1 The performance of colposcopy of the total group in identifying HSIL compared with the biopsy result Colposcopic assessment
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
1036 399 275
671 2234 1405
1707 2633 1680
Total
1710
4310
6020
Sensitivity of colposcopy to identify HSIL = 60.7% (1036 of 1707). PPV of colposcopy suggestive of HSIL = 60.6% (1036 of 1710).
Table 2 The performance of colposcopy in identifying HSIL for junior and senior colposcopists compared with the biopsy result Colposcopic assessment
From the database of the Royal Women’s Hospital (RWH), Carlton, Victoria, Australia, all 18,421 satisfactory colposcopies performed between 1999 and 2004 by 5 senior and 11 junior colposcopists, were analysed. These colposcopies were performed in both newly referred women with an abnormal Papanicolaou (Pap) smear, as well as women who came for follow up. Colposcopists in Australia are trained in colposcopy as a compulsory part of their obstetrics and gynaecology training, without specific examination and certification. Senior colposcopists were defined as colposcopists with >5 years of experience in colposcopy, performing >100 colposcopies/ year. Junior colposcopists all had 0–2 years of experience in colposcopy. During the study period none of the junior colposcopists worked longer than 3 years in the Dysplasia Department of the RWH, so no junior became a senior during the study period. At the start of the study, junior colposcopists had seen between approximately 50 and 150 new patients with cervical abnormalities, compared to approximately 100–300 at the end of their participation in the study. After exclusion of unsatisfactory colposcopies, 18,421 colposcopies were analysed of which 7000 were performed by junior colposcopists and 11,421 by senior colposcopists. Punch biopsies were taken during 6020 colposcopies and these data were further analysed. The histopathological results of the punch biopsies were used as the gold standard to calculate the PPV, as well as the sensitivity in detecting HSIL. The biopsy result was used because, if further treatment was deemed necessary, many women underwent laser treatment without a new histopathological specimen. At the end of each colposcopy, the colposcopists had to enter their colposcopic assessment in the database indicating the severity of the lesion, without having seen the histopathological outcome of the biopsy. It was impossible for colposcopists to alter data entered in the database at a later stage. The correlation between the colposcopic assessment and the histopathological outcome of the biopsy was studied for all colposcopists and comparisons were made between juniors and seniors, as well as between the first and last year for each colposcopist. Additionally, a comparison was made between referrals with a low grade abnormal smear, and referrals with a high grade abnormal smear. Statistical analyses was performed, using 95% confidence intervals (95% CI), x2-, McNemar-, and Kappa-tests. A value of p < 0.05 was considered significant. 3. Results Histopathological examination showed HSIL in 1707 (28%) of the 6020 biopsies, while LSIL was detected in 2633 (44%) biopsies, and no dysplasia in 1680 (28%).
Juniors HSIL
Seniors LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
408 184 136
208 855 582
616 1039 718
628 215 139
463 1379 823
1091 1594 962
Total
728
1645
2373
982
2665
3647
Sensitivity of colposcopy to identify HSIL is 66.7% (408 of 616) for juniors, and 57.5% (628 of 1091) for seniors (p < 0.001); PPV of colposcopy suggestive of HSIL is 56% (408 of 728) for juniors and 64% (628 of 982) for seniors (p < 0.001); the difference in sensitivity = 9.2% (95%CI 4.3–14.1%); the difference in PPV = 8.0% (95%CI 3.2– 12.8%).
The comparison of the colposcopic assessment and the histopathology of the biopsy is presented in Table 1. Of the total group of 1707 women with a histopathological diagnosis of HSIL, HSIL was predicted at colposcopy in 1036 women, indicating a sensitivity for the total group of 60.7%. In 1710 women the colposcopist interpreted the lesion as a HSIL lesion, which was correct in 1036, indicating a PPV of 60.6% in identifying HSIL at colposcopy. Table 2 shows the comparison between junior and senior colposcopists. Colposcopy by junior colposcopists had a significantly higher sensitivity (p < 0.001) for identifying HSIL (66.7% vs. 57.5%), but a significantly lower PPV (56% vs. 64%) (p < 0.001), compared with senior colposcopists (see Table 2). In order to assess whether a learning curve was present, the results of both juniors and seniors in their first and last year registered in the database were compared (see Tables 3 and 4). The juniors in their first and last year had, respectively a sensitivity of 68.8%, and 63.8%, and a PPV of 58.8%, and 53.8%. These differences were not significant. The seniors in their first and last year had, respectively a sensitivity of 56.4% and 57.3%, and a PPV of 64.8% and 66.6%. These differences were also not significant. Table 3 The performance of junior colposcopists in their first and last year Colposcopic assessment
First year
Last year
HSIL
LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy
163 70 44
74 330 207
237 400 251
134 58 57
76 280 264
210 338 321
Total
277
611
888
249
620
869
Sensitivity of colposcopy to identify HSIL is 68.8% (163 of 237) for juniors in the first year, and 63.8% (134 of 210) for juniors in the last year (not significant, 95%CI of this difference incorporates 0); PPV of colposcopy suggestive of HSIL is 58.8% (163 of 277) for juniors in the first year, and 53.8% 134 of 249) for juniors in the last year (not significant, 95% CI of this difference incorporates 0).
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78 Table 4 The performance of senior colposcopists in their first and last year Colposcopic assessment
First year
Last year
HSIL
LSIL
Total
HSIL
LSIL
Total
HSIL biopsy LSIL biopsy NEG biopsy Total
166 59 31 256
128 299 165 592
294 358 196 848
82 23 18 123
61 155 136 352
143 178 154 475
Sensitivity of colposcopy to identify HSIL is 56.4% (166 of 294) for seniors in the first year, and 57.3% (82 of 143) for seniors in the last year (not significant, 95%CI of this difference incorporates 0); PPV of colposcopy suggestive of HSIL is 64.8% (166 of 256) for seniors in the first year, and 66.6% (82 of 123) for seniors in the last year (not significant, 95%CI of this difference incorporates 0).
Table 5 Kappa values for colposcopy compared with biopsy for all 18,421 colposcopies First year
Juniors Seniors All
Last year
All colposcopies
Kappa
95%CI
Kappa
95%CI
Kappa
95%CI
0.58a 0.53b
0.53–0.64 0.47–0.58
0.53a 0.57b
0.47–0.59 0.49–0.64
0.55a 0.55b 0.55c
0.52–0.59 0.52–0.58 0.53–0.57
a McNemar test, p < 0.01 (chances of HSIL with colposcopy > chances of HSIL with biopsy). b McNemar test, p = 0.01 (chances of HSIL with colposcopy < chances of HSIL with biopsy). c McNemar test, p = 0.93.
Table 5 shows that there are no significant differences in Kappa values between junior and senior colposcopists. However, the chances of correctly identifying HSIL by colposcopy by juniors were significantly higher than having a HSIL biopsy, while for seniors the opposite was true. Referral smears were registered in the database in 3510 cases. In patients referred with a high-grade abnormality in the cervical smear, the sensitivity of colposcopy was 76%, and the PPV was 73% for the detection of HSIL. In patients referred with a low grade abnormality in the referral smear, the sensitivity and PPV were, respectively 26% and 48%. Junior colposcopists had a significantly higher sensitivity, and a similar PPV than senior colposcopists. Furthermore, in patients referred with a high grade smear, the sensitivity and PPV for identifying LSIL correctly were, respectively 46%, and 34%. 4. Discussion and conclusions Meta-analysis suggests a high average weighted sensitivity of colposcopy in identifying HSIL of 85%, however sensitivities in studies varied from 30% to 99% [10]. In this study an overall sensitivity of only 60% was found, based on over 6000 colposcopies with colposcopic guided biopsies. This difference in sensitivity may be explained by the relatively low prevalence of HSIL (28%) in this group of 6020 patients. Additionally, this study investigated retrospectively the colposcopy practice in a real-life situation. The colposcopists were not taking part in a study, a factor that may have positively influenced other study results with higher sensitivities [10]. Recent studies also found a low sensitivity for colposcopy in correctly identifying HSIL [9,15,16]. In fact, the 60% sensitivity found in this study may even be an overestimation of the true sensitivity, because additional cases of HSIL may have remained undetected in those patients in whom no biopsy was taken. Finally, the use of colposcopic guided biopsies as the gold standard may have resulted in this low sensitivity. Several studies have reported sensitivities of punch biopsies in correctly identifying HSIL in only 69–82% [5,6,15]. However, biopsies are
77
ideally taken from the most severely visually abnormal part of the lesion. If a biopsy under-represents the severity of the lesion, it has obviously been taken from the wrong site, indicating a poor colposcopic performance [5,15,17]. The PPV of colposcopy in identifying HSIL was 60% in this study. This is lower than reported by some [3,5,16], while others described a similar PPV of 57% in their meta-analysis [10]. Several authors have described a correlation between the PPV and the severity of the lesion being assessed. The highest PPVs were found in cases with the most severe SIL lesions, but not in those with micro-invasive disease [3,10]. The PPV is also related to the prevalence of HSIL in the study group. The low number of HSIL patients in this study may thus have influenced the PPV negatively. The overall colposcopy performance of both junior and senior colposcopists did not differ as illustrated by a Kappa value of 0.55 in both groups. This Kappa value is comparable to the Kappa values of other studies [4,8], which compared different colposcopists who assessed the same set of colposcopic images. Others also found no differences between more and less experienced colposcopists [4,15,16]. Despite a comparable overall performance between junior and senior colposcopists, significant differences in sensitivity and PPV were detected. Junior colposcopists had a significantly higher sensitivity, but a significantly lower PPV than senior colposcopists. This indicates that although the overall performance does not differ, there is a difference in colposcopic strategy. Juniors, feeling less sure may tend to more often call a lesion HSIL, and to more often take a biopsy when in doubt. This results in a higher sensitivity, but at the cost of a lower PPV [10]. In seniors the opposite may be true, a higher confidence in colposcopic assessment leads to a higher PPV at the cost of a lower sensitivity. No learning curve in colposcopic performance was observed in these groups. This lack of learning curve may be caused by the fact that juniors had sufficient training in colposcopy before starting colposcopies in the RWH. On the other hand, the fact that cytology results were known at time of the colposcopy, may have influenced the colposcopic assessment. Some studies have shown a high correlation between cytology and histopathology results of cervical abnormalities [7,17]. So, a bias in colposcopic assessment towards the cytology result may have masked any learning effect in colposcopic performance, as well as differences between junior and senior colposcopists. Indeed, knowledge about the referral smear had a high impact on the sensitivity and PPV of colposcopy in this study. The subanalysis showed that the sensitivity and PPV of colposcopy in identifying HSIL are higher in patients with a high grade abnormality in the referral smear, while both are very poor (26% and 48%, respectively) in patients with a low grade abnormality in the referral smear. In general, this study, as well as other studies [3,15–17] shows that colposcopy is a rather poor instrument in the assessment of the severity of cervical lesions. On the other hand, colposcopy remains very useful in assessing the size, site, and extent of cervical lesions, which are important factors in deciding on the best method of treatment in each individual patient. 5. Conclusion No difference in overall colposcopic performance between experienced and inexperienced colposcopists was observed in this study, but the sensitivity to identify HSIL was higher for inexperienced, and the PPV was higher for experienced colposcopists. This indicates a difference in colposcopic strategy between these groups. The rather low overall sensitivity (60%) and PPV (60%) of colposcopy in identifying HSIL, even worse in case of a low-grade referral smear, indicates that the role of colposcopy in
R.L. Bekkers et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 141 (2008) 75–78
78
detection and treatment of cervical abnormalities is to assess size, site, and extent of the lesion, rather than to assess the severity of an abnormality. Acknowledgements None. Conflict of interest None. References [1] Bekkers RL, Massuger LF, Bulten J, Melchers WJ. Epidemiological and clinical aspects of human papillomavirus detection in the prevention of cervical cancer. Rev Med Virol 2004;14(2):95–105. [2] Wright Jr TC, Cox JT, Massad LS, Twiggs LB, Wilkinson EJ. Consensus guidelines for the management of women with cervical cytological abnormalities. JAMA 2002;287(April (16)):2120–9. [3] Hopman EH, Kenemans P, Helmerhorst TJ. Positive predictive rate of colposcopic examination of the cervix uteri: an overview of literature. Obstet Gynecol Surv 1998;53(2):97–106. [4] Sheshadri V, O’Connor D. The agreement of colposcopic grading as compared to directed biopsy results. Journal of Lower Genital Tract Disease 1999;3(3):150–4 (ref type: Journal (full)).
[5] Byrom J, Douce G, Jones PW, et al. Should punch biopsies be used when highgrade disease is suspected at initial colposcopic assessment? A prospective study. Int J Gynecol Cancer 2006;16(1):253–6. [6] Denny LA, Soeters R, Dehaeck K, Bloch B. Does colposcopically directed punch biopsy reduce the incidence of negative LLETZ? Br J Obstet Gynaecol 1995;102(7):545–8. [7] Benedet JL, Matisic JP, Bertrand MA. The quality of community colposcopic practice. Obstet Gynecol 2004;103(1):92–100. [8] Hopman EH, Voorhorst FJ, Kenemans P, Meyer CJ, Helmerhorst TJ. Observer agreement on interpreting colposcopic images of CIN. Gynecol Oncol 1995;58(2):206–9. [9] Sideri M, Spolti N, Spinaci L, et al. Interobserver variability of colposcopic interpretations and consistency with final histologic results. J Low Genit Tract Dis 2004;8(3):212–6. [10] Mitchell MF, Schottenfeld D, Tortolero-Luna G, Cantor SB, Richards-Kortum R. Colposcopy for the diagnosis of squamous intraepithelial lesions: a metaanalysis. Obstet Gynecol 1998;91(4):626–31. [11] American Society for colposcopy and cervical pathology. Available from www.asccp.org. [12] The British Society of colposcopy and cervical pathology. Available from www.bsccp.org.uk. [13] Dutch oncology guidelines. Available from www.oncoline.nl. [14] Australian Society for colposcopy and cervical pathology. Available from www.asccp.com.au. [15] Gage JC, Hanson VW, Abbey K, et al. Number of cervical biopsies and sensitivity of colposcopy. Obstet Gynecol 2006;108(2):264–72. [16] Massad LS, Collins YC. Strength of correlations between colposcopic impression and biopsy histology. Gynecol Oncol 2003;89(3):424–8. [17] Massad LS. More is more: improving the sensitivity of colposcopy. Obstet Gynecol 2006;108(2):246–7.