Accuracy and reproducibility of pleural effusion cytology

Legal Medicine 10 (2008) 20–25 www.elsevier.com/locate/legalmed

Accuracy and reproducibility of pleural effusion cytology Ambrogio Fassina a

a,*

, Ugo Fedeli a, Matteo Corradin a, Monica Da Fre` b, Luigi Fabbris

b

Pathology Section, Department of Diagnostic Medical Sciences and Special Therapies, Faculty of Medicine, University of Padova, Via Gabelli 61, 35100 Padova, Italy b Department of Statistics, University of Padova, Italy Received 10 April 2007; received in revised form 27 May 2007; accepted 4 June 2007 Available online 16 August 2007

Abstract The increasing number of Malignant Mesothelioma (MM) cases that arrive for expert examinations to court for compensation reasons in subjects exposed to asbestos, in many instances rely exclusively on cytological smears of pleural effusion. We evaluated the accuracy and reproducibility of cytological pleural effusions, based on morphological criteria alone. Nine pathologists and eight residents from seven institutions in north-east Italy blindly examined 45 smears of MM (17), metastases (14) and benign effusions (14), in two rounds. Diagnoses had been confirmed by immunohistochemical and clinical follow-up, and eventually at autopsy. Diagnostic accuracy, interobserver and intraobserver agreement in the distinction of benign vs malignant cases, and in the differentiation of primary from metastatic malignancies, were evaluated. The distinction of benign from malignant smears resulted rather satisfactory (k = 0.514), but markedly decreased in differentiation of MM from metastases (overall agreement: k = 0.343), as well as when readings from residents were analyzed (k = 0.132). Cytology is a useful and reliable tool in the identification of malignancies, but when the distinction of primary from metastatic tumors is addressed morphological criteria alone are not sufficient for a definite diagnosis of MM and the use of cell blocks, immunohistochemistry (IHC) and molecular ancillary techniques are recommended. Ó 2007 Elsevier Ireland Ltd. All rights reserved. Keywords: Diagnostic accuracy; Effusion cytology; Intra–interobserver agreement; Litigation and compensation; Malignant mesothelioma; Metastatic pleural carcinoma

1. Introduction Effusions are pathological processes characterized by the collection of fluid in serosal cavities, with different causes and different degrees of clinical severity; acute or chronic inflammation or cardiac insufficiency and primitive or metastatic cancer are the most frequently encountered causes [1,2]. In the presence of effusions of unknown origin, it is mandatory to define with the highest accuracy the nature of the lesion and differentiate benign lesion from Malignant Mesothelioma (MM) and cancer metastases [3,4]. MM is a malignant tumor arising from the mesothelial lining of the serosal cavities, correlated with asbestos fibers exposure [5]. MM inci-

*

Corresponding author. Tel.: +39 049 8213780/1; fax: +39 049 8213782. E-mail address: [email protected] (A. Fassina).

dence is increasing worldwide, and is not expected to decrease for another 10–20 years [6,7]. In patients with a history of exposure, the combination of an unknown pleural effusion and pleural pain should raise the suspicion of MM, and the cytological examination of smears from effusion fluid represents the first step in the clinical work-up; however its accuracy and reproducibility are still matter of debate [8,9]. In many cases, sampling by fine needle aspiration or true-cut biopsy are required for the diagnosis. In the last decade, immunohistochemical reactions on cell blocks, ultrastructural and molecular studies have significantly contributed to better understand the biological nature and the characteristics of effusions [10]. The access to these techniques was restricted, until few years ago, to few centers, and the diagnosis rested on morphological criteria alone. In view of the long latency period after exposure, and the substantial personal and health care costs, MM is associated

1344-6223/$ - see front matter Ó 2007 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.legalmed.2007.06.001

A. Fassina et al. / Legal Medicine 10 (2008) 20–25

with compensation costs that represent a considerable problem for industry and government. The predicted total economic burden of malignant MM related to compensation for asbestos exposure in the next 40 years is up to $ 200 billion for the United States [11] and $ 80 billion for Europe [12]. In these years we assist in court to an increasing number of compensation trials for asbestos exposure, in which often the only material available is represented by the smears of the cytological effusions, with no biopsy, cell blocks, immunohistochemistry (IHC) or autopsy material. In consideration of the ‘‘mesothelioma epidemic’’ [5], the number of ‘‘old and cold’’ cases brought to court is expected to rise in number in the next future and in the expert witness position it can be relevant to refer to assessed values of the diagnostic accuracy and reproducibility of the available material. Moreover, no data are available in literature concerning interobserver agreement in pleural effusion cytology. This is indeed an ‘‘hot’’ spot also for the ethical need of a correct diagnosis, the identification of the best therapeutic approach, the value of diagnoses for cancer registries and relative risk exposure evaluation [13–15]. Aim of this study is to assess if cytological analysis, either obtained by centrifuge or cyto-spin and based on morphological criteria alone, is sufficient to differentiate benign from malignant pleural effusions, and distinguish MM from other malignancies. 2. Materials and methods 2.1. Patients We collected 45 pleural effusion specimens from the archives of the Pathology Department of Monfalcone Hospital, in the North-Eastern part of Italy. Smears were routinely stained with May–Grunwald–Giemsa and Papanicolau. All the cases were confirmed by IHC and clinical and pathological follow-up, and all the cases of MM underwent clinicopathological autopsy. Fourteen preparations were from benign effusions of 3 males and 11 females (aged 57–89, mean age 77); 4 of these were found to have benign pleural plaques at autopsy, a common finding in the shipyard area they were from [16]. Fourteen slides were from metastatic deposits (8 males and 6 females, aged 42–90, mean age 65). The primary tumours were breast ductal carcinoma (3 cases), hepatocellular carcinoma (1 case), non Hodgking lymphoma (1 case), lung cancer (9 cases: five adenocarcinomas, one squamous cell carcinoma, two large cell carcinomas, one small cell carcinoma). All the 17 malignant mesotheliomas affected male subjects (aged 62–83, mean age 71), who worked in a naval dockyards industrial area (Table 1). 2.2. Study design Nine pathologists from the Cytopathology Units of seven Pathology Laboratories in the North-Eastern part of Italy (Padova, Verona, Brescia, Belluno, Treviso, Pordenone, Trieste and Udine) independently reviewed the 45

21

Table 1 Patients profiles classified by age, gender, definitive diagnosis, and diagnostic procedure Case number

Age/ gender

Definitive diagnosis

Malignant mesothelioma 1 66/M MM 2 76/M MM 3 77/M MM 4 65/M MM 5 83/M MM 6 66/M MM 7 66/M MM 8 66/M MM 9 68/M MM 10 63/M MM 11 75/M MM 12 75/M MM 13 76/M MM 14 79/M MM 15 80/M MM 16 62/M MM 17 64/M MM Benign effusions 18 57/F 19 67/F 20 71/F 21 89/F 22 76/F 23 82/M 24 73/M 25 88/M 26 87/F 27 86/F 28 79/F 29 85/F 30 69/F 31 69/F Metastases 32 33 34 35 36 37

65/M 62/M 46/F 49/M 55/M 59/M

38 39 40 41 42 43 44 45

53/M 90/F 83/F 86/F 76/F 70/F 74/M 42/M

Benign Benign Benign Benign Benign Benign Benign Benign Benign Benign Benign Benign Benign Benign

Diagnostic procedure Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy

effusion effusion effusion effusion effusion effusion effusion effusion effusion effusion pleural plaques pleural plaques pleural plaques pleural plaques

Lung adenocarcinoma Lung adenocarcinoma Lung adenocarcinoma Lung adenocarcinoma Lung adenocarcinoma Squamous cell lung carcinoma Large cell lung carcinoma Large cell lung carcinoma Small cell lung carcinoma Ductal breast carcinoma Ductal breast carcinoma Ductal breast carcinoma Hepatocellular carcinoma Non Hodgking lymphoma

Biopsy Biopsy Biopsy Biopsy Biopsy Biopsy Biopsy Biopsy Biopsy Biopsy Autopsy Autopsy Autopsy Autopsy

Surgery Surgery Surgery Surgery Surgery Biopsy Surgery Biopsy Autopsy Autopsy Autopsy Autopsy Autopsy Autopsy

(M: male; F: female; MM: malignant mesothelioma).

slides. Slides were also examined by eight residents in Pathology, two from each of the Universities of Padova, Verona, Udine and Trieste. Reviewers were kept blind of the original clinical and pathological data and could not discuss the slides. In a preliminary meeting, the observers had been instructed to choose between the following diag-

22

A. Fassina et al. / Legal Medicine 10 (2008) 20–25

Table 2 Cytological criteria used for diagnosis High cellularity Uniform cell population Intercellular gap Central or paracentral nucleus Multinucleation with atypia Villosity Macronucleoli Blebbing Nuclear pleomorphism True papillae Pseudopapillary aggregates containing more than 20 cells (proliferative cell balls) Clear area in the periphery of cytoplasm (double cytoplasmic staining) Dense cytoplasms Macrovacuoli Vacuolizations Dense chromatin High mitotic index

nostic categories: 1. MM; 2. metastasis; 3. malignant cells not otherwise specified (not determined if primary or metastatic in origin); 4. benign effusion; 5. inadequate for a diagnosis, combining a list of widely accepted diagnostic criteria from the literature [5,15], summarized in Table 2. For each case the observers had to indicate which of these criteria were used for the diagnosis as the more convincing. So, we were made available of 45 independent nine-replicates of pathologists and other 45 independent eight-replicates of residents. After five months, the nine pathologists (not the residents) re-evaluated the same cases, with new labels and new identification numbers randomly assigned, unaware of their own previous evaluation. 2.3. Statistical analysis 2.3.1. Diagnostic accuracy Sensitivity, specificity, proportion of cases correctly classified by each observer, predictive value of a positive and negative study test results, prevalent diagnosis compared with true diagnosis (prevalent diagnosis = rendered by at least five reviewers) were calculated. 2.3.2. Interobserver agreement Kappa statistic of Cohen was employed to evaluate the agreement by couples of observers for categorical variables [17]; k statistic of David and Fleiss expressed interobserver overall agreement: k = (Po Pe)/(1 Pe), where Po is the proportion of empirical agreement and Pe is the proportion of agreement due to chance [18,19]. Kappa statistics compares the proportion of cases for which agreement can be attributed to the reproducibility of the observations rather than to mere chance (Po Pe) as a ratio with its maximum possible value (1 Pe). Its values vary from 1 (complete disagreement) to +1 (complete agreement), with 0 representing random agreement.

2.3.3. Intraobserver agreement The two readings (first and second round) of each expert reviewer were considered as performed by two independent observers, and Cohen Kappa statistic was re-calculated. Statistical indexes were firstly calculated for groups of benign (category 4) vs malignant readings (categories 1, 2, 3), and then for diagnosis of MM, metastases, and benign effusion considered as separate. 3. Results 3.1. Diagnostic accuracy The sensitivity of expert observers to recognize malignant effusions varied from 65.5% to 90.3%, with a mean value of 81.2%; specificity varied from 64.3% to 100%, with a mean value of 81.3%. The positive predictive value of a reading of malignancy varied from 84.4% to 100%, with a mean value of 91.2%, while the negative predictive value varied from 54.5% to 76.9%, with a mean of 67.4% (Table 3). Ten effusions out of 14 were properly identified as benign and 25 out of 31 were correctly identified as malignant by the majority of the reviewers (Table 4). Expert observers correctly provided the MM reading in a number of cases

Table 3 Intraobserver diagnostic accuracy measures in distinguishing malignant from benign cases by expert observer Expert Sensitivity Specificity Concordance Predictive observers value of malignant cases

Predictive value of benign cases

A B C D E F G H I

87.1 75.9 87.1 83.9 90.3 71.4 87.1 65.5 82.8

64.3 84.6 78.6 71.4 76.9 92.3 71.4 92.3 100

80.0 78.6 84.4 80.0 86.4 78.1 82.2 73.8 88.4

84.4 91.7 90.0 86.7 90.3 95.2 87.1 95.0 100

69.2 61.1 73.3 66.7 76.9 60.0 71.4 54.5 73.7

Mean 81.2 values

81.3

81.3

91.2

67.4

Table 4 Prevalent (rendered by at least five reviewers) compared with true diagnosis True diagnosis Benign

MM

Metastasis

Prevalent diagnosis Benign Malignant, not otherwise specified MM Metastasis No diagnosis

10 3 0 0 1

4 5 5 2 1

1 5 1 7 0

Total

14

17

14

A. Fassina et al. / Legal Medicine 10 (2008) 20–25

ranging from 1 to 11 (out of 17), and a diagnosis of metastatic deposit from 6 to 10 cases (out of 14). The diagnostic accuracy of residents was 71.0% in distinguishing malignancy from benign cases; sensitivity in the identification of neoplastic effusions was 82.8%, similar to that of expert reviewers, whereas specificity was by far worse (44.6%), due to the greater number of benign cases wrongly classified as malignant. For the same reason the positive and negative predictive values of a reading of malignancy resulted 77.0% and 57.5%, lower to that of experts. Each resident correctly rendered a definite diagnosis of MM in a number of cases that varied from three to eleven, and a diagnosis of metastatic deposit from one to nine. In the distinction of MM from metastatic pleural carcinoma, the troublesome cytological criteria included the presence of atypical nuclei, prominent nucleoli and frequent cytoplasmic vacuoles for the overlapping morphological features. 3.2. Interobserver agreement In distinguishing benign from malignant smears, overall agreement was rather good among expert reviewers; Kappa statistic resulted 0.514, with no differences between benign and neoplastic effusions. On the contrary, in the distinction of MM from metastases agreement was poor (k = 0.343), with a larger interobserver variability in the diagnosis of MM (k = 0.385 for metastases and k = 0.192 for MM). Agreement between couples of reviewers is shown in Tables 5 and 6: paired interobserver agreement ranges from 0.39 to 0.95 for benign versus malignant diagnosis and from 0.30 to 0.92 for the three categories benign/MM/metastases. In the resident group agreement was unsatisfactory not only in the distinction of primary from secondary tumours, but also in the differential diagnosis of malignant from benign smears (k = 0.132). 3.3. Intraobserver agreement Intraobserver agreement at 5 month distance was investigated only in the group of the expert reviewers. Except for two observers, it was rather good (Cohen’s k ranged from 0.31 to 0.70), but stability of the cytological diagnosis was Table 5 Cohen’s Kappa interobserver agreement of benign vs malignant diagnoses, by distinct pairs of expert observers

23

Table 6 Cohen’s Kappa interobserver agreement of benign vs MM vs metastases diagnoses, by distinct pairs of expert observers

Table 7 Cohen’s Kappa intraobserver agreement measures, by expert observer Expert observer

Benign vs malignant

Benign vs MM vs metastases

A B C D E F G H I

0.66 0.34 0.42 0.66 0.89 0.54 0.79 0.64 0.94

0.50 0.65 0.31 0.70 0.66 0.49 0.62 0.48 0.61

better when calculated only for groups of benign vs malignant readings (Table 7). 4. Discussion In this study we evaluated the role of cytological smears of pleural effusions in the distinction of MM from other malignancies and benign lesions. In court, for the compensation of exposure to asbestos fibers, the judge entrusts to the forensic physicians the evaluation of the correct diagnosis, exposition and causality of MM [20]. In many situations of critically ill patients with recurrent effusions and respiratory insufficiency, when thoracoscopy is difficult or too dangerous, the final diagnosis rests exclusively on the cytological examinations of the fluid obtained at thoracentesis, and remains the only material available for experts in court. The diagnosis of MM, metastasis, or benign mesothelial proliferation in effusion samples is often a hard task for the surgical and forensic pathologist, due to sampling problems (few malignant cells shedding, hemorrhagic or inflammatory effusion) and/or interpretative errors. Reported sensitivity for the cytological diagnosis of MM varies from 31.9% to 86.3% for malignancy without further specifications, and from 11.7% to 75.3% for a correct diagnosis of primary neoplasm [8]. The present study evidences satisfactory values of sensitivity and specificity only in the differentiation of benign from malignant cases, higher with experts and lower with residents. Nevertheless, a percentage of false negative cases and false positive cases is still present, with reactive mesothelial cells mimicking malig-

24

A. Fassina et al. / Legal Medicine 10 (2008) 20–25

nancy. The differentiation of MM from pleural metastasis was not satisfactory both in expert (k = 0.343) and in resident groups (k = 0.132). Ancillary techniques have been advocated for the correct identification of reactive mesothelial hyperplasia from malignancies: immunostaining alone [21,22], or in combination with silver-stained nucleolar organizer regions (AgNOR) [23], or DNA ploidy analysis obtain reliable results [24]. In our study, the further specifications of malignant smears, whether primary or metastatic in origin, lead to a great decrease in diagnostic accuracy and also the diagnostic reproducibility was markedly lower when diagnostic categories were examined as separated entities, not grouped in benign or malignant. These findings could be partially explained by the fact that Cohen’s Kappa tends of necessity to be lower for polytomies than for dichotomies [19]. Andrion et al., analyzing the consistency of histopathological diagnosis of pleural MM, found good agreement on material obtained at autopsy, surgery or endoscopy, whereas agreement for needle biopsy specimens was poor [13,14]. Skov et al. in 1994 examined interobserver variation of the histopathological diagnosis of adenocarcinoma of the lung and MM, and found that Kappa values positively increased in three successive diagnostic rounds from hematoxylin and eosin (H&E) stained sections alone, or with H&E plus histochemical mucin stains, and finally with a panel of specific antibodies [13]. Our data suggest that morphological criteria alone permit a first differentiation of benign mesothelial proliferation from malignancies, but the differential diagnosis between MM and metastatic tumors still remains doubtful, with a large subjective component, due to the involvement of the pleural surfaces and the presence both in MM and in metastatic adenocarcinoma of several overlapping features such as atypical nuclei, prominent nucleoli and frequent cytoplasmic vacuoles. IHC on cyto-spins or cell blocks with antibodies that specifically mark mesothelial cells (AMAD-2, calretinin and cytokeratin 5/6), cytogenetics and electron microscopy [25–27], and more recently also the evaluation of soluble mesothelin [28], along with accurate clinical data, can produce a definite and reproducible diagnosis. In a recent elegant paper, Kushitani reviewed a series of MM and Adenocarcinomas, suggesting that the combination of CEA, calretinin, WT1 or thrombomodulin is the most useful antibody panel for distinguishing between epithelioid mesothelioma and lung adenocarcinoma [29]. In court, expert examination based exclusively on cytological smears of pleural effusions cannot be considered sufficient for the definite conclusion of MM and consequently for asbestos exposure compensation. Acknowledgements The authors thank Prof. C. Bianchi, Dr. A. Brollo, Dr. A. Dei Tos, Dr. C. Doglioni, Dr. R. Vendraminelli, Prof. F. Bonetti, Prof. A. Beltrami, Prof. L. Di Bonito, Dr. G. Bortuzzo, Dr. P.L. Chiodera.

References [1] Light RW. The undiagnosed pleural effusion. Clin Chest Med 2006;27(2):309–19. [2] Lee YC, Light RW. Management of malignant pleural effusions. Respirology 2004;9(2):148–56. [3] Lee YC, Light RW, Musk AW. Management of malignant pleural mesothelioma: a critical review. Curr Opin Pulm Med 2000;6(4):267–74. [4] Barreiro TJ, Katzman PJ. Malignant mesothelioma: a case presentation and review. J Am Osteopath Assoc 2006;106(12):699–704. [5] Robinson BW, Lake RA. Advances in malignant mesothelioma. N Engl J Med 2005;353(15):1591–603. [6] Archer VE, Rom WN. Trends in mortality of diffuse malignant mesothelioma of pleura. Lancet 1983;2:112–3. [7] Peto J, Decarli A, La Vecchia C, Levi F, Negri E. The European mesothelioma epidemic. Br J Cancer 1999;79:666–72. [8] Di Bonito L, Falconieri G, Colautti I, Bonifacio Gori D, Dudine S, Giarelli L. Cytopathology of malignant mesothelioma: a study of its patterns and histological bases. Diagn Cytopathol 1993;9:25–31. [9] Renshaw AA, Dean BR, Antman KH, Sugarbaker DJ, Cibas ES. The role of cytologic evaluation of pleural fluid in the diagnosis of malignant mesothelioma. Chest 1997;111:106–9. [10] McLean AN, Bicknell SR, McAlpine LG, Peacock AJ. Investigation of pleural effusion: an evaluation of the new Olympus LTF semiflexible thoracofiberscope and comparison with Abram’s needle biopsy. Chest 1998;114(1):150–3. [11] Carroll S, Hensler D, Abrahamse A, Gross J, Ashwood S, Sloss E, et al. Asbestos litigation costs and compensation: an interim report, Available from: ; 2004 [Accessed 16.09.04]. [12] Shah N, Williams A. Surviving the asbestos epidemic. In: Insurance digest (European). London: PricewaterhouseCoopers; 2001. p. 14–8. [13] Skov BG, Lauritzen AF, Hirsch F, Nielsen HW. The histopathological diagnosis of malignant mesothelioma vs. pulmonary adenocarcinoma: reproducibility of the histopathological diagnosis. Histopathology 1994;24:553–7. [14] Andrion A, Magnani C, Betta PG, Donna A, Mollo F, Scelsi M, et al. Malignant mesothelioma of the pleura: interobserver variability. J Clin Pathol 1995;48:856–60. [15] Whitaker D. The cytology of malignant mesothelioma. Cytopathology 2000;11:139–51. [16] Bianchi C, Brollo A, Ramani L. Asbestos exposure in a shipyard area, northeastern Italy. Ind Health 2000;38:301–8. [17] Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46. [18] Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378–82. [19] Maclure M, Willet W. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol 1987;126:161–9. [20] Price B, Ware A. Mesothelioma: risk apportionment among asbestos exposure sources. Risk Anal 2005;25(4):937–43. [21] Motherby H, Friedrichs N, Kube M, Nadjari B, Knops K, Donner A, et al. Immunocytochemistry and DNA-image cytometry in diagnostic effusion cytology. II. Diagnostic accuracy in equivocal smears. Anal Cell Pathol 1999;19(2):59–66. [22] Attanoos RL, Griffin A, Gibbs AR. The use of immunohistochemistry in distinguishing reactive from neoplastic mesothelium. A novel use for desmin and comparative evaluation with epithelial membrane antigen, p53, platelet-derived growth factor-receptor, P-glycoprotein and Bcl-2. Histopathology 2003;43:231–8. [23] Wolanski KD, Whitaker D, Shilkin KB, Henderson DW. The use of epithelial membrane antigen and silver-stained nucleolar organizer regions testing in the differential diagnosis of mesothelioma from benign reactive mesothelioses. Cancer 1998;82:583–90. [24] Friedman MT, Gentile P, Tarectecan A, Fuchs A. Malignant mesothelioma: immunohistochemistry and DNA ploidy analysis as

A. Fassina et al. / Legal Medicine 10 (2008) 20–25 methods to differentiate mesothelioma from benign reactive mesothelial cell proliferation and adenocarcinoma in pleural and peritoneal effusions. Arch Pathol Lab Med 1996;120:959–66. [25] Montanaro F, Bray F, Gennaro V, Merler E, Tyczynski JE, Parkin DM, et al. ENCR working group. Pleural mesothelioma incidence in Europe: evidence of some deceleration in the increasing trends. Cancer Cause Control 2003;14:791–803. [26] Whitaker D. The cytology of malignant mesothelioma. Cytopathology 2000;11:139–51.

25

[27] Fetsch PA, Abati A. Immunocytochemistry in effusion cytology. Cancer Cytopathol 2001;93:293–308. [28] Creaney J, Yeoman D, Naumoff L, Hof M, Segal A, Musk WA, et al. Soluble mesothelin in effusions – a useful tool for the diagnosis of malignant mesothelioma. Thorax 2007;13 [in press]. [29] Kushitani K, Takeshima Y, Amatya VJ, Furonaka O, Sakatani A, Inai K. Immunohistochemical marker panels for distinguishing between epithelioid mesothelioma and lung adenocarcinoma. Pathol Int 2007;57(4):190–9.