- Email: [email protected]
Evaluation of automated multi-parametric indirect immunofluorescence assays to detect anti-neutrophil cytoplasmic antibodies (ANCA) in granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA)
AUTREV-01849; No of Pages 5 Autoimmunity Reviews xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Autoimmunity Reviews journal homepage: www.elsevier.com/locate/autrev
7
F
O
Elena Csernok a,1, Jan Damoiseaux b,1, Niels Rasmussen c, Bernhard Hellmich a, Pieter van Paassen d, Pieter Vermeersch e,f, Daniel Blockmans g, Jan-Willem Cohen Tervaert h, Xavier Bossuyt i,j,⁎
R O
6Q2
8 9 10 11 12 13 14 15 16 17 Q3 18 19
a
2 0
a r t i c l e
21 22 23 24 29 28 27 26 25 46 47 48 49 50
Article history: Received 19 February 2016 Accepted 3 March 2016 Available online xxxx
Department of Internal Medicine, Rheumatology and Immunology, Vasculitis Center Tübingen-Kirchheim, University Hospital Kirchheim, Germany Central Diagnostic Laboratory, Maastricht University Medical Center, Maastricht, The Netherlands Department of Autoimmune Serology, Statens Serum Institute, Copenhagen, Denmark d Department of Internal Medicine, Section of Nephrology and Immunology, Maastricht University Medical Center, Maastricht, The Netherlands e Clinical Department of Laboratory Medicine, University Hospitals Leuven, 3000 Leuven, Belgium f Department of Cardiovascular Sciences, KU Leuven, Leuven, ,Belgium g Clinical Department of General Internal Medicine, Research Department of Microbiology and Immunology, Laboratory of Clinical Infectious and Inflammatory Disorders, University Hospitals Leuven, Leuven, Belgium h Maastricht University, The Netherlands i Department of Microbiology and Immunology, KU Leuven, Leuven, Belgium j Department of Laboratory Medicine, University Hospitals Leuven, Leuven, Belgium b
E
D
P
c
57 58 59 60 61 62
Contents 1. 2.
U
53
a b s t r a c t
N C O
Keywords: Anti-neutrophil cytoplasmic antibodies (ANCA) Granulomatosis with polyangiitis (GPA) Microscopic polyangiitis (MPA) Indirect immunofluorescence
54 52 51 56 55
i n f o
T
5
Objectives: The aim of this multicenter EUVAS study was to evaluate the diagnostic performance of multiparametric indirect immunofluorescence (IIF) assays to detect anti-neutrophil cytoplasmic antibodies (ANCA) in granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA). Patients and methods: The study included 912 samples from diseased controls and 249 diagnostic samples from GPA (n = 183) and MPA (n = 66) patients. The performance of two automated multi-parametric assays [Aklides (Medipan/Generic Assays) and EuroPattern (Euroimmun)] combining IIF on cellular and purified antigen substrates was compared with two manual IIF analyses and with commercially available ELISAs for MPO- and PR3-ANCA (Euroimmun). Results: The area under the curve (AUC) of the receiver operating characteristics (ROC) curve to discriminate AAV from controls was 0.925, 0.848, 0.855 and 0.904 for, respectively, the two manual analyses, Aklides and EuroPattern, and 0.959, 0.921 and 0.886 for, respectively, antigen-specific ELISA, antigen-coated beads, and microdot, respectively. Variation in pattern assignment between IIF methods was observed. Conclusion: The performance of IIF depends on the substrate used and the definition of IIF patterns. The performance of automated IIF is improved by multi-parameter testing (combined IIF and antigen-specific testing). Given the variability between IIF methods, the diagnostic importance of this technique is questioned. © 2016 Published by Elsevier B.V.
C
4
E
3
Evaluation of automated multi-parametric indirect immunofluorescence assays to detect anti-neutrophil cytoplasmic antibodies (ANCA) in granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA)
R
2Q1
Review
R
1
Introduction . . . . . . . . . . . . . . . . Material and methods . . . . . . . . . . . 2.1. Patients . . . . . . . . . . . . . . 2.2. ANCA detection methods . . . . . . 2.2.1. Manual IIF . . . . . . . . . 2.2.2. Automated multi-parameter IIF
. . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
⁎ Corresponding author at: Laboratory Medicine, University Hospitals Gasthuisberg, Herestraat 49, 3000 Leuven, Belgium. Tel.: +32 16 347009; fax: +32 16 34 79 31. E-mail address: [email protected] (X. Bossuyt). 1 Share first authorship.
http://dx.doi.org/10.1016/j.autrev.2016.03.010 1568-9972/© 2016 Published by Elsevier B.V.
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
0 0 0 0 0 0
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
2
63 64 65 66 67 68
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
2.2.3. ELISA . . 2.3. Statistical analysis 3. Results . . . . . . . . 4. Discussion . . . . . . . Take-home messages . . . . References . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
0 0 0 0 0 0
69
2.1. Patients
93
110 111
The patients included are from a multicenter EUVAS (European Vasculitis Study Group). Patients (n = 249) with AAV (GPA and MPA) and controls (n = 912) with inflammatory disease were recruited at Klinikum Bad Bramstedt (Germany) (190 controls, 38 GPA, 26 MPA), Statens Serum Institute Copenhagen (Denmark) (237 controls, 38 GPA, 9 MPA), University Hospitals Leuven (Belgium) (235 controls, 39 GPA, 15 MPA) and Maastricht University Medical Center (The Netherlands) (250 controls, 68 GPA, 16 MPA). The samples included were diagnostic samples, i.e. obtained at the time the diagnosis was established. The disease controls recruited in Copenhagen, Leuven and Maastricht were consecutive patients in whom ANCA analysis was requested, but in whom AAV was excluded later. Patients in whom inflammatory bowel disease and/or autoimmune liver disease was considered were excluded. The disease controls in Bad Bramstedt included cohorts of patients with systemic lupus erythematosus (n = 59), rheumatoid arthritis (n = 89), systemic sclerosis (n = 11) and Sjögren's syndrome (n = 30). For analysis, patients with GPA and MPA were grouped as AAV. This study was approved by the ethics committee of each participating center.
112
2.2. ANCA detection methods
113
2.2.1. Manual IIF IF performed at Bad Bramstedt (Germany) was done on ethanolfixed neutrophils in combination with additional tests on formalinfixed neutrophils and HEp-2 cells to better discriminate between P-ANCA (or atypical-ANCA) and ANA, as previously described [3]. IIF performed at the Staten Serum Institut (Copenhagen, Denmark) used an ethanol-fixed mixture of neutrophils and lymphocytes, as previously described [8].
96 97 98 99 100 101 102 103 104 105 106 107 108 109
114 115 116 117 118 119 120
C
E
94 95
R
87 88
R
86
O
84 85
C
82 83
N
80 81
U
78 79
2.2.3. ELISA Antibodies against PR3 and MPO were detected by monospecific ELISA (anti-PR3-hn-hr.-ELISA (IgG) and anti-myeloperoxidase ELISA (IgG); Euroimmun AG).
150
2.3. Statistical analysis
154
F
92
76 77
O
2. Material and methods
74 75
R O
91
72 73
P
89 90
Anti-neutrophil cytoplasmic antibodies (ANCA) are recognized as important biomarkers for the diagnosis of ANCA-associated vasculitis (AAV), in particular, granulomatosis with polyangiitis (GPA) and microscopic polyangiitis [1–3]. According to the 1999 international consensus, indirect immunofluorescence (IIF) is the optimal assay for ANCA screening; if positive, reflex testing for both proteinase (PR)3- and myeloperoxidase (MPO)-ANCA is mandatory [4]. The methods for detection and clinical utility of ANCA are extensively reviewed [1–3]. The classical two-tier approach, i.e. IIF followed by antigen-specific immuno-assays, can be combined in a single multi-parametric IIF assay. The EuroPLUS™ ANCA biochip mosaic combines both ethanoland formalin-fixed neutrophil substrates with PR3 and MPO antigen microdots in a single assay [5]. The Medipan Cytobead IIF assay combines ethanol-fixed neutrophil substrates with beads that have been differentially coated with PR3 and MPO [6]. Importantly, both multiparametric IIF assays can be analyzed by automated pattern recognition software, the EuroPattern and Aklides, respectively [6,7]. In the recently performed multicenter EUVAS study, we evaluated the performance of both the ANCA biochip mosaic and the Cytobead assay, as analyzed by the respective pattern recognition software.
121
D
71
2.2.2. Automated multi-parameter IIF IIF at Euroimmun was performed on the EUROPLUS™ Granulocyte Mosaic (Euroimmun AG, Lübeck, Germany), which contains six different biochips per reaction field: ethanol-fixed human granulocytes, formalin-fixed granulocytes, ethanol-fixed mixtures of HEp-2 cells and granulocytes and PR3, MPO, and GBM microdots (purified antigens) [5]. Automated evaluation was performed by EUROPattern Suite [7]. With this system, a rule-based engine creates a result based on the following classifications: (1) C-ANCA (cytoplasmic staining on ethanoland formalin-fixed granulocytes); (2) P-ANCA (perinuclear staining on ethanol-fixed granulocytes confirmed by a cytoplasmic staining on formalin-fixed granulocytes); (3) “atypical” ANCA (ANCA patterns not associated with AAV, e.g. DNA-ANCA (a perinuclear staining on ethanol-fixed granulocytes, but no staining on formalin-fixed granulocytes; associated with diseases like IBD), ANA interference, atypical C-ANCA; and (4) negative. The microdots were reported independently from the cell substrates as PR3-, MPO-, GBM-positive or negative. The final result was accompanied by titers and confidence values and verified by an expert. According to the EuroPattern classifier, only C- and P-ANCA are to be considered ANCA-positive. Automated IIF on Aklides® and antigen-specific ANCA by CytoBead ANCA [Medipan/Generic Assays GmbH (Berlin, Germany)] was performed by Medipan according to the manufacturer's instructions [6,9]. Briefly, CytoBead® ANCA, a multiplex IIF test, combines the screening of ANCA on ethanol-fixed neutrophils and their confirmation with multiplex microbead immunoassays, using recombinant PR3 and MPO. The final IIF read out is expressed as arbitrary units. The patterns reported included C-ANCA, P-ANCA and unrecognized/atypical (labeled as atypical). All reactivities were considered ANCA positive.
E
1. Introduction
T
70
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149
151 152 153
ROC curve analysis was performed using Analyse-it for Microsoft 155 Excel 3.90. 156 3. Results
157
Table 1 gives an overview of (i) ANCA by manual [Bad Bramstedt and Copenhagen] and automated [Aklides (Medipan) and EuroPattern (Euroimmun)] IIF patterns and (ii) PR3-ANCA and MPO-ANCA by ELISA, antigen-coated beads (Aklides) and microdots (EuroPattern). The table shows all patterns in the way they were reported by their respective laboratories or by the respective software. For calculation of IIF performance characteristics, all reactivities were considered positive, except for EuroPattern for which atypical ANCA was considered negative (see Material and methods). The specificity (determined in 912 controls) of manual IIF was 94% for Bad Bramstedt and 79% for Copenhagen. The specificity of automated IIF patterns was 89% for Aklides and 90% for EuroPattern. Copenhagen assigned P-ANCA to a large fraction of controls (169/912). EuroPattern reported the simultaneous presence of ≥2 different ANCA patterns in 29/912 controls.
158 159
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
160 161 162 163 164 165 166 167 168 169 170 171 172
N
C
Table 1 Overview of the results for C-ANCA, P-ANCA, A-ANCA (atypical ANCA) by manual IIF performed at Bad Bramstedt (BB) and Copenhagen (Cop) and by automated EuroPattern (Euroimmun) and Aklides (Medipan). PR3-ANCA and MPO-ANCA was performed by ELISA (Euroimmun), by Aklides® Cytobead and by microdots (EuroPattern). Cutoff values proposed by the manufacturer were used. Diagnostic performance characteristics are given as well.
t1:4 t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15 t1:16 t1:17 t1:18 t1:19 t1:20 t1:21 t1:22 t1:23 t1:24 t1:25 t1:26 t1:27 t1:28 t1:29 t1:30 t1:31 t1:32 t1:33 t1:34
Controls n = 912
GPA n = 183
MPA n = 66
Specificity Sensitivity GPA Sensitivity MPA Sensitivity AAV LR(+) AAV LR(−) AAV AUC AAV (95% CI)
O
C-ANCA
P-ANCA
A-ANCA
BB IIF (manual)
Neg Neg Neg Neg Pos Pos Pos Pos Neg Neg Neg Neg Pos Pos Pos Neg Neg Neg Neg Pos Pos Pos
Neg Neg Pos Pos Neg Neg Pos Pos Neg Neg Pos Pos Neg Neg Pos Neg Neg Pos Pos Neg Neg Pos
Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Neg Pos Neg Pos Neg Pos Neg
857 6 32 17
Cop IIF (manual)
Aklides IIF (automated)
EuroPattern IIF (automated)
PR3-ANCA
MPO-ANCA
Euroimmun ELISA
Aklides beads (automated)
EuroPattern microdots (automated)
717
815 64 27
306 513 43 14 21 2 12 1 7 12 14 3 126
Neg
Neg
883
863
858
Neg
Pos
19
31
33
Pos
Neg
7
16
15
Pos
Pos
3
2
6
Neg
Neg
19
22
27
Neg
Pos
20
18
18
Pos
Neg
144
139
128
Pos Neg
Pos Neg
6
4 16
10 15
Neg
Pos
57
45
48
Pos
Neg
3
3
2
0.968 0.896 0.909 0.900 28.3 0.10 0.959 (0.942–0.977)
2 0.946 0.880 0.758 0.847 15.8 0.16 0.921 (0.894–0.947)
1 0.941 0.852 0.773 0.831 14.0 0.18 0.886 (0.862–0.911)
R 169 25
R
1
E 6
C
21
36
20
27
62 21 8
142
119
92
6
1 4
56
59
9 23 32
4
3
2
0.940 0.885 0.909 0.892 14.8 0.12 0.925 (0.904–0946)
0.786 0.803 0.939 0.839 3.9 0.20 0.848 (0.821–0, 876)
0.894 0.661 0.864 0.715 6.7 0.32 0.855 (0.828–0.882)
T
E
D
21 2 6 39 7 3 1 8 0.898 0.896 0.879 0.892 8.7 0.12 0.904 (0.882–0.927)
P
Pos
R O Pos
O
F
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
t1:1 t1:2 t1:3
U
3
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213
t2:1 t2:2 t2:3 t2:4 t2:5 t2:6 t2:7 t2:8 t2:9 t2:10 t2:11 t2:12 t2:13 t2:14 t2:15 t2:16 t2:17 t2:18 t2:19 t2:20 t2:21 t2:22 t2:23 t2:24 t2:25 t2:26
244
Nowadays, it is to be expected that automated ANCA IIF, which combines analysis of cellular and antigen-specific substrates, will increasingly be introduced in clinical laboratories. The benefits of these systems include a higher reproducibility and shorter hands-on time. In this EUVAS study we assessed the reliability of these multiparametric assays by evaluation of two automated pattern recognition systems. Analysis of neutrophil substrates is combined with a beadbased assay in the Aklides system and with PR3- and MPO-microdots in the EuroPattern system [Sowa 2014; Krause 2015]. For comparison, we included two manual established IIF methods and an ELISA for PR3- and MPO-ANCA. The results obtained in this study demonstrate that the automated IIF assays from both manufactures vary in their test characteristics. Differences observed may be explained by the use of different substrates, different algorithms for automatic pattern assignment, and different definitions of ANCA patterns. When applying the pattern definitions of the manufacturers, the Aklides system appeared less effective in recognition of both C- and P-ANCA patterns in sera of GPA and MPA patients, respectively. The EuroPattern Suite, on the other hand, identified these patterns equally good as both reference laboratories, but additional patterns, i.e. the reciprocal pattern and/or atypical ANCA, were reported simultaneously in about 10% of the AAV sera. This seems to be the resultant of the pattern recognition devices since these phenomena were not observed by manual reading of the respective ANCA slides in previous studies [5,10]. The results obtained by the automatic pattern recognition software devices, however, require confirmation by the operator. This enables correction of apparent misclassifications. The combination of ethanol- and formalin-fixed neutrophils and HEp-2 cells, as applied in Bad Bramstedt and in the ANCA biochip, enables better identification of vasculitis-associated ANCA patterns [1,2,11]. Indeed, these assays revealed similar ROC curves for the cellular substrates. Also the manual IIF method of Copenhagen and the ANCA patterns obtained by Aklides, both only relying on ethanol-fixed neutrophils, had similar ROC curves,
245
O
F
4. Discussion
R O
194 195
P
192 193
214 215
D
190 191
(13% of patients) was, respectively, 42 and 18. Similarly, with EuroPattern, the highest LR(+) (113) was found for the combination C-ANCA/PR3-ANCA (50% of patients). This was higher than the LR(+) of C-ANCA (32) or of PR3-ANCA (21) alone. The LR(+) for the P-ANCA/MPO-ANCA combination (21% of patients) was 14. Further investigation of the diagnostic performance characteristics of IIF on cellular and antigen-specific substrates was done by receiver operating characteristic (ROC) curve analysis. To perform such analysis, patients with GPA and MPA were clustered as AAV (n = 249). The highest level of reactivity, independent of ANCA pattern or of antigenspecificity, was selected for analysis. This was feasible as similar cutoff values for PR3- and MPO-ANCA were used. The results are shown in Fig. 1 and Table 1. The area under the curve (AUC) of ANCA patterns assigned by Aklides (0.85) was comparable to the AUC of manual IIF Copenhagen (0.85), but lower than the AUC of ANCA pattern recognition by EuroPattern (0.90) and Bad Bramstedt (0.92) (p b 0.0001). The highest AUC was found for the antigen-specific Euroimmun ELISAs (0.96) (p b 0.0001 for comparison with all IIF methods). The AUC for antigen-coated beads and microdot was, respectively, 0.92 and 0.89 (p = 0.003). For analysis of data generated by EuroPattern, atypical ANCA were not considered positive (see Material and methods). Possible alternative ways to analyze the data (i) consider only single positive C- and P-ANCA as AAV-related and any combination of patterns as AAV nonrelated or (ii) consider all reactivities, including atypical ANCA as positive. These options reveal an AUC (95% CI) of 0.839 (0.810–0.868) and 0.848 (0.822–0.874), respectively. If only C-ANCA, P-ANCA, or combinations of C-ANCA and P-ANCA or C-ANCA and atypical ANCA or P-ANCA and atypical ANCA are considered as related to AAV, as programmed in the EuroPattern software, the AUC is 0.904 (0.882–0.927).
T
188 189
C
186 187
E
184 185
R
182 183
R
180 181
Table 2 Likelihood ratios (LR) for (i) ANCA by IIF (C-ANCA, P-ANCA, A-ANCA); (ii) PR3-ANCA and MPO-ANCA by ELISA, CytoBead, and microdots; and (iii) a combination of IIF and antigenspecific multi-parameter analysis [Aklides (IIF and antigen-coated beads) and EuroPattern (IIF and microdots)].
O
179
Aklides
PR3
MPO
Neg Neg Pos Pos
Neg Pos Neg Pos
EuroPattern
PR3 Neg Neg Pos Pos
MPO Neg Pos Neg Pos
IIF Bad Bramstedt IIF Copenhagen ELISA
PR3 Neg Neg Pos
C
177 178
C-ANCA
P-ANCA
A-ANCA
Neg
1.8 0.0 111.1
0.0 12.2 0.0
0.7 42.1 18.3
57.4
5.4
2.5
0.1 0.0 16.8 0.0 0.32
C-ANCA 0.0
P-ANCA 0.6 14.1
C-, P-ANCA 0.0
7.3 7.3 P-ANCA 8.7 P-ANCA 1.9
10.1 6.7 A-ANCA 0.0 A-ANCA 3.7
N
175 176
The sensitivity of manual IIF for AAV (183 GPA and 66 MPA patients) was 89% for Bad Bramsted and 84% for Copenhagen. The sensitivity of automated IIF pattern analysis for AAV was 72% for Aklides and 89% for EuroPattern. Breakdown of AAV into GPA and MPA revealed that the low sensitivity of Aklides was related to the low sensitivity of C-ANCA in GPA and, to a lesser extent, P-ANCA in MPA. As a consequence, the sensitivity of Aklides for GPA was low (66%). Moreover, Aklides reported atypical ANCA in a substantial portion of GPA (21/183) and MPA patients (23/66). EuroPattern, on the other hand, reported the combination of ANCA patterns in 24/183 GPA and 16/66 MPA patients. The sensitivity of PR3- and MPO-ANCA for AAV was 90% for ELISA, 85% for the antigen-coated beads (Aklides) and 83% for microdots (EuroPattern). The lower sensitivity of the antigen-coated beads and microdots was mainly related to the low sensitivity of MPO-ANCA. As a consequence, these two assays had a low sensitivity for MPA (76% and 77%, respectively). The likelihood ratio (LR) for AAV of a positive test result [LR(+)] obtained by the cellular substrates was 14.8 for Bad Bramstedt, 3.9 for Copenhagen, 6.7 for Aklides and 8.7 for EuroPattern. For the antigenspecific methods, the LR(+) was 28.3 for ELISA, 15.8 for antigencoated beads (Aklides) and 14.0 for microdots (EuroPattern). The LR of a negative test result [LR(−)] obtained by the cellular substrates was 0.12 for Bad Bramstedt, 0.20 for Copenhagen, 0.32 for Aklides and 0.12 for EuroPattern. The high LR(−) for Aklides is related to the low sensitivity for C-ANCA. For the antigen-specific methods, the LR(−) was 0.10 for ELISA, 0.16 for antigen-coated beads and 0.18 for microdots. A breakdown of the LR according to IIF pattern and antigenspecificity is shown in Table 2. For all methods on cellular substrates, the highest LR(+) was found for C-ANCA, varying between 18 (Copenhagen) and 57 (Aklides). The LR(+) associated with P-ANCA varied between 4.5 and 8.7, except for Copenhagen for which it was 1.9. Similarly, for antigen-specific ANCA, the highest LR(+) was found for PR3-ANCA. It was 77 for ELISA and 32 for antigen-coated beads and microdots. The LR(+) associated with MPO-ANCA was 15 for ELISA and 7 for antigen-coated beads and microdots. With Aklides, the combined presence of C-ANCA and PR3-ANCA (36.5% of patients) had the highest LR(+) (111) for AAV, higher than the LR(+) of C-ANCA (57) or of PR3-ANCA (32) alone. The LR(+) of P-ANCA, MPO-ANCA or the P-ANCA/MPO-ANCA combination (16% of patients) was, respectively, 5.4, 7.4 and 12.2. The LR(+) for the combined presence of atypical ANCA and either MPO-ANCA or PR3-ANCA
U
173 174
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
E
4
MPO Neg Pos Neg
113.5 11.0 31.7 C-ANCA 31.5 C-ANCA 17.9
Neg 0.1 0.8 1.2 3.7 0.18 Neg 0.12 Neg 0.17
0.16 7.4 32.5 11.0
0.12 4.0 20.7 8.2
0.10 14.8 76.9
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx 1
1
A
0.9
B
0.9
0.8
0.8
0.7
0.7
PR3/MPO ELISA IIF Copenhagen
0.6
Sensitivity
Sensitivity
5
IIF Bad Bramstedt
0.5
Aklides IIF EuroPattern IIF
0.4
EUROPattern - C-ANCA and PANCA
0.6 0.5
EUROPattern - C-ANCA, P-ANCA and combinations
0.4
EUROPattern - all reactivities
0.3
0.3
0.2
0.2
0.1
0.1 0 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.1
0.2
0.3
1 - Specificity
0.4
0.5
0.6
0.7
0.8
O
0
F
Aklides PR3/MPO CytoBead
0.9
1
1 - Specificity
Take-home messages
292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308
312 313 314 315 316 317 318
C
E
290 291
R
288 289
R
286 287
N C O
284 285
U
282 283
D
311
280 281
• PR3- and MPO-ANCA ELISAs had the best performance characteristics for ANCA-associated vasculitis. • Given the variability between indirect immunofluorescence methods for ANCA, the value of this technique may be questioned.
T
309 310
although with lower AUCs. Detailed comparison of the four IIF approaches was somewhat hampered by the distinct definitions applied for the ANCA patterns. When other definitions were applied, including the ANCA pattern definitions of the international consensus [4], then this had a significant impact on the performance of the assays. Obviously, the use of different definitions for ANCA patterns does not benefit standardization and also hampers the use of ANCA patterns for other chronic inflammatory diseases [12]. Overall, PR3- and MPO-ANCA by ELISA had the best overall diagnostic performance characteristics with the highest AUC of ROC curve analysis, the highest LR(+) and lowest LR(−). Test characteristics for PR3and MPO-ANCA of both automated systems were quite similar to each other. Importantly, detection of MPO-ANCA by Aklides and EuroPattern was less sensitive than the MPO-ANCA ELISA. Data were also analyzed for the simultaneous recognition of C-ANCA and PR3-ANCA, or PANCA and MPO-ANCA, in the same sample. The specificity of these combinations for AAV was about 98.5% for both multi-parametric assays; this specificity was also reported in a meta-analysis of conventional ANCA assays [13]. The sensitivity of Aklides (53%) and EuroPattern (70%) was substantially lower than the sensitivity of 85% reported in the meta-analysis. Interestingly, both multi-parametric assays revealed that the positive likelihood ratio for the combination C-ANCA/PR3ANCA was about ten-fold higher than for the combination P-ANCA/ MPO-ANCA. Also in routine clinical practice the combined presence of P-ANCA and MPO-ANCA has appeared less specific for AAV [14]. In conclusion, our study demonstrates that the test characteristics of the automated IIF depends on the substrates used and the ANCA pattern definitions applied. The performance characteristics, in particular the LR(+), of the automated IIF is improved by multi-parameter testing, i.e. if the ANCA pattern is in agreement with the ANCA specificity. Furthermore, this study indicates that the diagnostic importance of IIF may be questioned since the combination of PR3- and MPO-ANCA ELISAs had the best performance for the diagnosis of AAV.
E
278 279
P
R O
Fig. 1. Panel A. Receiver operating characteristics (ROC) curve for two manual IIF methods [performed at Copenhagen (C) and at Bad Bramstedt (BB)], two automated IIF methods (EuroPattern and Aklides), and three antigen-specific immuno-assays [ELISA, antigen-coated beads and microdots] for ANCA detection. For automated IIF, the fluorescence intensity was used for calculations. For IIF, the highest level of reactivity, independent of pattern, was selected for analysis. For antigen-specific immuno-assays, the highest level of reactivity from the PR3- and MPO-ANCA determinations was selected for analysis. Panel B. ROC curve analysis for different approaches for evaluation of results obtained by EuroPattern. ROC curves for following approaches are given: (i) all reactivities were considered ANCA-positive (red), (ii) only single positive C- and P-ANCA were considered positive (green), and (iii) only C- and P-ANCA, including any combinations were considered positive (blue).
• Performance characteristics of indirect immunofluorescence for ANCA depend on the substrate used and the definition of patterns. • The performance of automated indirect immunofluorescence for ANCA is improved by multi-parameter testing. • The highest likelihood ratio for ANCA-associated vasculitis was found for the combination C-ANCA/PR3-ANCA.
319 320 321 322 323
References
324
[1] Cohen Tervaert JW, Damoiseaux J. Antineutrophil cytoplasmic autoantibodies: how are they detected and what is their use for diagnosis, classification and followup? Clin Rev Allergy Immunol 2012;43:211–219. [2] Radice A, Bianchi L, Sinico RA. Anti-neutrophil cytoplasmic autoantibodies: methodological aspects and clinical significance in systemic vasculitis. Autoimmun Rev 2013;12:487–495. [3] Csernok E, Moosig F. Current and emerging techniques for ANCA detection in vasculitis. Nat Rev Rheumatol 2014;10:494–501. [4] Savige J, Gillis D, Benson E, Davies D, Esnault V, Falk RJ, et al. International consensus statement on testing and reporting of antineutrophil cytoplasmic antibodies (ANCA). Am J Clin Pathol 1999;111:507–513. [5] Damoiseaux J, Steller U, Buschtez M, et al. EuroPLUS ANCA BIOCHIP mosaic: PR3 and MPO antigen microdots improve the laboratory diagnostics of ANCA-associated vasculitis. J Immunol Methods 2009;348:67–73. [6] Sowa M, Grossmann K, Knütter I, Hiemann R, Röber N, Anderer U, et al. Simultaneous automated screening and confirmatory testing for vasculitis-specific ANCA. PLoS One 2014;9, e107743. [7] Krause C, Ens K, Fechner K, Voigt J, Fraune J, Rohwäder E, et al. EuroPattern suite technology for computer-aided immunofluorescence microscopy in autoantibody diagnostics. Lupus 2015;24:516–529. [8] Wiik A, Rasmussen N, Wieslander J. Methods to detect autoantibodies to neutrophilic granulocytes. Manual Biol Markers Dis 1993;A9:1–14. [9] Knütter I, Hiemann R, Brumma T, Büttner T, Großmann K, Cusini M, et al. Automated interpretation of ANCA patterns—a new approach in the serology of ANCAassociated vasculitis. Arthritis Res Ther 2012;14:R271. [10] Damoiseaux J, Mallet K, Vaessen M, Austen J, Cohen Tervaert JW. Automatic reading of ANCA-slides: evaluation of the AKLIDES system. Clin Dev Immunol 2012;2012: 762–74. [11] Pollock W, Clarke K, Gallagher K, Hall J, Luckhurst E, McEvoy R, et al. Immunofluorescent patterns produced by antineutrophil cytoplasmic antibodies (ANCA) vary depending on neutrophil substrate and conjugate. J Clin Pathol 2002;55: 680–683. [12] Savige J, Dimech W, Fritzler M, Goeken J, Hagen EC, Jennette JC, et al. International Group for Consensus Statement on Testing and Reporting of Antineutrophil Cytoplasmic Antibodies (ANCA). Addendum to the International Consensus Statement on testing and reporting of antineutrophil cytoplasmic antibodies. Quality control guidelines, comments, and recommendations for testing in other autoimmune diseases. Am J Clin Pathol 2003;120:312–318. [13] Choi HK, Liu S, Merkel PA, Colditz GA, Niles JL. Diagnostic performance of antineutrophil cytoplasmic antibody tests for idiopathic vasculitides: metaanalysis with a focus on antimyeloperoxidase antibodies. J Rheumatol 2001;28:1584–1590. [14] Avery TY, Bons J, Van Paassen P, Damoiseaux J. Diagnostic ANCA algorithms in daily clinical practice: evidence, experience, and effectiveness. Lupus 2016 [in press].
325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 Q4
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
Contents lists available at ScienceDirect
Autoimmunity Reviews journal homepage: www.elsevier.com/locate/autrev
7
F
O
Elena Csernok a,1, Jan Damoiseaux b,1, Niels Rasmussen c, Bernhard Hellmich a, Pieter van Paassen d, Pieter Vermeersch e,f, Daniel Blockmans g, Jan-Willem Cohen Tervaert h, Xavier Bossuyt i,j,⁎
R O
6Q2
8 9 10 11 12 13 14 15 16 17 Q3 18 19
a
2 0
a r t i c l e
21 22 23 24 29 28 27 26 25 46 47 48 49 50
Article history: Received 19 February 2016 Accepted 3 March 2016 Available online xxxx
Department of Internal Medicine, Rheumatology and Immunology, Vasculitis Center Tübingen-Kirchheim, University Hospital Kirchheim, Germany Central Diagnostic Laboratory, Maastricht University Medical Center, Maastricht, The Netherlands Department of Autoimmune Serology, Statens Serum Institute, Copenhagen, Denmark d Department of Internal Medicine, Section of Nephrology and Immunology, Maastricht University Medical Center, Maastricht, The Netherlands e Clinical Department of Laboratory Medicine, University Hospitals Leuven, 3000 Leuven, Belgium f Department of Cardiovascular Sciences, KU Leuven, Leuven, ,Belgium g Clinical Department of General Internal Medicine, Research Department of Microbiology and Immunology, Laboratory of Clinical Infectious and Inflammatory Disorders, University Hospitals Leuven, Leuven, Belgium h Maastricht University, The Netherlands i Department of Microbiology and Immunology, KU Leuven, Leuven, Belgium j Department of Laboratory Medicine, University Hospitals Leuven, Leuven, Belgium b
E
D
P
c
57 58 59 60 61 62
Contents 1. 2.
U
53
a b s t r a c t
N C O
Keywords: Anti-neutrophil cytoplasmic antibodies (ANCA) Granulomatosis with polyangiitis (GPA) Microscopic polyangiitis (MPA) Indirect immunofluorescence
54 52 51 56 55
i n f o
T
5
Objectives: The aim of this multicenter EUVAS study was to evaluate the diagnostic performance of multiparametric indirect immunofluorescence (IIF) assays to detect anti-neutrophil cytoplasmic antibodies (ANCA) in granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA). Patients and methods: The study included 912 samples from diseased controls and 249 diagnostic samples from GPA (n = 183) and MPA (n = 66) patients. The performance of two automated multi-parametric assays [Aklides (Medipan/Generic Assays) and EuroPattern (Euroimmun)] combining IIF on cellular and purified antigen substrates was compared with two manual IIF analyses and with commercially available ELISAs for MPO- and PR3-ANCA (Euroimmun). Results: The area under the curve (AUC) of the receiver operating characteristics (ROC) curve to discriminate AAV from controls was 0.925, 0.848, 0.855 and 0.904 for, respectively, the two manual analyses, Aklides and EuroPattern, and 0.959, 0.921 and 0.886 for, respectively, antigen-specific ELISA, antigen-coated beads, and microdot, respectively. Variation in pattern assignment between IIF methods was observed. Conclusion: The performance of IIF depends on the substrate used and the definition of IIF patterns. The performance of automated IIF is improved by multi-parameter testing (combined IIF and antigen-specific testing). Given the variability between IIF methods, the diagnostic importance of this technique is questioned. © 2016 Published by Elsevier B.V.
C
4
E
3
Evaluation of automated multi-parametric indirect immunofluorescence assays to detect anti-neutrophil cytoplasmic antibodies (ANCA) in granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA)
R
2Q1
Review
R
1
Introduction . . . . . . . . . . . . . . . . Material and methods . . . . . . . . . . . 2.1. Patients . . . . . . . . . . . . . . 2.2. ANCA detection methods . . . . . . 2.2.1. Manual IIF . . . . . . . . . 2.2.2. Automated multi-parameter IIF
. . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
⁎ Corresponding author at: Laboratory Medicine, University Hospitals Gasthuisberg, Herestraat 49, 3000 Leuven, Belgium. Tel.: +32 16 347009; fax: +32 16 34 79 31. E-mail address: [email protected] (X. Bossuyt). 1 Share first authorship.
http://dx.doi.org/10.1016/j.autrev.2016.03.010 1568-9972/© 2016 Published by Elsevier B.V.
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
0 0 0 0 0 0
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
2
63 64 65 66 67 68
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
2.2.3. ELISA . . 2.3. Statistical analysis 3. Results . . . . . . . . 4. Discussion . . . . . . . Take-home messages . . . . References . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
0 0 0 0 0 0
69
2.1. Patients
93
110 111
The patients included are from a multicenter EUVAS (European Vasculitis Study Group). Patients (n = 249) with AAV (GPA and MPA) and controls (n = 912) with inflammatory disease were recruited at Klinikum Bad Bramstedt (Germany) (190 controls, 38 GPA, 26 MPA), Statens Serum Institute Copenhagen (Denmark) (237 controls, 38 GPA, 9 MPA), University Hospitals Leuven (Belgium) (235 controls, 39 GPA, 15 MPA) and Maastricht University Medical Center (The Netherlands) (250 controls, 68 GPA, 16 MPA). The samples included were diagnostic samples, i.e. obtained at the time the diagnosis was established. The disease controls recruited in Copenhagen, Leuven and Maastricht were consecutive patients in whom ANCA analysis was requested, but in whom AAV was excluded later. Patients in whom inflammatory bowel disease and/or autoimmune liver disease was considered were excluded. The disease controls in Bad Bramstedt included cohorts of patients with systemic lupus erythematosus (n = 59), rheumatoid arthritis (n = 89), systemic sclerosis (n = 11) and Sjögren's syndrome (n = 30). For analysis, patients with GPA and MPA were grouped as AAV. This study was approved by the ethics committee of each participating center.
112
2.2. ANCA detection methods
113
2.2.1. Manual IIF IF performed at Bad Bramstedt (Germany) was done on ethanolfixed neutrophils in combination with additional tests on formalinfixed neutrophils and HEp-2 cells to better discriminate between P-ANCA (or atypical-ANCA) and ANA, as previously described [3]. IIF performed at the Staten Serum Institut (Copenhagen, Denmark) used an ethanol-fixed mixture of neutrophils and lymphocytes, as previously described [8].
96 97 98 99 100 101 102 103 104 105 106 107 108 109
114 115 116 117 118 119 120
C
E
94 95
R
87 88
R
86
O
84 85
C
82 83
N
80 81
U
78 79
2.2.3. ELISA Antibodies against PR3 and MPO were detected by monospecific ELISA (anti-PR3-hn-hr.-ELISA (IgG) and anti-myeloperoxidase ELISA (IgG); Euroimmun AG).
150
2.3. Statistical analysis
154
F
92
76 77
O
2. Material and methods
74 75
R O
91
72 73
P
89 90
Anti-neutrophil cytoplasmic antibodies (ANCA) are recognized as important biomarkers for the diagnosis of ANCA-associated vasculitis (AAV), in particular, granulomatosis with polyangiitis (GPA) and microscopic polyangiitis [1–3]. According to the 1999 international consensus, indirect immunofluorescence (IIF) is the optimal assay for ANCA screening; if positive, reflex testing for both proteinase (PR)3- and myeloperoxidase (MPO)-ANCA is mandatory [4]. The methods for detection and clinical utility of ANCA are extensively reviewed [1–3]. The classical two-tier approach, i.e. IIF followed by antigen-specific immuno-assays, can be combined in a single multi-parametric IIF assay. The EuroPLUS™ ANCA biochip mosaic combines both ethanoland formalin-fixed neutrophil substrates with PR3 and MPO antigen microdots in a single assay [5]. The Medipan Cytobead IIF assay combines ethanol-fixed neutrophil substrates with beads that have been differentially coated with PR3 and MPO [6]. Importantly, both multiparametric IIF assays can be analyzed by automated pattern recognition software, the EuroPattern and Aklides, respectively [6,7]. In the recently performed multicenter EUVAS study, we evaluated the performance of both the ANCA biochip mosaic and the Cytobead assay, as analyzed by the respective pattern recognition software.
121
D
71
2.2.2. Automated multi-parameter IIF IIF at Euroimmun was performed on the EUROPLUS™ Granulocyte Mosaic (Euroimmun AG, Lübeck, Germany), which contains six different biochips per reaction field: ethanol-fixed human granulocytes, formalin-fixed granulocytes, ethanol-fixed mixtures of HEp-2 cells and granulocytes and PR3, MPO, and GBM microdots (purified antigens) [5]. Automated evaluation was performed by EUROPattern Suite [7]. With this system, a rule-based engine creates a result based on the following classifications: (1) C-ANCA (cytoplasmic staining on ethanoland formalin-fixed granulocytes); (2) P-ANCA (perinuclear staining on ethanol-fixed granulocytes confirmed by a cytoplasmic staining on formalin-fixed granulocytes); (3) “atypical” ANCA (ANCA patterns not associated with AAV, e.g. DNA-ANCA (a perinuclear staining on ethanol-fixed granulocytes, but no staining on formalin-fixed granulocytes; associated with diseases like IBD), ANA interference, atypical C-ANCA; and (4) negative. The microdots were reported independently from the cell substrates as PR3-, MPO-, GBM-positive or negative. The final result was accompanied by titers and confidence values and verified by an expert. According to the EuroPattern classifier, only C- and P-ANCA are to be considered ANCA-positive. Automated IIF on Aklides® and antigen-specific ANCA by CytoBead ANCA [Medipan/Generic Assays GmbH (Berlin, Germany)] was performed by Medipan according to the manufacturer's instructions [6,9]. Briefly, CytoBead® ANCA, a multiplex IIF test, combines the screening of ANCA on ethanol-fixed neutrophils and their confirmation with multiplex microbead immunoassays, using recombinant PR3 and MPO. The final IIF read out is expressed as arbitrary units. The patterns reported included C-ANCA, P-ANCA and unrecognized/atypical (labeled as atypical). All reactivities were considered ANCA positive.
E
1. Introduction
T
70
122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149
151 152 153
ROC curve analysis was performed using Analyse-it for Microsoft 155 Excel 3.90. 156 3. Results
157
Table 1 gives an overview of (i) ANCA by manual [Bad Bramstedt and Copenhagen] and automated [Aklides (Medipan) and EuroPattern (Euroimmun)] IIF patterns and (ii) PR3-ANCA and MPO-ANCA by ELISA, antigen-coated beads (Aklides) and microdots (EuroPattern). The table shows all patterns in the way they were reported by their respective laboratories or by the respective software. For calculation of IIF performance characteristics, all reactivities were considered positive, except for EuroPattern for which atypical ANCA was considered negative (see Material and methods). The specificity (determined in 912 controls) of manual IIF was 94% for Bad Bramstedt and 79% for Copenhagen. The specificity of automated IIF patterns was 89% for Aklides and 90% for EuroPattern. Copenhagen assigned P-ANCA to a large fraction of controls (169/912). EuroPattern reported the simultaneous presence of ≥2 different ANCA patterns in 29/912 controls.
158 159
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
160 161 162 163 164 165 166 167 168 169 170 171 172
N
C
Table 1 Overview of the results for C-ANCA, P-ANCA, A-ANCA (atypical ANCA) by manual IIF performed at Bad Bramstedt (BB) and Copenhagen (Cop) and by automated EuroPattern (Euroimmun) and Aklides (Medipan). PR3-ANCA and MPO-ANCA was performed by ELISA (Euroimmun), by Aklides® Cytobead and by microdots (EuroPattern). Cutoff values proposed by the manufacturer were used. Diagnostic performance characteristics are given as well.
t1:4 t1:5 t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15 t1:16 t1:17 t1:18 t1:19 t1:20 t1:21 t1:22 t1:23 t1:24 t1:25 t1:26 t1:27 t1:28 t1:29 t1:30 t1:31 t1:32 t1:33 t1:34
Controls n = 912
GPA n = 183
MPA n = 66
Specificity Sensitivity GPA Sensitivity MPA Sensitivity AAV LR(+) AAV LR(−) AAV AUC AAV (95% CI)
O
C-ANCA
P-ANCA
A-ANCA
BB IIF (manual)
Neg Neg Neg Neg Pos Pos Pos Pos Neg Neg Neg Neg Pos Pos Pos Neg Neg Neg Neg Pos Pos Pos
Neg Neg Pos Pos Neg Neg Pos Pos Neg Neg Pos Pos Neg Neg Pos Neg Neg Pos Pos Neg Neg Pos
Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Neg Pos Neg Pos Neg Pos Neg
857 6 32 17
Cop IIF (manual)
Aklides IIF (automated)
EuroPattern IIF (automated)
PR3-ANCA
MPO-ANCA
Euroimmun ELISA
Aklides beads (automated)
EuroPattern microdots (automated)
717
815 64 27
306 513 43 14 21 2 12 1 7 12 14 3 126
Neg
Neg
883
863
858
Neg
Pos
19
31
33
Pos
Neg
7
16
15
Pos
Pos
3
2
6
Neg
Neg
19
22
27
Neg
Pos
20
18
18
Pos
Neg
144
139
128
Pos Neg
Pos Neg
6
4 16
10 15
Neg
Pos
57
45
48
Pos
Neg
3
3
2
0.968 0.896 0.909 0.900 28.3 0.10 0.959 (0.942–0.977)
2 0.946 0.880 0.758 0.847 15.8 0.16 0.921 (0.894–0.947)
1 0.941 0.852 0.773 0.831 14.0 0.18 0.886 (0.862–0.911)
R 169 25
R
1
E 6
C
21
36
20
27
62 21 8
142
119
92
6
1 4
56
59
9 23 32
4
3
2
0.940 0.885 0.909 0.892 14.8 0.12 0.925 (0.904–0946)
0.786 0.803 0.939 0.839 3.9 0.20 0.848 (0.821–0, 876)
0.894 0.661 0.864 0.715 6.7 0.32 0.855 (0.828–0.882)
T
E
D
21 2 6 39 7 3 1 8 0.898 0.896 0.879 0.892 8.7 0.12 0.904 (0.882–0.927)
P
Pos
R O Pos
O
F
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
t1:1 t1:2 t1:3
U
3
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213
t2:1 t2:2 t2:3 t2:4 t2:5 t2:6 t2:7 t2:8 t2:9 t2:10 t2:11 t2:12 t2:13 t2:14 t2:15 t2:16 t2:17 t2:18 t2:19 t2:20 t2:21 t2:22 t2:23 t2:24 t2:25 t2:26
244
Nowadays, it is to be expected that automated ANCA IIF, which combines analysis of cellular and antigen-specific substrates, will increasingly be introduced in clinical laboratories. The benefits of these systems include a higher reproducibility and shorter hands-on time. In this EUVAS study we assessed the reliability of these multiparametric assays by evaluation of two automated pattern recognition systems. Analysis of neutrophil substrates is combined with a beadbased assay in the Aklides system and with PR3- and MPO-microdots in the EuroPattern system [Sowa 2014; Krause 2015]. For comparison, we included two manual established IIF methods and an ELISA for PR3- and MPO-ANCA. The results obtained in this study demonstrate that the automated IIF assays from both manufactures vary in their test characteristics. Differences observed may be explained by the use of different substrates, different algorithms for automatic pattern assignment, and different definitions of ANCA patterns. When applying the pattern definitions of the manufacturers, the Aklides system appeared less effective in recognition of both C- and P-ANCA patterns in sera of GPA and MPA patients, respectively. The EuroPattern Suite, on the other hand, identified these patterns equally good as both reference laboratories, but additional patterns, i.e. the reciprocal pattern and/or atypical ANCA, were reported simultaneously in about 10% of the AAV sera. This seems to be the resultant of the pattern recognition devices since these phenomena were not observed by manual reading of the respective ANCA slides in previous studies [5,10]. The results obtained by the automatic pattern recognition software devices, however, require confirmation by the operator. This enables correction of apparent misclassifications. The combination of ethanol- and formalin-fixed neutrophils and HEp-2 cells, as applied in Bad Bramstedt and in the ANCA biochip, enables better identification of vasculitis-associated ANCA patterns [1,2,11]. Indeed, these assays revealed similar ROC curves for the cellular substrates. Also the manual IIF method of Copenhagen and the ANCA patterns obtained by Aklides, both only relying on ethanol-fixed neutrophils, had similar ROC curves,
245
O
F
4. Discussion
R O
194 195
P
192 193
214 215
D
190 191
(13% of patients) was, respectively, 42 and 18. Similarly, with EuroPattern, the highest LR(+) (113) was found for the combination C-ANCA/PR3-ANCA (50% of patients). This was higher than the LR(+) of C-ANCA (32) or of PR3-ANCA (21) alone. The LR(+) for the P-ANCA/MPO-ANCA combination (21% of patients) was 14. Further investigation of the diagnostic performance characteristics of IIF on cellular and antigen-specific substrates was done by receiver operating characteristic (ROC) curve analysis. To perform such analysis, patients with GPA and MPA were clustered as AAV (n = 249). The highest level of reactivity, independent of ANCA pattern or of antigenspecificity, was selected for analysis. This was feasible as similar cutoff values for PR3- and MPO-ANCA were used. The results are shown in Fig. 1 and Table 1. The area under the curve (AUC) of ANCA patterns assigned by Aklides (0.85) was comparable to the AUC of manual IIF Copenhagen (0.85), but lower than the AUC of ANCA pattern recognition by EuroPattern (0.90) and Bad Bramstedt (0.92) (p b 0.0001). The highest AUC was found for the antigen-specific Euroimmun ELISAs (0.96) (p b 0.0001 for comparison with all IIF methods). The AUC for antigen-coated beads and microdot was, respectively, 0.92 and 0.89 (p = 0.003). For analysis of data generated by EuroPattern, atypical ANCA were not considered positive (see Material and methods). Possible alternative ways to analyze the data (i) consider only single positive C- and P-ANCA as AAV-related and any combination of patterns as AAV nonrelated or (ii) consider all reactivities, including atypical ANCA as positive. These options reveal an AUC (95% CI) of 0.839 (0.810–0.868) and 0.848 (0.822–0.874), respectively. If only C-ANCA, P-ANCA, or combinations of C-ANCA and P-ANCA or C-ANCA and atypical ANCA or P-ANCA and atypical ANCA are considered as related to AAV, as programmed in the EuroPattern software, the AUC is 0.904 (0.882–0.927).
T
188 189
C
186 187
E
184 185
R
182 183
R
180 181
Table 2 Likelihood ratios (LR) for (i) ANCA by IIF (C-ANCA, P-ANCA, A-ANCA); (ii) PR3-ANCA and MPO-ANCA by ELISA, CytoBead, and microdots; and (iii) a combination of IIF and antigenspecific multi-parameter analysis [Aklides (IIF and antigen-coated beads) and EuroPattern (IIF and microdots)].
O
179
Aklides
PR3
MPO
Neg Neg Pos Pos
Neg Pos Neg Pos
EuroPattern
PR3 Neg Neg Pos Pos
MPO Neg Pos Neg Pos
IIF Bad Bramstedt IIF Copenhagen ELISA
PR3 Neg Neg Pos
C
177 178
C-ANCA
P-ANCA
A-ANCA
Neg
1.8 0.0 111.1
0.0 12.2 0.0
0.7 42.1 18.3
57.4
5.4
2.5
0.1 0.0 16.8 0.0 0.32
C-ANCA 0.0
P-ANCA 0.6 14.1
C-, P-ANCA 0.0
7.3 7.3 P-ANCA 8.7 P-ANCA 1.9
10.1 6.7 A-ANCA 0.0 A-ANCA 3.7
N
175 176
The sensitivity of manual IIF for AAV (183 GPA and 66 MPA patients) was 89% for Bad Bramsted and 84% for Copenhagen. The sensitivity of automated IIF pattern analysis for AAV was 72% for Aklides and 89% for EuroPattern. Breakdown of AAV into GPA and MPA revealed that the low sensitivity of Aklides was related to the low sensitivity of C-ANCA in GPA and, to a lesser extent, P-ANCA in MPA. As a consequence, the sensitivity of Aklides for GPA was low (66%). Moreover, Aklides reported atypical ANCA in a substantial portion of GPA (21/183) and MPA patients (23/66). EuroPattern, on the other hand, reported the combination of ANCA patterns in 24/183 GPA and 16/66 MPA patients. The sensitivity of PR3- and MPO-ANCA for AAV was 90% for ELISA, 85% for the antigen-coated beads (Aklides) and 83% for microdots (EuroPattern). The lower sensitivity of the antigen-coated beads and microdots was mainly related to the low sensitivity of MPO-ANCA. As a consequence, these two assays had a low sensitivity for MPA (76% and 77%, respectively). The likelihood ratio (LR) for AAV of a positive test result [LR(+)] obtained by the cellular substrates was 14.8 for Bad Bramstedt, 3.9 for Copenhagen, 6.7 for Aklides and 8.7 for EuroPattern. For the antigenspecific methods, the LR(+) was 28.3 for ELISA, 15.8 for antigencoated beads (Aklides) and 14.0 for microdots (EuroPattern). The LR of a negative test result [LR(−)] obtained by the cellular substrates was 0.12 for Bad Bramstedt, 0.20 for Copenhagen, 0.32 for Aklides and 0.12 for EuroPattern. The high LR(−) for Aklides is related to the low sensitivity for C-ANCA. For the antigen-specific methods, the LR(−) was 0.10 for ELISA, 0.16 for antigen-coated beads and 0.18 for microdots. A breakdown of the LR according to IIF pattern and antigenspecificity is shown in Table 2. For all methods on cellular substrates, the highest LR(+) was found for C-ANCA, varying between 18 (Copenhagen) and 57 (Aklides). The LR(+) associated with P-ANCA varied between 4.5 and 8.7, except for Copenhagen for which it was 1.9. Similarly, for antigen-specific ANCA, the highest LR(+) was found for PR3-ANCA. It was 77 for ELISA and 32 for antigen-coated beads and microdots. The LR(+) associated with MPO-ANCA was 15 for ELISA and 7 for antigen-coated beads and microdots. With Aklides, the combined presence of C-ANCA and PR3-ANCA (36.5% of patients) had the highest LR(+) (111) for AAV, higher than the LR(+) of C-ANCA (57) or of PR3-ANCA (32) alone. The LR(+) of P-ANCA, MPO-ANCA or the P-ANCA/MPO-ANCA combination (16% of patients) was, respectively, 5.4, 7.4 and 12.2. The LR(+) for the combined presence of atypical ANCA and either MPO-ANCA or PR3-ANCA
U
173 174
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx
E
4
MPO Neg Pos Neg
113.5 11.0 31.7 C-ANCA 31.5 C-ANCA 17.9
Neg 0.1 0.8 1.2 3.7 0.18 Neg 0.12 Neg 0.17
0.16 7.4 32.5 11.0
0.12 4.0 20.7 8.2
0.10 14.8 76.9
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277
E. Csernok et al. / Autoimmunity Reviews xxx (2016) xxx–xxx 1
1
A
0.9
B
0.9
0.8
0.8
0.7
0.7
PR3/MPO ELISA IIF Copenhagen
0.6
Sensitivity
Sensitivity
5
IIF Bad Bramstedt
0.5
Aklides IIF EuroPattern IIF
0.4
EUROPattern - C-ANCA and PANCA
0.6 0.5
EUROPattern - C-ANCA, P-ANCA and combinations
0.4
EUROPattern - all reactivities
0.3
0.3
0.2
0.2
0.1
0.1 0 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.1
0.2
0.3
1 - Specificity
0.4
0.5
0.6
0.7
0.8
O
0
F
Aklides PR3/MPO CytoBead
0.9
1
1 - Specificity
Take-home messages
292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308
312 313 314 315 316 317 318
C
E
290 291
R
288 289
R
286 287
N C O
284 285
U
282 283
D
311
280 281
• PR3- and MPO-ANCA ELISAs had the best performance characteristics for ANCA-associated vasculitis. • Given the variability between indirect immunofluorescence methods for ANCA, the value of this technique may be questioned.
T
309 310
although with lower AUCs. Detailed comparison of the four IIF approaches was somewhat hampered by the distinct definitions applied for the ANCA patterns. When other definitions were applied, including the ANCA pattern definitions of the international consensus [4], then this had a significant impact on the performance of the assays. Obviously, the use of different definitions for ANCA patterns does not benefit standardization and also hampers the use of ANCA patterns for other chronic inflammatory diseases [12]. Overall, PR3- and MPO-ANCA by ELISA had the best overall diagnostic performance characteristics with the highest AUC of ROC curve analysis, the highest LR(+) and lowest LR(−). Test characteristics for PR3and MPO-ANCA of both automated systems were quite similar to each other. Importantly, detection of MPO-ANCA by Aklides and EuroPattern was less sensitive than the MPO-ANCA ELISA. Data were also analyzed for the simultaneous recognition of C-ANCA and PR3-ANCA, or PANCA and MPO-ANCA, in the same sample. The specificity of these combinations for AAV was about 98.5% for both multi-parametric assays; this specificity was also reported in a meta-analysis of conventional ANCA assays [13]. The sensitivity of Aklides (53%) and EuroPattern (70%) was substantially lower than the sensitivity of 85% reported in the meta-analysis. Interestingly, both multi-parametric assays revealed that the positive likelihood ratio for the combination C-ANCA/PR3ANCA was about ten-fold higher than for the combination P-ANCA/ MPO-ANCA. Also in routine clinical practice the combined presence of P-ANCA and MPO-ANCA has appeared less specific for AAV [14]. In conclusion, our study demonstrates that the test characteristics of the automated IIF depends on the substrates used and the ANCA pattern definitions applied. The performance characteristics, in particular the LR(+), of the automated IIF is improved by multi-parameter testing, i.e. if the ANCA pattern is in agreement with the ANCA specificity. Furthermore, this study indicates that the diagnostic importance of IIF may be questioned since the combination of PR3- and MPO-ANCA ELISAs had the best performance for the diagnosis of AAV.
E
278 279
P
R O
Fig. 1. Panel A. Receiver operating characteristics (ROC) curve for two manual IIF methods [performed at Copenhagen (C) and at Bad Bramstedt (BB)], two automated IIF methods (EuroPattern and Aklides), and three antigen-specific immuno-assays [ELISA, antigen-coated beads and microdots] for ANCA detection. For automated IIF, the fluorescence intensity was used for calculations. For IIF, the highest level of reactivity, independent of pattern, was selected for analysis. For antigen-specific immuno-assays, the highest level of reactivity from the PR3- and MPO-ANCA determinations was selected for analysis. Panel B. ROC curve analysis for different approaches for evaluation of results obtained by EuroPattern. ROC curves for following approaches are given: (i) all reactivities were considered ANCA-positive (red), (ii) only single positive C- and P-ANCA were considered positive (green), and (iii) only C- and P-ANCA, including any combinations were considered positive (blue).
• Performance characteristics of indirect immunofluorescence for ANCA depend on the substrate used and the definition of patterns. • The performance of automated indirect immunofluorescence for ANCA is improved by multi-parameter testing. • The highest likelihood ratio for ANCA-associated vasculitis was found for the combination C-ANCA/PR3-ANCA.
319 320 321 322 323
References
324
[1] Cohen Tervaert JW, Damoiseaux J. Antineutrophil cytoplasmic autoantibodies: how are they detected and what is their use for diagnosis, classification and followup? Clin Rev Allergy Immunol 2012;43:211–219. [2] Radice A, Bianchi L, Sinico RA. Anti-neutrophil cytoplasmic autoantibodies: methodological aspects and clinical significance in systemic vasculitis. Autoimmun Rev 2013;12:487–495. [3] Csernok E, Moosig F. Current and emerging techniques for ANCA detection in vasculitis. Nat Rev Rheumatol 2014;10:494–501. [4] Savige J, Gillis D, Benson E, Davies D, Esnault V, Falk RJ, et al. International consensus statement on testing and reporting of antineutrophil cytoplasmic antibodies (ANCA). Am J Clin Pathol 1999;111:507–513. [5] Damoiseaux J, Steller U, Buschtez M, et al. EuroPLUS ANCA BIOCHIP mosaic: PR3 and MPO antigen microdots improve the laboratory diagnostics of ANCA-associated vasculitis. J Immunol Methods 2009;348:67–73. [6] Sowa M, Grossmann K, Knütter I, Hiemann R, Röber N, Anderer U, et al. Simultaneous automated screening and confirmatory testing for vasculitis-specific ANCA. PLoS One 2014;9, e107743. [7] Krause C, Ens K, Fechner K, Voigt J, Fraune J, Rohwäder E, et al. EuroPattern suite technology for computer-aided immunofluorescence microscopy in autoantibody diagnostics. Lupus 2015;24:516–529. [8] Wiik A, Rasmussen N, Wieslander J. Methods to detect autoantibodies to neutrophilic granulocytes. Manual Biol Markers Dis 1993;A9:1–14. [9] Knütter I, Hiemann R, Brumma T, Büttner T, Großmann K, Cusini M, et al. Automated interpretation of ANCA patterns—a new approach in the serology of ANCAassociated vasculitis. Arthritis Res Ther 2012;14:R271. [10] Damoiseaux J, Mallet K, Vaessen M, Austen J, Cohen Tervaert JW. Automatic reading of ANCA-slides: evaluation of the AKLIDES system. Clin Dev Immunol 2012;2012: 762–74. [11] Pollock W, Clarke K, Gallagher K, Hall J, Luckhurst E, McEvoy R, et al. Immunofluorescent patterns produced by antineutrophil cytoplasmic antibodies (ANCA) vary depending on neutrophil substrate and conjugate. J Clin Pathol 2002;55: 680–683. [12] Savige J, Dimech W, Fritzler M, Goeken J, Hagen EC, Jennette JC, et al. International Group for Consensus Statement on Testing and Reporting of Antineutrophil Cytoplasmic Antibodies (ANCA). Addendum to the International Consensus Statement on testing and reporting of antineutrophil cytoplasmic antibodies. Quality control guidelines, comments, and recommendations for testing in other autoimmune diseases. Am J Clin Pathol 2003;120:312–318. [13] Choi HK, Liu S, Merkel PA, Colditz GA, Niles JL. Diagnostic performance of antineutrophil cytoplasmic antibody tests for idiopathic vasculitides: metaanalysis with a focus on antimyeloperoxidase antibodies. J Rheumatol 2001;28:1584–1590. [14] Avery TY, Bons J, Van Paassen P, Damoiseaux J. Diagnostic ANCA algorithms in daily clinical practice: evidence, experience, and effectiveness. Lupus 2016 [in press].
325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 Q4
Please cite this article as: Csernok E, et al, Evaluation of automated multi-parametric indirect immunofluorescence assays to detect antineutrophil cytoplasmic antibodies (ANCA) ..., Autoimmun Rev (2016), http://dx.doi.org/10.1016/j.autrev.2016.03.010