- Email: [email protected]
Diagnostic accuracy of solid phase HLA antibody assays for prediction of crossmatch strength
Human Immunology 73 (2012) 706–710
Contents lists available at SciVerse ScienceDirect
www.ashi-hla.org
journal homepage: www.elsevier.com/locate/humimm
Diagnostic accuracy of solid phase HLA antibody assays for prediction of crossmatch strength q Thomas M. Ellis a,⇑, Jennifer J. Schiller a, Allan M. Roza b, David C. Cronin b, Brian D. Shames b,1, Christopher P. Johnson b a b
Histocompatibility and Immunogenetics Laboratory, BloodCenter of Wisconsin, 638 N., 18th Street, P.O. Box 2178, Milwaukee, WI 53201, USA Division of Transplant Surgery, Medical College of Wisconsin, 8701 Watertown Plank Rd., Milwaukee, WI 53226, USA
a r t i c l e
i n f o
Article history: Received 25 October 2011 Accepted 16 April 2012 Available online 23 April 2012
a b s t r a c t Solid phase antibody assays are increasingly used to provide quantitative measures of donor-specific HLA antibodies for assessment of pretransplant risk, although cell-based crossmatches continue to serve as gold standards for determination of donor HLA antibody strength. This study determined the ability of HLA antibody solid phase assays to predict the strength of cell-based flow cytometric (FC) and complement-dependent cytotoxicity (CDC) crossmatches. Eighty-two recipient/donors pairs were analyzed using receiver operating characteristic (ROC) curve analyses to determine the accuracy of donor-specific median fluorescence intensity values (R MFI) from single antigen bead assays for predicting strong FC and CDC crossmatches. Diagnostic sensitivity and specificity of optimal R MFI values were highest for predicting strong T cell FCs. R MFI values showed good sensitivity for predicting positive direct and AHG-augmented CDC crossmatches (91% and 94%, respectively), but with lower specificity (67% each). Specificity and sensitivity for predicting positive B cell CDC crossmatches were 73% and 84%. R MFI values derived from single antigen bead assays can predict strong flow and positive CDC crossmatches, but with tradeoffs between sensitivity and specificity. The results support the use of solid phase assays for quantitative virtual crossmatching and as a replacement for cell-based crossmatching. Ó 2012 American Society for Histocompatibility and Immunogenetics. Published by Elsevier Inc. All rights reserved.
1. Introduction Successful transplantation of allosensitized recipients has been facilitated by the availability of solid phase HLA antibody assays that provide sensitive and specific HLA antibody identification. The ability of solid phase antibody assays to accurately detect and identify HLA antibodies allows for the prediction of recipient/donor crossmatch compatibility by virtual crossmatching Abbreviations: AHG, anti-human globulin; AUC, area under the curve; CDC, complement-dependent cytotoxicity; CI, confidence interval; DSA, donor-specific antibody; FC, flow cytometric; MCS, median channel shift; MFI, mean fluorescence intensity; ROC, receiver operating characteristic. q The work described in this article has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. ⇑ Corresponding author. Present address: Department of Pathology and Lab Medicine, University of Wisconsin-Madison, 600 Highland Avenue, Madison, WI 53792-2472, USA. E-mail address: [email protected] (T.M. Ellis). 1 Present Address: Hartford Transplant and Surgical Specialists, 85 Seymour St., Suite 301, Hartford, CT 06106, USA.
[1,2]. With the improved ability to transplant patients with donor-specific antibodies (DSA), a positive virtual or flow crossmatch (FC) alone is not a contraindication for transplant [3]. Many centers rely on quantitative measures of relative strength of donor-specific HLA antibody to assess allosensitization risk, such as flow crossmatch median channel shifts (MCS), FC titers, and reactivity in complement-dependent cytotoxic (CDC) crossmatches [4–6]. However, the strength of cell-based crossmatch continues to be widely regarded as the gold standard for assessing recipient/donor compatibility and level of risk for acute humoral rejection. Solid phase antibody assays offer a number of potential advantages over the use of cellular targets for quantitation of donor-specific antibodies, including ready availability, decreased biologic variability, lower non-specific reactivity, and potential for standardization [6–8]. In this study we assessed the diagnostic performance of solid phase antibody assays in predicting DSA strength measured using classical cell-based crossmatch assays. The longterm objective is to determine the feasibility of replacing cellbased crossmatches with solid phase antibody testing for assessment of allosensitization risk.
0198-8859/$36.00 - see front matter Ó 2012 American Society for Histocompatibility and Immunogenetics. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.humimm.2012.04.007
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
2. Materials and methods 2.1. Patients The study population comprises 82 recipient donor pairs evaluated for living donor renal transplantation from 2005 through 2009. All pairs were tested for T and/or B cell reactivity by FC crossmatch, and positive samples were also evaluated using FC crossmatch titrations (n = 61 T cell; n = 71 B cell) and by CDC crossmatching using direct, anti-human globulin (AHG)-augmented, and B cell techniques (n = 70). Thirty pairs proceeded to transplantation with crossmatched donors. 2.2. High resolution HLA antibody analysis HLA antibodies were identified using Luminex single antigen beads (One Lambda, Canoga Park CA), used according to the manufacturer’s instructions. DSA strength was were calculated by summing the unadjusted mean fluorescence intensity (MFI) of positive beads bearing donor-mismatched antigens and expressed as the R MFI value; background MFI values were subtracted only when these values exceeded 500. Positive beads were defined as those having MFIs of at least 1000, although this threshold could be adjusted between 800 and 1300 in select circumstances when supported by clear patterns of antibody epitope specificity. All background values were less than 2500. In cases where there were multiple beads in the panel for a given donor HLA antigen, only the bead with the highest MFI value was used to calculate the R MFI. 2.3. Crossmatching For flow cytometric crossmatches, peripheral blood mononuclear cells were isolated by treatment with carbonyl iron and dex-
707
tran followed by sedimentation on Ficoll-hypaque gradients. Cells were treated with pronase and 3-color FC crossmatches were performed according to established methods [9]. T and B cells were identified using PerCP-anti-CD3 and PE-anti-CD19 (BD Biosciences, San Jose, CA). The secondary antibody was a FITC-conjugated F(ab0 )2 fragment of goat anti-human IgG (heavy chain specific; Jackson ImmunoResearch, West Grove, PA). Cells were analyzed using a Becton-Dickinson FACSCalibur flow cytometer. Crossmatch titrations were performed using recipient sera diluted in fetal calf serum. MCS were calculated by subtracting median channel values obtained with donor cells incubated with pooled normal human serum from values obtained after staining with recipient serum. FC crossmatch titers were the reciprocal of the highest dilution of recipient serum yielding a positive crossmatch. Strong T and B cell crossmatches were those with a MCS P 150 and P 250, respectively. CDC crossmatch assays were performed using the NIH, B cell, and AHG-augmented techniques as described [9]. Cytolysis was determined by vital staining and a positive test defined as one with at least 50% target cell lysis. 2.4. Receiver operating characteristic (ROC) analysis ROC curves were generated by plotting the relationship between the true positive fraction (sensitivity) against the false positive fraction (1-specificity) at various R MFI values and calculating the resulting area under the curve (AUC) [10–12]. The variable in each case was the R MFI value, which was calculated using either donor-specific class I specificities alone (for T cell crossmatches) or with class II specificities (for B cell crossmatches). Gold standard classifiers were defined for each crossmatch assay analyzed. For CDC crossmatches, classifiers were simply a positive or negative CDC crossmatch result. Flow crossmatches were classified as
Fig. 1. ROC curves for R MFI values for discriminating recipient/donor pairs with and without strong T (a and c) and B cell (b and d) FC crossmatches. Panels a and b show curves for positive FC crossmatches with T cell MCS values P 150 (a) and B cell MCS values P 250 (b). Panels c and d show ROC curves for flow crossmatches with T cell titers P 8 (c) and B cell titers P 128 (d).
708
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
‘‘strongly positive’’ defined using either MCS cutoff values or titers. A test with an AUC of 1.0 is considered an ideal predictor, whereas an AUC of 0.5 indicates a random association between the test measure (R MFI) and the gold standard (crossmatch cutoff). ROC analyses were performed using MedCalc for Windows (v. 9.5.0.0, MedCalc Software, Mariakerke, Belgium). Optimal R MFI values were those calculated as those with optimal combined sensitivity and specificity for predicting the flow or CDC crossmatch result. 3. Results The relationship between solid phase MFI values and FC crossmatch results was determined using ROC curve analysis to determine the diagnostic accuracy of the R MFI value to predict weak and strong crossmatches defined using MCS value and titer cutoffs. ROC curve analysis is used increasingly in diagnostic testing to measure the ability of a diagnostic test measure to predict a disease or other outcome. ROC curves plot the sensitivity (true positive rate) against the false positive rate (100-specificity) for individual test values in a dataset (see Figs. 1 and 3). A test with perfect sensitivity and specificity will generate a plot close to the upper left corner, whereas a weaker association will trend toward a straight diagonal line from the origin toward the upper right, indicative of a random (50%) association between the diagnostic test and outcome. Thus, the accuracy of the test for predicting an end point can be represented by the resulting area under the curve and the strength of predictive ability determined if it differs significantly to the random association line (AUC = 50). ROC curve analyses of the R MFI value’s true versus false positive predictions are shown for T cell flow crossmatches with MCS > 150 (Fig. 1a) or titers >8 (Fig. 1c). R MFI values showed strong predictive accuracy for identifying T cell flow crossmatches with an MCS > 150, with an area under the curve (AUC) of 0.964 (95% confidence interval (CI) = 0.882–0.995; p = 0 .0001; Table 1). An AUC of 95% indicates that the R MFI predicted T cell crossmatch MCS values >150 in more than 95% of the comparisons [10–12]. The R MFI value (R MFI = 6282) with the highest diagnostic accuracy for distinguishing weak and strong crossmatches has a sensitivity (true positive rate) and specificity (true negative rate) of 90% and 95%, respectively (Table 1). R MFI values were also highly predictive of T cell flow crossmatch titers >8 (AUC = 0.915; 95% CI = 0.814–0.971; p = 0.0001). The lower AUC value (0.915) indicates R MFI values have lower accuracy for predicting strong T cell FC crossmatch titers than for MCS values. Further, the sensitivity and specificity for the R MFI cut point with the highest predictive accuracy (R MFI = 7964) were 76% and 100%, respectively. A sensitivity of 100% is obtained when the threshold titer is increased to 128, but with a reduction in specificity (69%). R MFI values showed good predictive accuracy for B cell crossmatches with MCS P 250 (AUC = 0.87; 95% CI = 0.764–0.940; p = 0.0001) and with high specificity (100%), but the sensitivity
P Fig. 2. Plots depicting MFI values with T MCS values (a) and B MCS values from FC crossmatching. Horizontal lines demarcate MCS cutoff values used to distinguish P strong T cell (150) and B cell (250) FC crossmatches. Vertical lines demarcate MFI values determined by ROC analysis to have the highest accuracy for predicting strong T and B cell FC crossmatches based on these MCS values.
was low (57%; Fig. 1b and d and Table 1). Expectedly, sensitivity increased to 85% when the MCS threshold was increased to 375, but with a significant decrease in specificity (56.4%). The predictive ability of R MFI values for B crossmatch titers P 512 was also low and statistically not significant (AUC = 0.61; 95% CI = 0.487– 0.723; p = 0.1110). A significant predictive relationship was achieved when the B cell titer was decreased to P 128 (p = 0.0004), although with reduced diagnostic specificity (56%) and sensitivity (80%). Overall, R MFI values showed lower predictive accuracy for B cell flow crossmatches than for T cells, regardless of endpoint chosen. The relationship between individual R MFI values and associated crossmatch MCS values is shown in Fig. 2a and b for T and B cell crossmatches, respectively. The R MFI values with the highest
Table 1 Summary statistics of ROC curve analysis of R MFI values and flow crossmatch results. Crossmatch criteria
AUC
p Value
R MFI with highest predictive accuracy
Sensitivity (%)
Specificity (%)
T cell MCS P 150 Titer P 8 Titer P 128
0.964 0.915 0.893
0.0001 0.0001 0.0001
>6282 >7964 >8248
90 76 100
95 100 69
B cell MCS P 250 MCS P 375 Titer P 128 Titer P 512
0.870 0.729 0.722 0.610
0.0001 0.0002 0.0004 0.1110
>12,693 >6282 >9608 >11,267
57 85 80 68
100 56 64 56
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
predictive value determined by ROC curve analyses and their associated T and B cell MCS cutoffs are depicted as dotted lines in each plot. The higher false negative (20/78; 26%) for R MFI values in predicting strong B cell crossmatches is evidenced by the higher numbers of events falling in the upper left (false positive) quadrants of Fig. 2b. For T cell crossmatches, false negatives (i.e., a R MFI value below that with highest predictive accuracy and T MCS > 150) comprised the majority of incorrect predictions (5/78; 6.4%; Fig. 2a). The ability of R MFI values to predict positive CDC crossmatches was analyzed in a subset of patients with positive flow crossmatches. The accuracy of R MFI values for predicting CDC crossmatch outcomes for T-NIH, AHG-augmented, and B cell CDC crossmatch methods are summarized in Table 2 and by the ROC curves shown in Fig. 3a,b and c. R MFI values showed a statistically significant ability to predict a positive CDC crossmatch for each method tested, although diagnostic sensitivity and specificity varied considerably among the methods. R MFI exhibited diagnostic sensitivities of 91%, 94% and 73% for predicting positive NIH, AHG and B cell CDC methods, respectively, while specificities were 67% (NIH and AHG CDC methods) and 84% (B cell CDC crossmatches). The lower specificity for R MFI values in predicting CDC crossmatches reflects a high false positive rate for predicting positive NIH and AHG CDC results. Thus, while R MFI values predict a significant majority of positive CDC+ crossmatches, it is less robust at identifying true CDC negative sera (larger false positive fraction). 4. Discussion The objective of this study was to determine whether solid phase HLA antibody measurements of donor-specific antibodies could accurately predict the strength of cellular crossmatch assays
709
used to assess level of donor-specific allosensitization. Although it is common practice to use single antigen bead assays to assess levels of HLA antibodies, these reagents are not intended by the manufacturers for this purpose [4–6]. Here we demonstrate the validity of using single antigen beads as a surrogate for cell-based crossmatching for use in identifying high risk recipient/donor pairs defined by crossmatch strength. ROC curve analysis provides a powerful tool for calibrating solid phase antibody results with cell-based assays of donor-specific HLA antibodies, which historically have served as the gold standard indicators for risk of early humoral rejection. In general, donor-specific HLA antibodies measured by solid phase assays and expressed as R MFI values showed good diagnostic accuracy for identifying recipient/donor pairs with either a strong positive FC crossmatch or positive CDC crossmatch. The R MFI value correlated most closely with results of T cell crossmatches, and showed high sensitivity and specificity for prediction of T cell crossmatches defined as ‘‘strong’’ (MCS P 150). Although the predictive value of the R MFI for B cell crossmatch results achieved statistical significance, this association had lower sensitivity, specificity, and correlative strength than observed for T cell flow crossmatches. While the weaker correlation with B cell crossmatches might reflect the decreased accuracy of R MFI values calculated using both Class I and Class II MFI values, it more likely reflects the increased biologic variability inherent in B cell flow crossmatches due to higher nonspecific backgrounds and variability [13–15]. The lower variability and backgrounds associated with solid phase assays represent a potential advantage over cell-based assays for detecting and quantifying donor-specific HLA antibodies. In our own practice, we have experienced excellent one year graft survivals in sensitized patients who have positive flow crossmatches and but were negative for donor-specific antibodies by solid phase assays [16].
Fig. 3. ROC curves for R MFI values for discriminating positive CDC crossmatches using T-NIH (a), AHG (b), and B cell (c) CDC crossmatch methods.
710
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710 Table 2 Summary statistics of ROC curve analysis of R MFI values and CDC crossmatch results. CDC method
AUC
p Value
R MFI with highest predictive accuracy
Sensitivity (%)
Specificity (%)
NIH+ AHG+ CDC-B+
0.837 0.860 0.810
0.0001 0.0001 0.0001
>8248 >8248 >10,374
94 94 73
69 67 84
Positive CDC crossmatches occur when high levels of complement-fixing, donor-specific antibodies are present. The sensitivity for predicting positive CDC crossmatches was greater than 90% for both T cell-based methods (NIH, AHG-augmented) but was low when B cells were used as targets. The low specificity reflects a significant proportion of test sera that had R MFIs above the calculated threshold, yet were negative in the CDC crossmatch. Thus, while a high R MFI value (>6000) was characteristic of all CDC positive crossmatches, not all high R MFI sera were CDC positive. Thus, positive CDC crossmatches cannot be predicted by antibody levels alone, and false negative predictions for high R MFI sera could occur due to the presence of high levels of non-complement fixing IgG isotypes although, to our knowledge, this has not been demonstrated directly. Alternatively, the low specificity may reflect the high variability of the CDC assay itself. In a survey of ASHI proficiency challenges for CDC crossmatching from 2008 to 2009, more than 30% positive samples tested failed to achieve 80% consensus positive among all labs, despite the fact that testing was performed using preselected sera and target cells (data not shown). Although additional outcomes-based studies are required to determine whether R MFI values provide a more stable and consistent measure of the level of donor-specific allosensitization and transplant risk than traditional CDC crossmatches, our studies indicate that solid phase MFI values may represent a more consistent and reproducible measure than the CDC crossmatch. The high sensitivity of R MFI values for predicting positive CDC crossmatches supports the diagnostic potential of using solid phase antibody assays to identify sera likely to be CDC positive. Previous groups documented the relationship between baseline levels of donor specific antibody and the risk for antibody-mediated allograft injury using either flow MCS or R MFI values [4,5]. Gloor et al. observed general agreement between high sum R MFI values and flow crossmatch MCS values >300, although the predictive relationship between crossmatch and MFI values was not directly measured [4]. Other studies also showed a correlation between DSA levels determined using single antigen beads and the likelihood of a positive flow crossmatch and risk for antibody-mediated rejection, although again the direct relationship between MFI values crossmatch strength was not determined [5,17]. In contrast to our studies, Morris et al. found that solid phase DSA measures can accurately predict a negative crossmatch but had limited sensitivity for predicting positive crossmatches [18]. This latter observation may reflect the greater sensitivity of single antigen beads for detecting DSA, as we have observed in our own experience (data not shown). This study supports the utility of R MFI values for quantitative virtual crossmatching in assessing donor-specific allosensitization levels and predicting crossmatch strength. While additional work
is needed to standardize R MFI values and determine clinically significant thresholds, these studies demonstrate the potential of R MFI in supporting transplantation where assessing recipient-donor compatibility where donor cells are unavailable or of low quality. Additional advantages over cell-based crossmatching include the ability to provide a rapid assessment of recipient compatibility with local or remote donor offers where a weak positive crossmatch is acceptable, and for initial screening and selection of potential living donors. References [1] Zangwill S, Ellis T, Stendahl G, Zahn A, Berger S, Tweddell J. Practical application of the virtual crossmatch. Pediatr Transplant 2007;11:650. [2] Zangwill SD, Ellis TM, Zlotocha J, Jaquiss RD, Tweddell JS, Mussatto KA, et al. The virtual crossmatch – a screening tool for sensitized pediatric heart transplant recipients. Pediatr Transplant 2006;10:38. [3] Gebel HM, Bray RA, Nickerson P. Pre-transplant assessment of donor-reactive, HLA-specific antibodies in renal transplantation: contraindication vs. risk. Am J Transplant 2003;3:1488. [4] Gloor JM, Winters JL, Cornell LD, Fix LA, Degoey SR, Knauer RM, et al. Baseline donor-specific antibody levels and outcomes in positive crossmatch kidney transplantation. Am J Transplant 2010;10:582. [5] Tambur AR, Ramon DS, Kaufman DB, Friedewald J, Luo X, Ho B, et al. Perception versus reality? Virtual crossmatch – how to overcome some of the technical and logistic limitations. Am J Transplant 2009;9:1886. [6] Cecka JM, Kucheryavaya AY, Reinsmoen NL, Leffell MS. Calculated PRA: initial results show benefits for sensitized patients and a reduction in positive crossmatches. Am J Transplant 2011;11:719. [7] El-Awar N, Lee J, Terasaki PI. HLA antibody identification with single antigen beads compared to conventional methods. Hum Immunol 2005;66:989. [8] Lefaucheur C, Loupy A, Hill GS, Andrade J, Nochy D, Antoine C, et al. Preexisting donor-specific HLA antibodies predict outcome in kidney transplantation. J Am Soc Nephrol 2010;21:1398. [9] Hahn A, Land G, Strothman R. ASHI laboratory manual; 2000. [10] Empson MB. Statistics in the pathology laboratory: characteristics of diagnostic tests. Pathology 2001;33:93. [11] Fawcett T. An introduction to ROC analysis. Pattern Recogn 2006;27:861. [12] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29. [13] Bray RA, Gebel HM. Strategies for human leukocyte antigen antibody detection. Curr Opin Organ Transplant 2009;14:392. [14] Gebel HM, Bray RA. Sensitization and sensitivity: defining the unsensitized patient. Transplantation 2000;69:1370. [15] Gebel HM, Bray RA, Ruth JA, Zibari GB, McDonald JC, Kahan BD, et al. Flow PRA to detect clinically relevant HLA antibodies. Transplant Proc 2001;33:477. [16] Johnson CP, Shames BD, Brown S, Roza AM, Cronin DCE. T.M. A Single Center Experience. Can a Virtual Crossmatch Replace the Prospective Crossmatch? Am J Transplant 2009;9:334. [17] Reinsmoen NL, Lai C-H, Vo A, Cao K, Ong G, Naim M, et al. Acceptable donorspecific antibody levels allowing for successful deceased and living donor kidney transplantation after desensitization therapy. Transplantation 2008;86:820. [18] Morris GP, Phelan DL, Jendrisak MD, Mohanakumar T. Virtual crossmatch by identification of donor-specific anti-human leukocyte antigen antibodies by solid-phase immunoassay: a 30-month analysis in living donor kidney transplantation. Hum Immunol 2010;71:268.
Contents lists available at SciVerse ScienceDirect
www.ashi-hla.org
journal homepage: www.elsevier.com/locate/humimm
Diagnostic accuracy of solid phase HLA antibody assays for prediction of crossmatch strength q Thomas M. Ellis a,⇑, Jennifer J. Schiller a, Allan M. Roza b, David C. Cronin b, Brian D. Shames b,1, Christopher P. Johnson b a b
Histocompatibility and Immunogenetics Laboratory, BloodCenter of Wisconsin, 638 N., 18th Street, P.O. Box 2178, Milwaukee, WI 53201, USA Division of Transplant Surgery, Medical College of Wisconsin, 8701 Watertown Plank Rd., Milwaukee, WI 53226, USA
a r t i c l e
i n f o
Article history: Received 25 October 2011 Accepted 16 April 2012 Available online 23 April 2012
a b s t r a c t Solid phase antibody assays are increasingly used to provide quantitative measures of donor-specific HLA antibodies for assessment of pretransplant risk, although cell-based crossmatches continue to serve as gold standards for determination of donor HLA antibody strength. This study determined the ability of HLA antibody solid phase assays to predict the strength of cell-based flow cytometric (FC) and complement-dependent cytotoxicity (CDC) crossmatches. Eighty-two recipient/donors pairs were analyzed using receiver operating characteristic (ROC) curve analyses to determine the accuracy of donor-specific median fluorescence intensity values (R MFI) from single antigen bead assays for predicting strong FC and CDC crossmatches. Diagnostic sensitivity and specificity of optimal R MFI values were highest for predicting strong T cell FCs. R MFI values showed good sensitivity for predicting positive direct and AHG-augmented CDC crossmatches (91% and 94%, respectively), but with lower specificity (67% each). Specificity and sensitivity for predicting positive B cell CDC crossmatches were 73% and 84%. R MFI values derived from single antigen bead assays can predict strong flow and positive CDC crossmatches, but with tradeoffs between sensitivity and specificity. The results support the use of solid phase assays for quantitative virtual crossmatching and as a replacement for cell-based crossmatching. Ó 2012 American Society for Histocompatibility and Immunogenetics. Published by Elsevier Inc. All rights reserved.
1. Introduction Successful transplantation of allosensitized recipients has been facilitated by the availability of solid phase HLA antibody assays that provide sensitive and specific HLA antibody identification. The ability of solid phase antibody assays to accurately detect and identify HLA antibodies allows for the prediction of recipient/donor crossmatch compatibility by virtual crossmatching Abbreviations: AHG, anti-human globulin; AUC, area under the curve; CDC, complement-dependent cytotoxicity; CI, confidence interval; DSA, donor-specific antibody; FC, flow cytometric; MCS, median channel shift; MFI, mean fluorescence intensity; ROC, receiver operating characteristic. q The work described in this article has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. ⇑ Corresponding author. Present address: Department of Pathology and Lab Medicine, University of Wisconsin-Madison, 600 Highland Avenue, Madison, WI 53792-2472, USA. E-mail address: [email protected] (T.M. Ellis). 1 Present Address: Hartford Transplant and Surgical Specialists, 85 Seymour St., Suite 301, Hartford, CT 06106, USA.
[1,2]. With the improved ability to transplant patients with donor-specific antibodies (DSA), a positive virtual or flow crossmatch (FC) alone is not a contraindication for transplant [3]. Many centers rely on quantitative measures of relative strength of donor-specific HLA antibody to assess allosensitization risk, such as flow crossmatch median channel shifts (MCS), FC titers, and reactivity in complement-dependent cytotoxic (CDC) crossmatches [4–6]. However, the strength of cell-based crossmatch continues to be widely regarded as the gold standard for assessing recipient/donor compatibility and level of risk for acute humoral rejection. Solid phase antibody assays offer a number of potential advantages over the use of cellular targets for quantitation of donor-specific antibodies, including ready availability, decreased biologic variability, lower non-specific reactivity, and potential for standardization [6–8]. In this study we assessed the diagnostic performance of solid phase antibody assays in predicting DSA strength measured using classical cell-based crossmatch assays. The longterm objective is to determine the feasibility of replacing cellbased crossmatches with solid phase antibody testing for assessment of allosensitization risk.
0198-8859/$36.00 - see front matter Ó 2012 American Society for Histocompatibility and Immunogenetics. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.humimm.2012.04.007
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
2. Materials and methods 2.1. Patients The study population comprises 82 recipient donor pairs evaluated for living donor renal transplantation from 2005 through 2009. All pairs were tested for T and/or B cell reactivity by FC crossmatch, and positive samples were also evaluated using FC crossmatch titrations (n = 61 T cell; n = 71 B cell) and by CDC crossmatching using direct, anti-human globulin (AHG)-augmented, and B cell techniques (n = 70). Thirty pairs proceeded to transplantation with crossmatched donors. 2.2. High resolution HLA antibody analysis HLA antibodies were identified using Luminex single antigen beads (One Lambda, Canoga Park CA), used according to the manufacturer’s instructions. DSA strength was were calculated by summing the unadjusted mean fluorescence intensity (MFI) of positive beads bearing donor-mismatched antigens and expressed as the R MFI value; background MFI values were subtracted only when these values exceeded 500. Positive beads were defined as those having MFIs of at least 1000, although this threshold could be adjusted between 800 and 1300 in select circumstances when supported by clear patterns of antibody epitope specificity. All background values were less than 2500. In cases where there were multiple beads in the panel for a given donor HLA antigen, only the bead with the highest MFI value was used to calculate the R MFI. 2.3. Crossmatching For flow cytometric crossmatches, peripheral blood mononuclear cells were isolated by treatment with carbonyl iron and dex-
707
tran followed by sedimentation on Ficoll-hypaque gradients. Cells were treated with pronase and 3-color FC crossmatches were performed according to established methods [9]. T and B cells were identified using PerCP-anti-CD3 and PE-anti-CD19 (BD Biosciences, San Jose, CA). The secondary antibody was a FITC-conjugated F(ab0 )2 fragment of goat anti-human IgG (heavy chain specific; Jackson ImmunoResearch, West Grove, PA). Cells were analyzed using a Becton-Dickinson FACSCalibur flow cytometer. Crossmatch titrations were performed using recipient sera diluted in fetal calf serum. MCS were calculated by subtracting median channel values obtained with donor cells incubated with pooled normal human serum from values obtained after staining with recipient serum. FC crossmatch titers were the reciprocal of the highest dilution of recipient serum yielding a positive crossmatch. Strong T and B cell crossmatches were those with a MCS P 150 and P 250, respectively. CDC crossmatch assays were performed using the NIH, B cell, and AHG-augmented techniques as described [9]. Cytolysis was determined by vital staining and a positive test defined as one with at least 50% target cell lysis. 2.4. Receiver operating characteristic (ROC) analysis ROC curves were generated by plotting the relationship between the true positive fraction (sensitivity) against the false positive fraction (1-specificity) at various R MFI values and calculating the resulting area under the curve (AUC) [10–12]. The variable in each case was the R MFI value, which was calculated using either donor-specific class I specificities alone (for T cell crossmatches) or with class II specificities (for B cell crossmatches). Gold standard classifiers were defined for each crossmatch assay analyzed. For CDC crossmatches, classifiers were simply a positive or negative CDC crossmatch result. Flow crossmatches were classified as
Fig. 1. ROC curves for R MFI values for discriminating recipient/donor pairs with and without strong T (a and c) and B cell (b and d) FC crossmatches. Panels a and b show curves for positive FC crossmatches with T cell MCS values P 150 (a) and B cell MCS values P 250 (b). Panels c and d show ROC curves for flow crossmatches with T cell titers P 8 (c) and B cell titers P 128 (d).
708
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
‘‘strongly positive’’ defined using either MCS cutoff values or titers. A test with an AUC of 1.0 is considered an ideal predictor, whereas an AUC of 0.5 indicates a random association between the test measure (R MFI) and the gold standard (crossmatch cutoff). ROC analyses were performed using MedCalc for Windows (v. 9.5.0.0, MedCalc Software, Mariakerke, Belgium). Optimal R MFI values were those calculated as those with optimal combined sensitivity and specificity for predicting the flow or CDC crossmatch result. 3. Results The relationship between solid phase MFI values and FC crossmatch results was determined using ROC curve analysis to determine the diagnostic accuracy of the R MFI value to predict weak and strong crossmatches defined using MCS value and titer cutoffs. ROC curve analysis is used increasingly in diagnostic testing to measure the ability of a diagnostic test measure to predict a disease or other outcome. ROC curves plot the sensitivity (true positive rate) against the false positive rate (100-specificity) for individual test values in a dataset (see Figs. 1 and 3). A test with perfect sensitivity and specificity will generate a plot close to the upper left corner, whereas a weaker association will trend toward a straight diagonal line from the origin toward the upper right, indicative of a random (50%) association between the diagnostic test and outcome. Thus, the accuracy of the test for predicting an end point can be represented by the resulting area under the curve and the strength of predictive ability determined if it differs significantly to the random association line (AUC = 50). ROC curve analyses of the R MFI value’s true versus false positive predictions are shown for T cell flow crossmatches with MCS > 150 (Fig. 1a) or titers >8 (Fig. 1c). R MFI values showed strong predictive accuracy for identifying T cell flow crossmatches with an MCS > 150, with an area under the curve (AUC) of 0.964 (95% confidence interval (CI) = 0.882–0.995; p = 0 .0001; Table 1). An AUC of 95% indicates that the R MFI predicted T cell crossmatch MCS values >150 in more than 95% of the comparisons [10–12]. The R MFI value (R MFI = 6282) with the highest diagnostic accuracy for distinguishing weak and strong crossmatches has a sensitivity (true positive rate) and specificity (true negative rate) of 90% and 95%, respectively (Table 1). R MFI values were also highly predictive of T cell flow crossmatch titers >8 (AUC = 0.915; 95% CI = 0.814–0.971; p = 0.0001). The lower AUC value (0.915) indicates R MFI values have lower accuracy for predicting strong T cell FC crossmatch titers than for MCS values. Further, the sensitivity and specificity for the R MFI cut point with the highest predictive accuracy (R MFI = 7964) were 76% and 100%, respectively. A sensitivity of 100% is obtained when the threshold titer is increased to 128, but with a reduction in specificity (69%). R MFI values showed good predictive accuracy for B cell crossmatches with MCS P 250 (AUC = 0.87; 95% CI = 0.764–0.940; p = 0.0001) and with high specificity (100%), but the sensitivity
P Fig. 2. Plots depicting MFI values with T MCS values (a) and B MCS values from FC crossmatching. Horizontal lines demarcate MCS cutoff values used to distinguish P strong T cell (150) and B cell (250) FC crossmatches. Vertical lines demarcate MFI values determined by ROC analysis to have the highest accuracy for predicting strong T and B cell FC crossmatches based on these MCS values.
was low (57%; Fig. 1b and d and Table 1). Expectedly, sensitivity increased to 85% when the MCS threshold was increased to 375, but with a significant decrease in specificity (56.4%). The predictive ability of R MFI values for B crossmatch titers P 512 was also low and statistically not significant (AUC = 0.61; 95% CI = 0.487– 0.723; p = 0.1110). A significant predictive relationship was achieved when the B cell titer was decreased to P 128 (p = 0.0004), although with reduced diagnostic specificity (56%) and sensitivity (80%). Overall, R MFI values showed lower predictive accuracy for B cell flow crossmatches than for T cells, regardless of endpoint chosen. The relationship between individual R MFI values and associated crossmatch MCS values is shown in Fig. 2a and b for T and B cell crossmatches, respectively. The R MFI values with the highest
Table 1 Summary statistics of ROC curve analysis of R MFI values and flow crossmatch results. Crossmatch criteria
AUC
p Value
R MFI with highest predictive accuracy
Sensitivity (%)
Specificity (%)
T cell MCS P 150 Titer P 8 Titer P 128
0.964 0.915 0.893
0.0001 0.0001 0.0001
>6282 >7964 >8248
90 76 100
95 100 69
B cell MCS P 250 MCS P 375 Titer P 128 Titer P 512
0.870 0.729 0.722 0.610
0.0001 0.0002 0.0004 0.1110
>12,693 >6282 >9608 >11,267
57 85 80 68
100 56 64 56
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710
predictive value determined by ROC curve analyses and their associated T and B cell MCS cutoffs are depicted as dotted lines in each plot. The higher false negative (20/78; 26%) for R MFI values in predicting strong B cell crossmatches is evidenced by the higher numbers of events falling in the upper left (false positive) quadrants of Fig. 2b. For T cell crossmatches, false negatives (i.e., a R MFI value below that with highest predictive accuracy and T MCS > 150) comprised the majority of incorrect predictions (5/78; 6.4%; Fig. 2a). The ability of R MFI values to predict positive CDC crossmatches was analyzed in a subset of patients with positive flow crossmatches. The accuracy of R MFI values for predicting CDC crossmatch outcomes for T-NIH, AHG-augmented, and B cell CDC crossmatch methods are summarized in Table 2 and by the ROC curves shown in Fig. 3a,b and c. R MFI values showed a statistically significant ability to predict a positive CDC crossmatch for each method tested, although diagnostic sensitivity and specificity varied considerably among the methods. R MFI exhibited diagnostic sensitivities of 91%, 94% and 73% for predicting positive NIH, AHG and B cell CDC methods, respectively, while specificities were 67% (NIH and AHG CDC methods) and 84% (B cell CDC crossmatches). The lower specificity for R MFI values in predicting CDC crossmatches reflects a high false positive rate for predicting positive NIH and AHG CDC results. Thus, while R MFI values predict a significant majority of positive CDC+ crossmatches, it is less robust at identifying true CDC negative sera (larger false positive fraction). 4. Discussion The objective of this study was to determine whether solid phase HLA antibody measurements of donor-specific antibodies could accurately predict the strength of cellular crossmatch assays
709
used to assess level of donor-specific allosensitization. Although it is common practice to use single antigen bead assays to assess levels of HLA antibodies, these reagents are not intended by the manufacturers for this purpose [4–6]. Here we demonstrate the validity of using single antigen beads as a surrogate for cell-based crossmatching for use in identifying high risk recipient/donor pairs defined by crossmatch strength. ROC curve analysis provides a powerful tool for calibrating solid phase antibody results with cell-based assays of donor-specific HLA antibodies, which historically have served as the gold standard indicators for risk of early humoral rejection. In general, donor-specific HLA antibodies measured by solid phase assays and expressed as R MFI values showed good diagnostic accuracy for identifying recipient/donor pairs with either a strong positive FC crossmatch or positive CDC crossmatch. The R MFI value correlated most closely with results of T cell crossmatches, and showed high sensitivity and specificity for prediction of T cell crossmatches defined as ‘‘strong’’ (MCS P 150). Although the predictive value of the R MFI for B cell crossmatch results achieved statistical significance, this association had lower sensitivity, specificity, and correlative strength than observed for T cell flow crossmatches. While the weaker correlation with B cell crossmatches might reflect the decreased accuracy of R MFI values calculated using both Class I and Class II MFI values, it more likely reflects the increased biologic variability inherent in B cell flow crossmatches due to higher nonspecific backgrounds and variability [13–15]. The lower variability and backgrounds associated with solid phase assays represent a potential advantage over cell-based assays for detecting and quantifying donor-specific HLA antibodies. In our own practice, we have experienced excellent one year graft survivals in sensitized patients who have positive flow crossmatches and but were negative for donor-specific antibodies by solid phase assays [16].
Fig. 3. ROC curves for R MFI values for discriminating positive CDC crossmatches using T-NIH (a), AHG (b), and B cell (c) CDC crossmatch methods.
710
T.M. Ellis et al. / Human Immunology 73 (2012) 706–710 Table 2 Summary statistics of ROC curve analysis of R MFI values and CDC crossmatch results. CDC method
AUC
p Value
R MFI with highest predictive accuracy
Sensitivity (%)
Specificity (%)
NIH+ AHG+ CDC-B+
0.837 0.860 0.810
0.0001 0.0001 0.0001
>8248 >8248 >10,374
94 94 73
69 67 84
Positive CDC crossmatches occur when high levels of complement-fixing, donor-specific antibodies are present. The sensitivity for predicting positive CDC crossmatches was greater than 90% for both T cell-based methods (NIH, AHG-augmented) but was low when B cells were used as targets. The low specificity reflects a significant proportion of test sera that had R MFIs above the calculated threshold, yet were negative in the CDC crossmatch. Thus, while a high R MFI value (>6000) was characteristic of all CDC positive crossmatches, not all high R MFI sera were CDC positive. Thus, positive CDC crossmatches cannot be predicted by antibody levels alone, and false negative predictions for high R MFI sera could occur due to the presence of high levels of non-complement fixing IgG isotypes although, to our knowledge, this has not been demonstrated directly. Alternatively, the low specificity may reflect the high variability of the CDC assay itself. In a survey of ASHI proficiency challenges for CDC crossmatching from 2008 to 2009, more than 30% positive samples tested failed to achieve 80% consensus positive among all labs, despite the fact that testing was performed using preselected sera and target cells (data not shown). Although additional outcomes-based studies are required to determine whether R MFI values provide a more stable and consistent measure of the level of donor-specific allosensitization and transplant risk than traditional CDC crossmatches, our studies indicate that solid phase MFI values may represent a more consistent and reproducible measure than the CDC crossmatch. The high sensitivity of R MFI values for predicting positive CDC crossmatches supports the diagnostic potential of using solid phase antibody assays to identify sera likely to be CDC positive. Previous groups documented the relationship between baseline levels of donor specific antibody and the risk for antibody-mediated allograft injury using either flow MCS or R MFI values [4,5]. Gloor et al. observed general agreement between high sum R MFI values and flow crossmatch MCS values >300, although the predictive relationship between crossmatch and MFI values was not directly measured [4]. Other studies also showed a correlation between DSA levels determined using single antigen beads and the likelihood of a positive flow crossmatch and risk for antibody-mediated rejection, although again the direct relationship between MFI values crossmatch strength was not determined [5,17]. In contrast to our studies, Morris et al. found that solid phase DSA measures can accurately predict a negative crossmatch but had limited sensitivity for predicting positive crossmatches [18]. This latter observation may reflect the greater sensitivity of single antigen beads for detecting DSA, as we have observed in our own experience (data not shown). This study supports the utility of R MFI values for quantitative virtual crossmatching in assessing donor-specific allosensitization levels and predicting crossmatch strength. While additional work
is needed to standardize R MFI values and determine clinically significant thresholds, these studies demonstrate the potential of R MFI in supporting transplantation where assessing recipient-donor compatibility where donor cells are unavailable or of low quality. Additional advantages over cell-based crossmatching include the ability to provide a rapid assessment of recipient compatibility with local or remote donor offers where a weak positive crossmatch is acceptable, and for initial screening and selection of potential living donors. References [1] Zangwill S, Ellis T, Stendahl G, Zahn A, Berger S, Tweddell J. Practical application of the virtual crossmatch. Pediatr Transplant 2007;11:650. [2] Zangwill SD, Ellis TM, Zlotocha J, Jaquiss RD, Tweddell JS, Mussatto KA, et al. The virtual crossmatch – a screening tool for sensitized pediatric heart transplant recipients. Pediatr Transplant 2006;10:38. [3] Gebel HM, Bray RA, Nickerson P. Pre-transplant assessment of donor-reactive, HLA-specific antibodies in renal transplantation: contraindication vs. risk. Am J Transplant 2003;3:1488. [4] Gloor JM, Winters JL, Cornell LD, Fix LA, Degoey SR, Knauer RM, et al. Baseline donor-specific antibody levels and outcomes in positive crossmatch kidney transplantation. Am J Transplant 2010;10:582. [5] Tambur AR, Ramon DS, Kaufman DB, Friedewald J, Luo X, Ho B, et al. Perception versus reality? Virtual crossmatch – how to overcome some of the technical and logistic limitations. Am J Transplant 2009;9:1886. [6] Cecka JM, Kucheryavaya AY, Reinsmoen NL, Leffell MS. Calculated PRA: initial results show benefits for sensitized patients and a reduction in positive crossmatches. Am J Transplant 2011;11:719. [7] El-Awar N, Lee J, Terasaki PI. HLA antibody identification with single antigen beads compared to conventional methods. Hum Immunol 2005;66:989. [8] Lefaucheur C, Loupy A, Hill GS, Andrade J, Nochy D, Antoine C, et al. Preexisting donor-specific HLA antibodies predict outcome in kidney transplantation. J Am Soc Nephrol 2010;21:1398. [9] Hahn A, Land G, Strothman R. ASHI laboratory manual; 2000. [10] Empson MB. Statistics in the pathology laboratory: characteristics of diagnostic tests. Pathology 2001;33:93. [11] Fawcett T. An introduction to ROC analysis. Pattern Recogn 2006;27:861. [12] Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29. [13] Bray RA, Gebel HM. Strategies for human leukocyte antigen antibody detection. Curr Opin Organ Transplant 2009;14:392. [14] Gebel HM, Bray RA. Sensitization and sensitivity: defining the unsensitized patient. Transplantation 2000;69:1370. [15] Gebel HM, Bray RA, Ruth JA, Zibari GB, McDonald JC, Kahan BD, et al. Flow PRA to detect clinically relevant HLA antibodies. Transplant Proc 2001;33:477. [16] Johnson CP, Shames BD, Brown S, Roza AM, Cronin DCE. T.M. A Single Center Experience. Can a Virtual Crossmatch Replace the Prospective Crossmatch? Am J Transplant 2009;9:334. [17] Reinsmoen NL, Lai C-H, Vo A, Cao K, Ong G, Naim M, et al. Acceptable donorspecific antibody levels allowing for successful deceased and living donor kidney transplantation after desensitization therapy. Transplantation 2008;86:820. [18] Morris GP, Phelan DL, Jendrisak MD, Mohanakumar T. Virtual crossmatch by identification of donor-specific anti-human leukocyte antigen antibodies by solid-phase immunoassay: a 30-month analysis in living donor kidney transplantation. Hum Immunol 2010;71:268.