- Email: [email protected]
Multicenter evaluation of the Bayer VERSANT™ HIV-1 RNA 3.0 assay: analytical and clinical performance
Journal of Clinical Virology 25 (2002) 205– 216 www.elsevier.com/locate/jcv
Multicenter evaluation of the Bayer VERSANT™ HIV-1 RNA 3.0 assay: analytical and clinical performance Curt A. Gleaves a,*, John Welle a, Mary Campbell a, Tarek Elbeik b, Valerie Ng b, Patricia E. Taylor c, Ken Kuramoto d, Sherri Aceituno d, Eva Lewalski d, Barbara Joppa e, Lynette Sawyer f, Carl Schaper g, Denise McNairn h, Thomas Quinn h a b
Infectious Disease Laboratory, Pro6idence Portland Medical Center, 4805 NE Glisan Street, Portland, OR 97213, USA Department of Laboratory Medicine, Uni6ersity of California and Clinical Laboratories, San Francisco General Hospital, San Francisco, CA, USA c Laboratory of Epidemiology, New York Blood Center, New York, NY, USA d Center for Blood Research, Sacramento, CA, USA e Assay De6elopment Department of the Nucleic Acid Diagnostics Di6ision of Bayer Diagnostics, Emery6ille, CA, USA f Bayer Reference Testing Laboratory, Emery6ille, CA, USA g Biostatistics Department of the Nucleic Acid Diagnostics Di6ision of Bayer Diagnostics, Emery6ille, CA, USA h Johns Hopkins Uni6ersity, Baltimore, MD, USA Received 20 August 2001; received in revised form 8 February 2001; accepted 22 February 2001
Abstract Background: The use of quantitative HIV-1 RNA assays is part of the standard of care for the management of HIV-1-infected individuals. Objecti6e: The Bayer VERSANT™ HIV-1 RNA 3.0 Assay (bDNA) was evaluated for reproducibility, linearity, limits of detection and quantitation, effects of potentially interfering substances and conditions, effects of plasma collection and handling conditions, clinical sensitivity and specificity, and biologic variability. Study design: Anti-HIV-1-positive specimens, patient specimens containing potentially interfering substances, and anti-HIV-negative specimens were collected from several HIV clinics, blood centers, or commercial companies across the United States. Specimen panels used to evaluate nonclinical performance of the assay were prepared at Bayer Diagnostics. Bayer Assay Development personnel performed two of the nonclinical studies— effect of freeze–thaw cycles using ‘spiked’ HIV-1 RNA-positive samples and effect of other disease organisms. All other studies were conducted at seven external sites. In some of the studies performed, specimens were tested in parallel with the Roche AMPLICOR® HIV-1 MONITOR™ version 1.0 PCR test. Results/conclusions: The results of these studies showed that the Bayer Assay has excellent reproducibility, a broad linear range (75– 500,000 HIV-1 RNA copies/ml), throughput of 168 patient results per two-plate run in a 22-h period, and few limitations for use. Because this test is designed for use only in individuals who are known to be HIV-1-positive, the clinical specificity of 97.6% is adequate
* Corresponding author. Tel.: + 1-503-215-6194; fax: + 1-503-215-6052. E-mail address: [email protected] (C.A. Gleaves). 1386-6532/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 1 3 8 6 - 6 5 3 2 ( 0 2 ) 0 0 0 1 1 - 2
206
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
for its intended use. These characteristics make it an attractive method for general laboratory use of monitoring HIV-1-infected patients. © 2002 Elsevier Science B.V. All rights reserved. Keywords: HIV-1 viral load; bDNA; PCR; Performance evaluation
1. Introduction The use of quantitative HIV-1 RNA assays is part of the standard of care for the management of individuals who are infected with the HIV-1 virus (Henry J. Kaiser Foundation, 2001; Carpenter et al., 2000). However, in order to interpret any given assay result or serial results, it is important to understand the performance characteristics of the assay generating the result. This paper will describe the outcome of a multi-center evaluation of the Bayer VERSANT™ HIV-1 RNA 3.0 Assay (bDNA), referred to as the Bayer Assay (Collins et al., 1997). Assay parameters reported here include reproducibility, linearity, limits of detection and quantitation, effects of potentially interfering substances and conditions, effects of plasma collection and handling conditions, clinical sensitivity and specificity, and biologic variability (within-patient results over time). In some studies, specimens were tested in parallel using the FDA-approved Roche AMPLICOR® HIV-1 MONITOR™ Test (Version 1.0, Roche Molecular Systems, Pleasanton, CA), referred to as the Roche Test (Mulder et al., 1994; Sun et al., 1998).
2. Materials and methods
2.1. Samples and testing facilities Clinical specimens were collected from antiHIV-1-positive patients enrolled at the HIV clinics of Johns Hopkins University, University of California (San Francisco, CA), University of Pittsburgh (Pittsburgh, PA), Veterans Administration Hospital (Palo Alto, CA), and ViRx Inc. (San Francisco, CA). Some anti-HIV-1-positive EDTA plasma specimens were also purchased from BioQual (Franklin, MA). Anti-HIV-negative clinical specimens were collected from repeat volunteer blood donors at New York Blood Center and at the Sacramento Blood Center
(Sacramento, CA). Patient specimens with potentially interfering substances were collected at University of Pittsburgh (cytomegalovirus, CMV), the PA Veterans Administration Hospital (hepatitis C virus, HCV), Sacramento Blood Center (hepatitis B virus (HBV), human T lymphotrophic virus (HTLV)), ProMedX, Norwood, MA (anti-nuclear antibody, ANA), BioQual (rheumatoid factor, HBV). All of the panels used to evaluate nonclinical performance of the assay, with the exception of the freeze–thaw study using patient specimens, were made at Bayer Diagnostics. Bayer Assay Development personnel tested the specimens in two nonclinical studies (effect of freeze–thaws using spiked samples and effect of other disease organisms). All other panels were tested at external sites. All subjects providing specimens did so under IRB-approved informed consent. Testing sites included: Providence Medical Center, Portland, OR; University of California, San Francisco, CA; New York Blood Center, New York, NY; Center for Blood Research, Sacramento, CA; Johns Hopkins University, Baltimore, MD; Bayer Reference Testing Laboratory, Emeryville, CA; and the Assay Development Department of the Nucleic Acid Diagnostics Business Segment of Bayer Diagnostics, Emeryville, CA. All individuals performing testing using the Bayer Assay or Roche Test were certified as proficient by the respective organizations.
2.2. Assessment of reproducibility, linearity, limit of detection and limit of quantitation A 7 member panel (Reproducibility Panel) was used to assess the reproducibility, linearity, limit of detection (LoD), and limit of quantitation (LoQ) of the Bayer Assay. This panel covered the reportable range of the assay and was created by serially diluting replication-defective mutant HIV1LAV/8E5 (Folks et al., 1986) into anti-HIVnegative EDTA plasma. In addition, two
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
acid –citrate–dextrose (ACD) panel members were created using anti-HIV-negative ACD plasma spiked with replication-defective mutant HIV-1LAE/8E5. One of these panel members quantitated close to the assay LoD; the other quantitated in the mid-range of the assay. The concentrations of the panel members were determined by direct comparison with the Reference Standard Curve prepared from the Reference Standard HIV-1 RNA (a 3.6 Kb RNA transcript containing most of the pol gene of HIV-1SF2 strain). Two operators at each of the three sites tested this panel. Each operator performed four runs, with three replicates per panel member, using each of the three kit lots of the Bayer Assay, for a total of 216 determinations per panel member. Linearity was also verified using serial dilutions of an HIV-1 RNA-positive EDTA patient specimen and an HIV-1 RNA-positive ACD patient specimen. Each panel was tested once in each of the four runs. The reproducibility study variance components were estimated using restricted maximum likelihood estimation, and the total variance | 2 (which accounts for all of site, lot, day, operator, run and within-run effects) was estimated as the sum of the individual variance components. The %CV for each panel member was calculated using the formula %CV = 10
|2 ln(10)
−1
where ln denotes natural log; this is the formula for %CV when the quantitations are log-normally distributed. The linearity of quantitation was evaluated by examining how accurately the assay measured differences between specimens with known dilutional relationships, i.e. by constructing the best-fitting 45° line to the data on log scale (the line for which observed differences exactly match the expected differences) and calculating the absolute value of the log difference between the observed values of each linearity panel member and the best-fitting 45° line. The LoD was defined as the lowest concentration of virus that yields a quantifiable result 95% of the time; thus, signal at the LoD can be reliably distinguished from background signal in HIV-
207
negative specimens. The LoQ was defined as the concentration of virus that is detected 95% of the time with a total %CV of B 45%.1 These limits were defined in conjunction with the reporting threshold (RT) of 35 HIV-1 RNA copies/ml, which was defined as the cutoff value that delivers a specificity of 95%.
2.3. Assessment of concordance of quantitation and precision of the Bayer assay and the Roche test A second, 3-member panel (referred to as the Concordance Panel), with members at the low, middle, and high end of the assay range, was tested at three sites using three lots of the Bayer Assay and at two sites using two lots of the Roche Test to estimate the concordance and precision of the two assays. This panel was also made using the replication-defective mutant HIV-1LAV/8E5 virus. The panel member at the low end of the assay range was tested using the UltraSensitive procedure. Five replicates of each panel member were tested on each run of the Bayer Assay and the Roche Test, for a total of 45 determinations per panel member using the Bayer Assay and 30 determinations per panel member using the Roche Test.
2.4. Potentially interfering substances and conditions The effect of endogenous substances (hemoglobin, triglycerides, bilirubin), therapeutic antiviral, antifungal and antibacterial drugs, common coexisting pathogens, and freeze–thaw cycles were evaluated in 25 HIV-1 RNA-positive and 25 HIV-1 RNA-negative panels. The HIV-1 RNApositive panels were created by spiking anti-HIV negative EDTA plasma with b-propiolactone (BPL)-treated viral stock (HIV-1SF2) to a target concentration of 75–100 HIV-1 RNA copies/ml. Hemoglobin was tested to a final concentration of 1 The constraint of 95% sensitivity at the LoD was required by the US FDA. The Bayer Assay that is approved for use in many other countries has a reportable assay range of 50 – 500 000 HIV-1 RNA copies/ml.
208
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
200 mg/dl (normal range B3 mg/dl); bilirubin was tested to a final concentration of 60 mg/dl (normal range 0.2– 1.3 mg/dl); and triglycerides were tested to a final concentration of 2000 mg/dl (normal range B 200 mg/dl). Therapeutic drugs (AZT, acyclovir, fluconazole, gancyclovir, ddI, trimethoprim/sulfamethoxazole, d4T, foscarnet, ciprofloxacin, 3TC, ritonavir, nevirapine, indinavir, ddC, nelfinavir, saquinavir, azithromycin dihydrate, cidofovir, delavirdine, rifampin, rifabutin, ethambutol, INH, clarithromycin, hydroxyurea) and disease pathogens (Streptococcus pneumoniae, Mycobacterium a6ium, herpes simplex virus type 2, CMV, HBV, HCV, Epstein – Barr virus, E. cloacae, Escherichia coli, Haemophilus influenzae, Klebsiella pneumoniae, P. mirabilis, Psuedomonas aeruginosa, Serratia marcescens, Staphylococcus aureus, Staphylococcus epidermidis, Candida albicans, Cryptococcus neoformans, and Streptococcus group B species) were grouped and tested in pools. Therapeutic drugs were tested at a final concentration of five times the peak serum level. Disease pathogens were tested at a final concentration of approximately 103 CFU or PFU/ml, except for HBV, HCV, and CMV, which were tested at concentrations of 108 DNA copies/ml, 106 RNA copies/ml and 60 DNA copies/ml, respectively, as measured in the Bayer bDNA assays for these viruses. For the freeze–thaw study, specimens were tested after one (reference), two, and three freeze– thaw cycles. The freeze–thaw study was also performed using EDTA plasma from HIV-1-infected individuals. These studies were designed to assess equivalence between a reference condition and a test condition (i.e. a specimen with and without a potential interfering substance or condition). In these studies, equivalence testing in HIV-1 RNAnegative specimens required rejecting the hypothesis that the specificity in the test condition was less than or equal to 95%. The surrogate measure that was used to test the hypothesis was based on the average increase in the relative light units (RLUs) induced by the test condition. For HIV-1 RNA-positive specimens, the measure of equivalence between reference and test conditions was percent concordance. Quantifica-
tion values from the reference and test condition were defined to be concordant if they were not statistically significantly different given the variability of the two assays; i.e. using a significance level of 5%, one would expect that 5% of the results would be judged discordant by chance alone, even if the specimens were tested twice in the reference condition; i.e. the expected percent concordance would be 95%. A claim of equivalence required rejecting the hypothesis that the percent concordance in the test condition was less than or equal to 90%. The surrogate measure used to test the hypothesis was based on the average change in quantitation induced by the test condition as compared to the reference condition.
2.5. Plasma collection and handling The effects of different anticoagulants and handling conditions were evaluated in the following collection tubes from Becton-Dickinson and Company (BD Diagnostic Systems, Sparks, MD): K3 EDTA glass Vacutainer® tubes, K2 EDTA plastic Vacutainer tubes, K2 Plasma Preparation Tubes (PPT™), and ACD Solution A (ACD-A) glass Vacutainer tubes. The EDTA and ACD-A Vacutainer tubes were centrifuged within 4 h of collection. One aliquot of plasma from each of the tubes was frozen at − 60 to − 80 °C immediately, while three other aliquots were held at 2–8 °C for 8, 24, and 48 h, respectively, and then frozen. PPTs were centrifuged within 2 h and either frozen immediately, held for 24 h at room temperature (RT), or held at 2–8 °C for 24 h prior to freezing. Equivalence between the reference condition (K3 EDTA Vacutainer tubes, centrifuged and frozen at − 60 to − 80 °C within 4 h of collection) and different collection devices and handling procedures were tested as described above.
2.6. Clinical specificity Clinical specificity was determined by testing 912 repeat volunteer HIV-seronegative blood donors from two blood centers. Testing was performed at three sites, and specimens were apportioned between the three investigational lots at
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
each site. Results that were above the LoQ of 75 HIV-1 RNA copies/ml in the Bayer Assay were assumed to be false positive. A 2-test was used to determine if specificity was independent of gender.
2.7. Clinical non-specificity EDTA plasma specimens were collected from anti-HIV-negative subjects who were infected with HCV, HBV, HTLV, or CMV or who had elevated levels of rheumatoid factor or ANA. Specimens from 10 subjects in each disease category were tested. Equivalence between the specimens from the disease groups and from the anti-HIV negative repeat volunteer blood donors was tested as described above. This study was performed to supplement the data from the nonclinical studies on the effect of other disease pathogens, which used spikes of cultured pathogens into HIV-1 RNA-positive and HIV-1 RNA-negative specimens.
2.8. Clinical sensiti6ity Clinical sensitivity was determined in EDTA or ACD plasma specimens from 984 subjects (698 males and 286 females) with documented HIV-1 or HIV-1/2 ELISA and supplemental test reactivity. Subjects were stratified by CD4+ cell count into three categories for analysis: CD4+ B200 cells/mm3, CD4+ between 200 and 500 cells/mm3, and CD4+ \ 500 cells/mm3. Specimens were apportioned between the three investigational lots of the Bayer Assay. The majority of these specimens (936) were also tested with the Roche Test. The UltraSensitive procedure of the Roche Test was used where appropriate. Within each CD4+ group, a 2-test was used to determine if the percent of specimens with levels of HIV-1 RNA less than the LoQ was independent of gender. Assay sensitivity was calculated as the percent of specimens with levels of HIV-1 RNA]LoQ. Two-sided 95% confidence intervals were determined, using the binomial distribution, for each CD4+ category by gender and for both genders combined.
209
2.9. Within-patient 6ariability Within-patient variability (WPV) (i.e. biological variability) was assessed weekly for 8 weeks in 33 subjects (24 males and nine females) using the Bayer Assay and the Roche Test. Subjects were required to have no change in therapy and no clinical events within 3 months of study enrollment, and were considered clinically stable by the investigator. Any change in therapy or clinical event during the 8-week study period disqualified them from the study. Subjects were required to provide at least six of the eight weekly specimens to remain eligible. Testing was performed at three sites by each assay method, using three lots of the Bayer Assay and two lots of the Roche Test. Each of the 6–8 successive specimens from a given subject was tested in a different run (but within the same kit lot). The ultimate objective of this study was to determine the minimum fold-change (i.e. the ratio of the largest to smallest of two successive measurements on the arithmetic scale) between two successive measurements that would signal a potential change in a subject’s clinical status. The minimum fold-change was estimated such that 95% of observed changes from subjects who experience no actual change in viral load would be expected to be less than the estimate of the minimum fold-change. The sources of variability that contribute to differences between successive measurements are biological (within-patient) and assay-related (site, lot, operator, within-run and between-run). In this study, testing of each subject’s specimens was done at the same site by the same operator and lot. To obtain a generalizable fold-change that included all sources of variability, data from the WPV study (biological variation) were used in conjunction with the data from reproducibility study (Bayer Assay) and concordance study (Roche Test) to obtain variability estimates for site, operator, lot, run and withinrun variation. An estimate of total variability was calculated as the sum of WPV and all other sources of variability estimated from the reproducibility or concordance studies. The total variability was used to estimate the 95th percentile of the distri-
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
210
bution of fold-changes (‘minimum fold-change’) between two measurements that would be observed even if there were no underlying changes in viral load. A change between two successive measurements that is larger than this 95th percentile would be unlikely to occur by chance and can be indicative of a change in clinical status.
3. Results
3.1. Reproducibility, linearity, LoD and LoQ 3.1.1. Reproducibility Table 1 presents data on assay reproducibility, including overall between-run and within-run variance components, as well as total percent coefficient of variation (%CV). The concentration of HIV-1 RNA in panel members used in this study ranged between 62 and about 620,000 copies/ml. Over most of the assay range (124 to \ 500,000 HIV-1 RNA copies/ml), the total %CV due to all sources of variation ranged from 23.7 to 28.9%. For the three panel members below 100 HIV-1 RNA copies/ml (EDTA panel members 6 and 7, and ACD panel member 5A), the variation ranged between 30.0 and 39.3%. In general, the
overall between-run component of variance contributed more to total variance than did the within-run component. However, with two panel members (6 and 7) that contained 93 and 62 copies/ml, the within-run component of variance was the greater contributor to total variance.
3.1.2. Linearity Fig. 1 shows the results of the linearity evaluation for the Bayer Assay. This figure was generated by plotting expected versus observed geometric mean (geomean; log10) quantitations for panel members used in the reproducibility study described above. The line is the best-fitting 45° line. The absolute deviations from the bestfitting 45° line were less than 0.05 log10 over the entire range of 62–620,282 HIV-1 RNA copies/ ml. Analysis of data from the two serially diluted patient specimens that were collected in ACD and EDTA showed that the slopes were not significantly different from 1.0 (1.069 and 1.047, respectively; data not shown). The ACD and EDTA patient specimens, undiluted, had geomean values of 635,000 and 438,000 HIV-1 RNA copies/ ml, respectively. Thus, linearity of the product was observed both with cell culture-derived HIV-1 and with HIV-1 obtained directly from patient specimens.
Table 1 Coefficient of variation (%CV) for overall between-run, within-run and total variation Panel members
1 2 3 4 5 6 7 3A 5A
Value assignment (HIV-1 RNA copies/ml)
620,282 62,028 6203 620 124 93 62 5640 94
Variance Overall between run (%)
Within run (%)
Total %CV
20.4 22.4 19.2 19.6 18.6 18.6 19.5 23.8 30.1
14.1 12.9 13.6 15.5 18.7 23.1 25.2 16.0 24.2
24.9 26.0 23.7 25.1 26.6 30.0 32.2 28.9 39.3
The variance components for lot, site, operator, and day were combined for the overall between-run estimate. Within the ‘‘overall’’ between-run component, the majority of the variability came from the between-run (within-day) component, followed by the between-day, between-lot, between-operator, and lastly, between-site terms. There were 216 determinations of each panel member (3 replicates/run×4 runs×3 operators×3 sites×3 lots). Panel members 1–7 were prepared in EDTA; 3A and 5A were prepared in ACD.
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
211
Fig. 1. Expected versus observed values for HIV-1 RNA in the Bayer assay.
3.1.3. LoD and LoQ The LoD was based on the percent detection of reproducibility of panel members at the lower end of the assay range. The concentration of HIV-1 RNA with 95% detection was estimated to be 67 HIV-1 copies/ml. The upper one-sided 95% confidence limit of the concentration with 95% detection was calculated using the one-sided upper 95% confidence limit for the 95th percentile of a normal distribution (Hahn and Meeker, 1991), and was 73 copies/ml. This was rounded up to the nearest 5 HIV-1 RNA copies/ml to obtain the LoD of 75 HIV-1 RNA copies/ml. Because the total %CV was B 40% at this concentration, 75 copies/ml was also the LoQ for the assay. 3.2. Assessment of concordance and precision for the Bayer and Roche assays The mean quantitation values and %CVs of each of the three Concordance Panel members were estimated using the Bayer Assay and the Roche Test. The results are shown in Table 2. The quantitations were evaluated to test whether the
means were equivalent, and the precision of quantitation was estimated for each assay method to test whether the variances were equal. The geomean quantitations from the Bayer assay and the Roche Test for the high and medium panel members were not statistically different, whereas the lowest panel member quantitated significantly higher in the Roche Test. The Bayer Assay quantitated all three panel members, on average, 29% lower than did the Roche Test. The %CVs obtained from the Bayer Assay were lower than the %CVs from the Roche Test for the high, medium, and low panel members.
3.3. Potentially interfering substances and conditions There was no effect on test results from HIV-1 RNA-positive specimens or on anti-HIV-negative specimens by elevated hemoglobin (400 times upper limit of normal), elevated bilirubin (20 times upper limit of normal), elevated triglycerides (five times upper limit of normal), therapeutic drugs used in HIV-infected patients (5 times peak
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
212
serum levels), disease pathogens, and multiple freeze –thaw cycles. However, grossly icteric (\ 20 mg/dl bilirubin) or chylomicronemic (\ 1000 mg/dl triglycerides) specimens (visually greenish or milky, respectively) should not be used in the Bayer Assay, as they may produce a decreased signal.
3.4. Plasma collection and handling All of the EDTA tube types (glass K3 EDTA Vacutainer™ tubes, plastic K2 EDTA Vacutainer tubes, K2 PPT) provided statistically equivalent HIV-1 RNA quantitations for the anti-HIV-1positive specimens and equivalent RLUs for the anti-HIV-negative specimens when the plasma was frozen at −60 to −80 °C within 2 (PPT) or 4 h of collection. There was no difference in RLUs from anti-HIV-negative specimens collected in tubes with K3 EDTA or ACD Solution A. In the 126 anti-HIV-1-positive specimens evaluated in this study, quantitations from specimens collected in ACD-A were, on average, 15% lower (0.07 log10) than the quantitations from specimens collected in K3 EDTA. This difference was consistent across the reportable range of the assay (slope = 0.99, 95% confidence interval 0.97– 1.01). The volume of the ACD-A solution in collection tubes results in a 17.6% dilution of the blood, so the 15% decrease in viral load seen in this study was expected. The specimen handling studies showed that for all tubes studied except the PPTs, plasma can be stored at 2–8 °C for up to 48 h before freezing.
Plasma collected using an EDTA PPT can be stored on the separator gel at RT for up to 24 h before freezing.
3.5. Clinical specificity Clinical specificity was evaluated by testing specimens from 912 repeat volunteer blood donors. Data were analyzed separately for males (n= 546) and females (n=366). There was no statistically significant difference in assay specificity between the genders (P= 0.83); therefore, the data were combined for this analysis. The percent of specimens with results below the LoQ was 97.6% (95% confidence interval 96.4%, 98.5%).
3.6. Clinical non-specificity Specimens from anti-HIV-negative individuals with other diseases were tested to confirm the results seen in the nonclinical experiments (potentially interfering substances) that used anti-HIV negative specimens spiked with disease organisms. Additionally, specimens from subjects with autoimmune diseases were tested. The specimens in the disease categories tested [HBV, HCV, HTLV, CMV, rheumatoid factor, anti-nuclear antibodies] had RLU values that were statistically equivalent to the RLUs of the repeat volunteer blood donor specimens (see Section 3.5). The mean RLU for the blood donor group was 0.32, while the mean RLU for the 60 subjects in the other disease group was 0.29.
Table 2 Coefficient of variation (%CV) of quantification of HIV-1 RNA by the Bayer Assay and the Roche Test Panel member
Bayer Assay
Roche Test
Number of tests
HIV-1 RNA copies/ml (95% CIa)
Total %CV
Number of tests
HIV-1 RNA copies/ml (95% CI)
Total %CV
1
45
25.1%
30
45 44
15.9% 31.6%
29 28
69,527 (53,907–89,674) 6515 (5265–8077) 92 (88–97)
39.5%
2 3
49,644 (38,199–64,518) 5138 (4818–5480) 59 (55–60)
a
CI= confidence interval.
43.7% 46.2%
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
213
Table 3 Clinical sensitivity: percentage of specimens with detectable HIV-1 RNA in patient groups stratified based on number of CD4+ cells
3.8. Comparison of HIV-1 RNA quantitations in clinical specimens using the Bayer Assay and the Roche Test
CD4+
Gender
N
% with results]LoQ (95% CIa)
CD4+B200
Males Females Combined
269 33 302
98.5% (96.3–99.6) 100.0% (91.3–100) 98.7% (96.7–99.6)
CD+ = 200
Males
251
87.6% (82.9–91.4)
Females Combined
120 371
90.0% (83.2–94.7) 88.4% (83.2–94.7)
Males Females Combined
178 133 311
67.4% (60.0–74.2) 62.4% (53.5–70.6) 65.3% (59.7–70.6)
Of the 936 specimens tested, 794 had quantitations that were in the reportable range of both assays. The geomean quantitation obtained by the Bayer Assay was 12,383 HIV-1 RNA copies/ ml, while the geomean quantitation by the Roche Test was 20,349 HIV-1 RNA copies/ml. There were 12 data points that were significantly different from the other data, in that the Bayer Assay quantitation was higher than the Roche test quantitation. Seven of these results came from one patient in the WPV study, and two results were from a single patient tested in a clinical utility study. The remaining three specimens were from three additional subjects. These 12 data points were excluded from the regression analyses and are not shown in Fig. 2. Additional information on these 12 data points is presented in Section 4. Fig. 2 is a plot of the log differences between the two assays versus the mean log quantitations. Regression analysis showed a slope of 0.003 (95% confidence interval; −0.017, 0.022). These data indicate that quantification values from the Bayer Assay are, on average, 39% lower than the values from the Roche Test. The slope of 0.003 cannot be distinguished from a slope of 0 (P= 0.78) therefore, we conclude that differences in quantitation values from the two assays are consistent across the common reportable range of the two assays.
– 500
CD+\500
a
CI= confidence interval.
3.7. Clinical sensiti6ity The clinical sensitivity of the assay was assessed in 984 subjects who were stratified into three groups based on their CD4+ cell count. This group was a representative cross-section of HIV-1-infected patients in the US. The mean age was 39 years (range 15– 77), 71% were males, 51% were Caucasian, 41.5% were AfricanAmerican, and 4.6% were Hispanic. Of the 984 subjects, 675 (69%) were receiving anti-retroviral therapy at the time of blood collection and 309 (31%) were not. Table 3 summarizes the number of subjects tested in this study by gender (and combined) and CD4+ category and shows the percentage of specimens within each category with HIV-1 RNA levels that were at or above the LoQ of 75 HIV-1 RNA copies/ml). There were no statistically significant differences between males and females in terms of sensitivity (P \ 0.4 for each of the 3 CD4+ groups). The results show that, as expected, sensitivity results were inversely related to CD4+ counts. The highest sensitivity (98.7%) was found for subjects with CD4+ counts below 200 cells/mm3 and the lowest (65.3%) for subjects with CD4+ counts above 500 cells/mm3.
Fig. 2. Log differences between the Bayer Assay and Roche Test versus the mean log quantitation for the 782 anti-HIV-1positive clinical specimens.
214
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
3.9. WPV
4. Discussion
Eight of the 33 subjects who were enrolled in the study were eliminated from the variance analyses for the two test methods. Five of these were eliminated because they had a 20-fold or greater change in assay quantitation by both assays during the 8-week study period and were thus not considered to satisfy the ‘clinically stable’ requirement for this study. Four of these five subjects were intravenous drug users, so the use of illicit drugs or lack of compliance with medication might have caused these large fluctuations in viral load. Three additional subjects were eliminated from the variance analyses because either all specimens (Bayer) or all but one specimen (Roche) had viral levels below the respective assay’s LoQ. The variance estimates from the WPV study, which included biological, run, and within-run sources of variance, were combined with variance estimates for site, operator, and lot from the reproducibility study for the Bayer assay or the concordance study for the Roche test, to estimate the total assay plus biological variability for each assay method. These estimates were then used to calculate the minimum change between two successive measurements, for each assay method, that could be considered to be significant and not attributed to chance. The total variance estimate for the Bayer Assay was 0.0276. Based on this estimate, the minimum fold-change between two successive measurements of HIV-1 RNA using the Bayer Assay that is unlikely to have happened by chance was estimated to be 2.9-fold (0.46 log10) with an upper confidence limit of 3.2-fold. The total variance estimate for the Roche Test was 0.0365. The minimum fold-change between two successive measurements of HIV-1 RNA using the Roche Test that is unlikely to have happened by chance was estimated to be 3.4-fold (0.53log10) with an upper confidence limit of 6.3-fold (0.80 log10). Because the smaller concordance study was used to obtain the variance estimates for the Roche Test, the actual degrees of freedom were lower leading to a bigger difference between the estimate of fold-change and the upper confidence limit.
The studies described in the current paper represent the validation trial performed for US product registration. They differ from several earlier studies, discussed below, in that the kit lots used were manufactured with diverse raw materials using the final, validated manufacturing processes. They also differ in the number and types of specimens tested, the apportionment of all specimens within a study (e.g. sensitivity, specificity, WPV, etc.) between three different kit lots, and the number of operators and sites performing the testing. Thus, the results seen in this study should reflect the future performance characteristics of the assay across sites and kit lots. The results described here, however, are similar in many aspects to those seen in previous studies (Highbarger et al., 1999; Erice et al., 2000; Murphy et al., 2000). Highbarger et al. (1999) reported on a comparison study between the Bayer Assay and two versions of the UltraSensitive procedure of the Roche Test. A total of 381 specimens from 59 HIV-1 infected individuals were tested by the Bayer Assay and an in-house UltraSensitive procedure for the Roche Test that used 1.0 ml of plasma. The results showed a slope that was statistically equivalent to 1.0 (1.0052) and an intercept of −0.0915, which was statistically different from 0.0. A subset of 47 specimens was tested by the Bayer Assay and the Roche UltraSensitive procedure, where 0.5 ml of plasma is used. In this comparison, the slope was 1.1214 and the intercept was −0.608, representing a 1.048-fold difference between the two assays on the log10 scale. In the study reported here, with a larger group of clinical specimens (n= 794), the slope for modeling the difference in log10 results as a function of the average of the log10 results was not significantly different from zero. There was, however, an overall 39% difference in quantitation between the two assays, with the Bayer Assay having lower quantitations than the Roche Test. Data from the smaller Concordance Panel Study, where replication defection mutant HIV-1LAV/8E5 virus was used, showed a 29% difference between the two assays. In each of the studies, three kit lots of the Bayer Assay and two kit lots of the Roche Test
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
were used for testing. Based on the lots used in the current studies, different lot combinations may produce differences between the two assays that range from 0 to 50%. A study by Erice et al. (2000) evaluated the performance of the research use version of the Bayer Assay, including LoD, specificity, and linearity in four laboratories. The current study had similar results for assay linearity, but showed better specificity, probably due to different cutoffs in the two assay versions (50 copies/ml in the research use version, 75 copies/ml in the current version). Both the Erice study and the current study found that the Bayer Assay had better reproducibility than the Roche Test. Murphy et al. (2000) reported on a multicenter comparison of the Roche 1.5 test, the Organon Teknika NucliSens QT Assay and the Bayer assay. The specificity of the ‘research use’ Bayer assay in the Murphy study, using 100 anti-HIV-1negative specimens, was 98%. The data are similar to the specificity results of 97.6% reported here, despite the higher LoD of 75 copies/ml. Murphy reported a combined between-run and within-run %CV of 18.2%, a between-lot %CV of 21.9% (using two lots), and a between-site %CV of 21.3% (using two sites). Because the comparison of lots and sites includes different runs, the Murphy estimates of between-lot and between-site %CVs include between-run and within-run %CVs. Murphy’s results are very similar to the results obtained in the clinical trial reported here. When the clinical trial data were analyzed using a method similar to that used by Murphy, the combined between-run and within-run %CV was 21.4%, the between-lot %CV was 22.4%, and the between-site %CV was 21.5% Because different manufacturers use different standardization methods for assigning HIV-1 RNA quantitations in their assays, the various guidance documents caution against changing assay methods during serial measurements. The results from the assay comparison study using clinical specimens indicate, however, that the relationship between Roche viral load units and Bayer viral load units is consistent across the shared reportable range. This suggests that a laboratory could establish a conversion factor using
215
currently available lots from both manufacturers and use this factor to convert between Roche viral load units to Bayer viral load units. There will be some exceptions to the application of this conversion factor, as seen with the 12 data points (representing five individuals) in the clinical specimen comparison where the Bayer Assay quantitation was higher than the Roche Test quantitation. Sufficient specimen volume was available to sequence specimens from two of these subjects. An analysis of the gag sequences compared to the Roche primers and probe found five and six mismatches between the forward primer and the gag sequence in two specimens from the WPV subject. An A:C nucleotide substitution was seen in the gag region of the specimen from the subject in the clinical utility study. These mismatches may account for the lower than expected quantitation in the Roche Test. Both subjects were infected with HIV-1 subtype B. Our current knowledge of HIV replication and development of resistance, and the need to reduce viral load as much as possible for as long as possible, makes viral load determinations essential for patient management. Although the Bayer assay is not quite as sensitive as the Roche Test (LoDs of 75 and 50 copies/ml, respectively), the Bayer assay has been shown to have superior reproducibility, particularly near the LoD (current study and Erice et al., 2000). Assay reproducibility is critical for determining when an assay result is different enough from a previous result to indicate a significant change in viral load, perhaps due to development of resistance to a drug or class of drugs. Total variability of the Bayer Assay, including biological and assay components indicated that a change of 2.9-fold was a statistically significant change. Therefore, the Bayer assay can be used in accordance with the Henry J. Kaiser Foundation (2001), which states (p. 3) that ‘A minimally significant change in plasma viremia is considered to be a 3-fold or 0.5 log10 increase or decrease.’ In summary, the results of the current studies showed that the Bayer assay has excellent reproducibility, a broad linear range (75–500,000 HIV1 RNA copies/ml), throughput of 168 patient results per two plate run, and few limitations for
216
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
use. Because this test is designed for use only in individuals who are known to be HIV-1 positive, the clinical specificity of 97.6% is adequate for its intended use. As reported by Elbeik et al. (2002), the Bayer Assay has similar detection of HIV-1 Group M subtypes tested (A– F) compared to the Roche test (version 1.5). However, in the Elbeik study, the quantitation of multiple replicas from serial dilutions of each subtype demonstrated higher reliability in test results for the Bayer assay as compared to the Roche Test (version 1.5). This finding was, in part, explained by the difference in acceptable ranges for the kit controls (low positive 0.7 and high positive 0.6 log10 for the Bayer assay, low positive 0.95 and high positive 0.84log10 for the Roche test version 1.5). In addition, a comparative cost analysis (Elbeik et al., 2000) demonstrated savings of up to 53% for labor, 67% for disposables, and 50% for biohazardous waste labor for the Bayer Assay as compared to the Roche Test (version 1.5). These characteristics make the Bayer Assay an attractive method for clinical laboratory use.
References Carpenter CC, Cooper DA, Fischl MA, et al. Antiretroviral therapy in adults: updated recommendations of the International AIDS Society –USA Panel. J Am Med Assoc 2000;283:381 – 90. Collins ML, Irvine B, Tyner D, et al. A branched DNA signal amplification assay for quantification of nucleic acid targets below 100 molecules/ml. Nucleic Acids Res 1997;25:2979 –84. Elbeik T, Charlebois E, Nassos P, et al. Quantitative and cost comparison of ultrasensitive human immunodeficiency virus type 1 RNA viral load Assays: Bayer bDNA Quan-
tiplex Versions 3.0 and 2.0 and Roche PCR Amplicor Monitor Version 1.5. J Clin Microbiol 2000;38:1113 – 20. Elbeik, T., Alvord, W.G., Trichavoroj, R. et al., 2002. Comparative analysis of HIV-1 viral load assays on subtype quantification: Bayer Versant HIV-1 RNA 3.0 Versus Roche Amplicor HIV-1 Monitor Version 15. J Acq Immune Defic Syndr 2002;29:330 – 339. Erice A, Brambilla D, Bremer J, et al. Performance characteristics of the QUANTIPLEX HIV-1 RNA 3.0 Assay for detection and quantitation of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 2000;38:2837 – 45. Folks TM, Powell D, Lightfoote M, et al. Biological and biochemical characterization of a cloned LEU- – cell surviving infection with the acquired immune deficiency syndrome retrovirus. J Exp Med 1986;164:280 – 90. Guidelines for the Use of Antiretroviral Agents in HIV-Infected Adults and Adolescents. Department of Health & Human Services and the Henry J. Kaiser Foundation, 2001. Available at: http://www.hivatis.org/ trtgdlns.htmlc AdultAdolescent, accessed March 14, 2001. Hahn GJ, Meeker WQ. Statistical Intervals, A Guide for Practitioners. New York: Wiley, 1991 The critical values were obtained from Tables A12a – d. Highbarger HC, Alvord WG, Jiang MK, et al. Comparison of the Quantiplex Version 3.0 Assay and a sensitized Amplicor Monitor Assay for measurement of human immunodeficiency virus type 1 RNA levels in plasma samples. J Clin Microbiol 1999;37:3612 – 4. Mulder J, McKinney N, Christopherson C, Sninsky J, Greenfield L, Kwok S. Rapid and simple PCR assay for quantitation of human immunodeficiency virus type 1 RNA in plasma: application to acute retroviral infection. J Clin Microbiol 1994;32:292 – 300. Murphy DG, Coˆ te L, Fauvel M, Rene P, Vincelette J. Multicenter comparison of Roche COBAS AMPLICOR MONITOR Version 1.5, Organon Teknika NucliSen QT with Extractor, and Bayer Quantiplex Version 3.0 for quantification of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 2000;38:4034 – 41. Sun R, Ku J, Jayakar H, et al. Ultrasensitive reverse transcription-PCR assay for quantitation of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 1998;36:2964 – 9.
Multicenter evaluation of the Bayer VERSANT™ HIV-1 RNA 3.0 assay: analytical and clinical performance Curt A. Gleaves a,*, John Welle a, Mary Campbell a, Tarek Elbeik b, Valerie Ng b, Patricia E. Taylor c, Ken Kuramoto d, Sherri Aceituno d, Eva Lewalski d, Barbara Joppa e, Lynette Sawyer f, Carl Schaper g, Denise McNairn h, Thomas Quinn h a b
Infectious Disease Laboratory, Pro6idence Portland Medical Center, 4805 NE Glisan Street, Portland, OR 97213, USA Department of Laboratory Medicine, Uni6ersity of California and Clinical Laboratories, San Francisco General Hospital, San Francisco, CA, USA c Laboratory of Epidemiology, New York Blood Center, New York, NY, USA d Center for Blood Research, Sacramento, CA, USA e Assay De6elopment Department of the Nucleic Acid Diagnostics Di6ision of Bayer Diagnostics, Emery6ille, CA, USA f Bayer Reference Testing Laboratory, Emery6ille, CA, USA g Biostatistics Department of the Nucleic Acid Diagnostics Di6ision of Bayer Diagnostics, Emery6ille, CA, USA h Johns Hopkins Uni6ersity, Baltimore, MD, USA Received 20 August 2001; received in revised form 8 February 2001; accepted 22 February 2001
Abstract Background: The use of quantitative HIV-1 RNA assays is part of the standard of care for the management of HIV-1-infected individuals. Objecti6e: The Bayer VERSANT™ HIV-1 RNA 3.0 Assay (bDNA) was evaluated for reproducibility, linearity, limits of detection and quantitation, effects of potentially interfering substances and conditions, effects of plasma collection and handling conditions, clinical sensitivity and specificity, and biologic variability. Study design: Anti-HIV-1-positive specimens, patient specimens containing potentially interfering substances, and anti-HIV-negative specimens were collected from several HIV clinics, blood centers, or commercial companies across the United States. Specimen panels used to evaluate nonclinical performance of the assay were prepared at Bayer Diagnostics. Bayer Assay Development personnel performed two of the nonclinical studies— effect of freeze–thaw cycles using ‘spiked’ HIV-1 RNA-positive samples and effect of other disease organisms. All other studies were conducted at seven external sites. In some of the studies performed, specimens were tested in parallel with the Roche AMPLICOR® HIV-1 MONITOR™ version 1.0 PCR test. Results/conclusions: The results of these studies showed that the Bayer Assay has excellent reproducibility, a broad linear range (75– 500,000 HIV-1 RNA copies/ml), throughput of 168 patient results per two-plate run in a 22-h period, and few limitations for use. Because this test is designed for use only in individuals who are known to be HIV-1-positive, the clinical specificity of 97.6% is adequate
* Corresponding author. Tel.: + 1-503-215-6194; fax: + 1-503-215-6052. E-mail address: [email protected] (C.A. Gleaves). 1386-6532/02/$ - see front matter © 2002 Elsevier Science B.V. All rights reserved. PII: S 1 3 8 6 - 6 5 3 2 ( 0 2 ) 0 0 0 1 1 - 2
206
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
for its intended use. These characteristics make it an attractive method for general laboratory use of monitoring HIV-1-infected patients. © 2002 Elsevier Science B.V. All rights reserved. Keywords: HIV-1 viral load; bDNA; PCR; Performance evaluation
1. Introduction The use of quantitative HIV-1 RNA assays is part of the standard of care for the management of individuals who are infected with the HIV-1 virus (Henry J. Kaiser Foundation, 2001; Carpenter et al., 2000). However, in order to interpret any given assay result or serial results, it is important to understand the performance characteristics of the assay generating the result. This paper will describe the outcome of a multi-center evaluation of the Bayer VERSANT™ HIV-1 RNA 3.0 Assay (bDNA), referred to as the Bayer Assay (Collins et al., 1997). Assay parameters reported here include reproducibility, linearity, limits of detection and quantitation, effects of potentially interfering substances and conditions, effects of plasma collection and handling conditions, clinical sensitivity and specificity, and biologic variability (within-patient results over time). In some studies, specimens were tested in parallel using the FDA-approved Roche AMPLICOR® HIV-1 MONITOR™ Test (Version 1.0, Roche Molecular Systems, Pleasanton, CA), referred to as the Roche Test (Mulder et al., 1994; Sun et al., 1998).
2. Materials and methods
2.1. Samples and testing facilities Clinical specimens were collected from antiHIV-1-positive patients enrolled at the HIV clinics of Johns Hopkins University, University of California (San Francisco, CA), University of Pittsburgh (Pittsburgh, PA), Veterans Administration Hospital (Palo Alto, CA), and ViRx Inc. (San Francisco, CA). Some anti-HIV-1-positive EDTA plasma specimens were also purchased from BioQual (Franklin, MA). Anti-HIV-negative clinical specimens were collected from repeat volunteer blood donors at New York Blood Center and at the Sacramento Blood Center
(Sacramento, CA). Patient specimens with potentially interfering substances were collected at University of Pittsburgh (cytomegalovirus, CMV), the PA Veterans Administration Hospital (hepatitis C virus, HCV), Sacramento Blood Center (hepatitis B virus (HBV), human T lymphotrophic virus (HTLV)), ProMedX, Norwood, MA (anti-nuclear antibody, ANA), BioQual (rheumatoid factor, HBV). All of the panels used to evaluate nonclinical performance of the assay, with the exception of the freeze–thaw study using patient specimens, were made at Bayer Diagnostics. Bayer Assay Development personnel tested the specimens in two nonclinical studies (effect of freeze–thaws using spiked samples and effect of other disease organisms). All other panels were tested at external sites. All subjects providing specimens did so under IRB-approved informed consent. Testing sites included: Providence Medical Center, Portland, OR; University of California, San Francisco, CA; New York Blood Center, New York, NY; Center for Blood Research, Sacramento, CA; Johns Hopkins University, Baltimore, MD; Bayer Reference Testing Laboratory, Emeryville, CA; and the Assay Development Department of the Nucleic Acid Diagnostics Business Segment of Bayer Diagnostics, Emeryville, CA. All individuals performing testing using the Bayer Assay or Roche Test were certified as proficient by the respective organizations.
2.2. Assessment of reproducibility, linearity, limit of detection and limit of quantitation A 7 member panel (Reproducibility Panel) was used to assess the reproducibility, linearity, limit of detection (LoD), and limit of quantitation (LoQ) of the Bayer Assay. This panel covered the reportable range of the assay and was created by serially diluting replication-defective mutant HIV1LAV/8E5 (Folks et al., 1986) into anti-HIVnegative EDTA plasma. In addition, two
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
acid –citrate–dextrose (ACD) panel members were created using anti-HIV-negative ACD plasma spiked with replication-defective mutant HIV-1LAE/8E5. One of these panel members quantitated close to the assay LoD; the other quantitated in the mid-range of the assay. The concentrations of the panel members were determined by direct comparison with the Reference Standard Curve prepared from the Reference Standard HIV-1 RNA (a 3.6 Kb RNA transcript containing most of the pol gene of HIV-1SF2 strain). Two operators at each of the three sites tested this panel. Each operator performed four runs, with three replicates per panel member, using each of the three kit lots of the Bayer Assay, for a total of 216 determinations per panel member. Linearity was also verified using serial dilutions of an HIV-1 RNA-positive EDTA patient specimen and an HIV-1 RNA-positive ACD patient specimen. Each panel was tested once in each of the four runs. The reproducibility study variance components were estimated using restricted maximum likelihood estimation, and the total variance | 2 (which accounts for all of site, lot, day, operator, run and within-run effects) was estimated as the sum of the individual variance components. The %CV for each panel member was calculated using the formula %CV = 10
|2 ln(10)
−1
where ln denotes natural log; this is the formula for %CV when the quantitations are log-normally distributed. The linearity of quantitation was evaluated by examining how accurately the assay measured differences between specimens with known dilutional relationships, i.e. by constructing the best-fitting 45° line to the data on log scale (the line for which observed differences exactly match the expected differences) and calculating the absolute value of the log difference between the observed values of each linearity panel member and the best-fitting 45° line. The LoD was defined as the lowest concentration of virus that yields a quantifiable result 95% of the time; thus, signal at the LoD can be reliably distinguished from background signal in HIV-
207
negative specimens. The LoQ was defined as the concentration of virus that is detected 95% of the time with a total %CV of B 45%.1 These limits were defined in conjunction with the reporting threshold (RT) of 35 HIV-1 RNA copies/ml, which was defined as the cutoff value that delivers a specificity of 95%.
2.3. Assessment of concordance of quantitation and precision of the Bayer assay and the Roche test A second, 3-member panel (referred to as the Concordance Panel), with members at the low, middle, and high end of the assay range, was tested at three sites using three lots of the Bayer Assay and at two sites using two lots of the Roche Test to estimate the concordance and precision of the two assays. This panel was also made using the replication-defective mutant HIV-1LAV/8E5 virus. The panel member at the low end of the assay range was tested using the UltraSensitive procedure. Five replicates of each panel member were tested on each run of the Bayer Assay and the Roche Test, for a total of 45 determinations per panel member using the Bayer Assay and 30 determinations per panel member using the Roche Test.
2.4. Potentially interfering substances and conditions The effect of endogenous substances (hemoglobin, triglycerides, bilirubin), therapeutic antiviral, antifungal and antibacterial drugs, common coexisting pathogens, and freeze–thaw cycles were evaluated in 25 HIV-1 RNA-positive and 25 HIV-1 RNA-negative panels. The HIV-1 RNApositive panels were created by spiking anti-HIV negative EDTA plasma with b-propiolactone (BPL)-treated viral stock (HIV-1SF2) to a target concentration of 75–100 HIV-1 RNA copies/ml. Hemoglobin was tested to a final concentration of 1 The constraint of 95% sensitivity at the LoD was required by the US FDA. The Bayer Assay that is approved for use in many other countries has a reportable assay range of 50 – 500 000 HIV-1 RNA copies/ml.
208
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
200 mg/dl (normal range B3 mg/dl); bilirubin was tested to a final concentration of 60 mg/dl (normal range 0.2– 1.3 mg/dl); and triglycerides were tested to a final concentration of 2000 mg/dl (normal range B 200 mg/dl). Therapeutic drugs (AZT, acyclovir, fluconazole, gancyclovir, ddI, trimethoprim/sulfamethoxazole, d4T, foscarnet, ciprofloxacin, 3TC, ritonavir, nevirapine, indinavir, ddC, nelfinavir, saquinavir, azithromycin dihydrate, cidofovir, delavirdine, rifampin, rifabutin, ethambutol, INH, clarithromycin, hydroxyurea) and disease pathogens (Streptococcus pneumoniae, Mycobacterium a6ium, herpes simplex virus type 2, CMV, HBV, HCV, Epstein – Barr virus, E. cloacae, Escherichia coli, Haemophilus influenzae, Klebsiella pneumoniae, P. mirabilis, Psuedomonas aeruginosa, Serratia marcescens, Staphylococcus aureus, Staphylococcus epidermidis, Candida albicans, Cryptococcus neoformans, and Streptococcus group B species) were grouped and tested in pools. Therapeutic drugs were tested at a final concentration of five times the peak serum level. Disease pathogens were tested at a final concentration of approximately 103 CFU or PFU/ml, except for HBV, HCV, and CMV, which were tested at concentrations of 108 DNA copies/ml, 106 RNA copies/ml and 60 DNA copies/ml, respectively, as measured in the Bayer bDNA assays for these viruses. For the freeze–thaw study, specimens were tested after one (reference), two, and three freeze– thaw cycles. The freeze–thaw study was also performed using EDTA plasma from HIV-1-infected individuals. These studies were designed to assess equivalence between a reference condition and a test condition (i.e. a specimen with and without a potential interfering substance or condition). In these studies, equivalence testing in HIV-1 RNAnegative specimens required rejecting the hypothesis that the specificity in the test condition was less than or equal to 95%. The surrogate measure that was used to test the hypothesis was based on the average increase in the relative light units (RLUs) induced by the test condition. For HIV-1 RNA-positive specimens, the measure of equivalence between reference and test conditions was percent concordance. Quantifica-
tion values from the reference and test condition were defined to be concordant if they were not statistically significantly different given the variability of the two assays; i.e. using a significance level of 5%, one would expect that 5% of the results would be judged discordant by chance alone, even if the specimens were tested twice in the reference condition; i.e. the expected percent concordance would be 95%. A claim of equivalence required rejecting the hypothesis that the percent concordance in the test condition was less than or equal to 90%. The surrogate measure used to test the hypothesis was based on the average change in quantitation induced by the test condition as compared to the reference condition.
2.5. Plasma collection and handling The effects of different anticoagulants and handling conditions were evaluated in the following collection tubes from Becton-Dickinson and Company (BD Diagnostic Systems, Sparks, MD): K3 EDTA glass Vacutainer® tubes, K2 EDTA plastic Vacutainer tubes, K2 Plasma Preparation Tubes (PPT™), and ACD Solution A (ACD-A) glass Vacutainer tubes. The EDTA and ACD-A Vacutainer tubes were centrifuged within 4 h of collection. One aliquot of plasma from each of the tubes was frozen at − 60 to − 80 °C immediately, while three other aliquots were held at 2–8 °C for 8, 24, and 48 h, respectively, and then frozen. PPTs were centrifuged within 2 h and either frozen immediately, held for 24 h at room temperature (RT), or held at 2–8 °C for 24 h prior to freezing. Equivalence between the reference condition (K3 EDTA Vacutainer tubes, centrifuged and frozen at − 60 to − 80 °C within 4 h of collection) and different collection devices and handling procedures were tested as described above.
2.6. Clinical specificity Clinical specificity was determined by testing 912 repeat volunteer HIV-seronegative blood donors from two blood centers. Testing was performed at three sites, and specimens were apportioned between the three investigational lots at
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
each site. Results that were above the LoQ of 75 HIV-1 RNA copies/ml in the Bayer Assay were assumed to be false positive. A 2-test was used to determine if specificity was independent of gender.
2.7. Clinical non-specificity EDTA plasma specimens were collected from anti-HIV-negative subjects who were infected with HCV, HBV, HTLV, or CMV or who had elevated levels of rheumatoid factor or ANA. Specimens from 10 subjects in each disease category were tested. Equivalence between the specimens from the disease groups and from the anti-HIV negative repeat volunteer blood donors was tested as described above. This study was performed to supplement the data from the nonclinical studies on the effect of other disease pathogens, which used spikes of cultured pathogens into HIV-1 RNA-positive and HIV-1 RNA-negative specimens.
2.8. Clinical sensiti6ity Clinical sensitivity was determined in EDTA or ACD plasma specimens from 984 subjects (698 males and 286 females) with documented HIV-1 or HIV-1/2 ELISA and supplemental test reactivity. Subjects were stratified by CD4+ cell count into three categories for analysis: CD4+ B200 cells/mm3, CD4+ between 200 and 500 cells/mm3, and CD4+ \ 500 cells/mm3. Specimens were apportioned between the three investigational lots of the Bayer Assay. The majority of these specimens (936) were also tested with the Roche Test. The UltraSensitive procedure of the Roche Test was used where appropriate. Within each CD4+ group, a 2-test was used to determine if the percent of specimens with levels of HIV-1 RNA less than the LoQ was independent of gender. Assay sensitivity was calculated as the percent of specimens with levels of HIV-1 RNA]LoQ. Two-sided 95% confidence intervals were determined, using the binomial distribution, for each CD4+ category by gender and for both genders combined.
209
2.9. Within-patient 6ariability Within-patient variability (WPV) (i.e. biological variability) was assessed weekly for 8 weeks in 33 subjects (24 males and nine females) using the Bayer Assay and the Roche Test. Subjects were required to have no change in therapy and no clinical events within 3 months of study enrollment, and were considered clinically stable by the investigator. Any change in therapy or clinical event during the 8-week study period disqualified them from the study. Subjects were required to provide at least six of the eight weekly specimens to remain eligible. Testing was performed at three sites by each assay method, using three lots of the Bayer Assay and two lots of the Roche Test. Each of the 6–8 successive specimens from a given subject was tested in a different run (but within the same kit lot). The ultimate objective of this study was to determine the minimum fold-change (i.e. the ratio of the largest to smallest of two successive measurements on the arithmetic scale) between two successive measurements that would signal a potential change in a subject’s clinical status. The minimum fold-change was estimated such that 95% of observed changes from subjects who experience no actual change in viral load would be expected to be less than the estimate of the minimum fold-change. The sources of variability that contribute to differences between successive measurements are biological (within-patient) and assay-related (site, lot, operator, within-run and between-run). In this study, testing of each subject’s specimens was done at the same site by the same operator and lot. To obtain a generalizable fold-change that included all sources of variability, data from the WPV study (biological variation) were used in conjunction with the data from reproducibility study (Bayer Assay) and concordance study (Roche Test) to obtain variability estimates for site, operator, lot, run and withinrun variation. An estimate of total variability was calculated as the sum of WPV and all other sources of variability estimated from the reproducibility or concordance studies. The total variability was used to estimate the 95th percentile of the distri-
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
210
bution of fold-changes (‘minimum fold-change’) between two measurements that would be observed even if there were no underlying changes in viral load. A change between two successive measurements that is larger than this 95th percentile would be unlikely to occur by chance and can be indicative of a change in clinical status.
3. Results
3.1. Reproducibility, linearity, LoD and LoQ 3.1.1. Reproducibility Table 1 presents data on assay reproducibility, including overall between-run and within-run variance components, as well as total percent coefficient of variation (%CV). The concentration of HIV-1 RNA in panel members used in this study ranged between 62 and about 620,000 copies/ml. Over most of the assay range (124 to \ 500,000 HIV-1 RNA copies/ml), the total %CV due to all sources of variation ranged from 23.7 to 28.9%. For the three panel members below 100 HIV-1 RNA copies/ml (EDTA panel members 6 and 7, and ACD panel member 5A), the variation ranged between 30.0 and 39.3%. In general, the
overall between-run component of variance contributed more to total variance than did the within-run component. However, with two panel members (6 and 7) that contained 93 and 62 copies/ml, the within-run component of variance was the greater contributor to total variance.
3.1.2. Linearity Fig. 1 shows the results of the linearity evaluation for the Bayer Assay. This figure was generated by plotting expected versus observed geometric mean (geomean; log10) quantitations for panel members used in the reproducibility study described above. The line is the best-fitting 45° line. The absolute deviations from the bestfitting 45° line were less than 0.05 log10 over the entire range of 62–620,282 HIV-1 RNA copies/ ml. Analysis of data from the two serially diluted patient specimens that were collected in ACD and EDTA showed that the slopes were not significantly different from 1.0 (1.069 and 1.047, respectively; data not shown). The ACD and EDTA patient specimens, undiluted, had geomean values of 635,000 and 438,000 HIV-1 RNA copies/ ml, respectively. Thus, linearity of the product was observed both with cell culture-derived HIV-1 and with HIV-1 obtained directly from patient specimens.
Table 1 Coefficient of variation (%CV) for overall between-run, within-run and total variation Panel members
1 2 3 4 5 6 7 3A 5A
Value assignment (HIV-1 RNA copies/ml)
620,282 62,028 6203 620 124 93 62 5640 94
Variance Overall between run (%)
Within run (%)
Total %CV
20.4 22.4 19.2 19.6 18.6 18.6 19.5 23.8 30.1
14.1 12.9 13.6 15.5 18.7 23.1 25.2 16.0 24.2
24.9 26.0 23.7 25.1 26.6 30.0 32.2 28.9 39.3
The variance components for lot, site, operator, and day were combined for the overall between-run estimate. Within the ‘‘overall’’ between-run component, the majority of the variability came from the between-run (within-day) component, followed by the between-day, between-lot, between-operator, and lastly, between-site terms. There were 216 determinations of each panel member (3 replicates/run×4 runs×3 operators×3 sites×3 lots). Panel members 1–7 were prepared in EDTA; 3A and 5A were prepared in ACD.
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
211
Fig. 1. Expected versus observed values for HIV-1 RNA in the Bayer assay.
3.1.3. LoD and LoQ The LoD was based on the percent detection of reproducibility of panel members at the lower end of the assay range. The concentration of HIV-1 RNA with 95% detection was estimated to be 67 HIV-1 copies/ml. The upper one-sided 95% confidence limit of the concentration with 95% detection was calculated using the one-sided upper 95% confidence limit for the 95th percentile of a normal distribution (Hahn and Meeker, 1991), and was 73 copies/ml. This was rounded up to the nearest 5 HIV-1 RNA copies/ml to obtain the LoD of 75 HIV-1 RNA copies/ml. Because the total %CV was B 40% at this concentration, 75 copies/ml was also the LoQ for the assay. 3.2. Assessment of concordance and precision for the Bayer and Roche assays The mean quantitation values and %CVs of each of the three Concordance Panel members were estimated using the Bayer Assay and the Roche Test. The results are shown in Table 2. The quantitations were evaluated to test whether the
means were equivalent, and the precision of quantitation was estimated for each assay method to test whether the variances were equal. The geomean quantitations from the Bayer assay and the Roche Test for the high and medium panel members were not statistically different, whereas the lowest panel member quantitated significantly higher in the Roche Test. The Bayer Assay quantitated all three panel members, on average, 29% lower than did the Roche Test. The %CVs obtained from the Bayer Assay were lower than the %CVs from the Roche Test for the high, medium, and low panel members.
3.3. Potentially interfering substances and conditions There was no effect on test results from HIV-1 RNA-positive specimens or on anti-HIV-negative specimens by elevated hemoglobin (400 times upper limit of normal), elevated bilirubin (20 times upper limit of normal), elevated triglycerides (five times upper limit of normal), therapeutic drugs used in HIV-infected patients (5 times peak
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
212
serum levels), disease pathogens, and multiple freeze –thaw cycles. However, grossly icteric (\ 20 mg/dl bilirubin) or chylomicronemic (\ 1000 mg/dl triglycerides) specimens (visually greenish or milky, respectively) should not be used in the Bayer Assay, as they may produce a decreased signal.
3.4. Plasma collection and handling All of the EDTA tube types (glass K3 EDTA Vacutainer™ tubes, plastic K2 EDTA Vacutainer tubes, K2 PPT) provided statistically equivalent HIV-1 RNA quantitations for the anti-HIV-1positive specimens and equivalent RLUs for the anti-HIV-negative specimens when the plasma was frozen at −60 to −80 °C within 2 (PPT) or 4 h of collection. There was no difference in RLUs from anti-HIV-negative specimens collected in tubes with K3 EDTA or ACD Solution A. In the 126 anti-HIV-1-positive specimens evaluated in this study, quantitations from specimens collected in ACD-A were, on average, 15% lower (0.07 log10) than the quantitations from specimens collected in K3 EDTA. This difference was consistent across the reportable range of the assay (slope = 0.99, 95% confidence interval 0.97– 1.01). The volume of the ACD-A solution in collection tubes results in a 17.6% dilution of the blood, so the 15% decrease in viral load seen in this study was expected. The specimen handling studies showed that for all tubes studied except the PPTs, plasma can be stored at 2–8 °C for up to 48 h before freezing.
Plasma collected using an EDTA PPT can be stored on the separator gel at RT for up to 24 h before freezing.
3.5. Clinical specificity Clinical specificity was evaluated by testing specimens from 912 repeat volunteer blood donors. Data were analyzed separately for males (n= 546) and females (n=366). There was no statistically significant difference in assay specificity between the genders (P= 0.83); therefore, the data were combined for this analysis. The percent of specimens with results below the LoQ was 97.6% (95% confidence interval 96.4%, 98.5%).
3.6. Clinical non-specificity Specimens from anti-HIV-negative individuals with other diseases were tested to confirm the results seen in the nonclinical experiments (potentially interfering substances) that used anti-HIV negative specimens spiked with disease organisms. Additionally, specimens from subjects with autoimmune diseases were tested. The specimens in the disease categories tested [HBV, HCV, HTLV, CMV, rheumatoid factor, anti-nuclear antibodies] had RLU values that were statistically equivalent to the RLUs of the repeat volunteer blood donor specimens (see Section 3.5). The mean RLU for the blood donor group was 0.32, while the mean RLU for the 60 subjects in the other disease group was 0.29.
Table 2 Coefficient of variation (%CV) of quantification of HIV-1 RNA by the Bayer Assay and the Roche Test Panel member
Bayer Assay
Roche Test
Number of tests
HIV-1 RNA copies/ml (95% CIa)
Total %CV
Number of tests
HIV-1 RNA copies/ml (95% CI)
Total %CV
1
45
25.1%
30
45 44
15.9% 31.6%
29 28
69,527 (53,907–89,674) 6515 (5265–8077) 92 (88–97)
39.5%
2 3
49,644 (38,199–64,518) 5138 (4818–5480) 59 (55–60)
a
CI= confidence interval.
43.7% 46.2%
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
213
Table 3 Clinical sensitivity: percentage of specimens with detectable HIV-1 RNA in patient groups stratified based on number of CD4+ cells
3.8. Comparison of HIV-1 RNA quantitations in clinical specimens using the Bayer Assay and the Roche Test
CD4+
Gender
N
% with results]LoQ (95% CIa)
CD4+B200
Males Females Combined
269 33 302
98.5% (96.3–99.6) 100.0% (91.3–100) 98.7% (96.7–99.6)
CD+ = 200
Males
251
87.6% (82.9–91.4)
Females Combined
120 371
90.0% (83.2–94.7) 88.4% (83.2–94.7)
Males Females Combined
178 133 311
67.4% (60.0–74.2) 62.4% (53.5–70.6) 65.3% (59.7–70.6)
Of the 936 specimens tested, 794 had quantitations that were in the reportable range of both assays. The geomean quantitation obtained by the Bayer Assay was 12,383 HIV-1 RNA copies/ ml, while the geomean quantitation by the Roche Test was 20,349 HIV-1 RNA copies/ml. There were 12 data points that were significantly different from the other data, in that the Bayer Assay quantitation was higher than the Roche test quantitation. Seven of these results came from one patient in the WPV study, and two results were from a single patient tested in a clinical utility study. The remaining three specimens were from three additional subjects. These 12 data points were excluded from the regression analyses and are not shown in Fig. 2. Additional information on these 12 data points is presented in Section 4. Fig. 2 is a plot of the log differences between the two assays versus the mean log quantitations. Regression analysis showed a slope of 0.003 (95% confidence interval; −0.017, 0.022). These data indicate that quantification values from the Bayer Assay are, on average, 39% lower than the values from the Roche Test. The slope of 0.003 cannot be distinguished from a slope of 0 (P= 0.78) therefore, we conclude that differences in quantitation values from the two assays are consistent across the common reportable range of the two assays.
– 500
CD+\500
a
CI= confidence interval.
3.7. Clinical sensiti6ity The clinical sensitivity of the assay was assessed in 984 subjects who were stratified into three groups based on their CD4+ cell count. This group was a representative cross-section of HIV-1-infected patients in the US. The mean age was 39 years (range 15– 77), 71% were males, 51% were Caucasian, 41.5% were AfricanAmerican, and 4.6% were Hispanic. Of the 984 subjects, 675 (69%) were receiving anti-retroviral therapy at the time of blood collection and 309 (31%) were not. Table 3 summarizes the number of subjects tested in this study by gender (and combined) and CD4+ category and shows the percentage of specimens within each category with HIV-1 RNA levels that were at or above the LoQ of 75 HIV-1 RNA copies/ml). There were no statistically significant differences between males and females in terms of sensitivity (P \ 0.4 for each of the 3 CD4+ groups). The results show that, as expected, sensitivity results were inversely related to CD4+ counts. The highest sensitivity (98.7%) was found for subjects with CD4+ counts below 200 cells/mm3 and the lowest (65.3%) for subjects with CD4+ counts above 500 cells/mm3.
Fig. 2. Log differences between the Bayer Assay and Roche Test versus the mean log quantitation for the 782 anti-HIV-1positive clinical specimens.
214
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
3.9. WPV
4. Discussion
Eight of the 33 subjects who were enrolled in the study were eliminated from the variance analyses for the two test methods. Five of these were eliminated because they had a 20-fold or greater change in assay quantitation by both assays during the 8-week study period and were thus not considered to satisfy the ‘clinically stable’ requirement for this study. Four of these five subjects were intravenous drug users, so the use of illicit drugs or lack of compliance with medication might have caused these large fluctuations in viral load. Three additional subjects were eliminated from the variance analyses because either all specimens (Bayer) or all but one specimen (Roche) had viral levels below the respective assay’s LoQ. The variance estimates from the WPV study, which included biological, run, and within-run sources of variance, were combined with variance estimates for site, operator, and lot from the reproducibility study for the Bayer assay or the concordance study for the Roche test, to estimate the total assay plus biological variability for each assay method. These estimates were then used to calculate the minimum change between two successive measurements, for each assay method, that could be considered to be significant and not attributed to chance. The total variance estimate for the Bayer Assay was 0.0276. Based on this estimate, the minimum fold-change between two successive measurements of HIV-1 RNA using the Bayer Assay that is unlikely to have happened by chance was estimated to be 2.9-fold (0.46 log10) with an upper confidence limit of 3.2-fold. The total variance estimate for the Roche Test was 0.0365. The minimum fold-change between two successive measurements of HIV-1 RNA using the Roche Test that is unlikely to have happened by chance was estimated to be 3.4-fold (0.53log10) with an upper confidence limit of 6.3-fold (0.80 log10). Because the smaller concordance study was used to obtain the variance estimates for the Roche Test, the actual degrees of freedom were lower leading to a bigger difference between the estimate of fold-change and the upper confidence limit.
The studies described in the current paper represent the validation trial performed for US product registration. They differ from several earlier studies, discussed below, in that the kit lots used were manufactured with diverse raw materials using the final, validated manufacturing processes. They also differ in the number and types of specimens tested, the apportionment of all specimens within a study (e.g. sensitivity, specificity, WPV, etc.) between three different kit lots, and the number of operators and sites performing the testing. Thus, the results seen in this study should reflect the future performance characteristics of the assay across sites and kit lots. The results described here, however, are similar in many aspects to those seen in previous studies (Highbarger et al., 1999; Erice et al., 2000; Murphy et al., 2000). Highbarger et al. (1999) reported on a comparison study between the Bayer Assay and two versions of the UltraSensitive procedure of the Roche Test. A total of 381 specimens from 59 HIV-1 infected individuals were tested by the Bayer Assay and an in-house UltraSensitive procedure for the Roche Test that used 1.0 ml of plasma. The results showed a slope that was statistically equivalent to 1.0 (1.0052) and an intercept of −0.0915, which was statistically different from 0.0. A subset of 47 specimens was tested by the Bayer Assay and the Roche UltraSensitive procedure, where 0.5 ml of plasma is used. In this comparison, the slope was 1.1214 and the intercept was −0.608, representing a 1.048-fold difference between the two assays on the log10 scale. In the study reported here, with a larger group of clinical specimens (n= 794), the slope for modeling the difference in log10 results as a function of the average of the log10 results was not significantly different from zero. There was, however, an overall 39% difference in quantitation between the two assays, with the Bayer Assay having lower quantitations than the Roche Test. Data from the smaller Concordance Panel Study, where replication defection mutant HIV-1LAV/8E5 virus was used, showed a 29% difference between the two assays. In each of the studies, three kit lots of the Bayer Assay and two kit lots of the Roche Test
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
were used for testing. Based on the lots used in the current studies, different lot combinations may produce differences between the two assays that range from 0 to 50%. A study by Erice et al. (2000) evaluated the performance of the research use version of the Bayer Assay, including LoD, specificity, and linearity in four laboratories. The current study had similar results for assay linearity, but showed better specificity, probably due to different cutoffs in the two assay versions (50 copies/ml in the research use version, 75 copies/ml in the current version). Both the Erice study and the current study found that the Bayer Assay had better reproducibility than the Roche Test. Murphy et al. (2000) reported on a multicenter comparison of the Roche 1.5 test, the Organon Teknika NucliSens QT Assay and the Bayer assay. The specificity of the ‘research use’ Bayer assay in the Murphy study, using 100 anti-HIV-1negative specimens, was 98%. The data are similar to the specificity results of 97.6% reported here, despite the higher LoD of 75 copies/ml. Murphy reported a combined between-run and within-run %CV of 18.2%, a between-lot %CV of 21.9% (using two lots), and a between-site %CV of 21.3% (using two sites). Because the comparison of lots and sites includes different runs, the Murphy estimates of between-lot and between-site %CVs include between-run and within-run %CVs. Murphy’s results are very similar to the results obtained in the clinical trial reported here. When the clinical trial data were analyzed using a method similar to that used by Murphy, the combined between-run and within-run %CV was 21.4%, the between-lot %CV was 22.4%, and the between-site %CV was 21.5% Because different manufacturers use different standardization methods for assigning HIV-1 RNA quantitations in their assays, the various guidance documents caution against changing assay methods during serial measurements. The results from the assay comparison study using clinical specimens indicate, however, that the relationship between Roche viral load units and Bayer viral load units is consistent across the shared reportable range. This suggests that a laboratory could establish a conversion factor using
215
currently available lots from both manufacturers and use this factor to convert between Roche viral load units to Bayer viral load units. There will be some exceptions to the application of this conversion factor, as seen with the 12 data points (representing five individuals) in the clinical specimen comparison where the Bayer Assay quantitation was higher than the Roche Test quantitation. Sufficient specimen volume was available to sequence specimens from two of these subjects. An analysis of the gag sequences compared to the Roche primers and probe found five and six mismatches between the forward primer and the gag sequence in two specimens from the WPV subject. An A:C nucleotide substitution was seen in the gag region of the specimen from the subject in the clinical utility study. These mismatches may account for the lower than expected quantitation in the Roche Test. Both subjects were infected with HIV-1 subtype B. Our current knowledge of HIV replication and development of resistance, and the need to reduce viral load as much as possible for as long as possible, makes viral load determinations essential for patient management. Although the Bayer assay is not quite as sensitive as the Roche Test (LoDs of 75 and 50 copies/ml, respectively), the Bayer assay has been shown to have superior reproducibility, particularly near the LoD (current study and Erice et al., 2000). Assay reproducibility is critical for determining when an assay result is different enough from a previous result to indicate a significant change in viral load, perhaps due to development of resistance to a drug or class of drugs. Total variability of the Bayer Assay, including biological and assay components indicated that a change of 2.9-fold was a statistically significant change. Therefore, the Bayer assay can be used in accordance with the Henry J. Kaiser Foundation (2001), which states (p. 3) that ‘A minimally significant change in plasma viremia is considered to be a 3-fold or 0.5 log10 increase or decrease.’ In summary, the results of the current studies showed that the Bayer assay has excellent reproducibility, a broad linear range (75–500,000 HIV1 RNA copies/ml), throughput of 168 patient results per two plate run, and few limitations for
216
C.A. Glea6es et al. / Journal of Clinical Virology 25 (2002) 205–216
use. Because this test is designed for use only in individuals who are known to be HIV-1 positive, the clinical specificity of 97.6% is adequate for its intended use. As reported by Elbeik et al. (2002), the Bayer Assay has similar detection of HIV-1 Group M subtypes tested (A– F) compared to the Roche test (version 1.5). However, in the Elbeik study, the quantitation of multiple replicas from serial dilutions of each subtype demonstrated higher reliability in test results for the Bayer assay as compared to the Roche Test (version 1.5). This finding was, in part, explained by the difference in acceptable ranges for the kit controls (low positive 0.7 and high positive 0.6 log10 for the Bayer assay, low positive 0.95 and high positive 0.84log10 for the Roche test version 1.5). In addition, a comparative cost analysis (Elbeik et al., 2000) demonstrated savings of up to 53% for labor, 67% for disposables, and 50% for biohazardous waste labor for the Bayer Assay as compared to the Roche Test (version 1.5). These characteristics make the Bayer Assay an attractive method for clinical laboratory use.
References Carpenter CC, Cooper DA, Fischl MA, et al. Antiretroviral therapy in adults: updated recommendations of the International AIDS Society –USA Panel. J Am Med Assoc 2000;283:381 – 90. Collins ML, Irvine B, Tyner D, et al. A branched DNA signal amplification assay for quantification of nucleic acid targets below 100 molecules/ml. Nucleic Acids Res 1997;25:2979 –84. Elbeik T, Charlebois E, Nassos P, et al. Quantitative and cost comparison of ultrasensitive human immunodeficiency virus type 1 RNA viral load Assays: Bayer bDNA Quan-
tiplex Versions 3.0 and 2.0 and Roche PCR Amplicor Monitor Version 1.5. J Clin Microbiol 2000;38:1113 – 20. Elbeik, T., Alvord, W.G., Trichavoroj, R. et al., 2002. Comparative analysis of HIV-1 viral load assays on subtype quantification: Bayer Versant HIV-1 RNA 3.0 Versus Roche Amplicor HIV-1 Monitor Version 15. J Acq Immune Defic Syndr 2002;29:330 – 339. Erice A, Brambilla D, Bremer J, et al. Performance characteristics of the QUANTIPLEX HIV-1 RNA 3.0 Assay for detection and quantitation of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 2000;38:2837 – 45. Folks TM, Powell D, Lightfoote M, et al. Biological and biochemical characterization of a cloned LEU- – cell surviving infection with the acquired immune deficiency syndrome retrovirus. J Exp Med 1986;164:280 – 90. Guidelines for the Use of Antiretroviral Agents in HIV-Infected Adults and Adolescents. Department of Health & Human Services and the Henry J. Kaiser Foundation, 2001. Available at: http://www.hivatis.org/ trtgdlns.htmlc AdultAdolescent, accessed March 14, 2001. Hahn GJ, Meeker WQ. Statistical Intervals, A Guide for Practitioners. New York: Wiley, 1991 The critical values were obtained from Tables A12a – d. Highbarger HC, Alvord WG, Jiang MK, et al. Comparison of the Quantiplex Version 3.0 Assay and a sensitized Amplicor Monitor Assay for measurement of human immunodeficiency virus type 1 RNA levels in plasma samples. J Clin Microbiol 1999;37:3612 – 4. Mulder J, McKinney N, Christopherson C, Sninsky J, Greenfield L, Kwok S. Rapid and simple PCR assay for quantitation of human immunodeficiency virus type 1 RNA in plasma: application to acute retroviral infection. J Clin Microbiol 1994;32:292 – 300. Murphy DG, Coˆ te L, Fauvel M, Rene P, Vincelette J. Multicenter comparison of Roche COBAS AMPLICOR MONITOR Version 1.5, Organon Teknika NucliSen QT with Extractor, and Bayer Quantiplex Version 3.0 for quantification of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 2000;38:4034 – 41. Sun R, Ku J, Jayakar H, et al. Ultrasensitive reverse transcription-PCR assay for quantitation of human immunodeficiency virus type 1 RNA in plasma. J Clin Microbiol 1998;36:2964 – 9.