- Email: [email protected]
A Comparison of the performance of nine commercially available anti-HTLV-I screening assays
Journal of Virological Methods, 45 (1993) 83-91 0 1993 Elsevier Science Publishers B.V.
Journal of Virological Methods
All rights reserved / 0166-0934/93/$06.00 VIRMET 01549
A Comparison of the performance of nine commercially available anti-HTLV-I screening assays Anna Karopoulos*,
Colin Silvester and Elizabeth
National HIV Reference
Laboratory,
Melbourne
M. Dax
(Australia)
(Accepted 6 April 1993)
Summary
The performance of eight anti-HTLV-I enzyme immunoassays (EIAs) and one particle agglutination assay was compared with respect to sensitivity, specificity and delta values, by testing a panel containing 99 anti-HTLV-I positive and 126 anti-HTLV-I negative samples which had been characterised by western blot and some by radioimmunoprecipitation assay. The estimated sensitivities produced by these assays ranged between 99% and 100% and estimated specificities were between 95.2% and 100%. The performance of the EIAs was further differentiated by using the delta value which measures the ability of an assay to separate the positive and negative populations from the cutoff value. A delta value could not be calculated for the particle agglutination assay (Serodia) because the test readings were not quantitative. The EIAs most likely to correctly identify anti-HTLV-I positive and anti-HTLV-I negative samples included the Cambridge Biotech, DuPont, Genetic Systems and Olympus assays. Our findings suggest that there may be some difficulty in correctly identifying anti-HTLV-I negative samples using the Abbott, Cellular Products Incorporated (CPI), Coulter and Diagnostic Biotechnology assays. The Serodia assay produced comparable sensitivity and specificity to the eight EIAs. HTLV-I screening assay; Delta value; Anti-HTLV-I
‘Corresponding author. Address: National HIV Reference Laboratory, Fairfield Hospital, Yarra Bend Road, Fairfield VIC 3078, Australia. Fax: +61 3 482 4352.
84
Introduction Human T-cell lymphotropic virus type I (HTLV-I) has been associated aetiologically with adult T-cell leukaemia/lymphoma and a neurological disease, tropical spastic paraparesis otherwise known as HTLV-I associated myelopathy (Yoshida et al., 1982; Osame et al., 1987). HTLV-I infection has been reported in Japan, the Caribbean, areas of the USA (Yamamoto et al., 1985) and Africa (Verdier et al., 1989). More recently HTLV-I has been reported in remote populations in Papua New Guinea (Yanagihara et al., 1990), regions of the southwestern Pacific (Brindle et al., 1988; Yanagihara et al., 1991) and Australian Aboriginies (Bastian et al., 1993). HTLV-I infection in blood transfusion recipients has been well documented and a seroconversion rate of 63% has been reported in recipients of contaminated whole blood and blood components (Okochi et al., 1984). Transmission of the virus has also been reported from mother to child through are by breast feeding (Guroff et al., 1986). Other modes of transmission intravenous drug users (IVDUs) who share contaminated needles and syringes (Guroff et al., 1986) and by sexual transmission (Murphy et al., 1989). In contrast to HTLV-I, the transmission and disease associations of human T-cell lymphotropic virus type II (HTLV-II) are not well defined. HTLV-II has been isolated from two individuals with hairy cell leukaemia and antibody to the virus has been detected in IVDUs in the United States (Kalyanaraman et al., 1982; Rosenblatt et al., 1986; Lee et al., 1989). The increasing awareness of the prevalence of HTLV has prompted the development of many screening tests to detect antibody to the virus. Some have been compared previously (Kline et al., 1991; Cossen et al., 1992; Tosswill et al., 1992). Three EIAs (Abbott, CPI and DuPont) were licensed in 1988 by the Food and Drug Administration, USA for screening antibody to HTLV-I in serum or plasma. We assessed the performance of 9 commercially available anti-HTLV-I screening assays using a panel of well characterised sera.
Methods Panel samples
Two groups of specimens, totalling 225 samples, formed the panel. The first group consisted of 99 Japanese blood donor samples which were anti-HTLV-I positive by western blot and radioimmunoprecipiatation assay (RIPA). The sources of each of the samples were known, however no clinical data was available. The negative population consisted of 126 Australian blood donor samples that were anti-HTLV-I negative by western blot. Questionnaires and interviews used by Australian blood banks also ensured that the donors were at low risk of infection by blood-borne viruses. 45 of the 126 specimens were non-reactive
85
by the Genetic Systems HIV-1 EIA. This subset of samples had been stored at 4°C and had not been subjected to multiple freeze thaw cycles. The remaining 81 Australian blood donor samples produced false positive reactions in either viral lysate or recombinant based anti-HIV-l EIAs. These samples were collected between 1985 and 1989 and had been freeze thawed more than once. Characterization of the Panel
The panel samples were further characterised at our laboratory using the Diagnostic Biotechnology HTLV-I western biot (Diagnostic Biotechnology Ltd, Singapore) utilising HTLV-I viral lysate antigens and the Diagnostic Biotechnology version 2.2 HTLV-I western blot utilising both HTLV-I viral lysate antigens and recombinant glycoprotein antigens. Western blot indeterminate samples were tested by RIPA using HTLV-I antigen prepared from the HUT102-B2 cell line (Abbott Laboratories, Chicago, USA) (Lee et al., 1991). Western blot and RIPA results were interpreted according to strict criteria. For a sample to be designated anti-HTLV-I positive, the sample had to demonstrate reactivity to the gag protein ~24, two additional viral proteins (~19, ~26, ~28, ~32, ~36, ~53) and at least one env protein (gp46 or gp68) or two recombinant e?zv products (Rgp46 and Rgp21) by a combination of western blot and RIPA. Samples with no reactivity to viral bands were considered negative for antibodies to HTLV-I. Results of these tests confirmed the original anti-HTLV-I status of the panel samples. Assays
The 9 HTLV-I/II screening assays that underwent analysis are outlined in Table I. Analysis
Assay procedures were performed according to the respective kit inserts and only test runs that were valid according to the manufacturers’ instructions were included in the comparison. Absorbance to cutoff (A/Co) ratios were calculated for data obtained in the form of absorbance readings, so that specimens with A/Co ratios greater than or equal to one were considered reactive for antibodies to HTLV-I. The sensitivity, or the ability of an assay to detect anti-HTLV-I in a presumed anti-HTLV-I positive sample, was determined by testing the 99 seropositive Japanese samples. An estimate of sensitivity in any given assay was made by determining the percentage of specimens found reactive compared with the total number of presumed anti-HTLV-I positive specimens tested. The specificity was determined by the ability of an assay to identify a presumed anti-HTLV-I negative sample as negative and was expressed as the percentage
86 TABLE I Anti-HTLV-I
Screening Assays Antigen
Test kit name
Manufacturer
Abbott HTLV-I EIA
Abbott Laboratories,
Detect HTLV-I/II
Manufactured by IAF BioChem for Coulter Corp., FL, USA
synthetic peptide
Diagnostic Biotechnology HTLV-I ELISA
Diagnostic
viral lysate
DuPont HTLV-I ELISA’
DuPont, DE, USA
viral lysate (HUT-102B2)
Genetic Systems HTLV-I EIA
Genetic Systems Corp., WA, USA
Recombinant
Cambridge
EIA
viral lysate (HUT-102B2)
Chicago, USA
Biotechnology
Ltd., Singapore
Retrotek HTLV-I ELISA’
Cellular Products Inc., NY, USA
viral lysate (HB,) recombinant gene (gp21) viral lysate
Serodia HTLV-I particle agglutination assay
Fujirebio Inc., Tokyo, Japan
viral lysate
SynthEIA
Manufacturered by United Biomedical Inc. for Olympus Corp., NT, USA
viral lysate and synthetic peptide
HTLV-I/II
EIA
HTLV-I ELISA
Biotech Corp., MA, USA
env
*The format of the DuPont and CPI assays have been modified since this comparison was performed. Therefore the results presented in this study regard the performance of the pre-modified assays.
of the total number of specimens found negative compared with the total number tested. Delta values were also calculated for the presumed anti-HTLV-I status positive and negative populations. Calculating delta values is useful to differentiate performance of assays with similar levels of sensitivity and specificity and to give an indication of the expected level of performance of an assay. Calculation and full description of delta has been reported previously (Crofts et al., 1988; Maskill et al., 1988). Briefly, the delta value measures the ability of an assay to separate the positive and negative population from the cutoff and is calculated by dividing the mean loglo A/Co ratio by the standard deviation of each population. Therefore an assay with the highest absolute delta positive (6 +) and delta negative (6 -) value would be the test with the highest probability of separating positive and negative populations.
Results
Anti-HTLV-I
positive population
The sensitivity produced
by these assays was calculated
for the positive panel
87
TABLE II Sensitivity, specificity and delta values produced by the HTLV screening assays Screening assay
Sensitivity (%)
95% CI (%)
S+
Diagnostic Biotechnology DuPont CPI Cambridge Biotech. Abbott Genetic Systems Coulter Olympus Serodia
100.0 100.0 100.0 99.0 99.0 100.0 99.0 100.0 100.0
96.2-100.0 96.2-100.0 96.2-100.0 94.4100.0 94.4100.0 96.2-100.0 94.4100.0 96.2-100.0 96.2-100.0
9.57 7.05 6.28 4.05 3.96 2.98 2.79 2.67 N/A
Cambridge Biotech Genetic Systems Olympus DuPont Abbott CPI Diagnostic Biotechnology Coulter Serodia
100.0 100.0 100.0 100.0 100.0 96.0 96.8 95.2* 97.6
97.0-100.0 97.&100.0 97.&100.0 97.&100.0 97.&100.0 91.g 98.3 92.0- 98.8 89.9- 97.8 93.2- 99.2
S4.85 4.23 2.62 2.29 1.77 1.44 1.43 1.30 N/A
N/A: Not-Applicable CI: Confidence Interval l Specificity and 95% CI calculated according to the initial reactor rate, because there were insufficient tests to complete testing in duplicate. Therefore the specificity calculated for this assay may include technical error.
samples. Six assays, CPI, Diagnostic Biotechnology, DuPont, Genetic Systems, Olympus and Serodia correctly identified all 99 anti-HTLV-I positive samples yielding an estimated sensitivity of 100% with a 95% confidence interval of 96.2 to 100%. The remaining assays Abbott, Cambridge Biotech and Coulter, repeatably found 1 anti-HTLV-I positive sample negative thus producing an estimated sensitivity of 99% with a 95% confidence interval of 94.4 to 100% (Table II). The Abbott and Coulter assays incorrectly identified the same seropositive sample while the Cambridge assay failed to detect a different seropositive panel member. The 6 + values for these assays are also shown in Table II. The 6 + values ranged from 9.57 to 2.67. A delta value could not be calculated for the Serodia assay because test readings are not quantitative. The mean and 95% confidence interval of the A/Co ratio produced by the EIAs with the anti-HTLV-I positive samples are shown in Fig. 1. The Coulter and Olympus assays produced the greatest variation of A/Co ratios and the Diagnostic Biotechnology assay showed the least variation with these samples. Anti-HTLV-I The Abbott,
negative population Cambridge
Biotech,
DuPont,
Genetic
Systems and Olympus
88
10 9
a
I
T
1
2-1
0’1,
T
T
I
~
/ ~
i
I
I
PN Abbott
I,,
I
PN CB
/
PN CPI
I
I,
PN Coulter
Screening
I
I,,‘,
PN DB
I
PN DuPont
PN OS
PN Olympus
assays
Fig. 1. Mean and 95% confidence interval of absorbance/cutoff ratio produced by the enzyme immunoassays when the anti-HLTV-I positive and anti-HTLV-I negative panels were tested. CB, Cambridge Biotech; CPI, Cellular Products Incorporated; DB, Diagnostic Biotechnology; GS, Genetic Systems; P, anti-HTLV-I positive population; N, anti-HTLV-I negative population.
assays identified all 126 anti-HTLV-I negative samples as negative producing an estimated specificity of 100% with a 95% confidence interval of 97.0 to 100% with these samples. Serodia identified 123 of these samples as negative, Diagnostic Biotechnology 122, CPI 121 and Coulter 120 samples. The specificity and 6- values for these assays are shown in Table II. The mean and 95% confidence intervals of the A/Co ratios obtained by each assay are in Fig. 1. The Coulter assay produced the largest variation and the Cambridge Biotech assay produced the least variation of A/Co ratios with the anti-HTLV-I negative samples.
Discussion Nine commercially available anti-HTLV-I screening assays were compared in a panel of sera containing presumed anti-HTLV-I status positive and presumed anti-HTLV-I status negative samples. Although the composition of this panel was limited, it was used to make a comparative assessment of performance in the assays particularly with respect to the delta value for each assay. The sensitivity, specificity and delta values were calculated and the information was used to compare these assays according to their performance in the panel. We emphasise that the information presented in our study refers to the performance of these assays on a small panel of serologically characterised sera and should not be extrapolated to estimate in-field assay performance. For example, samples from infected individuals with a range of clinical manifestations or from a range of geographic areas were not used. Furthermore, a number of the presumed anti-HTLV-I negative samples had
89
been freeze thawed. In the present study, the Cambridge Biotech, DuPont, Genetic Systems and Olympus assays produced by the highest sensitivity and specificity. Because the greatest absolute S + and 6 - values were also achieved in these assays, they are likely to produce the best separation of anti-HTLV-I negative and positive populations from the cutoff, Although the Abbott, CPI, Coulter and Diagnostic Biotechnology assays produced comparable levels of sensitivity to the assays cited above, these assays may produce a higher false positive rate because of their low 6- value. The specificity produced by the CPI, Coulter and Diagnostic Biotechnology assays were among the lowest which was reflected in their low 6- values. By contrast, the specificity produced by the Abbott assay was among the highest, but there was a shift of the anti-HTLV-I negative popuIation towards the cutoff as indicated by the intermediate 6value. The variation in the A/Co ratios with the anti-HTLV-I negative samples in the second group of assays was higher than in the first group (Fig. 1). Based on-the use of the delta value, our study identified the assays most likely to have the highest level of sensitivity and specificity (DuPont, Cambridge Biotech, Genetic Systems, Olympus) and assays likely to show lower specificity (Abbott, CPI, Coulter and Diagnostic Biotechnology). The Serodia assay showed comparable levels of sensitivity and specificity to the eight EIAs. Our findings do not always support the findings of other groups. For example, using different populations of samples others have found that the DuPont assay was one of the least specific assays (Khabbaz et al., 1990; Kline et al., 1991) while we found that the DuPont assay correctly identified all 126 anti-HTLV-I negative samples. However, by comparing the 6- values, the DuPont assay is less likely to correctly identify anti-HTLV-I negative samples than the Cambridge, Genetic Systems and Olympus assays because the latter assays had higher 6-- values. In other instances our findings parallel observations in other studies. In the present study taken with those of Cossen et al., Kline et al and Tosswill et al., the Serodia, Cambridge and Genetic Systems assays have higher sensitivities. The differences observed in these various studies may be explained by the sample populations from various geographic regions used for evaluations. Despite the extensive use of anti-HTLV-I screening assays in blood transfusion services and for diagnostic purposes, there are no reports of large scale evaluations of these assays. Therefore, quoted sensitivities and specificities (Khabbaz et al., 1990; Kline et al., 1991; Cossen et al., 1992; Tosswill et al., 1992) are estimates at best. As shown in the present study, confidence limits for the sensitivity and specificity estimates are wide. Therefore, further evaluation of HTLV-I screening assays is required on a larger scale to determine the infield performance of these assays, using suitably characterised sera.
90
Acknowledgements
We thank Mr D.S. Healey for helpful discussions and the Australian Blood Transfusion Services and Fujirebio Inc., Japan for providing the Blood Donor specimens.
References Bastian LB., Hinuma, Y. and Doherty, R.R. (1993) HTLV-I among Northern Territory Aborigines Med. J. Aust. 159, 12-16. Brindle R.J., Eglin R.P., Parons A.J., Barin, F., Gershy-Damet, G., Rey, J.L., Soro. B., Leonard, G,, Mounier, M. and Hugon, J. (1988) HTLV-I, HIV-I hepatitis B and hepatitis delta in the Pacific and South East Asia: a serological survey. Epidemiol. Infect. 100, 1533156. Cossen C.. Hagens S., Fukuchi R., Forghani B., Gallo D. and Ascher M. (1992) A comparison of six commercial human T-cell Lymphotropic virus type I (HTLV-I) enzyme immunoassay kits for detection of antibody to HTLV-I and -II, J. Clin. Microbial. 30, 7244725. Crofts N., Maskill W.J. and Gust I.D. (1988) Evaluation of enzyme-linked immunosorbent assays: a method of data analysis. J. Virol. Methods 22, 51.-59. Guroff R.M., Weiss S.H., Giron J.A., Jennings, A., Ginzburg, H.M., Margolis, LB., Blattner, W.A. and Gallo, R.C. (1986) Prevalence of antibodies to HTLV-I, II and III in intravenous drug abusers from an AIDS endemic region. J. Am. Med. Assoc. 255, 3132-3137. Kalya~ar~man V.S., Sarngadhara M.G., Robert-Guroff M., Miyoshi~ I., Blayney, D., Golde, D. and Gallo, R.C. (1982) A new subtype of human T-cell leukemia virus (HTLV-II) associated with a T-cell variant of hairy cell leukemia, Science 218, 571-573. Khabbaz R.F., Hartley T.M., Lairmore M.D. and Kaplan J.E. (1990) Epidemiologic assessment of screening tests for antibody to human T lymphotropic virus type I (HTLV-I). Am. J. Public Health, 80, 19&192. Kline R.L., Brothers T., Halser N., Boulos, R., Lairmore, M.D. and Quinn T.C. (1991) Evaluation of enzyme immunoassays for antibody to human T-lymphotrophic viruses type I/II. Lancet 337, 3@-33. Lee H., Swanson, P., Shorty V.S., Zack, J.A., Rosenblatt, J.D. and Chen, I.S.Y. (1989) High rate of HTLV-II infection in seropositive IV drug abusers in New Orleans. Science 244, 471474. Lee H.H., Swanson P., Rosenblatt J.D., Chen IS.. Sherwood W.C., Smith D.E., Tegtmeier G.E., Fernando L.P., Fang CT., Osame M. and Kleinman S.H. (1991) Relative prevalence and risk factors of HTLV-I and HTLV-II infection in US blood donors. Lancet 337, 1435.-1439. Maskill W.J., Crofts N., Waldman E., Healey, D.S., Howard, T.S., Silvester, C. and Gust, I.D. (1988) An evaluation of competitive and second generation ELISA screening tests for-antibody to HIV. J. Virol. Methods 22, 61-73. Murphy E.L., Figueroa P.J., Gibbs W.N., Brathwaite, M.B., Holding-Cobham, H., Waters, D., Cranston, B., Hanchard, B. and Blattner, W.A. (1989) Sexual transmission of human T-cell lymphotrophic virus type I (HTLV-I). Ann. Int. Med. 111, 5555560. Nakano S., Ando Y., Saito K., Moriyama, I., Ichijo, M., Toyama, T., Sugamura, K., Imai, J. and Hinuma, Y. (1986) Primary infection of Japanese infants with adult T-cell leukemia-associated retrovirus (ATLV): evidence for viral transmission from mothers to children. J. Infect. 12, 205 212. Okochi K., Sato H. and Hinuma Y. (1984) A retrospective study on transmission of adult T-cell leukemia virus by blood transfusion: seroconversion in recipients, VOX Sanguinis 46, 2455253. Osame M., Matsumoto M., Usuku K., Izumo, S., Ijichi, N., Amitrani, H., Tara, M. and Igata. A. (1987) Chronic progressive myelopathy associated with elevated antibodies to human Tlymphotrophic virus type I and adult T-cell leukemia-like cells. Ann. Neural. 21, 117-122. Rosenblatt J.D., Golde D.W., Wachsman W., Giorgi, J.V., Jacobs, A., Schmidt, G.M., Gasson, J.C.
91 and Chen, I.S.Y. (1986) A second isolate of HTLV-II associated with atypical hairy-cell leukemia. New Engl. J. Med. 315, 372-377. Tosswill, J.H.C., Parry, J.V. and Weber, J.N. (1992) Application of screening and confirmatory assays for anti-HTLV-I/II in U.K. populations. J. Med. Virol. 36, 167-171. Verdier M., Denis F., Sangare A., Hill, A.V.S. and Selkon, J.B. et al. (1989) Prevalence of antibody to human T-cell leukemia virus type I (HTLV-I) in populations of Ivory Coast, West Africa. J. Infect. Dis. 160, 363-370. Yamamoto N. and Hinuma Y. (1985) Viral aetiology of adult T-cell leukemia - a review. J. Gen. Viral. 66, 1641-1660. Yanagihara R., Jenkins C.L., Alexander S.S., Mora, CA. and Garrato, R.M. (1990) Human Tlymphotropic virus type I infection in Papua New Guinea: high prevalence among the Hagahai confirmed by western analysis. J. Infect. Dis. 162, 649-654. Yanagihara R., Ajdukiewicz A.B., Garruto R.M., Sharlow, E.R., Wu, X.Y., Alemaena, O., Sale. H. Alexander, S.S. and Gajdusek, D.C. (1991) Human T-lymphotrop~c virus type I infection in the Solomon Islands. Am. J. Trop. Med. Hygiene 44, 122-130. Yoshida M., Miyoshi I. and Hinuma Y. (1982) Isolation and cha~cterisation of retrovirus (ATLV) from cell lines of human adult T-cell leukemia and its implication in the disease. Proc. Nat]. Acad. Sci. USA 19: 2031-2035.
Journal of Virological Methods
All rights reserved / 0166-0934/93/$06.00 VIRMET 01549
A Comparison of the performance of nine commercially available anti-HTLV-I screening assays Anna Karopoulos*,
Colin Silvester and Elizabeth
National HIV Reference
Laboratory,
Melbourne
M. Dax
(Australia)
(Accepted 6 April 1993)
Summary
The performance of eight anti-HTLV-I enzyme immunoassays (EIAs) and one particle agglutination assay was compared with respect to sensitivity, specificity and delta values, by testing a panel containing 99 anti-HTLV-I positive and 126 anti-HTLV-I negative samples which had been characterised by western blot and some by radioimmunoprecipitation assay. The estimated sensitivities produced by these assays ranged between 99% and 100% and estimated specificities were between 95.2% and 100%. The performance of the EIAs was further differentiated by using the delta value which measures the ability of an assay to separate the positive and negative populations from the cutoff value. A delta value could not be calculated for the particle agglutination assay (Serodia) because the test readings were not quantitative. The EIAs most likely to correctly identify anti-HTLV-I positive and anti-HTLV-I negative samples included the Cambridge Biotech, DuPont, Genetic Systems and Olympus assays. Our findings suggest that there may be some difficulty in correctly identifying anti-HTLV-I negative samples using the Abbott, Cellular Products Incorporated (CPI), Coulter and Diagnostic Biotechnology assays. The Serodia assay produced comparable sensitivity and specificity to the eight EIAs. HTLV-I screening assay; Delta value; Anti-HTLV-I
‘Corresponding author. Address: National HIV Reference Laboratory, Fairfield Hospital, Yarra Bend Road, Fairfield VIC 3078, Australia. Fax: +61 3 482 4352.
84
Introduction Human T-cell lymphotropic virus type I (HTLV-I) has been associated aetiologically with adult T-cell leukaemia/lymphoma and a neurological disease, tropical spastic paraparesis otherwise known as HTLV-I associated myelopathy (Yoshida et al., 1982; Osame et al., 1987). HTLV-I infection has been reported in Japan, the Caribbean, areas of the USA (Yamamoto et al., 1985) and Africa (Verdier et al., 1989). More recently HTLV-I has been reported in remote populations in Papua New Guinea (Yanagihara et al., 1990), regions of the southwestern Pacific (Brindle et al., 1988; Yanagihara et al., 1991) and Australian Aboriginies (Bastian et al., 1993). HTLV-I infection in blood transfusion recipients has been well documented and a seroconversion rate of 63% has been reported in recipients of contaminated whole blood and blood components (Okochi et al., 1984). Transmission of the virus has also been reported from mother to child through are by breast feeding (Guroff et al., 1986). Other modes of transmission intravenous drug users (IVDUs) who share contaminated needles and syringes (Guroff et al., 1986) and by sexual transmission (Murphy et al., 1989). In contrast to HTLV-I, the transmission and disease associations of human T-cell lymphotropic virus type II (HTLV-II) are not well defined. HTLV-II has been isolated from two individuals with hairy cell leukaemia and antibody to the virus has been detected in IVDUs in the United States (Kalyanaraman et al., 1982; Rosenblatt et al., 1986; Lee et al., 1989). The increasing awareness of the prevalence of HTLV has prompted the development of many screening tests to detect antibody to the virus. Some have been compared previously (Kline et al., 1991; Cossen et al., 1992; Tosswill et al., 1992). Three EIAs (Abbott, CPI and DuPont) were licensed in 1988 by the Food and Drug Administration, USA for screening antibody to HTLV-I in serum or plasma. We assessed the performance of 9 commercially available anti-HTLV-I screening assays using a panel of well characterised sera.
Methods Panel samples
Two groups of specimens, totalling 225 samples, formed the panel. The first group consisted of 99 Japanese blood donor samples which were anti-HTLV-I positive by western blot and radioimmunoprecipiatation assay (RIPA). The sources of each of the samples were known, however no clinical data was available. The negative population consisted of 126 Australian blood donor samples that were anti-HTLV-I negative by western blot. Questionnaires and interviews used by Australian blood banks also ensured that the donors were at low risk of infection by blood-borne viruses. 45 of the 126 specimens were non-reactive
85
by the Genetic Systems HIV-1 EIA. This subset of samples had been stored at 4°C and had not been subjected to multiple freeze thaw cycles. The remaining 81 Australian blood donor samples produced false positive reactions in either viral lysate or recombinant based anti-HIV-l EIAs. These samples were collected between 1985 and 1989 and had been freeze thawed more than once. Characterization of the Panel
The panel samples were further characterised at our laboratory using the Diagnostic Biotechnology HTLV-I western biot (Diagnostic Biotechnology Ltd, Singapore) utilising HTLV-I viral lysate antigens and the Diagnostic Biotechnology version 2.2 HTLV-I western blot utilising both HTLV-I viral lysate antigens and recombinant glycoprotein antigens. Western blot indeterminate samples were tested by RIPA using HTLV-I antigen prepared from the HUT102-B2 cell line (Abbott Laboratories, Chicago, USA) (Lee et al., 1991). Western blot and RIPA results were interpreted according to strict criteria. For a sample to be designated anti-HTLV-I positive, the sample had to demonstrate reactivity to the gag protein ~24, two additional viral proteins (~19, ~26, ~28, ~32, ~36, ~53) and at least one env protein (gp46 or gp68) or two recombinant e?zv products (Rgp46 and Rgp21) by a combination of western blot and RIPA. Samples with no reactivity to viral bands were considered negative for antibodies to HTLV-I. Results of these tests confirmed the original anti-HTLV-I status of the panel samples. Assays
The 9 HTLV-I/II screening assays that underwent analysis are outlined in Table I. Analysis
Assay procedures were performed according to the respective kit inserts and only test runs that were valid according to the manufacturers’ instructions were included in the comparison. Absorbance to cutoff (A/Co) ratios were calculated for data obtained in the form of absorbance readings, so that specimens with A/Co ratios greater than or equal to one were considered reactive for antibodies to HTLV-I. The sensitivity, or the ability of an assay to detect anti-HTLV-I in a presumed anti-HTLV-I positive sample, was determined by testing the 99 seropositive Japanese samples. An estimate of sensitivity in any given assay was made by determining the percentage of specimens found reactive compared with the total number of presumed anti-HTLV-I positive specimens tested. The specificity was determined by the ability of an assay to identify a presumed anti-HTLV-I negative sample as negative and was expressed as the percentage
86 TABLE I Anti-HTLV-I
Screening Assays Antigen
Test kit name
Manufacturer
Abbott HTLV-I EIA
Abbott Laboratories,
Detect HTLV-I/II
Manufactured by IAF BioChem for Coulter Corp., FL, USA
synthetic peptide
Diagnostic Biotechnology HTLV-I ELISA
Diagnostic
viral lysate
DuPont HTLV-I ELISA’
DuPont, DE, USA
viral lysate (HUT-102B2)
Genetic Systems HTLV-I EIA
Genetic Systems Corp., WA, USA
Recombinant
Cambridge
EIA
viral lysate (HUT-102B2)
Chicago, USA
Biotechnology
Ltd., Singapore
Retrotek HTLV-I ELISA’
Cellular Products Inc., NY, USA
viral lysate (HB,) recombinant gene (gp21) viral lysate
Serodia HTLV-I particle agglutination assay
Fujirebio Inc., Tokyo, Japan
viral lysate
SynthEIA
Manufacturered by United Biomedical Inc. for Olympus Corp., NT, USA
viral lysate and synthetic peptide
HTLV-I/II
EIA
HTLV-I ELISA
Biotech Corp., MA, USA
env
*The format of the DuPont and CPI assays have been modified since this comparison was performed. Therefore the results presented in this study regard the performance of the pre-modified assays.
of the total number of specimens found negative compared with the total number tested. Delta values were also calculated for the presumed anti-HTLV-I status positive and negative populations. Calculating delta values is useful to differentiate performance of assays with similar levels of sensitivity and specificity and to give an indication of the expected level of performance of an assay. Calculation and full description of delta has been reported previously (Crofts et al., 1988; Maskill et al., 1988). Briefly, the delta value measures the ability of an assay to separate the positive and negative population from the cutoff and is calculated by dividing the mean loglo A/Co ratio by the standard deviation of each population. Therefore an assay with the highest absolute delta positive (6 +) and delta negative (6 -) value would be the test with the highest probability of separating positive and negative populations.
Results
Anti-HTLV-I
positive population
The sensitivity produced
by these assays was calculated
for the positive panel
87
TABLE II Sensitivity, specificity and delta values produced by the HTLV screening assays Screening assay
Sensitivity (%)
95% CI (%)
S+
Diagnostic Biotechnology DuPont CPI Cambridge Biotech. Abbott Genetic Systems Coulter Olympus Serodia
100.0 100.0 100.0 99.0 99.0 100.0 99.0 100.0 100.0
96.2-100.0 96.2-100.0 96.2-100.0 94.4100.0 94.4100.0 96.2-100.0 94.4100.0 96.2-100.0 96.2-100.0
9.57 7.05 6.28 4.05 3.96 2.98 2.79 2.67 N/A
Cambridge Biotech Genetic Systems Olympus DuPont Abbott CPI Diagnostic Biotechnology Coulter Serodia
100.0 100.0 100.0 100.0 100.0 96.0 96.8 95.2* 97.6
97.0-100.0 97.&100.0 97.&100.0 97.&100.0 97.&100.0 91.g 98.3 92.0- 98.8 89.9- 97.8 93.2- 99.2
S4.85 4.23 2.62 2.29 1.77 1.44 1.43 1.30 N/A
N/A: Not-Applicable CI: Confidence Interval l Specificity and 95% CI calculated according to the initial reactor rate, because there were insufficient tests to complete testing in duplicate. Therefore the specificity calculated for this assay may include technical error.
samples. Six assays, CPI, Diagnostic Biotechnology, DuPont, Genetic Systems, Olympus and Serodia correctly identified all 99 anti-HTLV-I positive samples yielding an estimated sensitivity of 100% with a 95% confidence interval of 96.2 to 100%. The remaining assays Abbott, Cambridge Biotech and Coulter, repeatably found 1 anti-HTLV-I positive sample negative thus producing an estimated sensitivity of 99% with a 95% confidence interval of 94.4 to 100% (Table II). The Abbott and Coulter assays incorrectly identified the same seropositive sample while the Cambridge assay failed to detect a different seropositive panel member. The 6 + values for these assays are also shown in Table II. The 6 + values ranged from 9.57 to 2.67. A delta value could not be calculated for the Serodia assay because test readings are not quantitative. The mean and 95% confidence interval of the A/Co ratio produced by the EIAs with the anti-HTLV-I positive samples are shown in Fig. 1. The Coulter and Olympus assays produced the greatest variation of A/Co ratios and the Diagnostic Biotechnology assay showed the least variation with these samples. Anti-HTLV-I The Abbott,
negative population Cambridge
Biotech,
DuPont,
Genetic
Systems and Olympus
88
10 9
a
I
T
1
2-1
0’1,
T
T
I
~
/ ~
i
I
I
PN Abbott
I,,
I
PN CB
/
PN CPI
I
I,
PN Coulter
Screening
I
I,,‘,
PN DB
I
PN DuPont
PN OS
PN Olympus
assays
Fig. 1. Mean and 95% confidence interval of absorbance/cutoff ratio produced by the enzyme immunoassays when the anti-HLTV-I positive and anti-HTLV-I negative panels were tested. CB, Cambridge Biotech; CPI, Cellular Products Incorporated; DB, Diagnostic Biotechnology; GS, Genetic Systems; P, anti-HTLV-I positive population; N, anti-HTLV-I negative population.
assays identified all 126 anti-HTLV-I negative samples as negative producing an estimated specificity of 100% with a 95% confidence interval of 97.0 to 100% with these samples. Serodia identified 123 of these samples as negative, Diagnostic Biotechnology 122, CPI 121 and Coulter 120 samples. The specificity and 6- values for these assays are shown in Table II. The mean and 95% confidence intervals of the A/Co ratios obtained by each assay are in Fig. 1. The Coulter assay produced the largest variation and the Cambridge Biotech assay produced the least variation of A/Co ratios with the anti-HTLV-I negative samples.
Discussion Nine commercially available anti-HTLV-I screening assays were compared in a panel of sera containing presumed anti-HTLV-I status positive and presumed anti-HTLV-I status negative samples. Although the composition of this panel was limited, it was used to make a comparative assessment of performance in the assays particularly with respect to the delta value for each assay. The sensitivity, specificity and delta values were calculated and the information was used to compare these assays according to their performance in the panel. We emphasise that the information presented in our study refers to the performance of these assays on a small panel of serologically characterised sera and should not be extrapolated to estimate in-field assay performance. For example, samples from infected individuals with a range of clinical manifestations or from a range of geographic areas were not used. Furthermore, a number of the presumed anti-HTLV-I negative samples had
89
been freeze thawed. In the present study, the Cambridge Biotech, DuPont, Genetic Systems and Olympus assays produced by the highest sensitivity and specificity. Because the greatest absolute S + and 6 - values were also achieved in these assays, they are likely to produce the best separation of anti-HTLV-I negative and positive populations from the cutoff, Although the Abbott, CPI, Coulter and Diagnostic Biotechnology assays produced comparable levels of sensitivity to the assays cited above, these assays may produce a higher false positive rate because of their low 6- value. The specificity produced by the CPI, Coulter and Diagnostic Biotechnology assays were among the lowest which was reflected in their low 6- values. By contrast, the specificity produced by the Abbott assay was among the highest, but there was a shift of the anti-HTLV-I negative popuIation towards the cutoff as indicated by the intermediate 6value. The variation in the A/Co ratios with the anti-HTLV-I negative samples in the second group of assays was higher than in the first group (Fig. 1). Based on-the use of the delta value, our study identified the assays most likely to have the highest level of sensitivity and specificity (DuPont, Cambridge Biotech, Genetic Systems, Olympus) and assays likely to show lower specificity (Abbott, CPI, Coulter and Diagnostic Biotechnology). The Serodia assay showed comparable levels of sensitivity and specificity to the eight EIAs. Our findings do not always support the findings of other groups. For example, using different populations of samples others have found that the DuPont assay was one of the least specific assays (Khabbaz et al., 1990; Kline et al., 1991) while we found that the DuPont assay correctly identified all 126 anti-HTLV-I negative samples. However, by comparing the 6- values, the DuPont assay is less likely to correctly identify anti-HTLV-I negative samples than the Cambridge, Genetic Systems and Olympus assays because the latter assays had higher 6-- values. In other instances our findings parallel observations in other studies. In the present study taken with those of Cossen et al., Kline et al and Tosswill et al., the Serodia, Cambridge and Genetic Systems assays have higher sensitivities. The differences observed in these various studies may be explained by the sample populations from various geographic regions used for evaluations. Despite the extensive use of anti-HTLV-I screening assays in blood transfusion services and for diagnostic purposes, there are no reports of large scale evaluations of these assays. Therefore, quoted sensitivities and specificities (Khabbaz et al., 1990; Kline et al., 1991; Cossen et al., 1992; Tosswill et al., 1992) are estimates at best. As shown in the present study, confidence limits for the sensitivity and specificity estimates are wide. Therefore, further evaluation of HTLV-I screening assays is required on a larger scale to determine the infield performance of these assays, using suitably characterised sera.
90
Acknowledgements
We thank Mr D.S. Healey for helpful discussions and the Australian Blood Transfusion Services and Fujirebio Inc., Japan for providing the Blood Donor specimens.
References Bastian LB., Hinuma, Y. and Doherty, R.R. (1993) HTLV-I among Northern Territory Aborigines Med. J. Aust. 159, 12-16. Brindle R.J., Eglin R.P., Parons A.J., Barin, F., Gershy-Damet, G., Rey, J.L., Soro. B., Leonard, G,, Mounier, M. and Hugon, J. (1988) HTLV-I, HIV-I hepatitis B and hepatitis delta in the Pacific and South East Asia: a serological survey. Epidemiol. Infect. 100, 1533156. Cossen C.. Hagens S., Fukuchi R., Forghani B., Gallo D. and Ascher M. (1992) A comparison of six commercial human T-cell Lymphotropic virus type I (HTLV-I) enzyme immunoassay kits for detection of antibody to HTLV-I and -II, J. Clin. Microbial. 30, 7244725. Crofts N., Maskill W.J. and Gust I.D. (1988) Evaluation of enzyme-linked immunosorbent assays: a method of data analysis. J. Virol. Methods 22, 51.-59. Guroff R.M., Weiss S.H., Giron J.A., Jennings, A., Ginzburg, H.M., Margolis, LB., Blattner, W.A. and Gallo, R.C. (1986) Prevalence of antibodies to HTLV-I, II and III in intravenous drug abusers from an AIDS endemic region. J. Am. Med. Assoc. 255, 3132-3137. Kalya~ar~man V.S., Sarngadhara M.G., Robert-Guroff M., Miyoshi~ I., Blayney, D., Golde, D. and Gallo, R.C. (1982) A new subtype of human T-cell leukemia virus (HTLV-II) associated with a T-cell variant of hairy cell leukemia, Science 218, 571-573. Khabbaz R.F., Hartley T.M., Lairmore M.D. and Kaplan J.E. (1990) Epidemiologic assessment of screening tests for antibody to human T lymphotropic virus type I (HTLV-I). Am. J. Public Health, 80, 19&192. Kline R.L., Brothers T., Halser N., Boulos, R., Lairmore, M.D. and Quinn T.C. (1991) Evaluation of enzyme immunoassays for antibody to human T-lymphotrophic viruses type I/II. Lancet 337, 3@-33. Lee H., Swanson, P., Shorty V.S., Zack, J.A., Rosenblatt, J.D. and Chen, I.S.Y. (1989) High rate of HTLV-II infection in seropositive IV drug abusers in New Orleans. Science 244, 471474. Lee H.H., Swanson P., Rosenblatt J.D., Chen IS.. Sherwood W.C., Smith D.E., Tegtmeier G.E., Fernando L.P., Fang CT., Osame M. and Kleinman S.H. (1991) Relative prevalence and risk factors of HTLV-I and HTLV-II infection in US blood donors. Lancet 337, 1435.-1439. Maskill W.J., Crofts N., Waldman E., Healey, D.S., Howard, T.S., Silvester, C. and Gust, I.D. (1988) An evaluation of competitive and second generation ELISA screening tests for-antibody to HIV. J. Virol. Methods 22, 61-73. Murphy E.L., Figueroa P.J., Gibbs W.N., Brathwaite, M.B., Holding-Cobham, H., Waters, D., Cranston, B., Hanchard, B. and Blattner, W.A. (1989) Sexual transmission of human T-cell lymphotrophic virus type I (HTLV-I). Ann. Int. Med. 111, 5555560. Nakano S., Ando Y., Saito K., Moriyama, I., Ichijo, M., Toyama, T., Sugamura, K., Imai, J. and Hinuma, Y. (1986) Primary infection of Japanese infants with adult T-cell leukemia-associated retrovirus (ATLV): evidence for viral transmission from mothers to children. J. Infect. 12, 205 212. Okochi K., Sato H. and Hinuma Y. (1984) A retrospective study on transmission of adult T-cell leukemia virus by blood transfusion: seroconversion in recipients, VOX Sanguinis 46, 2455253. Osame M., Matsumoto M., Usuku K., Izumo, S., Ijichi, N., Amitrani, H., Tara, M. and Igata. A. (1987) Chronic progressive myelopathy associated with elevated antibodies to human Tlymphotrophic virus type I and adult T-cell leukemia-like cells. Ann. Neural. 21, 117-122. Rosenblatt J.D., Golde D.W., Wachsman W., Giorgi, J.V., Jacobs, A., Schmidt, G.M., Gasson, J.C.
91 and Chen, I.S.Y. (1986) A second isolate of HTLV-II associated with atypical hairy-cell leukemia. New Engl. J. Med. 315, 372-377. Tosswill, J.H.C., Parry, J.V. and Weber, J.N. (1992) Application of screening and confirmatory assays for anti-HTLV-I/II in U.K. populations. J. Med. Virol. 36, 167-171. Verdier M., Denis F., Sangare A., Hill, A.V.S. and Selkon, J.B. et al. (1989) Prevalence of antibody to human T-cell leukemia virus type I (HTLV-I) in populations of Ivory Coast, West Africa. J. Infect. Dis. 160, 363-370. Yamamoto N. and Hinuma Y. (1985) Viral aetiology of adult T-cell leukemia - a review. J. Gen. Viral. 66, 1641-1660. Yanagihara R., Jenkins C.L., Alexander S.S., Mora, CA. and Garrato, R.M. (1990) Human Tlymphotropic virus type I infection in Papua New Guinea: high prevalence among the Hagahai confirmed by western analysis. J. Infect. Dis. 162, 649-654. Yanagihara R., Ajdukiewicz A.B., Garruto R.M., Sharlow, E.R., Wu, X.Y., Alemaena, O., Sale. H. Alexander, S.S. and Gajdusek, D.C. (1991) Human T-lymphotrop~c virus type I infection in the Solomon Islands. Am. J. Trop. Med. Hygiene 44, 122-130. Yoshida M., Miyoshi I. and Hinuma Y. (1982) Isolation and cha~cterisation of retrovirus (ATLV) from cell lines of human adult T-cell leukemia and its implication in the disease. Proc. Nat]. Acad. Sci. USA 19: 2031-2035.