- Email: [email protected]
Evaluation of a new West Nile Virus lateral-flow rapid IgM assay
Journal of Virological Methods 157 (2009) 223–226
Contents lists available at ScienceDirect
Journal of Virological Methods journal homepage: www.elsevier.com/locate/jviromet
Short communication
Evaluation of a new West Nile Virus lateral-flow rapid IgM assay Anthony R. Sambol ∗ , Steven H. Hinrichs Nebraska Public Health Laboratory, Department of Pathology and Microbiology, University of Nebraska Medical Center, Omaha, NE, United States
a b s t r a c t Article history: Received 26 October 2008 Received in revised form 11 December 2008 Accepted 15 December 2008 Available online 10 January 2009 Keywords: West Nile Virus Immunoglobulin M Diagnostic tool Enzyme-linked immunoassay
This study evaluated the performance of a new Food and Drug Administration-approved lateral-flow diagnostic screening test for qualitative detection of West Nile Virus (WNV) immunoglobulin M (IgM) in serum or plasma. Five public health laboratories across the United States performed retrospective testing on blinded serum samples from patients with physician reported diagnoses of WNV infection. The results of the RapidWNTM WNV IgM assay were compared with two commercially available WNV IgM enzyme-linked immunosorbent assays (EIA) and two public health-developed WNV-IgM tests. After discrepancies were resolved, the RapidWNTM WNV IgM EIA demonstrated a 98.8% sensitivity (range: 96.0–100%), a 95.3% specificity (range: 90.9–100%), a positive predictive value of 96.3% (range: 94.7–100%), and a negative predictive value of 98.4% (range: 95.5–100%), as compared to the predicate assays. The study results suggest that the RapidWNTM WNV IgM EIA is an effective, qualitative screening test that produces results comparable to that of predicate assays and can be employed rapidly to detect WNV IgM in patients suspected of having WNV infection. Published by Elsevier B.V.
1. Introduction Since its onset in 1999, West Nile Virus (WNV) has spread throughout North America and has become a serious public health concern in the United States (US) and Canada. According to the Centers for Disease Control and Prevention, from 1999 through 2008 there have been over 28,900 cases of confirmed symptomatic WNV infections in the US with over 1000 fatalities. As it is desirable to recognize infection early to initiate available therapeutic supportive measures and to rule-out other agents that cause neurological disease, detection of WNV-specific antibodies has become a valuable tool. Since a growing population may have had previous exposure and have pre-existing IgG antibodies, immunoglobulin M (IgM) antibody in serum and cerebral spinal fluid (CSF) specimens is the preferred marker for WNV infection. At the time of this study, two public health-developed tests and three commercial laboratory tests marketed by Focus Diagnostics, PanBio, and InBios were available in the US for initial screening of WNV IgM in serum specimens; however, the PanBio test kit is no longer available in the US. Specificity and sensitivity of these predicate assays have been analyzed against the plaque-reduction neutralization test (PRNT) (InBios International, Inc. Package Insert, 2009; Johnson
∗ Corresponding author at: Nebraska Public Health Laboratory, University of Nebraska Medical Center, 986495 Nebraska Medical Center, Omaha, NE 68198-6495, United States. Tel.: +1 402 559 3032; fax: +1 402 559 7799. E-mail addresses: [email protected] (A.R. Sambol), [email protected] (S.H. Hinrichs). 0166-0934/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.jviromet.2008.12.011
et al., 2007; Martin et al., 2004; Prince and Hogrefe, 2003a; Tilley et al., 2005). There have been many well-documented studies showing the utility of these screening tests (Malan et al., 2004; Niedrig et al., 2007a,b; Prince and Hogrefe, 2003b; Prince et al., 2004; Tardei et al., 2000; Sambol et al., 2007); however, these are multistep enzyme-linked immunosorbent assays (EIA) that require from several hours to overnight for completion and often employ expensive instrumentation. Both private and public health laboratories would benefit from the availability of a rapid, sensitive screening assay for WNV IgM that could be used in situations where an urgent request for testing services was needed for samples that were insufficient in number to warrant automated, high throughput testing. The primary goal of this study was to compare the relative sensitivity and specificity of a new Food and Drug Administrationapproved lateral-flow assay with commercially available and public health laboratory-developed screening tests. In addition to the Nebraska Public Health Laboratory (NPHL), four additional public health laboratories (PHL) participated in this study including: the Texas Department of State Health Services, the North Dakota Department of Health, the Illinois Department of Health, and the University of Iowa Hygienic Laboratory. Each PHL performed testing with the RapidWNTM WNV IgM EIA (Spectral Diagnostics, Toronto, Ont., Canada) and their predicate screening assay as follows: • WNV IgM Capture DxSelectTM (Focus Diagnostics, Cypress, CA, USA); • WNV IgM Capture ELISA (PanBio, Qld., Australia);
224
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
Table 1 Initial testing results of the RapidWN IgM EIA compared to predicate IgM assays. PHL site
Predicate kit
Rapid WNa
No. tested
Sensitivity
1 and 5 2 and 3 2 and 3 4
WNV IgM Capture DxSelect MIA MAC-EIA WNV IgM Capture ELISA
All sites
Specificity
Predictive value Positive
Negative
49 31 44 25
88.9% (24/27) 90.0% (18/20) 95.7% (22/23) 100% (15/15)
95.5% (21/22) 90.9% (10/11) 95.2% (20/21) 100% (10/10)
96.0% (24/25) 94.7% (18/19) 95.6% (22/23) 100% (15/15)
87.5% (21/24) 83.3% (10/12) 95.2% (20/21) 100% (10/10)
149
92.9% (79/85)
95.3% (61/64)
96.3% (79/82)
91.0% (61/67)
Abbreviations: RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); WNV IgM Capture ELISA = West Nile Virus Immunoglobulin M Capture Enzyme-linked Immunosorbent Assay (PanBio). a The numbers in parentheses show number of sample agreement divided by total sample in each group.
• WNV-specific IgM antibody-capture (MAC) [Centers for Disease Control and Prevention (CDC)]; • WNV-specific IgM micro-immunoassay (MIA) (CDC). Each of the 5 PHLs tested 10 negative and 15 positive serum specimens, including low, mid and high level positives, which were available from their archives. Specimens were tested in a blinded manner. With the exception of one laboratory where both the MAC-EIA and the MIA served as predicate assays for the RapidWNTM IgM EIA and were performed on the same set of specimens, and another laboratory where both of these predicate tests were in use but against different sets of specimens, all other laboratories tested the RapidWNTM IgM EIA against one predicate test. Overall, 124 serum specimens were tested, comparing 74 positives and 50 negatives by known methods. Because ease-of-use and interpretation of results were parameters for this study, no advanced training was provided to the clinical laboratory scientists performing the RapidWNTM IgM EIA. The commercially prepared predicate EIAs were performed according to manufacturers’ instructions using the provided internal controls and calibrators (Focus Diagnostics Package Insert, 2009; PanBio Package Insert, 2009). No PHLs in this pilot study used the InBios EIA test. The MAC-EIA was performed using the testing protocols developed originally by the CDC (Martin et al., 2000, 2004), and validated by the individual PHL. For the MIA, testing was performed as described by Johnson et al. (2007). The results from the MIA were reported as being positive or negative for WNV or St. Louis encephalitis virus (SLE) using quadratic dis-
criminate analysis and specimens were considered positive if their WNV-adjusted (WNV Adj) mean fluorescent intensity (MFI) was greater than 3. In the RapidWNTM IgM EIA, serum specimens were diluted 1:30 in Sample Diluent warmed to room temperature (15–25 ◦ C). The diluted samples were vortexed and a 150 L aliquot dispensed into the device sample well. If present, WNV IgM antibody in the diluted specimen was captured and formed a complex with the recombinant WNV antigen and secondary mouse flavivirus-specific monoclonal antibody. Migration of the complex through the reaction strip was detected by an immobilizing reagent resulting in a visible pink–purple horizontal band in the test area (Shaikh et al., 2007; Spectral Diagnostics’ Rapid WNTM Kit Insert, 2009). For the purpose of this study, each predicate assay was used as the reference standard to determine the performance characteristics of the RapidWNTM IgM EIA. Specimens that gave discordant results were retested with the RapidWNTM IgM EIA. If results remained discordant, retesting with the predicate device was performed. Specimens that were discrepant repeatedly were blinded and sent to an outside PHL for additional testing by traditional IgG-depleted indirect immuno-fluorescent antibody (IFA) as discussed by Shi and Wong (2003), and an experimental IgM-Western blot (IgM-WB) (Oceguera et al., 2007). PRNT was not performed on samples with discrepant results because this assay measures total antibody and not the presence of IgM alone (Oceguera et al., 2007). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were determined for all test methods. In 4 instances, specimens that tested positive initially with the predicate assay did not repeat as positive and were counted as neg-
Table 2 Comparison WNV-IgM testing of discrepant specimens by RapidWN IgM EIA, predicate assays, IgM-Western blot, and indirect fluorescent antibody-IgM. PHL site
IgM test used
Predicate cut-off value
Predicate resulta
Rapid WN result
WB
IFAb
Testing consensus
Rapid WN performance
1 1 1 1 2 2 3 2 2
WNV IgM Capture DxSelect WNV IgM Capture DxSelect WNV IgM Capture DxSelect WNV IgM Capture DxSelect MIA MIA MIA MAC-EIA MAC-EIA
1.1 1.1 1.1 1.1 3.0 3.0 3.0 3.0 3.0
0.53; neg 1.20; pos 1.99; pos 3.47; pos 2.75; neg 5.70; pos 32.4; pos 1.8; neg 5.0; pos
Pos Neg Neg Neg Pos Neg Neg Pos Neg
Neg Neg Neg Pos Neg Neg Neg Neg Neg
<1:10; neg 1:10; pos 1:10; pos 1:20; pos <1:10; neg 1:40; pos 1:160; pos <1:10; neg 1:40; pos
Neg None None Pos Neg None None Neg None
FPc
FNd FP
FP
Abbreviations: WNV IgM = West Nile Virus Immunoglobulin M; RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); Pos = positive; Neg = negative; FP = false positive; FN = false negative. a Expressed as Index Values. b Expressed as dilution titers. c False positive as determined by consensus tests. d False negative as determined by consensus tests.
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
225
Table 3 Revised testing results of Spectral RapidWN compared to predicate assays. PHL site
Predicate kit
No. of tested
Resolved discrepants
Rapid WNa Sensitivity
1 and 5 2 and 3 2 and 3 4 All sites
WNV IgM Capture DxSelect MIA MAC-EIA WNV IgM Capture ELISA
Specificity
Predictive value Positive
Negative
47 29 43 25
1 FN; 1 FP 1 FP 1 FP None
96.0% (24/25) 100% (18/18) 100% (22/22) 100% (15/15)
95.5% (21/22) 90.9% (10/11) 95.2% (20/21) 100% (10/10)
96.0% (24/25) 94.7% (18/19) 95.5% (22/23) 100% (15/15)
95.5% (21/22) 100% (10/10) 100% (20/20) 100% (10/10)
144
1 FN; 3 FP
98.8% (79/80)
95.3% (61/64)
96.3% (79/82)
98.4% (61/62)
Abbreviations: RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); WNV IgM Capture ELISA = West Nile Virus Immunoglobulin M Capture Enzyme-linked Immunosorbent Assay (PanBio); FP = false positive; FN = false negative. a The numbers in parentheses show number of sample agreement divided by total sample in each group.
ative. As shown in Table 1, the RapidWNTM IgM EIA demonstrated a 92.9% sensitivity (range of 88.9–100%), a 95.3% specificity (range of 90.9–100%), a PPV of 96.3% (range of 94.7–100%), and a NPV of 91.0% (range of 83.3–100%) as compared to the predicate assays. The results of testing performed on 9 discrepant serum specimens, and confirmation testing by WB and IFA, are shown in Table 2. Of the 9 specimens, 3 were determined to be false positive (FP), 1 specimen was determined to be a false negative (FN), and there were 5 specimens for which no consensus test results could be reached. These 5 specimens were subsequently excluded from the final data. Of the specimens examined in the comparison with WNV IgM Capture DxSelect, 1 was determined to be a RapidWNTM IgM EIA FP with an Index Value of 0.53 and 1 a RapidWNTM IgM EIA FN, with a fairly high Index Value of 3.47. Three specimens examined in the comparison of RapidWNTM IgM EIA with the MIA gave discrepant test results. Of these, 1 specimen was determined to be a FP by both RapidWNTM IgM EIA and MIA, with a WN Adj MFI of 2.75. The 2 specimens for which no consensus results were found had a MIA WN Adj of 5.70 and 32.4, negative findings by RapidWNTM IgM EIA, negative findings by WB, and positive findings by IFA. Of the specimens examined in the comparison with the MAC-EIA, 1 specimen that gave a RapidWNTM IgM EIA FP compared to a MAC-EIA negative Index Value of 1.8. The other discrepant specimen examined in this comparison was considered to be a no consensus sample as it was positive by MAC-EIA (Index Value of 5.0) and IFA, but negative by RapidWNTM IgM EIA and WB. Table 3 shows the results after analysis of discrepant serum specimens. When the 5 specimens without consensus were removed from the analysis, the RapidWNTM IgM EIA showed an increase in sensitivity from 92.9% to 98.8%, specificity remained the same at 95.3%, the PPV remained constant at 96.3%, and the NPV increased from 91.0% to 98.4%. It is well recognized that with EIA and MIA screening formats cross-reactions do occur with a variety of other bacterial and viral agents, as well as anti-nuclear antibodies and Rheumatoid factor. The reference standard for specific confirmation of WNV or any flavivirus disease is the PRNT. However, because the PRNT tests for the presence of both neutralizing IgM and IgG, it was not used in this study to resolve discrepancies. Instead, traditional IgG-depletion IFA testing as well as an experimental IgM-WB was performed on the discrepant sera. Of the 9 serum specimens with discrepant results in the initial testing, no consensus could be reached in 5 serum samples and these samples were excluded from the final analysis of the data. As shown in Table 2, the results for these 5 specimens were near the limits of sensitivity for the predicate screening tests. Future studies will seek to obtain follow-up specimens or convalescent serum from patients or to examine these specimens for other cross-reactive factors know to cause false results.
A single use diagnostic screening device for the detection of WNV would hold the potential for use in a variety of settings. This study was conducted at 5 PHL on 124 different serum specimens obtained throughout North America from patients with physicianreported diagnoses of WNV. One limitation of this study was the number of specimens tested against multiple reference assays; however, this study provided the opportunity to examine performance at multiple sites. Overall, the sensitivity and specificity of RapidWNTM IgM EIA was comparable to predicate screening tests that are currently in use. In cases where results of screening tests may be questionable, definitive clinical diagnosis should include patient symptomology and the use of confirmatory tests such as the PRNT. In summary, this study demonstrated that the RapidWNTM IgM EIA: (1) performed comparably with commercially available and public health-developed assays; (2) offered several advantages as tests can be performed rapidly without automated equipment, tedious calculations, or special handling of the EIA-based device reagents; (3) is suitable for use in a situation where a small number of WNV IgM tests are necessary. Acknowledgments The authors would like to thank the directors and public health laboratories that assisted with this study: Dr. Susan Neill, Texas Department of State Health Services; Dr. Myra Kosse, North Dakota Department of Health; Dr. Bernard Johnson, Illinois Department of Health; and Dr. Chris Atchison, University of Iowa Hygienic Laboratory. Spectral Diagnostics provided the RapidWNTM WNV IgM assay kits at no charge and all remaining costs were born by the performing sites. No financial compensation was provided to the investigators. References Centers for Disease Control and Prevention (CDC), 2008. Division of Vector Borne Infectious Diseases (DVBID), www.\cdc.gov/ncidod/dvbid/westnile. Focus Diagnostics Package Insert, 2009. InBios International, Inc. Package Insert, 2009. Johnson, A.J., Cheshier, R.C., Cosentino, G., Masri, H., Mock, V., Oesterle, R., Lanciotti, R.S., Martin, D.A., Panella, A., Kosoy, O., Biggerstaff, B., 2007. Validation of a microsphere-based immunoassay for the detection of anti-West Nile virus and anti-St. Louis encephalitis virus immunoglobulin M antibodies. Clin. Vac. Immunol. 14, 1084–1093. Malan, A.K., Martins, T.B., Hill, H.R., Litwin, C.M., 2004. Evaluations of commercial West Nile virus immunoglobulin G (IgG) and IgM enzyme immunoassays show the value of continuous validation. J. Clin. Microbiol. 42, 727–733. Martin, D.A., Muth, D.A., Brown, T., Johnson, A.J., Karabatsos, N., Roehrig, J.T., 2000. Standardization of immunoglobulin M capture enzyme-linked immunosorbent assays for routine diagnosis of arboviral infections. J. Clin. Microbiol. 38, 1823–1826.
226
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
Martin, D.A., Noga, A., Kosoy, O., Johnson, A.J., Petersen, L.R., Lanciotti, R.S., 2004. Evaluation of a diagnostic algorithm using immunoglobulin M enzyme-linked immunosorbent assay to differentiate human West Nile Virus and St. Louis Encephalitis virus infections during the 2002 West Nile Virus epidemic in the United States. Clin. Diagn. Lab. Immunol. 11, 1130–1133. Niedrig, M., Donoso-Mantke, O., Altmann, D., Zeller, H., 2007a. First international diagnostic accuracy study for the serological detection of West Nile virus infection. BMC Infect. Dis. 7, 72. Niedrig, M., Sonnenberg, K., Steinhagen, K., Paweska, J.T., 2007b. Comparison of ELISA and immunoassays for measurement of IgG and IgM antibody to West Nile virus in human sera against virus neutralization. J. Virol. Methods 139, 103–105. Oceguera, L.F., Patiris, P.J., Chiles, R.E., Busch, M.P., Tobler, L.H., Hanson, C.V., 2007. Short report: flavivirus serology by western blot analysis. Am. J. Trop. Med. Hyg. 77, 159–163. PanBio Package Insert, 2009. Prince, H.E., Hogrefe, W.R., 2003a. Detection of West Nile virus (WNV)-specific immunoglobulin M in a reference laboratory setting during the 2002 WNV season in the United States. Clin. Diagn. Lab. Immunol. 10, 764–768. Prince, H.E., Hogrefe, W.R., 2003b. Performance characteristics of an in-house assay system used to detect West Nile Virus (WNV)-specific immunoglobulin M (IgM) during the 2001 WNV season in the United States. Clin. Diagn. Lab. Immunol. 10, 177–179.
Prince, H.E., Lape’-Nixon, M., Moore, R.J., Hogrefe, W.R., 2004. Utility of the Focus Technologies West Nile virus immunoglobulin M capture enzyme-linked immunosorbent assay for testing cerebrospinal fluid. J. Clin. Microbiol. 42, 12–15. Sambol, A.R., Hinrichs, S.H., Hogrefe, W.R., Schweitzer, B.K., 2007. Performance of a commercial immunoglobulin M antibody capture assay using analyte-specific reagents to screen for interfering factors during a West Nile virus epidemic season in Nebraska. Clin. Vac. Immunol. 14, 87–89. Shaikh, N.A., Ge, J., Zhao, Y.X., Walker, P., Drebot, M., 2007. Development of a novel, rapid, and sensitive immunochromatographic strip assay for West Nile virus (WNV) IgM and testing of its diagnostic accuracy in patients suspected of WNV infection. Clin. Chem. 53, 2031–2034. Spectral Diagnostics Rapid WNTM Kit Insert, 2009. Shi, P.Y., Wong, S.J., 2003. Serologic diagnosis of West Nile virus infection. Expert Rev Mol Diagn 3, 733–741. Tilley, P.A., Walle, R., Chow, A., Jayaraman, G.C., Fonseca, K., Drebot, M.A., Priksaitis, J., Fox, J., 2005. Clinical Utility of Commercial Enzyme Immunoassays during the Inaugural Season of West Nile Virus Activity, Alberta, Canada. J. Clin. Microbiol. 43, 4691–4695. Tardei, G., Ruta, S., Chitu, V., Rossi, C., Tsai, T.F., Cernescu, C., 2000. Evaluation of immunoglobulin M (IgM) and IgG enzyme immunoassays in serologic diagnosis of West Nile Virus infection. J. Clin. Microbiol. 38, 2232–2239.
Contents lists available at ScienceDirect
Journal of Virological Methods journal homepage: www.elsevier.com/locate/jviromet
Short communication
Evaluation of a new West Nile Virus lateral-flow rapid IgM assay Anthony R. Sambol ∗ , Steven H. Hinrichs Nebraska Public Health Laboratory, Department of Pathology and Microbiology, University of Nebraska Medical Center, Omaha, NE, United States
a b s t r a c t Article history: Received 26 October 2008 Received in revised form 11 December 2008 Accepted 15 December 2008 Available online 10 January 2009 Keywords: West Nile Virus Immunoglobulin M Diagnostic tool Enzyme-linked immunoassay
This study evaluated the performance of a new Food and Drug Administration-approved lateral-flow diagnostic screening test for qualitative detection of West Nile Virus (WNV) immunoglobulin M (IgM) in serum or plasma. Five public health laboratories across the United States performed retrospective testing on blinded serum samples from patients with physician reported diagnoses of WNV infection. The results of the RapidWNTM WNV IgM assay were compared with two commercially available WNV IgM enzyme-linked immunosorbent assays (EIA) and two public health-developed WNV-IgM tests. After discrepancies were resolved, the RapidWNTM WNV IgM EIA demonstrated a 98.8% sensitivity (range: 96.0–100%), a 95.3% specificity (range: 90.9–100%), a positive predictive value of 96.3% (range: 94.7–100%), and a negative predictive value of 98.4% (range: 95.5–100%), as compared to the predicate assays. The study results suggest that the RapidWNTM WNV IgM EIA is an effective, qualitative screening test that produces results comparable to that of predicate assays and can be employed rapidly to detect WNV IgM in patients suspected of having WNV infection. Published by Elsevier B.V.
1. Introduction Since its onset in 1999, West Nile Virus (WNV) has spread throughout North America and has become a serious public health concern in the United States (US) and Canada. According to the Centers for Disease Control and Prevention, from 1999 through 2008 there have been over 28,900 cases of confirmed symptomatic WNV infections in the US with over 1000 fatalities. As it is desirable to recognize infection early to initiate available therapeutic supportive measures and to rule-out other agents that cause neurological disease, detection of WNV-specific antibodies has become a valuable tool. Since a growing population may have had previous exposure and have pre-existing IgG antibodies, immunoglobulin M (IgM) antibody in serum and cerebral spinal fluid (CSF) specimens is the preferred marker for WNV infection. At the time of this study, two public health-developed tests and three commercial laboratory tests marketed by Focus Diagnostics, PanBio, and InBios were available in the US for initial screening of WNV IgM in serum specimens; however, the PanBio test kit is no longer available in the US. Specificity and sensitivity of these predicate assays have been analyzed against the plaque-reduction neutralization test (PRNT) (InBios International, Inc. Package Insert, 2009; Johnson
∗ Corresponding author at: Nebraska Public Health Laboratory, University of Nebraska Medical Center, 986495 Nebraska Medical Center, Omaha, NE 68198-6495, United States. Tel.: +1 402 559 3032; fax: +1 402 559 7799. E-mail addresses: [email protected] (A.R. Sambol), [email protected] (S.H. Hinrichs). 0166-0934/$ – see front matter. Published by Elsevier B.V. doi:10.1016/j.jviromet.2008.12.011
et al., 2007; Martin et al., 2004; Prince and Hogrefe, 2003a; Tilley et al., 2005). There have been many well-documented studies showing the utility of these screening tests (Malan et al., 2004; Niedrig et al., 2007a,b; Prince and Hogrefe, 2003b; Prince et al., 2004; Tardei et al., 2000; Sambol et al., 2007); however, these are multistep enzyme-linked immunosorbent assays (EIA) that require from several hours to overnight for completion and often employ expensive instrumentation. Both private and public health laboratories would benefit from the availability of a rapid, sensitive screening assay for WNV IgM that could be used in situations where an urgent request for testing services was needed for samples that were insufficient in number to warrant automated, high throughput testing. The primary goal of this study was to compare the relative sensitivity and specificity of a new Food and Drug Administrationapproved lateral-flow assay with commercially available and public health laboratory-developed screening tests. In addition to the Nebraska Public Health Laboratory (NPHL), four additional public health laboratories (PHL) participated in this study including: the Texas Department of State Health Services, the North Dakota Department of Health, the Illinois Department of Health, and the University of Iowa Hygienic Laboratory. Each PHL performed testing with the RapidWNTM WNV IgM EIA (Spectral Diagnostics, Toronto, Ont., Canada) and their predicate screening assay as follows: • WNV IgM Capture DxSelectTM (Focus Diagnostics, Cypress, CA, USA); • WNV IgM Capture ELISA (PanBio, Qld., Australia);
224
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
Table 1 Initial testing results of the RapidWN IgM EIA compared to predicate IgM assays. PHL site
Predicate kit
Rapid WNa
No. tested
Sensitivity
1 and 5 2 and 3 2 and 3 4
WNV IgM Capture DxSelect MIA MAC-EIA WNV IgM Capture ELISA
All sites
Specificity
Predictive value Positive
Negative
49 31 44 25
88.9% (24/27) 90.0% (18/20) 95.7% (22/23) 100% (15/15)
95.5% (21/22) 90.9% (10/11) 95.2% (20/21) 100% (10/10)
96.0% (24/25) 94.7% (18/19) 95.6% (22/23) 100% (15/15)
87.5% (21/24) 83.3% (10/12) 95.2% (20/21) 100% (10/10)
149
92.9% (79/85)
95.3% (61/64)
96.3% (79/82)
91.0% (61/67)
Abbreviations: RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); WNV IgM Capture ELISA = West Nile Virus Immunoglobulin M Capture Enzyme-linked Immunosorbent Assay (PanBio). a The numbers in parentheses show number of sample agreement divided by total sample in each group.
• WNV-specific IgM antibody-capture (MAC) [Centers for Disease Control and Prevention (CDC)]; • WNV-specific IgM micro-immunoassay (MIA) (CDC). Each of the 5 PHLs tested 10 negative and 15 positive serum specimens, including low, mid and high level positives, which were available from their archives. Specimens were tested in a blinded manner. With the exception of one laboratory where both the MAC-EIA and the MIA served as predicate assays for the RapidWNTM IgM EIA and were performed on the same set of specimens, and another laboratory where both of these predicate tests were in use but against different sets of specimens, all other laboratories tested the RapidWNTM IgM EIA against one predicate test. Overall, 124 serum specimens were tested, comparing 74 positives and 50 negatives by known methods. Because ease-of-use and interpretation of results were parameters for this study, no advanced training was provided to the clinical laboratory scientists performing the RapidWNTM IgM EIA. The commercially prepared predicate EIAs were performed according to manufacturers’ instructions using the provided internal controls and calibrators (Focus Diagnostics Package Insert, 2009; PanBio Package Insert, 2009). No PHLs in this pilot study used the InBios EIA test. The MAC-EIA was performed using the testing protocols developed originally by the CDC (Martin et al., 2000, 2004), and validated by the individual PHL. For the MIA, testing was performed as described by Johnson et al. (2007). The results from the MIA were reported as being positive or negative for WNV or St. Louis encephalitis virus (SLE) using quadratic dis-
criminate analysis and specimens were considered positive if their WNV-adjusted (WNV Adj) mean fluorescent intensity (MFI) was greater than 3. In the RapidWNTM IgM EIA, serum specimens were diluted 1:30 in Sample Diluent warmed to room temperature (15–25 ◦ C). The diluted samples were vortexed and a 150 L aliquot dispensed into the device sample well. If present, WNV IgM antibody in the diluted specimen was captured and formed a complex with the recombinant WNV antigen and secondary mouse flavivirus-specific monoclonal antibody. Migration of the complex through the reaction strip was detected by an immobilizing reagent resulting in a visible pink–purple horizontal band in the test area (Shaikh et al., 2007; Spectral Diagnostics’ Rapid WNTM Kit Insert, 2009). For the purpose of this study, each predicate assay was used as the reference standard to determine the performance characteristics of the RapidWNTM IgM EIA. Specimens that gave discordant results were retested with the RapidWNTM IgM EIA. If results remained discordant, retesting with the predicate device was performed. Specimens that were discrepant repeatedly were blinded and sent to an outside PHL for additional testing by traditional IgG-depleted indirect immuno-fluorescent antibody (IFA) as discussed by Shi and Wong (2003), and an experimental IgM-Western blot (IgM-WB) (Oceguera et al., 2007). PRNT was not performed on samples with discrepant results because this assay measures total antibody and not the presence of IgM alone (Oceguera et al., 2007). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were determined for all test methods. In 4 instances, specimens that tested positive initially with the predicate assay did not repeat as positive and were counted as neg-
Table 2 Comparison WNV-IgM testing of discrepant specimens by RapidWN IgM EIA, predicate assays, IgM-Western blot, and indirect fluorescent antibody-IgM. PHL site
IgM test used
Predicate cut-off value
Predicate resulta
Rapid WN result
WB
IFAb
Testing consensus
Rapid WN performance
1 1 1 1 2 2 3 2 2
WNV IgM Capture DxSelect WNV IgM Capture DxSelect WNV IgM Capture DxSelect WNV IgM Capture DxSelect MIA MIA MIA MAC-EIA MAC-EIA
1.1 1.1 1.1 1.1 3.0 3.0 3.0 3.0 3.0
0.53; neg 1.20; pos 1.99; pos 3.47; pos 2.75; neg 5.70; pos 32.4; pos 1.8; neg 5.0; pos
Pos Neg Neg Neg Pos Neg Neg Pos Neg
Neg Neg Neg Pos Neg Neg Neg Neg Neg
<1:10; neg 1:10; pos 1:10; pos 1:20; pos <1:10; neg 1:40; pos 1:160; pos <1:10; neg 1:40; pos
Neg None None Pos Neg None None Neg None
FPc
FNd FP
FP
Abbreviations: WNV IgM = West Nile Virus Immunoglobulin M; RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); Pos = positive; Neg = negative; FP = false positive; FN = false negative. a Expressed as Index Values. b Expressed as dilution titers. c False positive as determined by consensus tests. d False negative as determined by consensus tests.
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
225
Table 3 Revised testing results of Spectral RapidWN compared to predicate assays. PHL site
Predicate kit
No. of tested
Resolved discrepants
Rapid WNa Sensitivity
1 and 5 2 and 3 2 and 3 4 All sites
WNV IgM Capture DxSelect MIA MAC-EIA WNV IgM Capture ELISA
Specificity
Predictive value Positive
Negative
47 29 43 25
1 FN; 1 FP 1 FP 1 FP None
96.0% (24/25) 100% (18/18) 100% (22/22) 100% (15/15)
95.5% (21/22) 90.9% (10/11) 95.2% (20/21) 100% (10/10)
96.0% (24/25) 94.7% (18/19) 95.5% (22/23) 100% (15/15)
95.5% (21/22) 100% (10/10) 100% (20/20) 100% (10/10)
144
1 FN; 3 FP
98.8% (79/80)
95.3% (61/64)
96.3% (79/82)
98.4% (61/62)
Abbreviations: RapidWN IgM EIA = Rapid West Nile Immunoglobulin M Immunoassay (Spectral Diagnostics); IgM = Immunoglobulin M; PHL = Public Health Laboratory; WNV IgM Capture DxSelect = West Nile Virus Immunoglobulin M Capture Deluxe Select Immunoassay (Focus Diagnostics); MIA = Microimmunoassay (Centers for Disease Control and Prevention); MAC-EIA = West Nile Virus-specific Immunoglobulin M Antibody-capture Immunoassay (Centers for Disease Control and Prevention); WNV IgM Capture ELISA = West Nile Virus Immunoglobulin M Capture Enzyme-linked Immunosorbent Assay (PanBio); FP = false positive; FN = false negative. a The numbers in parentheses show number of sample agreement divided by total sample in each group.
ative. As shown in Table 1, the RapidWNTM IgM EIA demonstrated a 92.9% sensitivity (range of 88.9–100%), a 95.3% specificity (range of 90.9–100%), a PPV of 96.3% (range of 94.7–100%), and a NPV of 91.0% (range of 83.3–100%) as compared to the predicate assays. The results of testing performed on 9 discrepant serum specimens, and confirmation testing by WB and IFA, are shown in Table 2. Of the 9 specimens, 3 were determined to be false positive (FP), 1 specimen was determined to be a false negative (FN), and there were 5 specimens for which no consensus test results could be reached. These 5 specimens were subsequently excluded from the final data. Of the specimens examined in the comparison with WNV IgM Capture DxSelect, 1 was determined to be a RapidWNTM IgM EIA FP with an Index Value of 0.53 and 1 a RapidWNTM IgM EIA FN, with a fairly high Index Value of 3.47. Three specimens examined in the comparison of RapidWNTM IgM EIA with the MIA gave discrepant test results. Of these, 1 specimen was determined to be a FP by both RapidWNTM IgM EIA and MIA, with a WN Adj MFI of 2.75. The 2 specimens for which no consensus results were found had a MIA WN Adj of 5.70 and 32.4, negative findings by RapidWNTM IgM EIA, negative findings by WB, and positive findings by IFA. Of the specimens examined in the comparison with the MAC-EIA, 1 specimen that gave a RapidWNTM IgM EIA FP compared to a MAC-EIA negative Index Value of 1.8. The other discrepant specimen examined in this comparison was considered to be a no consensus sample as it was positive by MAC-EIA (Index Value of 5.0) and IFA, but negative by RapidWNTM IgM EIA and WB. Table 3 shows the results after analysis of discrepant serum specimens. When the 5 specimens without consensus were removed from the analysis, the RapidWNTM IgM EIA showed an increase in sensitivity from 92.9% to 98.8%, specificity remained the same at 95.3%, the PPV remained constant at 96.3%, and the NPV increased from 91.0% to 98.4%. It is well recognized that with EIA and MIA screening formats cross-reactions do occur with a variety of other bacterial and viral agents, as well as anti-nuclear antibodies and Rheumatoid factor. The reference standard for specific confirmation of WNV or any flavivirus disease is the PRNT. However, because the PRNT tests for the presence of both neutralizing IgM and IgG, it was not used in this study to resolve discrepancies. Instead, traditional IgG-depletion IFA testing as well as an experimental IgM-WB was performed on the discrepant sera. Of the 9 serum specimens with discrepant results in the initial testing, no consensus could be reached in 5 serum samples and these samples were excluded from the final analysis of the data. As shown in Table 2, the results for these 5 specimens were near the limits of sensitivity for the predicate screening tests. Future studies will seek to obtain follow-up specimens or convalescent serum from patients or to examine these specimens for other cross-reactive factors know to cause false results.
A single use diagnostic screening device for the detection of WNV would hold the potential for use in a variety of settings. This study was conducted at 5 PHL on 124 different serum specimens obtained throughout North America from patients with physicianreported diagnoses of WNV. One limitation of this study was the number of specimens tested against multiple reference assays; however, this study provided the opportunity to examine performance at multiple sites. Overall, the sensitivity and specificity of RapidWNTM IgM EIA was comparable to predicate screening tests that are currently in use. In cases where results of screening tests may be questionable, definitive clinical diagnosis should include patient symptomology and the use of confirmatory tests such as the PRNT. In summary, this study demonstrated that the RapidWNTM IgM EIA: (1) performed comparably with commercially available and public health-developed assays; (2) offered several advantages as tests can be performed rapidly without automated equipment, tedious calculations, or special handling of the EIA-based device reagents; (3) is suitable for use in a situation where a small number of WNV IgM tests are necessary. Acknowledgments The authors would like to thank the directors and public health laboratories that assisted with this study: Dr. Susan Neill, Texas Department of State Health Services; Dr. Myra Kosse, North Dakota Department of Health; Dr. Bernard Johnson, Illinois Department of Health; and Dr. Chris Atchison, University of Iowa Hygienic Laboratory. Spectral Diagnostics provided the RapidWNTM WNV IgM assay kits at no charge and all remaining costs were born by the performing sites. No financial compensation was provided to the investigators. References Centers for Disease Control and Prevention (CDC), 2008. Division of Vector Borne Infectious Diseases (DVBID), www.\cdc.gov/ncidod/dvbid/westnile. Focus Diagnostics Package Insert, 2009. InBios International, Inc. Package Insert, 2009. Johnson, A.J., Cheshier, R.C., Cosentino, G., Masri, H., Mock, V., Oesterle, R., Lanciotti, R.S., Martin, D.A., Panella, A., Kosoy, O., Biggerstaff, B., 2007. Validation of a microsphere-based immunoassay for the detection of anti-West Nile virus and anti-St. Louis encephalitis virus immunoglobulin M antibodies. Clin. Vac. Immunol. 14, 1084–1093. Malan, A.K., Martins, T.B., Hill, H.R., Litwin, C.M., 2004. Evaluations of commercial West Nile virus immunoglobulin G (IgG) and IgM enzyme immunoassays show the value of continuous validation. J. Clin. Microbiol. 42, 727–733. Martin, D.A., Muth, D.A., Brown, T., Johnson, A.J., Karabatsos, N., Roehrig, J.T., 2000. Standardization of immunoglobulin M capture enzyme-linked immunosorbent assays for routine diagnosis of arboviral infections. J. Clin. Microbiol. 38, 1823–1826.
226
A.R. Sambol, S.H. Hinrichs / Journal of Virological Methods 157 (2009) 223–226
Martin, D.A., Noga, A., Kosoy, O., Johnson, A.J., Petersen, L.R., Lanciotti, R.S., 2004. Evaluation of a diagnostic algorithm using immunoglobulin M enzyme-linked immunosorbent assay to differentiate human West Nile Virus and St. Louis Encephalitis virus infections during the 2002 West Nile Virus epidemic in the United States. Clin. Diagn. Lab. Immunol. 11, 1130–1133. Niedrig, M., Donoso-Mantke, O., Altmann, D., Zeller, H., 2007a. First international diagnostic accuracy study for the serological detection of West Nile virus infection. BMC Infect. Dis. 7, 72. Niedrig, M., Sonnenberg, K., Steinhagen, K., Paweska, J.T., 2007b. Comparison of ELISA and immunoassays for measurement of IgG and IgM antibody to West Nile virus in human sera against virus neutralization. J. Virol. Methods 139, 103–105. Oceguera, L.F., Patiris, P.J., Chiles, R.E., Busch, M.P., Tobler, L.H., Hanson, C.V., 2007. Short report: flavivirus serology by western blot analysis. Am. J. Trop. Med. Hyg. 77, 159–163. PanBio Package Insert, 2009. Prince, H.E., Hogrefe, W.R., 2003a. Detection of West Nile virus (WNV)-specific immunoglobulin M in a reference laboratory setting during the 2002 WNV season in the United States. Clin. Diagn. Lab. Immunol. 10, 764–768. Prince, H.E., Hogrefe, W.R., 2003b. Performance characteristics of an in-house assay system used to detect West Nile Virus (WNV)-specific immunoglobulin M (IgM) during the 2001 WNV season in the United States. Clin. Diagn. Lab. Immunol. 10, 177–179.
Prince, H.E., Lape’-Nixon, M., Moore, R.J., Hogrefe, W.R., 2004. Utility of the Focus Technologies West Nile virus immunoglobulin M capture enzyme-linked immunosorbent assay for testing cerebrospinal fluid. J. Clin. Microbiol. 42, 12–15. Sambol, A.R., Hinrichs, S.H., Hogrefe, W.R., Schweitzer, B.K., 2007. Performance of a commercial immunoglobulin M antibody capture assay using analyte-specific reagents to screen for interfering factors during a West Nile virus epidemic season in Nebraska. Clin. Vac. Immunol. 14, 87–89. Shaikh, N.A., Ge, J., Zhao, Y.X., Walker, P., Drebot, M., 2007. Development of a novel, rapid, and sensitive immunochromatographic strip assay for West Nile virus (WNV) IgM and testing of its diagnostic accuracy in patients suspected of WNV infection. Clin. Chem. 53, 2031–2034. Spectral Diagnostics Rapid WNTM Kit Insert, 2009. Shi, P.Y., Wong, S.J., 2003. Serologic diagnosis of West Nile virus infection. Expert Rev Mol Diagn 3, 733–741. Tilley, P.A., Walle, R., Chow, A., Jayaraman, G.C., Fonseca, K., Drebot, M.A., Priksaitis, J., Fox, J., 2005. Clinical Utility of Commercial Enzyme Immunoassays during the Inaugural Season of West Nile Virus Activity, Alberta, Canada. J. Clin. Microbiol. 43, 4691–4695. Tardei, G., Ruta, S., Chitu, V., Rossi, C., Tsai, T.F., Cernescu, C., 2000. Evaluation of immunoglobulin M (IgM) and IgG enzyme immunoassays in serologic diagnosis of West Nile Virus infection. J. Clin. Microbiol. 38, 2232–2239.