Validation of a fluorescence polarization assay (FPA) performed in microplates and comparison with other tests used for diagnosing B. melitensis infection in sheep and goats

Journal of Immunological Methods 320 (2007) 94 – 103 www.elsevier.com/locate/jim

Research paper

Validation of a fluorescence polarization assay (FPA) performed in microplates and comparison with other tests used for diagnosing B. melitensis infection in sheep and goats A. Minas a,⁎, A. Stournara b , M. Minas c , J. Stack d , E. Petridou e , G. Christodoulopoulos f , V. Krikelis a a

d

Technological Educational Institution of Larissa, Faculty of Health Professions, Laboratory of Microbiology, Larissa, 411 10, Greece b Veterinary Laboratory of Larissa National Reference Laboratory of Brucellosis, 6th Kilometer of National Road Larissa, Trikala, Larissa, 411 10, Greece c University of Thessally, Medical Faculty of Larissa, Larissa, 411 10, Greece FAO/WHO Collaborating Centre for Reference and Research on Brucellosis, Veterinary Laboratories Agency, Weybridge, Surrey, KT15 3NB, UK e Aristotelian University of Thessaloniki, Veterinary Faculty, Laboratory of Microbiology, Thessaloniki, 541 24, Greece f University of Thessally, Veterinary Faculty of Karditsa, Karditsa, 431 00, Greece Received 12 May 2006; received in revised form 27 November 2006; accepted 13 December 2006 Available online 16 January 2007

Abstract Fluorescence polarization assay (FPA) is a relatively new test for the serological diagnosis of Brucella spp. infection in animals. FPA, carried out in 96-well microplate format, was validated here for diagnosing B. melitensis infection in sheep and goats. This study included sera from 1933 sheep and goats, from animals reared in naturally infected flocks (verified by culture) and showing a positive reaction to two different tests conducted in parallel. In addition, 2154 sera originating from healthy sheep and goats, reared in areas where B. melitensis had never been isolated, were assayed. The optimum cut-off value offering the highest diagnostic sensitivity (DSn) and diagnostic specificity (DSp) was determined at 15 mP over the mean value of the buffer control used in each microplate as determined by receiver operating characteristic analysis. The DSn and DSp of the FPA for small ruminants carried out in microplates at this cut-off value were calculated to be 95.9% and 97.9% with 95% confidence intervals (95% CI) of 94.9– 97.7% and 97.2–98.4%, respectively. The accuracy of the FPA, as expressed by determination of the area under the curve, was 0.991. Indirect ELISA and FPA tests offered the highest DSn when compared with the Rose Bengal test, the complement fixation test, the modified Rose Bengal test and competitive ELISA. The parallel or serial combination of FPA with indirect ELISA offers the highest DSn and DSp. As temperature can affect the results of the FPA, all reagents must be at the same temperature and the standard for comparison must always be read under the same conditions as the sera under test. FPA performed in microplates is a promising assay; the DSn and accuracy are better than those of the tests currently approved for diagnosing B. melitensis in small ruminants. Because of its simplicity, speed, and accuracy, this test can improve capacity for laboratory testing and the efficacy of an eradication program based on a test-and-slaughter policy. © 2006 Elsevier B.V. All rights reserved. Keywords: B. melitensis; Fluorescence polarization assay; Brucellosis in small ruminants; Validation; Serology

⁎ Corresponding author. 18 Argyrokastrou Street, 412 22 Larissa, Greece. Tel.: +30 2410 612 325; fax: +30 2410 617 982. E-mail address: [email protected] (A. Minas). 0022-1759/$ - see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.jim.2006.12.008

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

1. Introduction Brucellosis eradication programs are based on presumptive diagnosis by serology of Brucella spp. infection. A panel of tests has been validated and approved (OIE, 2004a) for diagnosing brucellosis in cattle. For ovine and caprine brucellosis, however, only the Rose Bengal test (RBT) and the complement fixation test (CFT) are currently accepted by the EU and OIE (Garin-Bastuji and Blasco, 1997; ECD, 1991; OIE, 2004b). RBT is suggested only for identifying infected flocks (flock screening test) (Blasco et al., 1994). CFT is regarded as more sensitive than RBT (ECD, 1991; Nicoletti, 1969; MacMillan, 1990) and is recommended as a confirmatory test and for testing of individual animals in infected flocks. When RBT and CFT are used singly or in combination (serial or parallel), they are very effective as flock screening tests. However, these tests cannot detect all the infected animals in a flock when used individually. This lack of diagnostic sensitivity (DSn) impedes eradication of brucellosis in sheep and goats using a test-and-slaughter policy (Nicoletti, 1969). Efforts to improve serological diagnosis of brucellosis in small ruminants have led to the development of new tests, such as indirect ELISA (i-ELISA), competitive ELISA (c-ELISA) and fluorescence polarization assay (FPA). These tests appear to be more efficacious than RBT and CFT in identifying infected animals (Jaques et al., 1998; Nielsen and Gall, 2001). FPA for detecting antibodies against Brucella spp. has recently been developed based on Perrin's theory and improvements have been made by Weber and Steiner (Dandliker and De Saussure, 1970). The test measures the return photons in the planes parallel and perpendicular to the polarized light that excites the fluorescence molecule (fluorophore), and thus allows the assessment of the fluorophore's rotation rate, which is inversely proportional to its size. Thus, the rotation rate of the Brucella O-polysaccharide molecule, labelled with fluoroscein isothiocyanide (FITC), changes if antiBrucella lipopolysacharide (LPS) antibody binds to it, because of the increase in size of the molecule (Nasir and Jolley, 1999). FPA is a rapid, homogenous, species-independent assay, which was initially developed and validated for the detection of antibodies to B. abortus in cattle (Nielsen et al., 2001; McGiven et al., 2003; Nielsen et al., 1996, 2000; OIE, 2004a). Information on its performance in detecting antibodies to B. melitensis in sheep and goats is limited (Nielsen and Gall, 2001; Minas et al. 2005; Ramirez-Pfeiffer et al., 2006). To

95

date, in the studies conducted to validate FPA for diagnosis of B. melitensis infection in small ruminants, the assay was carried out in single glass test tubes using a volume of 1 ml, and the results read by an instrument designed to read a single test tube. Recently, new instruments that can read microplates in fluorescence polarization mode have been developed, and these have opened up new prospects for FPA. The aim of the present study was to validate an FPA carried out in microplates, for the diagnosis B. melitensis infection in sheep and goats. A multi-mode microplate reader was used to analyse the results. The study focused on FPA in microplates, because published information on the validity of the assay performed in this manner for the diagnosis of B. melitensis infection in sheep and goats, is limited (Bahn and Nöckler, 2005). The objective was to determine the appropriate cut-off that offers the highest performance index [diagnostic sensitivity (DSn) + diagnostic specificity (DSp)] and to compare it with those of other tests under the same and different epidemiological conditions. 2. Materials and methods 2.1. Negative reference sera The negative reference sera used for validating FPA were collected from 15 sheep and 12 goat flocks reared on islands of the Aegean Sea. The flocks were selected using random number tables after the numeration of all flocks in these islands. These islands are considered to be officially free from brucellosis, according to Office International des Epizooties (OIE) stipulations (OIE, 2004c). In addition, vaccination against brucellosis in this area has not been used since 1990 and Brucella spp. has never been isolated from any of the abortions that were investigated in the area. From the selected flocks, blood samples from the jugular vein of 1117 sheep and 1037 goats were collected in Vacutainer™ tubes. The samples were allowed to clot; sera were then separated after centrifugation at 1000 ×g for 20 min, aliquoted and stored at − 75 °C until tested. 2.2. Positive reference sera Although isolation of B. melitensis constitutes the “reference standard”, the use of sera from culture positive animals for validation of a serological assay has some limitations because they are not representative of the test's target population (Jacobson, 1998).

96

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

To create a more representative panel of positive reference sera, 16 sheep and 21 goat flocks, naturally infected with B. melitensis, sited in different prefectures of the country, were randomly selected. In each selected flock, the infection had been confirmed by isolating and identifying B. melitensis in at least one animal. Animal vaccination had never been implemented in these flocks. Seven out of 16 sheep and 10 out of 21 goat flocks had been newly identified as infected. The remaining samples represented animals that were at second or third serological test in the eradication program. The animals that were seropositive to RBT or CFT had been removed previously. All animals in the study were > 6 months old. Blood samples were collected from the jugular vein into Vacutainer™ tubes, allowed to clot; sera were separated after centrifugation at 1000 ×g for 20 min, aliquoted and stored at − 75 °C until tested. A total of 2415 sheep and 3339 goat sera samples were collected. All sera were tested in parallel by RBT, modified RBT (m-RBT), CFT, i-ELISA, c-ELISA and FPA. Any sample testing positive in at least two of these tests was considered to have originated from an infected animal and included in the panel of positive reference sera (Greiner and Gardner, 2000). Following this procedure, sera of 913 sheep and 1020 goats were selected and used as positive references to validate the FPA. In addition, to assess the DSn, sheep and goat sera were selected as positive reference standards for RBT (2119 samples), m-RBT (2099 samples), CFT (2109 samples), i-ELISA (1904) and c-ELISA (2088 samples). The samples that were positive in both RBT and m-RBT were not selected, because these two tests are considered similar. To avoid any bias, the test under validation was not included as a criterion for selection. The cut-off used for the FPA was > 15 mP above the mean value of the buffer controls used in each microplate, as determined in this study for sheep and goats. 2.3. Serological tests FPA was conducted in 96-well flat-bottom black polystyrene microtitre plates, type COS96fb, manufactured by Corning USA. Initially, 200 μl of dilution buffer was placed in each of the three wells in the first column (A1, B1, and C1) and the wells were designated buffer controls. In each remaining well, 180 μl of dilution buffer and 20 μl of test sera were added. The dilution buffer was provided by the manufacturer in 25× concentrated form and the working dilution was prepared using ultra clean sterile water. In each microplate, positive and negative control sera of bovine

origin, provided by the manufacturer, were used in duplicate. Buffer and serum samples were mixed by setting the microplate on a rotating microplate shaker for 2 min at room temperature. After the initial mixing, a background reading was taken in fluorescence polarization mode by a multi-mode microplate reader (Tecan Genios Pro) connected to a laptop computer. Subsequently, 10 μl of antigen (O-polysaccharide from B. abortus strain 1119.3 prepared and conjugated with FITC), as described by Lin and Nielsen (1997), was added to all wells. After mixing for 3 min at room temperature, a second reading was taken. The reader automatically subtracted the background reading and calculated the value for every sample in millipolarization units (mP). The reagents used in this assay were manufactured by Diachemix (Whitefish Bay, WT, USA) and supplied by Prionics AG (Switzerland). The fluorescence polarization reader was calibrated using blank and low polarized standards, which were included in the kit along with the antigen. The calibration was automatic; the instrument takes the value of the low polarized standard (fixed at 25 mP) and automatically calculates the internal compensation factor (G-factor) utilized in the equation for calculating the final results. The measurement parameters of the instrument were set for gain at 55, integration time at 40 μs and at 10 flashes per well per second. The filter for excitation wavelength was at 485 nm and for emission at 535 nm. The results of each microplate measurement were accepted if the mP values were as follows: the positive control >180 mP, the negative control 60– 85 mP and buffer control >70 mP and <80 mP. For other tests used in the study, RBT was implemented as described in the Manual of Standards for Diagnostic Tests and Vaccines (OIE, 2004a). The mRBT, considered more sensitive than RBT, was used in addition by mixing 75 μl of serum and 25 μl of antigen (Blasco et al., 1994; OIE, 2004b). The antigen used in the RBT and m-RBT was standardized as described in the Manual of Standards for Diagnostic Tests and Vaccines and a positive value was recorded from a sample showing any degree of agglutination (OIE, 2004a). The reagents used in the CFT were standardized and the test was conducted as described in the Manual of Standards for Diagnostic Tests and Vaccines (OIE, 2004a) using the warm method. Any serum showing a value ≥ 20 ICFTU/ml was considered positive (ECD, 1991). Sera were tested by i-ELISA and c-ELISA utilizing commercial kits manufactured by Pourquier Institute and Svanova Biotech AB, respectively. According to the

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

manufacturers’ instructions, the cut-off for i-ELISA was set at ≥ 100% of the value of the positive control serum included in the kit and for c-ELISA at 26% inhibition (26% I) of the conjugated control, as suggested by Nielsen and Gall (2001). 2.4. Data analysis Until now, there has been no positive or negative international standard reference serum that could be used to calibrate the tests for diagnosing B. melitensis infection in small ruminants. Hence, in this study, the buffer control was used for comparison. As the components of the buffer control are constant, the mP value must remain constant if the reader is calibrated properly and factors like temperature, which can affect this, remain stable. The results of the FPA in each microplate were calculated as follows: the average value of the buffer control was calculated from wells A1, B1 and C1; it was subtracted from every value and the difference reported. Using these differences, the cut-off value for FPA giving the highest sum of DSn and DSp values and the area under the curve (AUC) and their 95% CI were determined separately for sheep and goats, and for sheep and goats together by receiver operating characteristic (ROC) analysis. The performance of FPA on sheep and goat sera, as well as all other tests, was compared by Mann–Whitney non-parametric test for independent samples. The comparisons and ROC analyses were carried out using MedCalc software version 8.0 (Schoonjans et al., 1995). The DSn and DSp of the tests, as well as their performances under different disease prevalence rates and in paired combination (parallel or serial), were assessed by cross tabulation using Win Episcope software version 2.0 for windows.

97

A regression analysis was conducted so that the correlation of temperature with the mP value could be determined. The best-fit regression line with 95% prediction of mP values of buffer controls against temperature is shown in Fig. 1. The regression analysis shows a significant (P < 0.0001) negative linear correlation of the mP value of buffer controls with the temperature at which the measurement was obtained with the slope of the regression line at − 0.89. The optimum cut-off for FPA for testing sheep sera, goat sera and small ruminant sera (sheep and goats together) were calculated by ROC analysis at 15 mP, 14 mP and 15 mP, respectively, above the mean value of the buffer controls used in each microplate. The ROC curves for sheep, for goats and for sheep and goats together are shown in Fig. 2. The calculated DSn, DSp, AUC and their 95% CI for sheep, goats, and sheep and goats together, are listed in Table 1. The Mann–Whitney test reveals that although the DSn, DSp and AUC for sheep are higher than those for goats, the performance of the FPA does not differ significantly (P > 0.05) among sheep and goats. The DSn and DSp and their 95% CI of FPA at different cut-off values, as well as positive and negative likelihood ratios (+ LR, − LR) for sheep, goats, and sheep and goats together are given in Tables 2 and 3, respectively. The cut-off values in these tables were selected to cover the range from 100% DSn to 100% DSp. The appropriate cut-offs can therefore be selected to suit the demands of the animal species and the testing regime in which the assay is incorporated. DSn and DSp and Youden's J value, and their 95% CI, of all tests for sheep and goats together used in the study were determined (Table 4) using the positive and negative reference sera selected for each test.

3. Results The microplates were read in a temperature-controlled chamber incorporated in the instrument. The reading temperatures ranged from 19 °C to 27.3 °C with a mean of 24.77 ± 1.83 °C and 95% CI of 24.56–24.98 °C. During the study, 302 measurements were obtained for the buffer control on different microplates and days, and the values ranged from 67 mP to 79 mP. The arithmetic mean (expressed in mP) of these buffer control values, its 95% CI, the standard deviation (SD) and coefficient of variation (CV), which express the precision of the measurements, were calculated at 72.43 mP, 72.16 to 72.70 mP, ±2.4 mP and 3.31%, respectively.

Fig. 1. Best fit regression line of mP value against temperature and 95% CI prediction interval.

98

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

Fig. 2. ROC analysis curves of FPA results from positive and negative reference sera from sheep, goats and for sheep and goats together.

The results of all tests on positive and negative reference sera were compared using the non-parametric Mann–Whitney test for independent samples (Table 5). The efficacy of FPA, RBT, m-RBT, CFT, i-ELISA and c-ELISA to detect sheep and goats infected with B. melitensis under different prevalence rates was assessed. The apparent prevalence (AP) and positive and negative predicted values (+ PV, − PV) of each test for different B. melitensis infection prevalence rates were calculated (Table 6) using the DSn and DSp given in Table 4 for each test. The data in Table 6 reveal that in areas free from brucellosis, the use of FPA with a 15 mP cut-off above the mean value of buffer control is not suitable for testing sheep and goats, because the percentage of false positive results will be at least 2.1%. In areas where B. melitensis infection exists, the efficacy of RBT and CFT depends on the prevalence of infection in each flock. As the prevalence of infection increases in a flock, the number of infected animals increases too, and the current approved tests (RBT and CFT) cannot identify all the infected animals, since the apparent prevalence is

significantly lower than the real prevalence. The − PVs of RBT and CFT, in cases of high prevalence of disease, are the lowest among all the tests, indicating that under these circumstances, these tests produce a large number of false negative results. The tests showing the highest AP and −PV, where 90% of the animals are infected, are i-ELISA and FPA using a cut-off of 15 mP above the mean value of buffer control. In brucellosis eradication programs, usually two tests in combination (serial or parallel) are used to identify infected animals. Using the values of DSn and DSp for each test (Table 4), the DSn and DSp of combinations of different pairs of tests conducted in parallel or serial were assessed (Table 7). The data in Table 7 reveal that any combination of two tests in parallel shows DSn >94%; the highest is the combinations of m-RBT with iELISA and i-ELISA with FPA, whereas the DSp of these testing regimes is >95%. Results of the tests conducted in serial show that the DSn and DSps of the combinations of m-RBT with i-ELISA and i-ELISA with FPA are the highest. 4. Discussion FPA performed in microplates is validated for diagnosing brucellosis in small ruminants and the cut-off offering the highest performance index (sum of DSn and DSp) determined by ROC analysis. The determination of the most suitable cut-off value for a test by ROC analysis has many advantages, because DSn and DSp are calculated for different cut-off values. This enables the most appropriate cut-off to be used to provide the required DSn or DSp for a given epidemiological situation. For evaluating an assay, a representative panel of positive and negative reference sera must be used. The criterion for selecting positive reference sera is a crucial factor that influences the outcome of a test's evaluation. It is widely accepted that collection of sera from culture positive animals representing all disease stages (latent, incubation period, chronic) is a difficult task. In addition, most culture positive animals have generated a strong immune response, and hence the use of these samples for assessing a test's performance would lead,

Table 1 DSn, DSp, AUC and their 95% CI for the cut-offs determined for sheep, goats and sheep + goats Species

Cut-off (mP)

DSn (95% CI)

DSp (95% CI)

AUC (95% CI)

Sheep Goats Sheep + goats

15 14 15

97.5 (96.2–98.4) 95.4 (93.9–96.6) 95.9 (94.9–96.7)

98.3 (97.4–99.0) 96.4 (95.1–97.5) 97.9 (97.2–98.4)

0.994 (0.989–0.997) 0.987 (0.982–0.992) 0.991 (0.987–0.993)

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

99

Table 2 DSn, DSp and the 95% CI and positive and negative likehood ratios for sheep and goat sera at different cut-off values mP value >− 9 >− 8 >− 7 >− 6 >− 5 >− 4 >− 3 >− 2 >− 1 >0 >1 >2 >3 >4 >5 >6 >7 >8 >9 >10 >11 >12 >13 >14 a >15 b >16 >17 >18 >19 >20 >21 >22 >23 >24 >25 >26 >27 >28 >29 >30 >31 >32 >33 >34 a b

Sera from sheep DSn (95% CI)

100.0 (99.6–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.8 (99.2–100.0) 99.5 (98.7–99.8) 99.5 (98.7–99.8) 99.2 (98.4–99.7) 99.1 (98.3–99.6) 98.9 (98.0–99.5) 98.8 (97.9–99.4) 98.2 (97.2–99.0) 97.9 (96.8–98.7) 97.8 (96.6–98.7) 97.5 (96.2–98.4) 96.5 (95.1–97.6) 95.7 (94.2–96.9) 94.9 (93.2–96.2) 94.5 (92.8–95.9) 93.8 (92.0–95.2) 93.2 (91.4–94.8) 92.4 (90.5–94.1) 91.2 (89.2–93.0) 90.7 (88.6–92.5) 90.0 (87.9–91.9)

Sera from goats DSp (95% CI)

+LR

3.2 (2.3–4.4) 1.03 4.7 (3.5–6.1) 1.05 7.1 (5.6–8.7) 1.07 10.4 (8.7–12.3) 1.11 13.2 (11.3–15.4) 1.15 16.7 (14.5–19.0) 1.20 20.2 (17.9–22.7) 1.25 24.0 (21.5–26.6) 1.31 27.8 (25.2–30.6) 1.38 35.0 (32.2–37.9) 1.54 43.4 (40.5–46.4) 1.76 52.6 (49.6–55.5) 2.10 62.4 (59.5–65.2) 2.64 71.4 (68.7–74.1) 3.47 78.5 (76.0–80.9) 4.61 85.0 (82.8–87.1) 6.62 90.1 (88.2–91.8) 9.94 93.8 (92.2–95.2) 15.90 96.4 (95.2–97.4) 27.34 97.2 (96.1–98.1) 35.24 98.3 (97.4–99.0) 57.31 98.7 (97.9–99.3) 76.99 99.1 (98.4–99.6) 106.93 99.5 (98.8–99.8) 176.58 99.7 (99.2–99.9) 351.94 99.7 (99.2–99.9) 349.09 99.7 (99.2–99.9) 347.05 99.7 (99.2–99.9) 344.19 99.9 (99.5–100.0) 1019.12 99.9 (99.5–100.0) 1013.01 100.0 (99.7–100.0)

− LR

0.00 0.05 0.03 0.02 0.02 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.02 0.02 0.02 0.03 0.04 0.04 0.05 0.05 0.06 0.07 0.08 0.09 0.09 0.10

DSn (95% CI)

DSp (95% CI)

+LR

− LR

100.0 (99.6–100.0) 99.9 (99.5–100.0) 99.8 (99.3–100.0) 99.8 (99.3–100.0) 99.8 (99.3–100.0) 99.8 (99.3–100.0) 99.8 (99.3–100.0) 99.7 (99.1–99.9) 99.7 (99.1–99.9) 99.6 (99.0–99.9) 99.6 (99.0–99.9) 99.6 (99.0–99.9) 99.6 (99.0–99.9) 99.6 (99.0–99.9) 99.5 (98.9–99.8) 99.4 (98.7–99.8) 99.3 (98.6–99.7) 99.2 (98.5–99.7) 98.8 (98.0–99.4) 97.9 (96.9–98.7) 97.6 (96.5–98.5) 97.3 (96.1–98.2) 96.8 (95.5–97.8) 95.4 (93.9–96.6) 94.4 (92.8–95.7) 93.1 (91.4–94.6) 91.4 (89.5–93.0) 90.3 (88.3–92.0) 89.3 (87.3–91.1) 88.3 (86.2–90.2) 87.2 (84.9–89.1) 86.1 (83.8–88.1) 85.3 (83.0–87.4) 84.2 (81.8–86.4) 83.3 (80.9–85.6) 82.0 (79.5–84.3) 81.2 (78.6–83.5) 80.2 (77.6–82.6) 79.1 (76.5–81.6) 78.3 (75.7–80.8) 77.8 (75.2–80.4) 76.9 (74.2–79.4) 76.5 (73.7–79.0) 75.6 (72.8–78.2)

0.2 (0.0–0.7) 0.2 (0.0–0.7) 0.3 (0.1–0.8) 0.4 (0.1–1.0) 0.6 (0.2–1.3) 1.1 (0.5–1.9) 1.3 (0.7–2.1) 2.4 (1.6–3.5) 3.7 (2.6–5.0) 5.4 (4.1–7.0) 8.5 (6.9–10.4) 12.2 (10.2–14.3) 17.7 (15.5–20.2) 25.2 (22.6–27.9) 34.6 (31.7–37.6) 43.9 (40.8–47.0) 54.8 (51.7–57.8) 63.2 (60.1–66.1) 72.5 (69.7–75.2) 81.0 (78.5–83.3) 86.7 (84.5–88.7) 91.2 (89.3–92.9) 94.4 (92.8–95.7) 96.4 (95.1–97.5) 97.4 (96.2–98.3) 98.3 (97.3–99.0) 98.6 (97.7–99.3) 99.0 (98.2–99.5) 99.2 (98.5–99.7) 99.4 (98.7–99.8) 99.5 (98.9–99.8) 99.6 (99.0–99.9) 99.7 (99.2–99.9) 99.7 (99.2–99.9) 99.8 (99.3–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 99.9 (99.5–100.0) 100.0 (99.6–100.0)

1.00 1.00 1.00 1.00 1.00 1.01 1.01 1.02 1.03 1.05 1.09 1.13 1.21 1.33 1.52 1.77 2.20 2.69 3.60 5.16 7.34 11.08 17.30 26.74 36.26 53.66 67.68 93.64 115.77 152.67 180.76 223.16 294.83 291.11 432.08 849.93 841.80 831.63 820.45 812.32 807.23 797.07 793.00

0.00 0.51 0.68 0.51 0.34 0.18 0.16 0.12 0.08 0.07 0.05 0.03 0.02 0.02 0.01 0.01 0.01 0.01 0.02 0.03 0.03 0.03 0.03 0.05 0.06 0.07 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0.21 0.22 0.22 0.23 0.24 0.24

Cut-off for goats showing the highest performance index (DSn+DSp). Cut-off for sheep showing the highest performance index (DSn+DSp).

in most cases, to over estimation of DSn, which would be unrealistic for the test's target population (Thrusfield, 1995; Jacobson, 1998). One solution to overcome this drawback is to use randomly selected samples, seropositive at least in two different tests (Greiner and Gardner, 2000). In the present study, this alternative solution was adopted for selecting positive reference standards.

The reference standards used in this study can be considered more representative of the test's target population, because the flocks were selected randomly. All the factors that can influence a test's DSn and DSp, such as age, sex, stage of pregnancy, health status and stage of infection (latent, chronic, incubation period) were also represented. In addition, the sampled

100

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

Table 3 DSn, DSp and their 95% CI and positive and negative likehood ratio for sheep and goats sera at different cut-off values mP value DSn (95% CI) >− 9 >− 8 >− 7 >− 6 >− 5 >− 4 >− 3 >− 2 >− 1 >0 >1 >2 >3 >4 >5 >6 >7 >8 >9 >10 >11 >12 >13 >14 >15 a >16 >17 >18 >19 >20 >21 >22 >23 >24 >25 >26 a

DSp (95% CI)

+LR

100.0 (99.8–100.0) 0.5 (0.2–0.9) 1.00 99.9 (99.7–100.0) 0.6 (0.3–1.0) 1.01 99.9 (99.6–100.0) 0.9 (0.5–1.4) 1.01 99.9 (99.6–100.0) 1.3 (0.9–1.9) 1.01 99.9 (99.6–100.0) 1.9 (1.4–2.6) 1.02 99.8 (99.5–99.9) 2.9 (2.3–3.7) 1.03 99.8 (99.5–99.9) 4.3 (3.5–5.2) 1.04 99.7 (99.4–99.9) 6.5 (5.5–7.7) 1.07 99.7 (99.4–99.9) 8.6 (7.5–9.9) 1.09 99.7 (99.3–99.9) 11.2 (9.9–12.6) 1.12 99.7 (99.3–99.9) 14.6 (13.1–16.1) 1.17 99.7 (99.3–99.9) 18.3 (16.7–20.0) 1.22 99.7 (99.3–99.9) 23.0 (21.2–24.8) 1.29 99.7 (99.3–99.9) 30.3 (28.3–32.3) 1.43 99.6 (99.3–99.9) 39.2 (37.1–41.3) 1.64 99.4 (99.0–99.7) 48.4 (46.2–50.5) 1.93 99.4 (98.9–99.7) 58.7 (56.6–60.8) 2.41 99.2 (98.7–99.6) 67.5 (65.4–69.4) 3.05 99.0 (98.4–99.4) 75.6 (73.8–77.4) 4.06 98.4 (97.7–98.9) 83.1 (81.5–84.7) 5.82 98.2 (97.5–98.7) 88.4 (87.0–89.8) 8.49 97.7 (97.0–98.3) 92.6 (91.4–93.6) 13.16 97.3 (96.5–98.0) 95.5 (94.5–96.3) 21.39 96.5 (95.6–97.3) 96.8 (96.0–97.5) 30.58 95.9 (94.9–96.7) 97.9 (97.2–98.4) 44.89 94.7 (93.6–95.7) 98.5 (97.9–99.0) 63.76 93.4 (92.2–94.5) 98.9 (98.3–99.3) 83.85 92.4 (91.2–93.6) 99.3 (98.8–99.6) 124.45 91.8 (90.5–93.0) 99.5 (99.1–99.7) 179.70 90.9 (89.5–92.1) 99.6 (99.2–99.8) 217.53 90.0 (88.6–91.3) 99.6 (99.3–99.8) 242.35 89.1 (87.6–90.4) 99.7 (99.3–99.9) 274.11 88.1 (86.6–89.5) 99.8 (99.5–99.9) 474.39 87.3 (85.7–88.7) 99.8 (99.5–99.9) 469.93 86.5 (84.9–88.0) 99.9 (99.7–100.0) 931.50 85.3 (83.6–86.9) 100.0 (99.7–100.0) 1837.37

− LR 0.00 0.09 0.12 0.08 0.05 0.07 0.05 0.04 0.03 0.03 0.02 0.02 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.02 0.02 0.02 0.03 0.04 0.04 0.05 0.07 0.08 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15

Cut-off showing the highest performance index (DSn + DSp).

population did not differ from those in other parts of the country, because the type of husbandry, management and animal breeds were almost the same. Moreover, the prevalence of diseases among the country's prefectures

Table 5 Comparison of the tests by Mann–Whitney test Tests

P value of the difference

RBT vs m-RBT RBT vs CFT RBT vs i-ELISA RBT vs c-ELISA RBT vs FPA m-RBT vs CFT m-RBT vs i-ELISA m-RBT vs c-ELISA m-RBT vs FPA CFT vs i-ELISA CFT vs c-ELISA CFT vs FPA i-ELISA vs c-ELISA i-ELISA vs FPA c-ELISA vs FPA

<0.05 >0.05 <0.05 >0.05 <0.05 <0.05 >0.05 <0.05 <0.05 <0.05 >0.05 <0.05 >0.05 >0.05 <0.05

due to bacteria (Salmonella spp., Cambylobacter spp.) that might cross react with the antigens used for B. melitensis infection diagnosis in small ruminants, do not differ significantly (Veterinary Service, unpublished data). The DSn and DSp determined using these reference standards can be considered more realistic because the assay's performance is determined within the population to which it is going to be applied (Dohoo et al., 1986). Unfortunately, at the time of this study, there is no positive or negative International Standard Serum that can be used for calibration of serological tests used for diagnosis of B. melitensis infection in small ruminants. Therefore, the mean value of the buffer controls in each microplate was selected as the standard for comparison of positive results. This selection was based on the assumption that because the components and concentration of the buffer are always constant, the measurements will not deviate significantly if the external factors influencing the mP value do not change and the instrument is properly calibrated. Although the cut-off for sheep is not identical to that of goats, the results of the comparison reveal that a

Table 4 DSn and DSp and Youden's J value and their 95% CI for all tests on sheep and goats together calculated from negative reference sera (n = 2154) and positive reference sera with positive results in two serological tests in parallel Test

Positive reference

DSn (95% CI)

DSp (95% CI)

Youden's J (95% CI)

RBT M-RBT CFT i-ELISA c-ELISA FPA

2119 2099 2109 1904 2088 1933

75.8 (74.0–77.7) 86.6 (85.2–88.1) 80.6 (78.9–82.3) 98.2 (97.6–98.8) 76.5 (74.7–78.4) 95.9 (94.9–96.7)

99.7 (99.5–99.9) 97.6 (97.0–98.3) 99.1 (99.8–100) 99.5 (99.8–100) 98.5 (98.0–99.0) 97.9 (97.2–98.4)

0.76 (0.74–0.77) 0.84 (0.83–0.86) 0.80 (0.79–0.82) 0.98 (0.98–0.99) 0.75 (0.73–0.77) 0.94 (0.93–0.95)

2.10 11.48 20.86 30.24 39.62 49.00 58.38 67.76 77.14 86.52 95.90 AP, apparent prevalence for the test; +PV, positive predictive value; − PV, negative predictive value.

− PV AP (%) +PV − PV

0.00 100.0 85.00 97.42 92.73 94.37 95.63 90.72 97.14 86.28 98.08 80.74 98.71 73.64 99.17 64.24 99.51 51.17 99.78 31.77 100.0 0.00

+ CFT RBT + i-ELISA RBT + c-ELISA RBT + FPA m-RBT + CFT M-RBT + i-ELISA M-RBT + c-ELISA M-RBT + FPA CFT + i-ELISA CFT + c-ELISA CFT + FPA i-ELISA + c-ELISA i-ELISA + FPA c-ELISA + FPA

1.50 9.00 16.50 24.00 31.50 39.00 46.50 54.00 61.50 69.00 76.50

AP (%) +PV

Test combination

0.00 100.0 95.62 99.80 98.00 99.55 98.83 99.23 99.24 98.81 99.49 98.22 99.66 97.36 99.78 95.95 99.87 93.25 99.94 86.00 100.0 0.00

− PV

101

Table 7 Estimation of DSn, DSp and DSn+DSp index for parallel and serial combinations of tests used for diagnosis of B. melitensis infection in small ruminants

0.50 10.27 20.04 29.81 39.58 49.35 59.12 68.89 78.66 88.43 98.20

AP (%) +PV − PV

0.00 100.0 90.87 97.87 95.32 95.33 97.46 92.26 98.35 88.46 98.90 83.63 99.26 77.30 99.52 68.64 99.72 56.08 99.88 36.21 100.0 0.00 0.90 8.87 16.84 24.81 32.78 40.75 48.72 56.69 64.66 72.63 80.60

AP (%) +PV − PV

0.00 100.0 80.04 98.50 90.02 96.68 93.93 94.44 96.01 91.61 97.30 87.93 98.19 82.92 98.83 75.74 99.31 64.55 99.69 44.73 100.0 0.00 2.40 10.82 19.24 27.66 36.08 44.50 52.92 61.34 69.76 78.18 86.60

AP (%) +PV −PV

0.00 100.0 96,56 97.37 98.44 94.28 99.08 90.58 99.41 86.07 99.61 80.47 99.74 73.31 99.83 63.84 99.90 50.74 99.96 31.40 100.0 0.00

AP (%) +PV

0.30 7.85 15.40 22.95 30.50 38.05 45.60 53.15 60.70 68.25 75.80 0 10 20 30 40 50 60 70 80 90 100

FPA c-ELISA i-ELISA CFT m-RBT True prevalence (%) RBT

Table 6 Performance of serological tests for diagnosis of B. melitensis infection in small ruminants at different infection prevalence levels

0.00 100.0 83.54 99.54 91.95 98.96 95.14 98.24 96.82 97.28 97.86 95.98 98.56 94.09 99.07 91.10 99.46 85.65 99.76 72.63 100.0 0.00

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

Parallel testing

Serial testing

DSn

DSp

DSn + DSp

DSn

DSp

DSn + DSp

95.31 99.56 94.31 99.01 97.40 99.76 96.85 99.45 99.65 95.44 99.20 99.58 99.93 99.04

98.80 99.20 98.20 97.61 96.72 97.11 96.14 95.55 98.60 97.61 97.02 98.01 97.41 96.43

194.11 198.76 192.51 196.62 194.12 196.87 192.99 195.00 198.25 193.05 196.22 197.59 197.34 195.47

61.09 74.44 57.99 72.69 69.80 85.04 66.25 83.05 79.15 61.66 77.30 75.20 94.17 73.36

100.00 100.00 100.00 99.99 99.98 99.99 99.99 99.95 100.00 99.99 99.98 99.99 99.99 99.97

161.09 174.44 157.99 172.68 169.78 185.03 166.24 183.00 179.15 161.65 177.28 175.19 194.16 173.33

unique cut-off can be used for FPA for sheep and goat sera, without any significant effect on the test's DSn and DSp. However, FPA offers the advantage of selecting different cut-offs so that DSn and DSp can be adjusted according to the species origin of the sera, the demands of the epidemiological situation, and the purpose of the test. Determining DSn and DSp in the tests utilized in the study, using positive and negative reference standards selected the same way, permits comparison of their performance. The comparison reveals that i-ELISA and FPA offer the highest DSn, and that their DSps are also high. Although i-ELISA appears to be more sensitive and specific than FPA, the performance of the two tests does not differ significantly (P > 0.05). The DSns of RBT and CFT are low and this confirms the observations of other researchers that these tests are ineffective for individual testing of animals, especially when many infected animals exist in a flock (Nicoletti, 1969; Kolar, 1995). However, DSn determined for m-RBT is higher than that of RBT or CFT, and this test can be more effective than the current approved tests for individual testing of animals (Blasco et al., 1994). The DSn and DSp of FPA determined in this study differ from those reported by Nielsen and Gall (2001): 91.5% and 98.6% for sheep and 94.9% and 99.4% for goats, respectively. They also differ from those determined by Bahn and Nöckler (2005), who reported DSn and DSp of FPA performed in microplates of 92.3% for sheep and 87.8% for goats. These differences can be attributed to different selection criteria of positive reference standards and perhaps to the quality of sera used. It must be pointed

102

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

out that Nielsen and Gall (2001) conducted FPA in single glass tubes and not in microplates. However, the DSn of FPA carried out in microplates for sheep is close to what Minas et al. (2005) observed when the assay was carried out in single glass test tubes. As the negative and positive reference sera were selected using the same criteria in both studies, the results are comparable and lead to the conclusion that DSn of FPA remains constant and is independent of the way the test is carried out. DSp calculated in microplates for sheep is slightly lower than that calculated when FPA was performed in single test glass tubes, but their 95% CIs overlap. This must be attributed to the fact that in the study conducted by Minas et al. (2005), the negative reference standards giving positive results were retested after centrifugation, resulting in higher DSp. Although the performance of FPA differs among the studies, all the data show that FPA, independent of the way it is carried out (in single glass test tubes or in microplates), performs better than the current approved tests for diagnosing B. melitensis infection in small ruminants (Nielsen and Gall, 2001; Minas et al., 2005; Bahn and Nöckler, 2005). Moreover, the high value of AUC (0.991) indicates that FPA is a highly accurate test and in 99% of cases it correctly identifies a sheep or goat as positive, if the result is > 15 mP from the mean value of the buffer control in the microplate. It is widely accepted that eradication of brucellosis by test-and-slaughter policy is extremely difficult and prolonged in large herds and flocks with high prevalence of infection (Blasco et al., 1994; Kolar, 1995). From the data in this study (Table 6), it can be concluded that in small flocks, consisting of up to 50 animals with a prevalence rate of infection up to 10% (maximum five diseased animals in the flock), all tests can detect almost all the infected animals. As the prevalence rate of infection or the size of the flock increases, the number of infected animals increases and consequently, the number of infected animals not detected by RBT and CFT also increases. Where false negative animals are not removed from the flock, because they are considered healthy, they serve as a source of infection for the truly healthy animals. This results in the persistence of infection and consequently, eradication of the disease from the flock becomes almost impossible. Therefore, a test with higher DSn and DSp needs to be used. The results of the current study reveal that in flocks with a high prevalence rate (> 40%) of B. melitensis infection, FPA and i-ELISA can detect > 95% of infected animals with > 97% predicted value for positive results and hence, these tests could assist best in eradicating the disease.

In brucellosis eradication programs, two or more tests in combination are usually used to improve identification and diagnosis of infected animals. When the existing tests are used in parallel (seropositive in at least one of the tests conducted simultaneously) for any combination, the DSn achieved is >90%. However, the highest DSn, >99%, was observed by using combinations of i-ELISA with FPA, mRBT with FPA and m-RBT with i-ELISA. When the tests are combined in serial (seropositive in one test must be confirmed by the other), and first implementing the test with the highest DSn, the combination of i-ELISA with FPA offers the highest DSn. The results of the current study reveal that if quick elimination of B. melitensis infection from a flock is desired, a combination of the tests with the highest DSn must be used in parallel. On the contrary, for flocks or in areas free of brucellosis, it is better to use a serial combination of the tests. It is not possible to offer a standard set of recommendations for selecting the tests that are appropriate to every area or condition; epidemiological information, the current situation and targets of the program must be taken into account when selecting a testing regime. The crucial factors that interfere with the mP value obtained by the reader instrument are the temperature of the reagents and the temperature in the reading chamber. It was documented that the temperature affects the relaxation time of the molecule and consequently, the mP value calculated by the reading instrument (Nielsen et al., 2000). The negative linear correlation of mP value with the temperature indicates that the mP value decreases as the temperature increases. As the temperature affects all the samples in the microplate equally, for accurate and repeatable measurements, all reagents (dilution buffer, sera, and antigen) must be at the same temperature and the temperature at which the measurements are taken must be stable. Therefore, as all the samples must be compared simultaneously under the same test conditions, buffer controls must be tested in every microplate and the results from previous measurements of buffer controls cannot be used for comparison, because this may lead to false results. 5. Conclusions FPA performed in microplates is an accurate diagnostic test that has many advantages compared with other methods used for serological diagnosis of brucellosis in small ruminants. The test is easy to perform and the results are obtained in less than 5 min; the instrument calculates the mP value automatically and provides objectively

A. Minas et al. / Journal of Immunological Methods 320 (2007) 94–103

interpreted results. Prozone effects are not observed and haemolysed sera can be tested with the same degree of accuracy. For accurate results, all reagents must be at the same temperature. As the test is performed in microplate format, it can be easily automated resulting in significantly increased testing capacity of a field laboratory. The cut-off can be adjusted so that the desired combination of DSn and DSp can be obtained for different epidemiological situations and the test can be used accordingly as a screening or confirmatory test. As FPA in combination with m-RBT or i-ELISA offers high DSn and DSp, such testing regimes should increase the efficacy of an eradication program based on a test-and-slaughter policy. The data available for small ruminants reveal that FPA is a promising method with high accuracy, and hence, it should be considered for use in brucellosis eradication programs; its performance and contribution to brucellosis eradication can be validated in practice. Acknowledgements This study was supported by Prionics AG, Switzerland, which provided the antigens for the implementation of FPA and other serological tests. References Bahn, P., Nöckler, K., 2005. Validierung des Fluoreszenz Polarisations Assay (FPA) für die Serodiagnostik der brucellose. Berl. Münch. Tierärztl. Wochenchr. 118, 372. Blasco, J.M., Garin-Bastuji, B., Marin, C.M., Gerbier, G., Fanlo, J., Jimenes de Bagues, M.P., Cau, C., 1994. Efficacy of different Rose Bengal and Complement Fixation antigens for the diagnosis of Brucella melitensis infection in sheep and goats. Vet. Rec. 134, 415. Dandliker, W.B., De Saussure, V.A., 1970. Fluorescence polaization in immunochemistry. Immunochemistry 7, 799. Dohoo, I.R., Wright, P.F., Ruckerbauer, G.M., Samagh, B.S., Robertson, F.J., Forbes, L.B., 1986. A comparison of five serological tests for bovine brucellosis. Can. J. Vet. Res. 50, 485. ECD, 1991. 91/68/EEC. On animal health conditions governing intracommunity trade in ovine and caprine animals. European Council Directive. Garin-Bastuji, B., Blasco, J.M., 1997. . Caprine and ovine brucellosis (excluding Brucella ovis infection), Manual of Standards for Diagnostic Tests and Vaccines, third ed. Office International des Epizooties, Paris, France, p. 350. Greiner, M., Gardner, I.A., 2000. Epidemiologic issues in the validation of veterinary diagnostic tests. Prev. Vet. Med. 45, 3. Jacobson, R.H., 1998. Validation of serological assays for diagnosis of infectious diseases. Rev. Sci. Tech.-Off. Int. Epizoot. 17, 469. Jaques, I., Olivier-Bernardin, V., Dubray, G., 1998. Efficacy of ELISA compared to conventional tests (RBPT and CFT) for the diagnosis of Brucella melitensis infection in sheep. Vet. Microbiol. 64, 61.

103

Kolar, J., 1995. Some experience from brucellosis control with Rev.1 vaccine in a heavily infected country — Mongolia. FAO/WHO/ OIE Round Table on the Use of Rev.1 Vaccine in Small Ruminants and Cattle. CNEVA, Alfort, France. September 21–22. Lin, M., Nielsen, K., 1997. Binding of Brucella abortus lipopolysaccharide O-chain fragment to a monoclonal anibody. Biol. Chem. 272, 2821. MacMillan, A.P., 1990. Conventional serological tests. In: Nielsen, K., Duncan, J.R. (Eds.), Animal Brucellosis. CRC Press, Boca Raton, FL, p. 153. McGiven, J.A., Tucker, J.D., Perrett, L.L., Stack, J.A., Brew, S.D., MacMillan, A.P., 2003. Validation of FPA and cELISA for the detection of antibodies to Brucella abortus in cattle sera and comparison to SAT, CFT, and iELISA. J. Immunol. Methods 278, 171. Minas, A., Stournara, A., Minas, M., Papaioannou, A., Krikelis, V., Tselepidis, S., 2005. Validation of fluorescence polarization assay (FPA) and comparison with other tests used for diagnosis of B. melitensis infection in sheep. Vet. Microbiol. 111, 211. Nasir, M.S., Jolley, M.E., 1999. Fluorescence polarization: an analytical tool for immunoassay and drug discovery. Comb. Chem. High Throughput Screen. 2, 177. Nicoletti, P., 1969. Further evaluation of serologic test procedures used to diagnose brucellosis. Am. J. Vet. Res. 30, 1811. Nielsen, K., Gall, D., Jolley, M., Leishman, G., Balsevicius, S., Smith, P., Nicoletti, P., Thomas, F., 1996. A homogenous fluorescence polarization assay for detection antibody to Brucella abortus. J. Immunol. Methods 195, 161. Nielsen, K., Lin, M., Gall, D., Jolley, M., 2000. Fluorescence polarization immunoassay: detection antibody to Brucella abortus. Methods 22, 71. Nielsen, K., Gall, D., 2001. Fluorescence polarization assay for the diagnosis of brucellosis: a review. J. Immunoass. Immunochem. 22, 183. Nielsen, K., Gall, D., Smith, P., Kelly, W., Yeo, J., Kenny, K., Heneghan, T., McNamara, S., Maher, P., O'Connor, J., Walsh, B., Caroll, J., Rojas, X., Rojas, F., Perez, B., Wulff, O., Buffoni, L., Salustio, E., Gregoret, R., Samartino, L., Dajer, A., Luna-Martinez, E., 2001. Fluorescence polarization assay for the diagnosis of bovine brucellosis: adaptation to field use. Vet. Microbiol. 80, 163. OIE, 2004a. Bovine brucellosis. Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. Office International des Epizooties, Paris. OIE, 2004b. Caprine and ovine brucellosis (excluding Brucella ovis). Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. Office International des Epizooties, Paris. OIE, 2004c. Caprine and ovine brucellosis. Terrestrian Animal Health Code. Office International des Epizooties, Paris. Chapter 2.4.2. Ramirez-Pfeiffer, C., Nielsen, K., Marin-Ricalde, F., RodriguezPadilla, C., Gomez-Flores, R., 2006. Comparison of fluorescence polarization assay with card and complement fixation tests for the diagnosis of goat brucellosis in a high-prevalence area. Vet. Immunol. Immunopathol. 110, 121. Schoonjans, F., Zalata, A., Depuydt, C., Comhaize, F., 1995. Medcalc: a new computer program for medical statistics. Comput. Methods Programs Biomed. 48, 257. Thrusfield, M., 1995. Veterinary Epidemiology, second ed. Blackwell Science, UK. 276 pp.