Validation of fluorescence polarization assay (FPA) and comparison with other tests used for diagnosis of B. melitensis infection in sheep

Veterinary Microbiology 111 (2005) 211–221 www.elsevier.com/locate/vetmic

Validation of fluorescence polarization assay (FPA) and comparison with other tests used for diagnosis of B. melitensis infection in sheep A. Minas a,*, A. Stournara b, M. Minas c, A. Papaioannou d, V. Krikelis a, S. Tselepidis e a

Technological Educational Institution of Larissa, Faculty of Health Professions, Laboratory of Microbiology, Larissa, 41 110, Greece b Veterinary Laboratory of Larissa, National Reference Laboratory of Brucellosis, 6th Kilometer of National Road Larissa-Trikala, Larissa, 41 110, Greece c University of Thessally, Medical Faculty of Larissa, Larissa, 41 110, Greece d Technological Educational Institution of Larissa, Faculty of Health Professions, Laboratory of Biochemistry, Larissa, 41 110, Greece e Veterinary Clinic and Training Center, General Staff Medical Corporation, Hellenic Army, Plastira Camp, Larissa, 41 110, Greece Received 12 July 2005; received in revised form 24 September 2005; accepted 6 October 2005

Abstract Fluorescence polarization assay (FPA) is a new test for the serological diagnosis of Brucella spp. infection in animals. The FPA is validated for the diagnosis of B. melitensis infection in sheep. For this purpose, 166 sera originated from natural infected sheep (verified by culture) and 851 sera originated from healthy animals (reared in areas where B. melitensis was never been isolated) were tested. The optimum cut-off value that offers the highest diagnostic sensitivity (DSn) and diagnostic specificity (DSp) was determined at 87 mP with the use of ROC analysis. The DSn and DSp of FPA using this cut-off value are calculated at 97.6 and 98.9% with a 95% confidence interval (CI) of 93.9–99.3% and 98.0–99.5%, respectively. The DSn and DSp of FPA have been assessed also using as positive reference (n = 587), sera that gave positive results at least in two tests used for diagnosis of B. melitensis in sheep as Rose Bengal Test (RBT), modified Rose Bengal Test (m-RBT), complement fixation test (CFT), indirect Elisa (i-Elisa) and competition Elisa (c-Elisa) originated from animals reared in flocks infected by B. melitensis. The optimum cut-off value using the above panel of positive reference sera was the same offering a DSn of 95.9% with a 95% CI, 94.0–97.4%, since the DSp remains the same. The DSn and DSp as well as performance, accuracy and agreement of FPA’s result were compared with those of other tests used. The accuracy of FPA is very high, similar with that of i-Elisa. FPA is a promising assay, which offers a DSn and accuracy better that of those of the tests currently approved for the diagnosis of B. melitensis in sheep and goats. Due to its simplicity, the sort time that results can be obtained and its accuracy it * Corresponding author. Tel.: +30 2410 612 325; fax: +30 2410 617 982. E-mail address: [email protected] (A. Minas). 0378-1135/$ – see front matter # 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.vetmic.2005.10.009

212

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

can be used and improve the laboratory testing capacity as well as the efficacy of the eradication program based on test-andslaughter policy. # 2005 Elsevier B.V. All rights reserved. Keywords: B. melitensis; FPA; Sheep; Validation; Serology

1. Introduction Brucellosis in sheep and goats, caused by B. melitensis, one of the most virulent species of Brucella, is responsible for important economic losses in sheep and goats farming. It is also a widespread zoonosis constituting a serious public health problem wherever the infectious agent is endemic, like in Mediterranean and Middle-East countries (Nicoletti, 1989; Corbel, 1989). Diagnosis of Brucella spp. infection is mainly based on the detection of antibodies in serum by serological tests. The Rose Bengal Test (RBT) (Davies, 1971) and complement fixation test (CFT) (Alton et al., 1988) are the most accepted tests worldwide for this purpose (Garin-Bastuji and Blasco, 1997) and the only approved for certification of sheep and goats flocks due to brucellosis status in EU member states (ECD, 1991). The RBT, due to its low sensitivity on sheep and goats sera, is suggested to be used only for identification of infected flocks (flock screening test) and not for individual animals (Blasco et al., 1994). Since CFT is regarded as more sensitive and specific, is used for individual testing of animals in infected flocks as well as confirmatory test (ECD, 1991; Nicoletti, 1969; MacMillan, 1990). Although both tests when used singly or in combination (serial or parallel) are very effective as flock screening tests, they detect only 70% of the infected animals when these are tested individually. This makes the implementation of test-and-slaughter policy for brucellosis eradication in small ruminants not very effective (Nicoletti, 1969). It is accepted that for serological diagnosis of brucellosis to be improved there is a need of other tests to be used more sensitive than the currently used (Blasco et al., 1994). For this purpose, indirect Elisa (iElisa), fluorescence polarization assay (FPA) and competition Elisa (c-Elisa) have been developed, which ability of identifying infected animals appears

superior to RBT and CFT (Jaques et al., 1998; Nielsen and Gall, 2001). The aim of the present study is the validation of FPA for the diagnosis of B. melitensis infection in sheep with the determination of cut-off offering the highest performance index (diagnostic sensitivity (DSn) + diagnostic specificity (DSp)) and the comparison of its performance with that of the other tests used for this purpose. The study has been focused on FPA, since it is a relative new diagnostic test for brucellosis and there is limited published information about its performance in sheep sera (Nielsen et al., 2001, 2005). Additionally, due to its simplicity, the speed that results obtained and their objective interpretation, it can contribute significantly to improvement of eradication’s program efficacy if its performance is superior to those of the tests used at the present.

2. Materials and methods 2.1. Serological tests All sera used in the study were tested with RBT, a modification of RBT (m-RBT), CFT, i-Elisa, c-Elisa and FPA in parallel. FPAwas conducted as it is described by Nielsen et al. (2001). Briefly, 1.0 ml of 0.01 M Tris buffer, pH 7.2, containing 0.15 M sodium chloride, 15 mM EDTA and 0.05% Igepal A-630, was placed in a 10 mm  75 mm glass tube followed by 10 ml of serum. After mixing, a background reading was taken with the use of a portable Sentry Fluorescence polarization analyser (FPM Sentry, Diachemix Corp., Wisconsin) connected to a laptop computer. Then, 10 ml of antigen (o-polysaccharide from B. abortus strain 1119.3 prepared and conjugated with fluoroscein isothiocyanide (FITC)) as described by Lin and Nielsen (1997) was added and after mixing, a second reading was taken by the analyser, at least 2 min after the addition of antigen. The

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

analyser automatically subtracts the background reading and calculates a value for the sample in millipolarization units (mP). The antigen used in FPA was supplied by Diachemix, Whitefish Bay, WT and USA. The Sentry Fluorescence polarization analyser was calibrated with the use of blank, low and highpolarized standards provided by the manufacturer. The measurement of sera proceed only if the values of low and high polarized standards were between 23–27 and >360 mP units, respectively. In the beginning and end of each batch of sera tested, a series of controls consisted from buffer control, strong positive, weak positive and negative control sera, provided with the antigen by the manufacturer, were tested and the results obtained were accepted as valid if the control’s values were in the limits of 70.5– 80.5, 180–280, 95–110 and 59–85 mP units, respectively. RBT was implemented as it is described in Manual of Standards for Diagnostic Tests and Vaccines (OIE, 2000) and accepted by EU legislation (ECD, 1964). The m-RBT that considered as more sensitive than RBT was implemented mixing 75 ml of serum and 25 ml of antigen (Blasco et al., 1994; OIE, 2000). The antigen used in RBT and m-RBT was standardized to give positive reaction with OIE International Standard Serum (OIEISS) at dilution 1/45 and negative reaction at dilution 1/55. The reagents used in CFT, were standardized and the test conducted as described in Manual of Standards for Diagnostic Tests and Vaccines (OIE, 2000) with the warm method. Any serum showing a value 20 ICFTU/ml considered as positive (ECD, 1991). Regarding i-Elisa the sera tested with the use of a commercial kit dedicated for detection of antibodies against B. melitensis infection in sheep and goats (Pourquier Institute). According to manufacturer’s instruction, any serum showing a value 100% of the positive control serum provided with the kit, considered as positive. All sera were tested also with c-Elisa, a multispecies assay for the detection of antibodies against Brucella spp. infection (Nielsen et al., 1996) with the use of a commercial kit (Svanova Biotech AB). The cut-off for the positive result for sheep sera was set at 26% Inhibition (26% I) of conjugate control as suggested by Nielsen and Gall (2001).

213

2.2. Negative reference sera The negative reference sheep sera used for the validation of FPA and other tests were collected at year 2004 and originated from animals reared in 11 flocks sited in the islands of Naxos, Andros and Skopelos in Aegean Sea. These islands considered as brucellosis officially free according to OIE rules (OIE, 2004) since all animals >6 months of age in all flocks were tested in the past twice in a year period with negative results and the last 3 years they were tested annually with negative results too. Additionally, only animals originated from brucellosis officially free flocks are allowed to be introduced, the vaccination against brucellosis in this area was stopped at 1990, the introduction of vaccinated animals originated from mainland is not allowed since 1997 and bacteria of genus Brucella spp. have never been isolated from the abortions investigated in the area. The 11 sheep flocks were selected randomly with the use of random number tables after the numeration of the total flock population in these islands. The epidemiological data collected monthly by Veterinary Service in the Ministry of Agriculture about ovine diseases caused by Salmonella spp., Cambylobacter spp. and other bacteria, reveal that their prevalence does not differ significantly among the prefectures of the country (Veterinary Service of Ministry of Agriculture, unpublished data). Taking this in to account, can be assumed that there is not any significant difference among country’s prefectures of the prevalence of infectious agents that could cross-react with the antigens used for diagnosis of B. melitensis infection in sheep. Additionally, since the type of husbandry and management of sheep flocks, as well as sheep breeds are almost the same in whole country can be assumed that the selected flocks are representative of the test’s target population. From the selected flocks all animals >6 months of age, were bled and due that can be concluded also that all variables which can influence the DSp of a test as age, sex, stage of pregnancy and health status due to other diseases are represented in the sample. From the selected flocks, 851 blood samples were collected, left to clot and sera were separated after centrifugion and stored at 75 8C until tested.

214

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

2.3. Positive reference sera 2.3.1. Sera from sheep culture positive Due to the difficulty of acquisition, only 166 sera were collected from corresponding sheep from which B. melitensis was isolated by culture. These sera were collected at years 2002 and 2003 from 166 naturally infected sheep originated from non-vaccinated flocks sited in different prefectures in the mainland as well as in the islands of Rodos, Lesvos and Crete. The sera were aliquoted and stored at 75 8C and used as positive controls for the evaluation of serological tests in National Reference Laboratory for Brucellosis. The 166 sheep were identified as ‘‘infected’’ by serology during the implementation of eradication program. The animals, according to the measures foreseen in eradication program, were driven to slaughterhouse for culling. After slaughter, from every animal samples of mammary glad, mammary and retropharyngeal lymph nodes were collected and cultured in Fallel’s selective media in O2 and CO2 environment. From the cultures B. melitensis was identified taking into account growth characteristics, the results of biochemical tests (H2S production, nitrate reduction, urease test) and reaction with monospecific sera, obtained from VLA Weybridge. The transportation of animals to slaughterhouse and their slaughter were performed according to the rules laid down by EU about animal welfare. These rules had been respected during the handling of animals for blood collection too. 2.3.2. Sera from sheep positive to other serological tests Although the isolation of B. melitensis constitutes the ‘‘reference standard’’ that is the absolute proof of infection, the use of sera from culture positive animals has some limitations since they are not representative of the test’s target population. For the creation of more representative panel of positive reference sera all animals >6 months of age reared in 14 sheep flocks naturally infected by B. melitensis were bled and the sera collected stored at 75 8C until tested. The 14 flocks selected randomly during the implementation of eradication program in the country at years 2002, 2003 and 2004. The infection in each flock selected, was confirmed by culture, isolation and identification of B. melitensis at

least from one animal in each flock. From the selected flocks 5 out of 14 were sited in different prefectures in the mainland and the other 3, 4 and 2 in the islands of Rodos, Lesvos and Crete, respectively. In selected flocks vaccination has never been implemented and six of these were firstly identified as infected and the remaining eight were at the procedure of disease’s eradication at the second and third test after the removal of seropositive animals. The population of animals >6 months of age that tested in each flock as well as the serological prevalence estimated by RBT and CFT are presented in Table 1. From the above flocks 2243 sera were collected and tested by RBT, m-RBT, CFT, i-Elisa, c-Elisa and FPA in parallel. Every serum gave positive results at least in two of the above tests considered as originated from infected animal and included in the panel of positive reference sera (Greiner, 2000). In order incorporation bias to be avoided the test under validation is not included as selection criterion for positive reference sera. It is not considered as positive reference, also, the sera that gave positive results only in RBT and m-RBT simultaneously, since the two tests considered the same. Following this procedure, 645, 639, 635, 598, 633 and 587 sera were selected and used as positive reference for the validation of RBT, m-RBT, CFT, iElisa, c-Elisa and FPA, respectively. Since all animals >6 months of age were bled in selected flocks naturally infected from B. melitensis can be concluded that all the factors that can influence the performance of a single test like age, stage of pregnancy, health status as well as the stage of infection (latent, chronic and incubation period) are represented. The sera sample collected this way is more representative of tests target population than the sera originated from culture positive animals. 2.4. Data analysis The cut-off value for FPA that gives the highest sum of DSn and DSp values and the area under curve (AUC) and their 95% CI were determined from the analysis of results of negative reference sera (851 sera) and positive reference sera (166 sera) originated from culture positive animals and from sera positive to other tests (587 sera) with the use of receiver operator characteristics (ROC) analysis using MedCalc software version 8.0 (Schoonjans et al., 1995).

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

215

Table 1 Animals tested in every infected flock, number of sera with positive results in RBT and CFT and sero prevalence observed Flock number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Total a

Number of animals sampled

Sera positive to RBT

Prevalence % at RBT

Positive to CFT

Prevalence % at CFT

90 81 130 50 34 416 85 104 82 319 287 346 79 132

6 5 99 27 26 72 12 2 6 83 60 21 22 26

6.67 6.17 76.15 54.00 76.47 17.31 14.12 1.92 7.32 26.02 20.91 6.07 27.85 19.70

5 13 92 30 29 76 13/15a 2 6 98 66 28/62a 23 25

5.56 16.05 70.77 60.00 85.29 18.27 15.29/17.65a 1.92 7.32 30.72 23.00 8.09/17.92a 29.11 18.94

2235

467

20.89

506/542a

22.64/24.25a

Sera with anticomplementary reaction (AC) considered as positives.

The DSn, DSp, Youden’s J value and AUC and their 95% confidence interval (CI) of every test used in the study were calculated with cross-tabulation and ROC analysis of the results of negative reference sera and positive reference sera from culture positive animals and from those with positive results at least in two other tests (the test under validation was excluded). The agreement of test’s results was assessed with the analysis of the results of all sera tested in brucellosis free and infected flocks (n = 3094), by Kappa analysis and calculation of Kappa (K) statistic. The performance of the tests used compared by Friedman test for repeat measurements. The above calculations materialized with the use of Win Episcope version 2.0 and SPSS version 12.0 softwares.

0.002 with positive and negative likehood ratio (+LR) ( LR) at 92.28 and 0.02, respectively. The DSn and DSp of FPA at different cut-off values, their 95% CI as well as +LR and LR are presented in Table 2. The cut-off for FPA was determined also with ROC analysis using as positive reference sera the panel of sera (587 sera) that gave positive results at least in two different tests. The cut-off determined at 87 mP too.

3. Results The cut-off for FPA using as positive reference sera those originated from culture positive animals determined by ROC analysis at 87 mP. Any serum showing a value of exactly 87 mP units is considered as negative. The ROC diagram is presented in Fig. 1. The DSn and DSp of FPA using this cut-off determined at 97.6 and 98.9% with a 95% CI of 93.9–99.3% and 98.0–99.5%, respectively. The AUC calculated at 0.998 with a 95% standard error (S.E.) of

Fig. 1. ROC curve diagram of FPA using as positive reference sera originated from culture positive animals.

216

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

Table 2 DSn and DSp of FPA at different cut-off values calculated by ROC analysis using as positive reference sera originated from culture positive animals mP Units

DSn

CI at 95% confidence

DSp

CI at 95% confidence

+LR

84 85 86 87 * 88

100.0 98.8 98.8 97.6 95.2

97.8–100.0 95.7–99.8 95.7–99.8 93.9–99.3 90.7–97.9

87.4 90.7 95.3 98.9 100.0

85.0–89.6 88.6–92.6 93.7–96.6 98.0–99.5 99.6–100.0

7.95 10.64 21.02 92.28

*

LR 0.00 0.01 0.01 0.02 0.05

Cut-off offers the optimum combination of DSn and DSp.

The diagram of ROC curve is presented in Fig. 2. The test’s DSp remains the same and its relative DSn determined at 95.9% with 95% CI 94.0–97.4%. The AUC calculated at 0.987 with a 95% S.E. 0.003 with +LR and LR at 90.68 and 0.04, respectively. The DSn and DSp of FPA at different cut-off values, their 95% CI as well as +LR and LR are presented in Table 3. The calculated DSn and accuracy of the test, as it is expressed by AUC, using this panel of positive reference sera are less than those calculated using as positive reference sera originated from culture positive animals. The DSn, DSp, Youden’s J value the AUC and their 95% CI of all tests as they determined by crosstabulation and ROC analysis of the results of negative reference sera and culture positive reference sera as well as of the results of negative reference sera and positive reference those shown positive results at least in two other tests (test due to be validated was

excluded) are presented in Tables 4 and 5. The DSn’s, Youden’s J values and accuracy of the tests expressed by AUC, presented in Table 4 are higher than those presented in Table 5. The results presented in Table 4 reveal that when as positive reference sera are used these originated from culture positive animals almost all the tests perform the same because all the 95% CI of DSn overlap. Although under these conditions the tests DSn do not differ significant, RBT is the test with the lowest DSn, Youden’s J and accuracy. When a more representative panel of positive reference sera is used (sera positive in two other tests in parallel) the results of calculation of DSn’s, Youden’s J values and accuracy of the tests as they presented in Table 4 are different. The tests FPA, iElisa and m-RBT show higher DSn, Youden’s J value and accuracy than the other used and RBT remains the test with the lowest values for these variables. The results of comparison of test’s performances by Friedman test for repeat measurements and the agreement of the tests as it assessed by Kappa analysis are presented in Table 6. The results in Table 6 reveal that the performance of m-RBT, i-Elisa and FPA does not differ significant (P > 0.05) as well as that of CFT and c-Elisa (P > 0.05). The agreement of i-Elisa and FPA, CFT and c-Elisa, and RBT and CFT is very good (K > 0.8 and <1) since the agreement between the other tests is good (K > 0.6 <0.8).

4. Discussion

Fig. 2. ROC curve diagram of FPA using as positive reference sera with positive results in two serological tests in parallel.

The optimum cut-off value for FPA offering the highest ‘‘performance index’’ (sum of DSn and DSp) was determined by ROC analysis two ways, one using as positive reference, sera originated from culture

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

217

Table 3 DSn and DSp of FPA at different cut-off values calculated by ROC analysis using as positive reference sera with positive results in two serological tests in parallel mP Units

DSn

CI at 95% confidence

DSp

CI at 95% confidence

+LR

72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 * 88

100.0 99.8 99.8 99.5 99.1 99.1 99.0 98.6 98.6 98.5 98.1 97.6 97.4 96.4 95.9 95.9 94.4

99.4–100.0 99.0–100.0 99.0–100.0 98.5–99.9 98.0–99.7 98.0–99.7 97.8–99.6 97.3–99.4 97.3–99.4 97.1–99.3 96.7–99.1 96.0–98.7 95.8–98.6 94.6–97.8 94.0–97.4 94.0–97.4 92.2–96.1

17.3 22.2 27.7 32.0 37.1 43.4 49.6 56.1 63.1 70.2 76.7 83.0 87.4 90.7 95.3 98.9 100.0

14.8–20.0 19.5–25.2 24.7–30.9 28.8–35.2 33.9–40.5 40.0–46.8 46.2–53.0 52.6–59.4 59.8–66.4 67.0–73.2 73.7–79.5 80.3–85.4 85.0–89.6 88.6–92.6 93.7–96.6 98.0–99.5 99.6–100.0

1.21 1.28 1.38 1.46 1.58 1.75 1.96 2.24 2.67 3.30 4.22 5.73 7.75 10.39 20.40 90.68

*

LR 0.00 0.01 0.01 0.02 0.02 0.02 0.02 0.02 0.02 0.02 0.02 0.03 0.03 0.04 0.04 0.04 0.06

Cut-off offers the optimum combination of DSn and DSp.

positive animals (166 sera) and another using as positive reference, sera from animals reared in naturally infected flocks showing positive results simultaneously in two different tests conducted in parallel. Using the two different panels of positive reference sera, the accuracies of all tests were assessed from AUC values calculated by ROC analysis, as well as their performance from determination of Youden’s J value, which is the indicator of test’s DSn and DSp sum, by cross-tabulation (Armitage and Berry, 1994). The FPA’s cut-off, using as positive reference, sera originated from culture positive animals was determined at 87 mP, which is almost identical with that determined by Nielsen et al. (2005) in an evaluation study for FPA’s performance on goat sera. At this cutoff, DSp and DSn were determined at 98.9 and 97.6%,

respectively. The DSp calculated this way in this study does not differ significant from DSp of 98.6, 99.4 and 98.9% found in other studies in sheep and goats by Nielsen et al. (2001, 2005), while DSn is higher than 91.5, 94.9 and 88.7% found in the same studies. The higher DSn of FPA determined in this study must be attributed to selection criterion of positive reference sera. Such differences of FPA’s DSn among studies were observed also by Nielsen et al. (2001, 2005) who attributed these to selection criteria of positive reference sera also. The DSn, DSp, performance and accuracy of the other tests were assessed also, using as positive reference, sera originated from culture positive animals. The results reveal that DSn, performance and accuracy of all tests used, do not differ significant.

Table 4 Test’s performance calculated from the results of negative reference sera (n = 851) and positive reference sera originated from culture positive sheep (n = 166) Tests

DSn

RBT m-RBT CFT i-Elisa c-Elisa FPA

90.36 97.59 98.79 97.59 96.39 97.59

(85.87–94.85) (95.26–99.92) (97.14–100) (95.25–99.23) (93.55–99.23) (95.26–99.92)

Values in brackets presents the CI at 95% confidence.

DSp

Youden’s J

AUC

99.64 (99.25–100) 98.12 (97.21–99.03) 100 (100–100) 100 (100–100) 99.41 (98.9–99.93) 98.94 (98.26–99.63)

0.90 0.96 0.99 0.98 0.96 0.97

0.95 0.98 0.99 0.99 0.97 0.98

(0.86–0.95) (0.93–0.98) (0.97–1.0) (0.95–1.0) (0.93–0.99) (0.94–0.99)

(0.94–0.96) (0.97–0.99) (0.99–1.0) (0.98–0.99) (0.96–0.98) (0.97–0.99)

218

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

Table 5 Test’s performance calculated from the results of negative reference sera (n = 851) and positive reference sera with positive results in two serological tests in parallel Tests

Number of positive reference sera

DSn

DSp

Youden’s J

AUC

RBT m-RBT CFT

645 639 635

i-Elisa c-Elisa FPA

598 633 587

70.54 (67.03–74.06) 89.20 (86.80–91.61) 78.9 (75.72–82.10) 78.58 (75.39–81.77)a 94.48 (92.65–96.31) 80.10 (76.98–83.21) 95.74 (94.11–97.38)

99.65 (99.25–100) 98.12 (97.21–99.03) 100 (100–100) 100 (100–100) 100 (100–100) 99.4 (98.90–99.30) 98.94 (98.26–99.63)

0.70 0.87 0.79 0.79 0.94 0.80 0.95

0.85 0.93 0.89 0.89 0.97 0.90 0.97

(0.67–0.74) (0.85–0.90) (0.76–0.82) (0.75–0.82) (0.93–0.96) (0.76–0.83) (0.93–0.96)

(0.83–0.87) (0.92–0.95) (0.88–0.91) (0.88–0.91) (0.96–0.98) (0.88–0.91) (0.96–0.98)

Values in brackets present the CI at 95% confidence. a Sera with anticomplementary reaction (AC) considered as negative (samples with AC reaction were only in panel of positive reference sera for this reason DSp is not affected).

This would be attributed to the fact that culture positive animals have already generated strong immune response and the existing antibodies in their serum can be easily detected even by tests with poor performance leading to overestimation of their DSn, which leads consequently to overestimation of performance and accuracy. Due to this, CFT appears Table 6 Comparison of the performances of the tests with Friedman test for repeat measurements and agreement of tests with Kappa analysis (n = 3093) Tests

Friedman test

Kappa

RBT vs. m-RBT RBT vs. CFTa RBT vs. CFTb RBT vs. i-Elisa RBT vs. c-Elisa RBT vs. FPA m-RBT vs. CFTa m-RBT vs. CFTb m-RBT vs. i-Elisa m-RBT vs. c-Elisa m-RBT vs. FPA CFT vs. i-Elisa a CFT vs. c-Elisaa CFT vs. FPA a CFT vs. i-Elisa b CFT vs. c-Elisab CFT vs. FPA b i-Elisa vs. c-Elisa i-Elisa vs. FPA c-Elisa vs. FPA

P < 0.0001 P < 0.0001 P < 0.002 P < 0.0001 P < 0.0001 P < 0.0001 P < 0.0001 P < 0.0001 P > 0.05 P < 0.0001 P > 0.05 P < 0.0001 P > 0.05 P < 0.0001 P < 0.0001 P < 0.0001 P < 0.0001 P < 0.0001 P > 0.05 P < 0.0001

0.80 0.80 0.83 0.74 0.78 0.72 0.77 0.77 0.79 0.77 0.78 0.78 0.82 0.77 0.81 0.85 0.80 0.79 0.84 0.78

(0.77–0.84) (0.76–0.83) (0.80–0.87) (0.70–0.77) (0.75–0.82) (0.68–0.75) (0.73–0.80) (0.73–0.80) (0.76–0.83) (0.74–0.81) (0.74–0.81) (0.75–0.82) (0.79–0.86) (0.74–0.81) (0.78–0.84) (0.82–0.89) (0.77–0.84) (0.76–0.83) (0.81–0.88) (0.75–0.82)

The values in brackets is CI of Kappa statistic with 95% confidence. a CFT: the sera with AC reaction considered as positive. b CFT: the sera with AC reaction considered as negative.

to be the test with the highest DSn of 98.79%, which is not in agreement with that of 88.1 and 83.1% found by other researchers (Nielsen, 2001; Blasco et al., 1994). These results confirm the conclusion that determination of a test’s DSn using sera from culture positive animals not representing all the stages of a disease (latent, incubation period and chronic) is leading to overestimation of DSn, which is not realistic for the target population (Jacobson, 1998). Although there is overestimation of all test’s DSn, performance and accuracy, this with the lowest values of these variables is RBT. This justifies the suggestion, RBT not be used as individual test (Blasco et al., 1994). However, although FPA, i-Elisa and m-RBT have the same DSn, taking into account i-Elisa’s Youden’s J and AUC values seems to perform slightly better. Because culture positive animals representing all disease stages is difficult to be obtained from test’s target population, the simultaneous positivity at least in two different serological tests conducted in parallel, was used as alternative criterion for positive reference sera selection (Thrusfield, 1995). Although all serological tests used, yield false positive results, since they measure the same phenomenon, which is interaction of antigen-antibodies. However, in practice if a serum, originated from an animal reared in a flock which infection by B. melitensis has been confirmed by culture and isolation of etiological agent at least from one animal, gives positive results in two serological tests simultaneously conducted in parallel, is more likely to originate from a true infected animal than from one caring cross reacting antibodies.

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

The positive reference sera collected using this criterion can be considered as more representative of the test’s target population, since they were collected after testing all animals >6 months of age in naturally infected flocks selected randomly, and can be assumed that all stages of infection (latent, incubation period and chronic) and all variables influencing the performance of a test as age, pregnancy status, health condition and sex are represented. The cut-off value of FPA determined this way at 87 mP too, but DSn is lower calculated at 95.9%, since DSp remains the same because the negative reference sera are same. Using the above mentioned criterion for selection of positive reference, different panels of sera were collected which were used for assessing DSn, DSp, performance and accuracy of the other tests used in the study. The values of DSn, performance and accuracy of the tests determined using these panel of sera, are significantly lower (Table 5) than those determined with the use of positive reference, sera originated from culture positive animals (Table 4). This difference confirms the overestimation of these variables when it is used as positive reference, sera originated from culture positive animals having high titers of antibodies. The FPA and i-Elisa were the tests with the highest DSn, performance and accuracy, which do not differ significant. Both tests are highly accurate and have high discriminatory ability because their AUC values are between 0.9 and 1 (Greiner, 2000). The AUC value of FPA and i-Elisa at 0.97 indicates that the assays in 97% of the time will correctly identify an animal as positive if its serum has a value greater the determined cut-off. The DSn of RBT and CFT calculated this way are less than 80% classifying these tests as those with the worse performance and lowest accuracy. This finding confirms the observations of Nicoletti (1969) and Blasco et al. (1994), which pointed out the inefficacy of these tests for individual testing of animals. The low DSn observed for CFT may be attributed to inability of certain immunoglobulin isotype (IgG2) to bind guinea pig complement. In some cases this isotype is present in large quantities in sheep sera (McGuire et al., 1979). Concerning DSp of the tests, i-Elisa shows the highest DSp, which, however, does not differ significantly from that of CFTand c-Elisa. The high DSp of i-Elisa may be attributed to anti-IgG1 conjugate that

219

does not react with the immunoglobulins of IgM class, which may be responsible for false positive reactions (Wright and Nielsen, 1990). The DSp of RBT and CFT calculated in this study was the same with those observed in other studies when these tests are used for testing brucellosis officially free animals (Blasco et al., 1994). The test with the lowest DSp, which differs significant from those of other tests, is m-RBT. Since this test has higher DSn than classical RBT this happens in expense of DSp. The false positive reactions observed in this test may be due to improper inactivation of immunoglobulins of IgM class because of the different ratio of serum/antigen. The comparison of test’s performances by Friedman’s test for repeat measurements reveals that FPA, iElisa and m-RBT have equal performance. Also the results obtained by FPA are in very good agreement (K = 0.84) with those obtained by i-Elisa and in good agreement (K = 0.78) with those by m-RBT. During the study it was pointed out that sera of bad quality containing small clots or other particles could give false positive results. This happens because when the light beams pass threw the solution these particles impair their passage, interfering with the measurement. In most of cases these sera give negative results after centrifugion. False positive result can be obtained also if the serum under test is contaminated with bacteria. Such kind of serum gives constantly positive result that does not change after centrifugion perhaps due to interaction of bacteria with the antigen used. The results can be also influenced from the temperature of the reagents, mainly that of the buffer. If the reagents are in low temperature the mP values measured by analyzer are increased.

5. Conclusions FPA is a diagnostic test that can be performed easily and has many advantages compared with the other methods used for serological diagnosis of brucellosis in small ruminants. The results are obtained in a sort period of time, less than 5 min, the instrument calculates the mP value automatically and the interpretation is not subjective, anticomplementary or prozone effects are not observed and hemolysed sera can be tested with the same degree of

220

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221

accuracy. The test can be easily automated which could increase the testing capacity of a field laboratory significantly. Although its accuracy and performance is equal with those of i-Elisa it can be a test of choice because of its simplicity and speed of the results gotten. Its cut-off can be adjusted so the desirable combination of DSn and DSp can be succeeded for different epidemiological situations. Taking in to account that RBT and CFT have proved problems in DSn and CFT is very difficult to be standardized and time consuming there is a need of new tests offering higher DSn’s to be used in order the efficacy of eradication program for brucellosis in small ruminants to be increased. From the new tests developed, FPA is a promising method with high accuracy equal with that of i-Elisa. Since the information available for FPA’s performance in small ruminants are still limited it can be used in eradication programs additionally to the approved tests, so data to be collected from its implementation under field conditions, in order its performance and validity to be assessed more accurate after its extend use under field conditions. Acknowledgements This study was supported in part by Diachemix Corp., Wisconsin, USA, which provided the antigens for the implementation of FPA.

References Alton, G.G., Jones, L.M., Angus, R.D., Verger, J.M., 1988. Techniques for the Brucellosis Laboratory. INRA, Paris, France. Armitage, P., Berry, G., 1994. Statistical Methods in Medical Research, 3rd ed. Blackwell Science, Oxford, 523–524. Blasco, J.M., Garin-Bastuji, B., Marin, C.M., Gerbier, G., Fanlo, J., Jimenes de Bagues, M.P., Cau, C., 1994. Efficacy of different Rose Bengal and complement fixation antigens for the diagnosis of Brucella melitensis infection in sheep and goats. Vet. Rec. 134, 415–420. Corbel, M.J., 1989. Brucellosis epidemiology and prevalence worldwide. In: Young, E.J., Corbel, M.J. (Eds.), Brucellosis: Clinical and Laboratory Aspects. CRC Press, Boca Raton, pp. 25–40. Davies, G., 1971. The Rose Bengal Test. Vet. Rec. 88, 447–449. European Council Directive, 1964. 64/432/EEC. On health problems affecting intra-community trade in bovine animals and swine.

European Council Directive, 1991. 91/68/EEC. On animal health conditions governing intra-community trade in ovine and caprine animals. Garin-Bastuji, B., Blasco, J.M., 1997. Caprine and ovine brucellosis (excluding Brucella ovis infection). In: Manual of Standards for Diagnostic Tests and Vaccines, 3rd ed. Office International des Epizooties, Paris, France, 350–362. Greiner, M., Gardner, I.A., 2000. Epidemiologic issues in the validation of veterinary diagnostic tests. Prev. Vet. Med. 45, 3–22. Greiner, M., Pfeiffer, D., Smith, R.D., 2000. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev. Vet. Med. 45, 23–41. Jacobson, H.R., 1998. Validation of serological assays for the diagnosis of infectious diseases. Rev. Sci. Tech. Off. Int. Epiz. 17 (2), 469–486. Jaques, I., Olivier-Bernardin, V., Dubray, G., 1998. Efficacy of Elisa compared to conventional tests (RBPT and CFT) for the diagnosis of Brucella melitensis infection in sheep. Vet. Microbiol. 64, 61–73. Lin, M., Nielsen, K., 1997. Binding of Brucella abortus lipopolysaccharide O-chain fragment to a monoclonal anibody. Biol. Chem. 272, 2821–2827. MacMillan, A.P., 1990. Conventional serological tests. In: Nielsen, K., Duncan, J.R. (Eds.), Animal Brucellosis. CRC Press, Boca Raton, Florida, pp. 153–197. McGuire, T.C., Musoke, A.J., Kurtti, T., 1979. Functional properties of bovine IgG1 and IgG2: interaction with complement, macrophages, neutrofils and skin. Immunology 38 (2), 249–256. Nicoletti, P., 1969. Further evaluation of serologic test procedures used to diagnose brucellosis. Am. J. Vet. Res. 30, 1811– 1816. Nicoletti, P., 1989. Relationship between animal and human disease. In: Young, E.J., Corbel, M.J. (Eds.), Brucellosis: Clinical and Laboratory Aspects. CRC Press, Boca Raton, pp. 41–51. Nielsen, K., Gall, D., 2001. Fluorescence polarization assay for the diagnosis of brucellosis: a review. J. Immunoassay Immunochem. 22 (3), 183–201. Nielsen, K., Gall, D., Kelly, W., Vigliocco, A., Henning, D., Garcia, M., 1996. Immunoassay development: application to enzyme immunoassay for the diagnosis of brucellosis. Agriculture and Agri-Food monograph A62-45/1996E, ISBN 0662-24163-0. Nielsen, K., Gall, D., Smith, P., Kelly, W., Yeo, J., Kenny, K., Heneghan, T., McNamara, S., Maher, P., O’Connor, J., Walsh, B., Caroll, J., Rojas, X., Rojas, F., Perez, B., Wulff, O., Buffoni, L., Salustio, E., Gregoret, R., Samartino, L., Dajer, A., LunaMartinez, E., 2001. Fluorescence polarization assay for the diagnosis of bovine brucellosis: adaptation to field use. Vet. Microbiol. 80, 163–170. Nielsen, K., Gall, D., Smith, P., Bermudez, R., Moreno, F., Renteria, T., Ruiz, A., Aparicio, L., Vazquez, S., Dajer, A., Luna, E., Samartino, L., Halbert, G., 2005. Evaluation of serological tests for detection of caprine antibody to Brucella melitensis. Small Rumin. Res. 45, 253–258. OIE, 2000. Manual of Standards for Diagnostic Tests and Vaccines. OIE, Paris, 475–489.

A. Minas et al. / Veterinary Microbiology 111 (2005) 211–221 OIE, 2004. Caprine and ovine Brucellosis. In: Terrestrian Animal Health Code, OIE, Paris, Chapter 2.4.2. Schoonjans, F., Zalata, A., Depuydt, C., Comhaize, F., 1995. Medcalc: a new computer program for medical statistics. Comp. Meth. Prog. Biomed. 48, 257–262.

221

Thrusfield, M., 1995. Veterinary Epidemiology, 2nd ed. Blackwell Science, UK, 276. Wright, P.E., Nielsen, H.K., 1990. Current and future serological methods. In: Adams, L.G. (Ed.), Advances in Brucellosis Research. Texas A&M University Press, College Station, pp. 305–332.