A Bayesian approach to estimating the performance of a bovine virus diarrhoea virus (BVDV) antibody ELISA bulk-tank milk test

Preventive Veterinary Medicine 50 (2001) 71±87

A Bayesian approach to estimating the performance of a bovine virus diarrhoea virus (BVDV) antibody ELISA bulk-tank milk test Paul S. Vallea,b,c,*, S. Wayne Martind, Eystein Skjervee a

GENO Breeding and A.I. Association, Post Box 4123, N-2300 Hamar, Norway TINE Norwegian Dairies BA, Post Box 9051, Grùnland, N-0133 Oslo, Norway c Section of Preventive Veterinary Medicine, Department of Large Animal Clinical Sciences, The Norwegian School of Veterinary Sciences, Post Box 8146, Dep., N-0033 Oslo, Norway d Department of Population Medicine, Ontario Veterinary College, University of Guelph, Guelph, Ont., Canada N1G 2W1 e Section of Food Hygiene, Department of Pharmacology, Microbiology, and Food Hygiene, The Norwegian School of Veterinary Science, Post Box 8146, Dep., N-0033 Oslo, Norway b

Received 6 March 2000; accepted 10 April 2001

Abstract We investigated the operational performance (sensitivity and specificity) of a bovine virus diarrhoea virus (BVDV) antibody ELISA bulk-tank milk test for predicting the herd BVDV antibody status in young stock (as a relatively precise indicator of active BVDV infection). The study was based on results from the annual screenings under the Norwegian bovine virus diarrhoea (BVD) control and eradication program, lasting from 1993 to 1997. Empirical information from these annual screenings was the basis for prior assumptions about the true prevalence of youngstock-positive herds. Assumptions about prior distributions for sensitivity and specificity were based on the literature. Improved posterior test performance estimates were achieved applying a Bayesian approach using Gibbs sampling simulation. The simulations were run separately for each year, and yielded median values for sensitivity of 87% at the cut-off used in the BVD program. The posterior distributions were wide indicating much uncertainty in these estimates. The specificity estimates ranged from 79 to 92% and had narrower posterior estimates. The estimates differed by year. When running the same simulation procedures at a lower cut-off Ð after altering the sensitivity and specificity priors Ð the median sensitivity estimates increased to about 95%; the median specificity ranged from 71 to 83%. Due to low prevalence, the Bayesian method lacked power to assess the test sensitivity. A technically simpler descriptive graphing procedure (based on empirical information) provided * Corresponding author. Present address: Post Box 2038, N-6402 Molde, Norway. Tel.: ‡47-7125-1820; fax: ‡47-7124-7402. E-mail address: [email protected] (P.S. Valle).

0167-5877/01/$ ± see front matter # 2001 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 5 8 7 7 ( 0 1 ) 0 0 2 2 1 - 5

72

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

equally useful insight into the bulk-tank milk test performance. # 2001 Elsevier Science B.V. All rights reserved. Keywords: Cattle microbiological disease; Bovine virus diarrhoea virus; Population screening; Bulk-tank milk test; ELISA test; Bayesian; Gibbs sampling

1. Introduction The co-operative Norwegian bovine virus diarrhoea (BVD) control and eradication program (BVD program) carried out its first screening of the Norwegian cattle population in 1993 (Waage et al., 1994, 1997). Since then, the population has been screened annually. The main goals of the BVD program were (1) to map the prevalence of the infection, (2) to maintain a disease-free status in BVD-free areas and herds, and (3) to reduce the BVD incidence to the lowest possible level (Aamodt and Nyberg, 1993). Bovine virus diarrhoea virus (BVDV) infection is listed as a group-B disease (milder infectious disease) on the list of notifiable animal diseases. According to the Norwegian Animal Disease Act, herds diagnosed (or suspected) infected with group-B diseases are to be isolated. This legislation constitutes a key control instrument in the BVD program. Animals persistently infected (PI) with BVDV are regarded as the major source for spread of the infection both within and between herds (Straver et al., 1983; Hartley and Richards, 1988; Liess, 1990; Meyling et al., 1990; Tremblay, 1996; Houe, 1999). Under the BVD program, herds with BVDV antibody-positive young stock (YS, 8±15 months) were suspected to have an active infection (mainly in the form of a PI animal) and were isolated by imposing official movement restrictions (Waage et al., 1994; Valle et al., 2001). In the population of dairy herds, a three-stage test scheme (Alenius et al., 1992) (starting with screening for BVDV antibodies in the bulk-tank milk; BTM) was carried out to locate the herds with BVDV antibody-positive YS (Fig. 1). Herds negative on the BTM test were excluded from further testing that year. Hence, the ability of the BTM test to detect herds with antibody-positive YS was crucial for the entire test scheme performance. The BTM test is a low-cost screening option as opposed to the spot test or YS blood sampling test (a more precise, but also more laborious and costly method) (Houe, 1992, 1994; Bitsch and Rùnsholt, 1995). One problem given the initial use of a

Fig. 1. Flow chart for the co-operative Norwegian BVD control and eradication program three-stage test scheme, starting with an antibody ELISA BTM test. Follow-up tests were carried out by an FCM test (five animals) and subsequently an YS (8±12 (15) months old) blood test if the herd was defined positive on the previous test.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

73

BTM screening is that any herd newly infected and with a BTM result lower than the cutoff will not be detected until the following year. A lower cut-off could be used to include more herds from this group, which is essential for eradication of the agent from the population Ð but this raises the possibility of unnecessarily follow-up testing in BTMpositive but non-infected herds. In this study, we aimed at estimating the operational sensitivity and specificity of the BTM test for herd BVDV antibody YS status at the cut-off chosen by the BVD program. We also evaluated the performance of the test at a lower cut-off, which was used to define sero-conversion in studies of health and reproduction effects of BVDV in Norwegian cattle herds. 2. Materials and methods 2.1. BVD program test scheme Test results from the screening of the entire Norwegian dairy cattle population from 1993 to 1998 were provided by the BVD program in an SAS1 (SAS Institute, Inc., Cary, NC) file. The three-stage serial test scheme (Fig. 1) applied by the BVD program has been carried out annually since 1993. The individual first-calvers' milk (FCM) and YS samples were pooled before analysis. All samples were analysed using an indirect antibody enzyme-linked immunosorbent assay (ELISA) test kit (SVANOVIR1, Svanova Biotech, Uppsala, Sweden) (Juntti et al., 1987; Niskanen et al., 1989, 1991; Niskanen, 1993). On the continuous scale of sample-to-positive (S/P) ratio (i.e. calculated optical density, cOD) yielded by this test, cut-off values for a positive test were defined. The selected cut-offs for the BTM, FCM, and YS test were initially set to S/P ratios 0.25, 0.10, and 0.25, respectively. Each year, some herds with BTM S/P ratios below the selected cut-off had YS samples tested. There was an increasing tendency of such testing towards the end of the study period. 2.2. Prior assumptions 2.2.1. Young-stock prevalence The true YS status in the population of dairy herds for a given year was unknown. However, we believed that there was a low probability for an YS-infected herd to be missed by two consecutive annual BTM screenings. This belief was based on the following: (1) in the subsequent year, the YS would be 1-year older and in most herds would be subject to screening by the BTM as first calvers, and (2) infected positive YS are associated closely with the presence of a PI animal (Houe, 1992, 1994; Bitsch and Rùnsholt, 1995). In addition, the presence of a PI animal is expected to cause seroconversion of a number of the animals within the herd during a 1-year period (Barber et al., 1985; Meyling et al., 1990; Moerman et al., 1993). Based on these assumptions, we estimated the prior distributions for the prevalence of YS-infected herds. We set a likely lower (minimum) value for this prior to be equal to the proportion of herds actually testing YS-positive, and we estimated a likely upper (maximum) value by adding the

74

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

number of herds not tested by the YS test in a given year, but found FCM-positive the subsequent year Ð or YS-positive if the FCM test was missing. This last criterion was included because the FCM test was skipped in some circumstances (e.g. to avoid the consequential time delay before testing the YS). It was not likely that all of these latter herds were YS-positive at the time of the previous BTM test, because some herds would have become YS- and/or FCM-positive between the two consecutive BTM tests. Because no information was available to make a qualified guess whether the true prevalence was closer to the likely minimum or maximum, we decided to take the mean of these estimated values as an estimate for the mean proportion (figures given as percentage) of YS-positive herds. The 95% range of the distribution around the mean was set to 3% (i.e. two times a standard deviation (S.D.) of 1.5). In this way, the estimated likely minimum and maximum were included in the range of the prior distribution. Two additional sets of simulations were run for the S/P ratio 0.25 cut-off with the aim of evaluating the effect of the different prevalence priors (using the likely minimum and maximum prevalence estimates as the mean of the priors). The 95% range in these cases was increased to 6% (corresponding to an S.D. of 3%) Ð once again to include the likely minimum and maximum in the prior distribution. 2.2.2. Sensitivity and specificity The ELISA test used by the Norwegian BVD program was compared to a serumneutralisation test and had good agreement (Juntti et al., 1987), but its performance as a herd level test for the herd YS has not been assessed. However, Niskanen (1993) evaluated the test agreement with the individual antibody status of cows contributing to the BTM in a group of 123 Swedish dairy herds. We used these data to produce prior assumptions of the expected values for the sensitivity at the S/P ratios 0.25 and 0.10 of 85 and 92%, respectively. A wide distribution was assigned around these means (95% range ˆ mean of prior  20%; corresponding to an S.D. of 10%). The specificity for YS clearly will be less than the 100% levels calculated by Niskanen (1993) (100% at both cut-off's). If we assume (as a worst case) that all herds above 0.25 in 1993 and 1998 were false positives, then the specificity would be 70 and 90%, respectively. Clearly, some of these were actually infected so the actual specificity must be above these figures. Based on a study by Bitsch and Rùnsholt (1995) evaluating the Danish BTM ELISA blocking test (Rùnsholt et al., 1997) for herd YS status, the BTM test specificity corresponding to a sensitivity of 85 and 92% was set to 85 and 80%, respectively, for all years. Once again, we chose a wide prior distribution Ð though not as wide as for the sensitivity (95% range ˆ prior ofmean  14%, corresponding to an S.D. of 7%). The choice of a more narrow specificity prior relates to the likely lower limits for the specificity provided above. 2.2.3. Descriptive statistics Descriptive statistics for the BTM S/P ratio were calculated using PROC UNIVARIATE (SAS1) for (1) herds testing YS-positive in each of three BTM S/P ratio categories: S/P ratio: <0.10, 0.1 to <0.25, and 0.25 to (2) for herds not YS-tested, but found FCM- or YSpositive the subsequent year, and (3) for all other herds (essentially, YS-negative). S/Pratio histograms using PROC GPLOT (SAS1) were made for each of the categories above,

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

75

and for the group of herds in either category 1 or category 2. The latter group is the basis for the estimated maximum number of YS-positive herds. Also, the proportion of YSpositive herds in category 1 or in either category 1 or 2 above a given S/P ratio Ð mimicking the sensitivity Ð and the proportion of YS-negative herds (category 3) below a given S/P ratio Ð mimicking the specificity Ð were graphed. 2.2.4. Gibbs sampling simulation Joseph et al. (1995) suggested a simulation procedure using Gibbs sampling to estimate test performance in situations without a gold standard or knowledge about the true status of the disease of interest. Joseph also has provided an S-plus program (http:// www.epi.mcgill.ca/web2/bstJoseph.html), where the prior distributions for prevalence, sensitivity, specificity, and the number of test-positive and test-negative units (herds in this case) are the required input values. The prior distributions (i.e. mean and S.D.) were converted to Beta densities (a and b parameters) required by the program (Table 1). (A conversion macro is included in Joseph et al. (1995) program.) The region of this family of densities matches the ranges of the parameters of interest (0±1), and is also a flexible family allowing many density shapes depending on the mean and range (S.D.) for test parameters. Without a gold standard, we had a situation with missing information; analysis of such data can be analysed by a frequentist approach (Walter and Irwing, 1988; Enùe et al., 2000). However, when estimating test parameters using only one test and having no gold standard, constraints have to be imposed on a subset of the parameters when applying maximum likelihood methods. Joseph et al. (1995) state that ``The basic idea behind the Bayesian approach presented here is to eliminate the need for these constraints by first constructing a prior distribution over all unknown quantities. The data are then, through the likelihood function, combined with the prior distributions to derive Table 1 Estimates for the mean and standard deviation (S.D.) and corresponding Beta density parameters (a, b) of the prior distributions for the annual prevalence of BVDV antibody-positive YS in Norwegian dairy herds, and the applied prior distributions for the sensitivity and specificity of a ELISA BTM test at the S/P ratio cut-off used by the co-operative Norwegian BVD control and eradication program …S=P ratio ˆ 0:25† and at a selected lower cut-off …S=P ratio ˆ 0:10† Mean

S.D.

a

b

47.8 26.1 20.2 6.8 3.9

386.4 300.0 268.2 162.9 124.5

YS prevalence 1993 1994 1995 1996 1997

11 8 7 4 3

Sensitivity at S/P ratio 0.25 at S/P ratio 0.10

85 92

10 10

10.0 5.9

1.8 0.5

Specificity at S/P ratio 0.25 at S/P ratio 0.10

85 80

7 7

21.7 25.3

3.8 6.3

1.5 1.5 1.5 1.5 1.5

76

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

posterior distributions using Bayes' theorem''. By using the Gibbs sampler (Gelfand and Smith, 1990; Tanner, 1991) (which is an iterative Markov-chain Monte Carlo technique for approximating analytically intractable posterior densities), inferences about posterior distributions could be made. For further insight into the theoretical basis and algorithms, see Joseph et al. (1995). The resulting estimates for the posterior distributions of BTM test sensitivity and specificity were summarised using the median, first and third quartiles, and 2.5th and 97.5th percentiles. 3. Results 3.1. Descriptive statistics There was a decline in the number of herds tested over the study period due to farmers going out of business (Table 2). As expected, only a proportion of the herds above the BTM cut-off for follow-up testing had been YS-tested; the rest were likely FCMnegative. Herds actually testing YS-positive were present in all three BTM S/P ranges. Herds not YS-tested but testing FCM- or YS-positive the following year, also were found in all BTM ranges. Based on these data, minimum and maximum values for the prior prevalence distribution (with rounded means for the priors in brackets (all in percentages)) were calculated as 8 and 13% (11), 6 and 10% (8), 6 and 7% (7), 3 and 4% (4), 2 and 3% (3) for each year from 1993 to 1997 (Table 1). The numbers of BTMtest-positive and -test-negative herds for each year and both cut-offs assessed, are found in the first columns of Table 2. For the herds in the lowest BTM-S/P group, the median BTM values increased slightly over the 5-year period. Among the herds in the lowest BTM group and not YS-tested Ð but found FCM/YS-positive the subsequent year Ð 75% had an S=P ratio < 0:05 in the initial year. Out of the herds not YS-tested in a given year but found FCM- or YS-positive the next year, 1.4, 3.1, 2.9, 10.5, and 30.9% (actual numbers are 18/1293, 38/1240, 10/342, 32/305, and 34/110, respectively) did not have a BTM S/P ratio above the 0.25 cut-off. The histograms (Fig. 2) showed that the frequency for herds that were below the S/P ratio of 0.25, testing YS-positive (the lowest line) and herds testing YS-positive plus herds not YS-tested (but testing FCM/YS-positive the subsequent year; the line shifting higher) had a marked discrepancy (distance between the lines) for the first year of the program (1993). For the later years (e.g. 1997 in Fig. 2), the frequency of herds actually testing positive came closer to the line assumed to represent a maximum for the prior prevalence distribution. At higher S/P ratios, all three groups showed parallel shapes. The graphs mimicking the sensitivity and specificity (Fig. 3) show a similar discrepancy between the two positive groups (tested YS-positive and tested plus assumed YS-positive) for the first 2 years (i.e. the curves are not coincident). For the following years, the lines for the two groups comes together. We believe that the sensitivity should lie between the lines of these two positive groups Ð and over the period, we observed a shift in the shape of the distribution of positive herds. Within the first 3 years, the sensitivity at an S/P ratio of 0.25 appears to be >90%, but for the last year (1997), it drops

Table 2 BVDV antibody S/P ratio from a BVDV ELISA test applied on BTM samples from Norwegian dairy herds for the years 1993±1997a Year

No. of tested

Tested YS-positive

Subsequently found FCM/YS-positive

No. of positive/No. of tested (%)

S/P ratio Median

Q1, Q3

b

No. (%)

S/P ratio Median

Q1, Q3

Assumed YS-negative No.

S/P ratio Median

Q1, Q3

1993

1 2 3

17781 2529 6076

8/39 (21) 12/32 (38) 2078/3796 (55)

0.001 0.21 0.53

0, 0.02 0.14, 0.24 0.43, 1.14

414 (2) 336 (13) 1139 (19)

0.004 0.17 0.49

0, 0.04 0.14, 0.21 0.33, 0.62

17359 2181 2859

0 0.16 0.41

0, 0.006 0.13, 0.20 0.32, 0.52

1994

1 2 3

17580 2181 6387

15/53 (28) 25/64 (39) 1420/3330 (43)

0.01 0.15 0.58

0, 0.08 0.14, 0.21 0.45, 0.72

298 (2) 244 (11) 1298 (20)

0.002 0.18 0.57

0, 0.05 0.15, 0.22 0.44, 0.69

17267 1912 3669

0 0.17 0.45

0, 0.004 0.13, 0.21 0.34, 0.58

1995

1 2 3

17135 1873 6576

35/154 (23) 42/138 (30) 1486/4205 (35)

0.003 0.17 0.65

0, 0.09 0.16, 0.21 0.56, 0.80

76 (0.4) 51 (3) 656 (10)

0.001 0.16 0.64

0, 0.01 0.14, 0.20 0.48, 0.77

17024 1780 4434

0 0.17 0.49

0, 0.006 0.13, 0.21 0.36, 0.64

1996

1 2 3

19150 2448 3574

19/219 (9) 82/335 (24) 668/2238 (30)

0.04 0.19 0.49

0.004, 0.06 0.16, 0.221 0.39, 0.61

97 (0.5) 114 (5) 338 (9)

0.006 0.19 0.41

0, 0.05 0.14, 0.22 0.32, 0.54

19034 2252 2568

0 0.16 0.4

0, 0.009 0.13, 0.20 0.32, 0.52

1997

1 2 3

20024 2366 2480

28/305 (9) 66/365 (18) 439/1568 (30)

0.06 0.19 0.42

0.004, 0.08 0.15, 0.21 0.34, 0.51

49 (0.2) 52 (2) 163 (7)

0.02 0.19 0.42

0.001, 0.06 0.15, 0.22 0.32, 0.52

19947 2248 1878

0 0.16 0.36

0, 0.01 0.13, 0.2 0.3, 0.44

a

This BTM test was the initial test in the three-stage test scheme used by the co-operative Norwegian BVD control and eradication program. The results are provided separating the dairy herds into three categories (1) herds testing BVDV antibody-positive on an YS sample, (2) herds not YS-tested, but testing BVDV antibody-positive on FCM sample or YS-positive in the subsequent year, and (3) herds not in either of these two categories, hence, assumed YS-negative. Within these three categories, the herds are divided further into groups 1, 2, and 3 according to their BTM S/P ratio: <0.10, 0.10 to <0.25 and 0.25, respectively. b First and third quartile, respectively.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

BTM Group

77

78

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

Fig. 2. Histograms for S/P ratio from a BVDV antibody BTM ELISA test used by the co-operative Norwegian BVD control and eradication program to rule in dairy herds with a positive BVDV antibody YS status. Three YS categories are graphed: (1) herds testing YS-positive (line marked with diamond, ^); (2) herds testing YSpositive plus those not YS-tested, but found FCM- or YS-positive (if FCM missing) the subsequent year (line marked with open circles, *); (3) herds not being in the previous category (2) (line marked with stars, ). The histograms are truncated at 1200; the frequencies for category 3 in 1993±1997 (excluding 1995) at S/P ratio equals 0 were 16 258, 16 320, 17 640, and 18 427, respectively. The year 1995 is not presented due to test problems for this year.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

79

Fig. 2. (Continued ).

below this level; apparently, a larger proportion of these YS-positive herds had lower S/P ratios. At the same time, the specificity increased over the 5-year period. The year 1995 is distinct from the others with both a relatively low specificity and sensitivity compared to the other years. (The authors became aware that a laboratory materials problem had occurred during this year.) The sensitivities for the BTM using only the actual YS test results decreased from 99% for the first years to 82% for the last year.

80

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

Fig. 3. The proportion of YS-positive herds Ð in two categories Ð being above a given BVDV antibody S/P ratio as measured by a BVDV ELISA BTM test used by the co-operative Norwegian BVD control and eradication program, lasting from 1993 to 1997 Ð mimicking the BTM sensitivity. The two categories are (1) herds testing YS-positive (line marked with diamond, ^); (2) herds testing YS-positive plus herds not YS-tested but testing FCM-test-positive or YS-positive (if FCM missing) the following year (line marked with open circles, *). Further, the proportion of YS-negative herds (herds not being in category 2 above (line marked with stars, ) being below a given S/P ratio is graphed) mimicking the BTM specificity. Due to a test problem in 1995, this year is not included.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

81

Fig. 3. (Continued ).

3.2. Gibbs sampling simulation The simulation results are given for the high and low cut-off levels in Tables 3 and 4, respectively. The sensitivity posteriors showed wide ranges not very different from the range of priors. The median value for test sensitivity tended to lie around 87% for the cutoff chosen by the BVD program (i.e. 0.25). The median values for test specificity ranged

82

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

Table 3 Gibbs sampling simulation posterior estimates for the median, first and third quartile (Q1, Q3), and upper and lower 2.5% percentiles (i.e. 95% CI) for the prevalence of dairy herds with a BVDV antibody-positive YS for the years 1993 to 1997, are displayeda Year

1993 1994 1995 1996 1997

Prevalence

Sensitivity

Specificity

Median

Q2, Q3

CI

Median

Q2, Q3

CI

Median

Q2, Q3

CI

11 9 7 4 2

10, 8, 6, 3, 2,

8±14 6±13 4±11 2±7 1±5

85 87 88 89 86

76, 79, 80, 82, 78,

51±98 61±98 62±98 63±99 60±98

84 82 79 89 92

83, 81, 78, 88, 91,

80±88 79±85 77±82 87±91 91±94

12 10 8 5 3

91 92 93 94 92

85 83 80 90 93

a Also, posterior estimates for the performance (i.e. sensitivity, specificity, positive, and negative predictive value) of the BVDV antibody BTM ELISA test with respect to the herd YS status simulated at an S/P ratio of 0.25 are provided. The S/P ratio of 0.25 was the cut-off used for follow-up testing by the co-operative Norwegian BVD control and eradication program lasting from 1993 to 1997.

Table 4 Gibbs sampling simulation posterior estimates for the median, first and third quartile (Q1, Q3), and upper and lower 2.5% percentiles (i.e. 95% CI) for the prevalence of dairy herds with a BVDV antibody-positive YS for the years 1993±1997, are displayeda Year

1993 1994 1995 1996 1997

Prevalence

Sensitivity

Specificity

Median

Q2±Q3

CI

Median

Q2±Q3

CI

Median

Q2±Q3

CI

11 9 7 4 3

10±12 8±10 6±8 3±5 2±4

9±14 6±12 4±10 1±9 1±6

97 96 95 97 95

93±99 90±99 88±99 90±99 86±99

76±99 66±100 65±100 70±100 62±100

75 73 71 79 82

74±76 72±74 71±72 78±80 82±83

72±78 70±76 69±74 77±83 81±85

a Also, posterior estimates for the performance (i.e. sensitivity, specificity, positive, and negative predictive value) of a BVDV antibody BTM ELISA test with respect to the herd YS status simulated at an S/P ratio of 0.10 are provided. The S/P ratio of 0.10 was used as a cut-off for defining a herd as sero-converted in studies evaluating effects of BVDV on health, reproduction, and production, presented elsewhere.

from 79 to 92%, and showed improved posterior distributions (i.e. narrower limits). The specificity estimates differed between different years in the study period (nonoverlapping 2.5th to 97.5th percentile range). The estimated sensitivity at the selected lower cut-off (0.10) increased to a median value of about 95% whereas the specificity dropped, ranging from 71 to 83% (Table 4). When using the minimum for the prior prevalence distribution as the mean for an alternative prior-prevalence distribution, the sensitivity and specificity estimates (at an S/ P ratio of 0.25 for the 5 years) ranged from 86 to 89% and 78 to 91%, respectively; when using the maximum, these ranged from 84 to 88% and 81 to 91%. The medians for the estimates were at maximum three integers away from the original estimates (Table 3), …average ˆ 1:35†, indicating a low impact of these alterations in prior-prevalence distributions.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

83

4. Discussion BTM sampling is used in several countries for screening of dairy populations for BVDV (Lindberg and Alenius, 1999). Detection of the infective source Ð the PI animals (Tremblay, 1996; Houe, 1999) Ð is the main task for this screening. Due to their reduced life span, PI animals are mainly found among YS (Houe, 1993). Houe (1992, 1994) found the YS spot test to be relatively accurate (sensitivity and specificity of 93 and 100%, respectively; Houe, 1999). This was evaluated in Danish diary herds using a blockingELISA test (Rùnsholt et al., 1997). In high-prevalence areas, it is likely to be more cost efficient to use a more direct test such as the YS spot test (Houe, 1999) or the FCM test as the primary screen. However, individual animal sampling in every dairy farm in the national dairy herd population involves considerable expense. By using BTM sampling as the initial screening tool (as suggested by Alenius et al., 1992), the cost can be kept low. When evaluating the histograms of S/P ratios versus YS status, we observed an overlap of the distributions in negative and positive herds (most pronounced towards the end of the study period). An ideal test would be expected to show two distinct distributions with little or no overlap (Martin et al., 1987). Apparently, the BTM test does not distinguish the actively infected herds from herds earlier infected, but cleared. This is a direct consequence of the nature of serologic data being footprints of an infection that Ð due to a lengthy and maybe lifelong immunity (Karhs, 1981; Duffell and Harkness, 1985; Fredriksen et al., 1999) Ð can be long gone when the testing is done. Directly estimating the BTM test sensitivity and specificity of the BTM was difficult because the frequentist approaches (Walter and Irwing, 1988) require that either the specificity or the sensitivity and prevalence are known. When this study was initiated in 1997, the YS-positive herd prevalence was about 2%. Therefore, performing a field study to achieve a gold standard with regards to the BTM test performance would have demanded a large sample size (and could not be performed due to the costs). Based on the apparent prevalence, we knew that the specificity must be at least 70 and 90% for 1993 and 1997, respectively. Thus, we used the method suggested by Joseph et al. (1995) as an alternative approach to better estimate both sensitivity and specificity. However, the Bayesian approach also carries weaknesses, which we briefly will address later in this paper. Initially, due to cost considerations, the BVD program officials decided not use an S/P ratio of, e.g. 0.1 to achieve a higher sensitivity, because the inverse relation between sensitivity and specificity (Martin et al., 1987) would have lead to a drop in the specificity (causing a dramatic increase in the number of false-positive herds included for follow-up testing). Towards the end of the study period (given the decreased prevalence), the actual extra number enrolled for further testing would be less, and this was also the reason the BVD program officials found it possible to lower the cut-off from 0.25 to 0.15 for the 1998 screening. It can be argued that a high sensitivity is not a must given the repeated (annual) testing. A herd having an ongoing/active infection Ð but missed the first year Ð would (as we initially claimed) be detected the next year (the mean proportion of infected animals needed for an S=P ratio > 0:25 is reported to be only 6.5%; Niskanen, 1993). However, the concern associated with this approach Ð especially in the late stage in a control program with a low herd sero-prevalence (herd immunity) Ð is the harm the

84

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

recently infected herd may cause if not detected and isolated (Brownlie, 1996; Synge et al., 1995). During the BVD program, officials have recommended more testing of herds below the BVD program cut-off if infectious contacts were suspected because of risk factors such as purchase of animals, common pasture, or over-the-fence contact (Valle et al., 1999). Therefore, the available data give a better picture of the YS situation at the end of the study period compared to (especially) the first year. Apparently, there was a drop in the sensitivity for the last 2 years. The sensitivity was highest in 1995 and at the same time the specificity was at its lowest. Due to test problems reported in 1995, these data probably are not representative for the test. An assumption used for the frequentist approaches (reviewed by Walter and Irwing, 1988) is that sensitivity and specificity remain constant across population subgroups, and in particular, they do not vary with changes in disease prevalence. This assumption apparently does not hold for the BVDV antibody BTM test; however, this might be because we do not have a true gold standard. Chriel and Willeberg (1997) also found both the test sensitivity and specificity to be affected by the prevalence when applying the present test evaluation method on a slaughterhouse test procedure. We believe that the observed differences in specificity and sensitivity might be related to a shift in the population dynamics (nature of the disease) caused by the BVD program selectively only following up on herds above the S/P ratio of 0.25, and associated with the long immunity achieved in transiently infected animals (Karhs, 1981; Duffell and Harkness, 1985; Fredriksen et al., 1999). At the end of the study period, there were relatively more recently infected herds or herds that for other reasons would have low BTM titres. The method we used to obtain estimates of the sensitivity and specificity of the BTM test has been criticised for not behaving reasonably when the sample size increases, and also because the posterior distributions were extremely dependent on the priors (Andersen, 1997). Though some effect of sample size is present, it is evident that even with large sample sizes (as in this case), precautions must be taken when specifying priors (Joseph, 1997). The dependency on priors is an unfortunate ``state of nature'' for diagnostic test problems when having imperfect knowledge (Joseph, 1997). Because we had a situation with a low prevalence (causing the data to contain little information about the test sensitivity), more weight on the prior was required. The BTM test sensitivity was of major interest; therefore, we needed strong assumptions for at least one of the other parameters. Based on empirical information, we made the strongest prior assumptions regarding the prevalence, keeping the test parameter distributions more flexible. However, as a consequence of the low prevalences, the posterior ranges were very wide (signalling a large uncertainty). The estimation procedure returned median values similar to the means of the priors for the sensitivity estimate Ð reflecting, we believe, the dependency between prior and posterior distributions. By changing the prior prevalence distributions, we noted that a higher value caused a higher estimate for the posterior sensitivity and vice versa. Based on the simulation, estimates of the sensitivity of the BTM test appeared to remain stable over the period. However, the graphing procedure indicated otherwise, and we also observed that the sensitivity based on the actual YS test results for the last year was below the sensitivity yielded by the estimation procedure.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

85

For the specificity estimates, yearly differences were indicated both by the simulation procedure and the graphing procedure. For the posterior-specificity estimates, the dependency on prior estimates was not as great as with sensitivity. Also, the specificity estimates were more in agreement with the empirical information. As stated above, this is likely an effect of little power for sensitivity estimation at low prevalence. In addition to the method used to obtain the BTM test characteristics, other factors such as herd characteristics can affect the spread of the infection (Radostits and Littlejohns, 1988), and therefore also (probably) the sensitivity of the BTM. In Norway, most herds have tie-stalls, so less contact between animals would be expected compared to a cubicle (free-stall) system. However, YS are (as a rule) housed in the same open room with the milking cows, which would increase transmission of BVDV. Another factor potentially affecting the test sensitivity is the herd size. Norwegian dairy herds have an annual average of 12.9 cow-years, and hence an individual animal will represent a larger proportion of the BTM (and therefore, a smaller dilution effect). A higher sensitivity of the BTM test may, therefore, be expected compared to in dairy populations with larger herd sizes (unless the larger herd size also enhances the spread of infection within the herd). Clearly, the BVDV infection can enter a herd without initially affecting milking cows (e.g. by purchasing a PI YS or YS becoming infected at YS pasture; Valle et al., 1999). The different routes by which the agent enters the herd might affect the BTM test sensitivity. Also, a time lag between an active infection in the YS and this being reflected in the BTM would be expected in a proportion of the infected herds. On the other hand, the more effective spread of the infection in herds with PI animals (Meyling et al., 1990; Moerman et al., 1993; Tremblay, 1996; Houe, 1999) would make us expect a higher frequency of antibody-positive animals in YS-positive herds (as shown by Houe et al., 1995) causing a higher probability of being enrolled for follow-up testing. However, BVDV also can be present at relatively low BTM levels (Drew et al., 1999). Our simulation indicates a higher sensitivity for YS status than reported by Niskanen (1993). Whether this is related to the effect of the presence of an active infection or the different herd characteristics remains unknown. Based on the data in Table 2, we have no doubt that the BVD program using the BTM test has helped lower the frequency of YS-positive herds. The effects of the control program on the population dynamics and the changes in prevalence apparently affected the specificity and the sensitivity of the BTM test. We conclude that the applied Bayesian approach was useful for assessing test specificity. However, when the prevalence approaches 0 or 100%, the certainty of either the sensitivity or specificity estimates will suffer. It is also evident that the method depends on well-specified prior distributions Ð especially so for a one-test evaluation. There also appears to be valuable information in applying simple descriptive methods to empirical data (if these can approach a gold standard representative). One might be tempted to claim that for our data (and with respect to test sensitivity estimation), the simple graphing procedure mimicking the sensitivity (by using the data for minimum and maximum prevalence estimation) outperformed the technically more complex Bayesian Gibbs sampling simulation. (We refer the reader to the paper by McDermott (1995) recommending a better use of basic methods.)

86

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

Acknowledgements Paul S. Valle received financial support from GENO Breeding and A.I. Association, TINE Norwegian Dairies BA, and Norwegian Research Council. The BVD program provided data with assistance from BVD program manager Ola Nyberg. References Aamodt, O., Nyberg, O., 1993. Prosjektplan, Informasjon, Framdriftsplan og Ansvarsfordeling for Prosjekt: Kontroll og Bekjempelse av BVD/MD hos Storfe (in Norwegian). Veterinñrinstituttet, Oslo. Alenius, S., Larsson, B., Niskanen, R., Jacobsson, S.O., 1992. Bovint virusdiarreÂvirus hos noÈtkreatur: symtom, diagnostik, profylax och bekaÈmpande. Svensk Veterinñrtidning (in Swedish) 44, 51±62. Andersen, S., 1997. Bayesian estimation of disease prevalence: the parameters of diagnostic tests in the absence of a gold standard. Am. J. Epidemiol. 145, 290±291. Barber, D.M.L., Nettleton, P.F., Herring, J.A., 1985. Disease in a dairy herd associated with the introduction and spread of bovine virus diarrhoea virus. Vet. Rec. 117, 459±464. Bitsch, V., Rùnsholt, L., 1995. Control of bovine viral diarrhoea virus infection without application of vaccines. Vet. Clin. North Am. Food Anim. Pract. 11, 627±640. Brownlie, J., 1996. Bovine virus diarrhoea virus vaccines and vaccination. Cattle Pract. 4, 45±49. Chriel, M., Willeberg, P., 1997. Dependency between sensitivity, specificity. and prevalence analysed by means of Gibbs sampling. EpideÂmiologie et Sante Animale 31±32, 12.03.1±12.03.3. Drew, T.W., Yapp, F., Paton, D.J., 1999. The detection of bovine virus in bulk milk samples by the use of a single-tube RT-PCR. Vet. Microbiol. 64, 145±154. Duffell, S.J., Harkness, J.W., 1985. Bovine virus diarrhoea-mucosal disease infection in cattle. Vet. Rec. 117, 240±245. Enùe, C., Georgiadis, M.P., Johnson, W.O., 2000. Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Prev. Vet. Med. 45, 61±81. Fredriksen, B., Sandvik, T., Lùken, T., édegaard, S.A., 1999. Level and duration of serum antibodies in cattle infected experimentally and naturally with bovine virus diarrhoea virus. Vet. Rec. 144, 111±114. Gelfand, A.E., Smith, A.F.M., 1990. Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398±409. Hartley, P.E., Richards, M.S., 1988. A study of the transmission of bovine virus diarrhoea virus between and within cattle herds. Proc. 5th Int. Symp. Vet. Epidem. Econ. Acta Vet. Scand., (Suppl. 84) 164±166. Houe, H., 1992. Serologic analysis of a small herd sample to predict presence or absence of animals persistently infected with bovine viral diarrhoea virus (BVDV) in dairy herds. Res. Vet. Sci. 53, 320±323. Houe, H., 1993. Survivorship of animals persistently infected with bovine virus diarrhoea virus (BVDV). Prev. Vet. Med. 15, 275±283. Houe, H., 1994. Bovine virus diarrhoea virus: detection of Danish dairy herds with persistently infected animals by means of a screening test of 10 young stock. Prev. Vet. Med. 19, 241±248. Houe, H., 1999. Epidemiological features and economical importance of bovine virus diarrhoea virus (BVDV) infections. Vet. Microbiol. 64, 89±107. Houe, H., Baker, J.C., Maes, R.K., Ruegg, P.L., Lloyd, J.W., 1995. Application of antibody titers against bovine viral diarrhoea virus (BVDV) as a measure to detect herds with cattle persistently infected with BVDV. J. Vet. Diagn. Invest. 7, 327±332. Joseph, L., 1997. The first author reply. Am. J. Epidemiol. 145, 291. Joseph, L., Gyorkos, T., Coupal, L., 1995. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. Am. J. Epidemiol. 141, 263±272. Juntti, N., Larsson, B., Fossum, C., 1987. The use of monoclonal antibodies in enzyme-linked immunosorbent assays for detection of antibodies to bovine viral diarrhoea virus. J. Vet. Med. B 34, 356±363. Karhs, R.F., 1981. Viral Diseases of Cattle. Iowa State University Press, Ames, IA. Liess, B., 1990. Bovine viral diarrhoea virus. In: Dinter, Z., Morein, B. (Eds.), Virus Infections of Ruminants. Elsevier, Amsterdam, pp. 247±266.

P.S. Valle et al. / Preventive Veterinary Medicine 50 (2001) 71±87

87

Lindberg, A., Alenius, S., 1999. Principles for eradication of bovine viral diarrhoea virus (BVDV) infections in cattle populations. Vet. Microbiol. 64, 197±222. Martin, S.W., Meek, A.H., Willeberg, P., 1987. Veterinary Epidemiology: Principles and Methods. Iowa State University Press, Ames, IA. McDermott, J.J., 1995. Progress in Analytic Methods Ð More Sophistication or Back to Basics? Prev. Vet. Med. 25, 121±133. Meyling, A., Houe, H., Jensen, A.M., 1990. Epidemiology of bovine virus diarrhoea virus. Rev. Sci. Technol. Off. Int. Epizootiol. 9, 75±93. Moerman, A., Straver, P.J., DeJong, M.C.M., Quak, J., Baanvinger, T., VanOirschot, J.T., 1993. A long-term epidemiologic study of bovine viral diarrhoea in a large herd of dairy cattle. Vet. Rec. 132, 622±626. Niskanen, R., 1993. Relationship between the levels of antibodies to bovine viral diarrhoea virus in bulk tank milk and the prevalence of cows exposed to the virus. Vet. Rec. 133, 341±344. Niskanen, R., Alenius, S., Larsson, B., Juntti, N., 1989. Evaluation of an enzyme-linked immunosorbent assay for detection of antibodies to bovine virus diarrhoea virus in milk. J. Vet. Med. Ser. B 36, 113±118. Niskanen, R., Alenius, S., Larsson, B., Jacobsson, S.O., 1991. Determination of level of antibodies to bovine virus diarrhoea virus (BVDV) in bulk tank milk as a tool in the diagnosis and prophylaxis of BVDV infections in dairy herds. Arch. Virol. (Suppl. 3) 245±251. Radostits, O.M., Littlejohns, I.R., 1988. New concepts in the pathogenesis, diagnosis, and control of diseases caused by bovine viral diarrhoea virus. Can. Vet. J. 29, 513±528. Rùnsholt, L., Nylin, B., Bitsch, V., 1997. A BVDV antigen- and antibody-blocking ELISA (DVIV) system used in a Danish voluntary eradication program. In: Edwards, S., Wensvoort, G. (Eds.), Proceedings of the Third ESVV Symposium on Pestivirus Infections, Lelystad, Weybridge, 1996, pp. 150±153. Straver, P.J., Journee, D.L.H., Binkhorst, G.J., 1983. Neurological disorders, virus persistence, and hypomyelination in calves due to intrauterine infections with bovine virus diarrhoea virus. II. Virology and epizootiology. Vet. Quart. 5, 156±164. Synge, B.A., Bond, J.M., Nettleton, P.F., Herring, J.A., Moar, J.A.E., Murrey, L., Nicolson, J.T., 1995. A pilot scheme for the control of bovine virus diarrhoea virus in Shetland. Cattle Pract. 3, 385±391. Tanner, M.A., 1991. Tools for Statistical Inferences. Springer, New York. Tremblay, R., 1996. Transmission of bovine viral diarrhoea virus. Vet. Med. 91, 858. Valle, P.S., Martin, S.W., Tremblay, R., Bateman, K., 1999. Factors associated with being a BVD seropositive dairy herd in Mùre and Romsdal County of Norway. Prev. Vet. Med. 40, 165±177. Valle, P.S., Martin, S.W., Skjerve, E., 2001. The co-operative Norwegian bovine virus diarrhoea (BVD) control and eradication program, lasting from 1993 through 1997 Ð a descriptive study. Acta. Vet. Scand., in press. Waage, S., Krogsrud, J., Nyberg, O., 1994. The Norwegian program for eradication of bovine viral diarrhoea/ mucosal disease. In: Proceedings of the 18th World Buiatrics Congress: 26th Congress of the Italian Association of Buiatrics, Bologne, 773 pp. Waage, S., Krogsrud, J., Nyberg, O., Sandvik, T., 1997. Results achieved by a national programme for the eradication of bovine virus diarrhoea. In: Edwards, S., Wensvoort, G. (Eds.), Proceedings of the Third ESVV Symposium on Pestivirus Infections, Lelystad, Weybridge, 1996, pp. 170±172. Walter, S.D., Irwing, L.M., 1988. Estimation of test error rates, disease prevalence, and relative risk from misclassified data: a review. J. Clin. Epidemiol. 41, 923±937.