- Email: [email protected]
A new confidence interval for the difference of two binomial proportions
Computational North-Holland
Statistics & Data Analysts 9 (1990) 237-241
: 37
A new confidence interval for the difference of two binomial proportions Michael CON LON Unroersq01Flori&. Garneswlle,
FL 32610,
USA
Ronald G. THOMAS Ccwperarrue
hdies
Program
Corwdrnarrng
Cenrer,
VA Medrcol
Cenrer,
Palo Ah.
CA,
US.4
Received December 1988 Revised June 1989 Absrracr: A new confidence interval for the difference of two binomial proportions from independent samples is developed. Tbe derivation dwnds on neither asymptotic nor conditional argu-
mew. The inkrval is shown to be equivalent to a bootstrap interval constructed using infiite bootstrap sampling. but can also be motivated using fust principles. An example is given and numerical comparison with akmativc mshds is provided. Keywords: Bootstrap, Chi-square test, Comparative
trial.
1. Introduction
Consider the 2 x 2 comparative trial [Barnard (1947)]. In this experimental setting we have two independent samples of size n, and n2. Each sampled individual is categorized as either a success or a faihtre. Let S, denote the observed number of successes in the i tb sample, i = 1, 2. The model assumes that S, and s2 are observed outcomes of independent binomially distributed random variables with probabilities of success p, and pz respectively. In this note we are concerned with constructing a confidence interval for the difference of two binomial proportions, that is, A = p, - p2. A new procedure is motivated through an intuitive approach using first principles but can also also be derived through an infinite bootstrap argument.
2. Metboddog) For any observed data, (So, sz), our goal in constructing a confidence interval is to indicate which are the likely values for b. Values of A occur in the interval 0167~9473/9O/S3.50
0 1990. Elsevier Science Publishers B.V. (No&-Holland)
M. Cordon. R.G.
238
Thomas / A new confidence
inren~al
[ - I. I]. Values of 3 which are unlikely and small will constitute the portion of [ - 1, I] lo the left of the confidence interval for A. Values of A which are unlikely and large will constitute the portion of [ - 1. 11 to the right of the confidence interval for J. The remaining segment of [ - 1. 11 will be a confidence interval for J. In the following development we will be concerned with symmetric two-sided intervals. that is. fc, any confidence level CY,we will want the left segment to contain a/2 of the probability for A and the right segment to also contain a/2 of I he probability for A. Under the joint binomial model, each possible outcome, (s;. s;). is contained on a lattice of points 9= (0. . . . , n, ) x (0. . . . , n2 ). The observed data occurs on the lattice with an unknown probability which is a function of p,. p2, n,, and n2. Since II, and n2 are known, we need only estimate p, and pz to be able to estimate the probability of the data falling on any point on the lattice. The maximum likelihood estimate of p, is i=
a, = s,/n,.
1.2.
In the cases s, = 0 and s, = II, sui:able corrections for B, are applied [Louis (1981)]. Using these estimates we can consistently estimate the probability of observing any point. say (s,‘. si) on the lattice as
i( s, = s;. s, = s;, =
(:;1
_jj)“‘-‘;
j$(l
() :;
&$l
_j2)~?-G~
In addition. each point on the lattice can be used to construct a corresponding statistic d’ = s;/n, - s;/n2. Let D represent a discrete random variable defined over the possible values of d’. For each possible value d’ we can estimate the probability of observing D = d’ by summing the estimated probability defiied above over all the lattice points that produce the value d’. That is.
k(D=d’)=
c ((5;. si)E~.s;/n,-sj/n!=d’)
B(s,
= s;, s, = s;).
This amounts to an estimated probability density function for the discrete random variable D. For every possible value of A on the lattice Y WC can estimate a probability. We can easily find the segment of the interval [ -1, l] correspondi.~g to unlikely and small values of A by summing these probabilities starting from the smallest values of d’ until no more than a/2 of the estimated probability has been accumulated. In like manner we can sum the estimated probabilities starting from the highest values of A to accumulate at most a/2 of the estimated probability from above. Algebraically, we have 3, = maxd(
c
i(D=d’)
ax/Z).
(d’sd) ALP= min,. (
c (d’md)
b(D=d’)sa/Z).
M. Conlon. R.G. Thomar
/ A new confidence mren~af
239
3. An example Suppose a sample of 60 subjects are given a new drug and 18 of the subjects respond to the treatment. An additional 60 subjects are given a standard treatment and 6 of these subjects respond. We wish to form a confidence interval for the difference in the response rate under the two treatments. The set 9 contains 61 X 61 = 3721 possible tables. We estimate a probability for each one using @, = 18/60 = 0.3 and p2 = 6/60 = 0.1. Each possible table (i.e. each point on the lattice) yields a possible value of d’. Tables yielding equivalent values of d’ are used to estimate the probability of d’. The largest d’ which has an estimated cumulative probability of less than 0.025 is 0.05. For this d’. the estimated cumulative probability is 0.022. The next largest d’ is 0.067 with an estimated cumulative probability of 0.038. By linear interpolation, we estimate the value 0.062 has a cumulative probability of 0.025. The upper bound is calculated in an analogous fashion. The value 0.350 has an estimated cumulative probability of 0.978. The value 0.333 has an estimated cumulative probability of 0.962. By linear interpolation, we estimate the value 0.339 to have a cumulative probability of 0.975. The new confidence interval is thus (0.062. 0.339). A confidence interval calculated using the Hauck and Anderson (1986) best asymptotic corrected normal statistic is (0.069, 0.348). The technique has been implemented in a program for the IBM/PC which is available from the first author on request. The calculations performed for this example take less than 1 second to complete using a 12Mh.z 80286 computer. A recursive algorithm which avoids calculating results for tables wh.i:h have very small probability mass is used. For any k. summing only tables with P 1 k/( n,t~ z). produces an error in the calculations of less than k.
4. Equivalence to an infiiite bootstrap Consider
using a parametric boorstrap approach [Efron (1982)] for the formulainterval for A. la the parametric bootstrap. bootslrap sample data, say (SF, ST) are generated using the joint binomial probability distribution with p, and pz estimated as before through maximum Likelihood. That is. !Icn ~!f a confidence
wilh B, = s/n,.
i = 1. 2.
we generate B independent pairs (SF, s+ ). For each pair we calculate d l = SF/~, - s,*/n,. We accumulate these d l into an empirical distribution function G+. A confidence interval is formed by finding Lhe (ri2 and
To perform a bootstrap
M. Cottlon. R.G. Thomas / A new conpdence rrlrenwl
?a2
1 - a/2 percentage poinls of G*. We will show that G* converges to 6, the distribution function corresponding to jj used in the new interval as described previously. Consider any particular value of D. say do. Under the parametric bootstrap. ~*(d,)_#of(s;.~~)s~hthatd’~d,.
This is simply the fraction of bootstrap data points that fall on the lattice such that their corresponding d * values are less than or equal to d,. After B bootstrap samples, the number of bootstrap data points at vy lattice point (s;. si) is binornially distributed with parameters B and P(S, =s;, + = s;). As B goes to co the fraction of points at any (s;. s;) converges to P(S, =s;, s, = s;). Thus as B goes to co the fraction of bootstrap data points on any set of lattice converges to the sum of the probabilities of those lattice points. In particular.
=
6( d,).
Rather than generating B sample;, we can evaluate F directly at each point and calculate inte_rvals from G rather than G*. In fact, we can do than that since many P are very small and do not contribute amottnts accumulating. Thus the lattice can be investigated only in a neighbourhood
lattice better worth of (-r,,
s: )a
5. Performance The performance of confidence intervals can be assessed by establishing true values for the parameters, in this case, n,, n2. p, and p2. and determining the coverage probability and expected interval length. Other parameters can and have been considered. See Conlon and Thomas (1986) for a more e..tensive treatrnent of performance measures and performance results. Determining coverage probability f, and expected interval length E(L) can be done without resort to simulation. Exact values for these measures can be computed by enumerating all possible cases and weighting each by the probability that they occur given the actual values of the parameters. Table 1 shows the probability of coverage. P,. and the expected interval length, E(L). for three methods of forming a 95% mnfidence interval for A. The Conditional Method of Santner and Snell (1980), the Infinite Bootstrap method introduced in this article and the Asymptotic Corrected Normal interval of Hauck and Anderson (1986) are compared for a variety of binomial parameters.
M. Conlon. R.G. Thomas / A new conjrdenre men-al
241
Table I Performance results for three .\ intervals.
“2
“I
5
5
5 I5 40
5 I5 10 15
40 40 40
40 40 40
9
PI
0.50 0.95 0.65 0.80 0.95 0.35 0.50 0.65
P2
0.20 0.35 0.05 0.50 0.65 0.35 0.50 0.05
Conditional
fine-hod infinite bootstrap
Aqmplotlc
normal
P,
E(L)
p,
E( 1.1
P,
0.997 0.999 0.999 0.992 1.000 1.000 I .OOO 1.000
1.26 1.11 0.92 I.01 0.86 0.50 0.48 0.89
0.987 (I.999 0.955 0.942 0.932 0.948 0.943 0.947
I.12 I .09 0.55 0.77 0.48 0.41 0.43 0.32
0.999 0.999 0.955 0.960
1.17 1.06 0.56 0.82
0.938 0.943 0.954 0.950
0.50 0.42 0.44 0.33
The new method outperforms the best asymptolic interval in a variety of settings and consislenlly produces shorl intervals with near nominal coverage probability for larger sample sizes.
6. Summary A new method for computing confidence intervals for the difference of two binomial proportions has been described. Tlris intenlal is not dependent on asymptotic distribution theory and is easily motivated. The interval is equivalent to the results that would be obtained from using a parametric bootstrap confidence interval with infinite bootstrap samples.
Referemes Barnard, G.A. (1947). Significance tests for 2 x 2 tables. B~omerrrcs -34. 123-38. ConIon. M. and Thomas. RG. (1986). A comparison of the performance of confidence intervals for the difference of two binomial proportions, Technical Report 271. Depi. of Statistics. University 0r Florida.
Gainesville.
FL.
Efron, B. (1979). 7;he Jackknije, The Boorswap, and Other Resamphng Plans. SIAM. Pluladelphia. Hauck. W.W.. Anderson, S. (1986). A comparison of large-sample confidence interval methods for the difference of two biiormal probabii ties. 7%. Arnerrcun S~arlsrrrrun 40. 318 - 322. Louis, T.A. (1981) Confidence intervals for a binomial parameter after observmg no successes. The American Stattsfrcian
35. 181.
Sanmer, TJ. and Snell, M.K. (1980). SmaI sample confidence tntervals for p ,-p2 and pl /pr 2 x 2 wntingemcy tables. 1. Am4ncan Sratrsrical Assrrrrarron 15. 386-394.
rn
Statistics & Data Analysts 9 (1990) 237-241
: 37
A new confidence interval for the difference of two binomial proportions Michael CON LON Unroersq01Flori&. Garneswlle,
FL 32610,
USA
Ronald G. THOMAS Ccwperarrue
hdies
Program
Corwdrnarrng
Cenrer,
VA Medrcol
Cenrer,
Palo Ah.
CA,
US.4
Received December 1988 Revised June 1989 Absrracr: A new confidence interval for the difference of two binomial proportions from independent samples is developed. Tbe derivation dwnds on neither asymptotic nor conditional argu-
mew. The inkrval is shown to be equivalent to a bootstrap interval constructed using infiite bootstrap sampling. but can also be motivated using fust principles. An example is given and numerical comparison with akmativc mshds is provided. Keywords: Bootstrap, Chi-square test, Comparative
trial.
1. Introduction
Consider the 2 x 2 comparative trial [Barnard (1947)]. In this experimental setting we have two independent samples of size n, and n2. Each sampled individual is categorized as either a success or a faihtre. Let S, denote the observed number of successes in the i tb sample, i = 1, 2. The model assumes that S, and s2 are observed outcomes of independent binomially distributed random variables with probabilities of success p, and pz respectively. In this note we are concerned with constructing a confidence interval for the difference of two binomial proportions, that is, A = p, - p2. A new procedure is motivated through an intuitive approach using first principles but can also also be derived through an infinite bootstrap argument.
2. Metboddog) For any observed data, (So, sz), our goal in constructing a confidence interval is to indicate which are the likely values for b. Values of A occur in the interval 0167~9473/9O/S3.50
0 1990. Elsevier Science Publishers B.V. (No&-Holland)
M. Cordon. R.G.
238
Thomas / A new confidence
inren~al
[ - I. I]. Values of 3 which are unlikely and small will constitute the portion of [ - 1, I] lo the left of the confidence interval for A. Values of A which are unlikely and large will constitute the portion of [ - 1. 11 to the right of the confidence interval for J. The remaining segment of [ - 1. 11 will be a confidence interval for J. In the following development we will be concerned with symmetric two-sided intervals. that is. fc, any confidence level CY,we will want the left segment to contain a/2 of the probability for A and the right segment to also contain a/2 of I he probability for A. Under the joint binomial model, each possible outcome, (s;. s;). is contained on a lattice of points 9= (0. . . . , n, ) x (0. . . . , n2 ). The observed data occurs on the lattice with an unknown probability which is a function of p,. p2, n,, and n2. Since II, and n2 are known, we need only estimate p, and pz to be able to estimate the probability of the data falling on any point on the lattice. The maximum likelihood estimate of p, is i=
a, = s,/n,.
1.2.
In the cases s, = 0 and s, = II, sui:able corrections for B, are applied [Louis (1981)]. Using these estimates we can consistently estimate the probability of observing any point. say (s,‘. si) on the lattice as
i( s, = s;. s, = s;, =
(:;1
_jj)“‘-‘;
j$(l
() :;
&$l
_j2)~?-G~
In addition. each point on the lattice can be used to construct a corresponding statistic d’ = s;/n, - s;/n2. Let D represent a discrete random variable defined over the possible values of d’. For each possible value d’ we can estimate the probability of observing D = d’ by summing the estimated probability defiied above over all the lattice points that produce the value d’. That is.
k(D=d’)=
c ((5;. si)E~.s;/n,-sj/n!=d’)
B(s,
= s;, s, = s;).
This amounts to an estimated probability density function for the discrete random variable D. For every possible value of A on the lattice Y WC can estimate a probability. We can easily find the segment of the interval [ -1, l] correspondi.~g to unlikely and small values of A by summing these probabilities starting from the smallest values of d’ until no more than a/2 of the estimated probability has been accumulated. In like manner we can sum the estimated probabilities starting from the highest values of A to accumulate at most a/2 of the estimated probability from above. Algebraically, we have 3, = maxd(
c
i(D=d’)
ax/Z).
(d’sd) ALP= min,. (
c (d’md)
b(D=d’)sa/Z).
M. Conlon. R.G. Thomar
/ A new confidence mren~af
239
3. An example Suppose a sample of 60 subjects are given a new drug and 18 of the subjects respond to the treatment. An additional 60 subjects are given a standard treatment and 6 of these subjects respond. We wish to form a confidence interval for the difference in the response rate under the two treatments. The set 9 contains 61 X 61 = 3721 possible tables. We estimate a probability for each one using @, = 18/60 = 0.3 and p2 = 6/60 = 0.1. Each possible table (i.e. each point on the lattice) yields a possible value of d’. Tables yielding equivalent values of d’ are used to estimate the probability of d’. The largest d’ which has an estimated cumulative probability of less than 0.025 is 0.05. For this d’. the estimated cumulative probability is 0.022. The next largest d’ is 0.067 with an estimated cumulative probability of 0.038. By linear interpolation, we estimate the value 0.062 has a cumulative probability of 0.025. The upper bound is calculated in an analogous fashion. The value 0.350 has an estimated cumulative probability of 0.978. The value 0.333 has an estimated cumulative probability of 0.962. By linear interpolation, we estimate the value 0.339 to have a cumulative probability of 0.975. The new confidence interval is thus (0.062. 0.339). A confidence interval calculated using the Hauck and Anderson (1986) best asymptotic corrected normal statistic is (0.069, 0.348). The technique has been implemented in a program for the IBM/PC which is available from the first author on request. The calculations performed for this example take less than 1 second to complete using a 12Mh.z 80286 computer. A recursive algorithm which avoids calculating results for tables wh.i:h have very small probability mass is used. For any k. summing only tables with P 1 k/( n,t~ z). produces an error in the calculations of less than k.
4. Equivalence to an infiiite bootstrap Consider
using a parametric boorstrap approach [Efron (1982)] for the formulainterval for A. la the parametric bootstrap. bootslrap sample data, say (SF, ST) are generated using the joint binomial probability distribution with p, and pz estimated as before through maximum Likelihood. That is. !Icn ~!f a confidence
wilh B, = s/n,.
i = 1. 2.
we generate B independent pairs (SF, s+ ). For each pair we calculate d l = SF/~, - s,*/n,. We accumulate these d l into an empirical distribution function G+. A confidence interval is formed by finding Lhe (ri2 and
To perform a bootstrap
M. Cottlon. R.G. Thomas / A new conpdence rrlrenwl
?a2
1 - a/2 percentage poinls of G*. We will show that G* converges to 6, the distribution function corresponding to jj used in the new interval as described previously. Consider any particular value of D. say do. Under the parametric bootstrap. ~*(d,)_#of(s;.~~)s~hthatd’~d,.
This is simply the fraction of bootstrap data points that fall on the lattice such that their corresponding d * values are less than or equal to d,. After B bootstrap samples, the number of bootstrap data points at vy lattice point (s;. si) is binornially distributed with parameters B and P(S, =s;, + = s;). As B goes to co the fraction of points at any (s;. s;) converges to P(S, =s;, s, = s;). Thus as B goes to co the fraction of bootstrap data points on any set of lattice converges to the sum of the probabilities of those lattice points. In particular.
=
6( d,).
Rather than generating B sample;, we can evaluate F directly at each point and calculate inte_rvals from G rather than G*. In fact, we can do than that since many P are very small and do not contribute amottnts accumulating. Thus the lattice can be investigated only in a neighbourhood
lattice better worth of (-r,,
s: )a
5. Performance The performance of confidence intervals can be assessed by establishing true values for the parameters, in this case, n,, n2. p, and p2. and determining the coverage probability and expected interval length. Other parameters can and have been considered. See Conlon and Thomas (1986) for a more e..tensive treatrnent of performance measures and performance results. Determining coverage probability f, and expected interval length E(L) can be done without resort to simulation. Exact values for these measures can be computed by enumerating all possible cases and weighting each by the probability that they occur given the actual values of the parameters. Table 1 shows the probability of coverage. P,. and the expected interval length, E(L). for three methods of forming a 95% mnfidence interval for A. The Conditional Method of Santner and Snell (1980), the Infinite Bootstrap method introduced in this article and the Asymptotic Corrected Normal interval of Hauck and Anderson (1986) are compared for a variety of binomial parameters.
M. Conlon. R.G. Thomas / A new conjrdenre men-al
241
Table I Performance results for three .\ intervals.
“2
“I
5
5
5 I5 40
5 I5 10 15
40 40 40
40 40 40
9
PI
0.50 0.95 0.65 0.80 0.95 0.35 0.50 0.65
P2
0.20 0.35 0.05 0.50 0.65 0.35 0.50 0.05
Conditional
fine-hod infinite bootstrap
Aqmplotlc
normal
P,
E(L)
p,
E( 1.1
P,
0.997 0.999 0.999 0.992 1.000 1.000 I .OOO 1.000
1.26 1.11 0.92 I.01 0.86 0.50 0.48 0.89
0.987 (I.999 0.955 0.942 0.932 0.948 0.943 0.947
I.12 I .09 0.55 0.77 0.48 0.41 0.43 0.32
0.999 0.999 0.955 0.960
1.17 1.06 0.56 0.82
0.938 0.943 0.954 0.950
0.50 0.42 0.44 0.33
The new method outperforms the best asymptolic interval in a variety of settings and consislenlly produces shorl intervals with near nominal coverage probability for larger sample sizes.
6. Summary A new method for computing confidence intervals for the difference of two binomial proportions has been described. Tlris intenlal is not dependent on asymptotic distribution theory and is easily motivated. The interval is equivalent to the results that would be obtained from using a parametric bootstrap confidence interval with infinite bootstrap samples.
Referemes Barnard, G.A. (1947). Significance tests for 2 x 2 tables. B~omerrrcs -34. 123-38. ConIon. M. and Thomas. RG. (1986). A comparison of the performance of confidence intervals for the difference of two binomial proportions, Technical Report 271. Depi. of Statistics. University 0r Florida.
Gainesville.
FL.
Efron, B. (1979). 7;he Jackknije, The Boorswap, and Other Resamphng Plans. SIAM. Pluladelphia. Hauck. W.W.. Anderson, S. (1986). A comparison of large-sample confidence interval methods for the difference of two biiormal probabii ties. 7%. Arnerrcun S~arlsrrrrun 40. 318 - 322. Louis, T.A. (1981) Confidence intervals for a binomial parameter after observmg no successes. The American Stattsfrcian
35. 181.
Sanmer, TJ. and Snell, M.K. (1980). SmaI sample confidence tntervals for p ,-p2 and pl /pr 2 x 2 wntingemcy tables. 1. Am4ncan Sratrsrical Assrrrrarron 15. 386-394.
rn