Bayesian inference in mixtures of two exponentials

Microelectron. Reliab., Vol. 28, No. 2, pp. 217-221, 1988. Printed in Great Britain.

0026 2714/88S3.00+ .00 Pergamon Press plc

BAYESIAN INFERENCE IN MIXTURES OF TWO EXPONENTIALS M. PANDEY Department of Zoology, Banaras Hindu University, Varanasi 221005, India and S. K. UPADHYAY Department of Statistics, Banaras Hindu University,Varanasi 221005, India

(Received for publication 1 September 1987) Abstraet--This paper provides the Bayes estimators of scale parameters in two exponential distributions mixed in proportion p, which is assumed to be known. For deriving the Bayes estimators the prior distributions are chosen such that they are centered at the known prior values of the scale parameters. The Bayes estimators thus obtained are found to be equivalent to Tbompson's shrinkage estimators. The Bayes estimator of reliability function and another estimator of reliability using the Bayes estimators of scale parameters are given. The efficiencyof the proposed estimator of reliability with respect to its maximum likelihood estimator (mle) is studied by Monte Carlo simulation. Lastly, the Bayes confidence interval of reliability is obtained and its validity is examined by a Monte Carlo study of 500 samples.

1. I N T R O D U C T I O N

the most prominent applications are those in the field of life testing and reliability problems. The probability density function (pdf) of a one parameter exponential distribution is 1 f(x;O) =-~exp[-(x/O)], x >1 0; 0 > 0. (1)

Thompson [1] considered the problem of shrinking an unbiased estimator/~ of a parameter # towards a prior value #0 and suggested the shrunken estimator k/~+(1-k)/~0, with k a constant (0 ~< k~< 1). The estimator is more efficient than/~ if the true value of p is close to #o and is less efficient otherwise. This The parameter 0, called the scale parameter, gives the estimator was further modified by Mehta and average or mean life of the item under study. Thus it is Srinivasan [2], Pandey [3] and Lemmer [4], amongst assumed that the underlying population is a others. Lemmer [4] has considered the question homogeneous one with the failure time distribution whether it would be more realistic to postulate a prior given in (1), where the form f(.) is known but the distribution for # around /~o and use an ordinary parameter 0 is unknown. One should also consider Bayes estimator /1o instead of #o in the shrunken several other models for failure time distributions estimator. The Bayes estimator/i 0 was derived from a which are more complex than the model assumed in prior distribution which places a weight (I - b ) on the (1). Situations may occur when the assumption of prior value #o and distributes the rest of the homogeneity does not hold and the underlying probability mass b (0 ~< b ~< 1) according to some population may consist of several subpopulations probability distribution. He thus obtained the mixed in proportions Pl, P: ..... say. Further, the failure time distribution in each Bayesian shrinkage estimators of binomial, Poisson and normal parameters. The Bayesian shrinkage subpopulation may be given by Fj(t),j = 1, 2,..., with estimators of exponential and Weibull parameters pdt's f~(t), respectively. A more common situation were later considered by Pandey [5] and Pandey and would be when Fi(t ), j = 1, 2..... all have the same form F, but they differ in parameters. A random Upadhyay I-6,7], amongst others. In this paper, for deriving the Bayes estimators we sample of size n is drawn from such a population, have considered whether it would be more realistic to giving X1... X n as failure times of the n items included choose the priors which have means equal to some in the sample. Two different types of situations known prior values. It is found that the Bayes commonly arise. In one case it is possible to assign, estimator of the scale parameter of an exponential either after or at the start of experiment, each unit to distribution thus obtained reduces to the Thompson's the appropriate subpopulation, while in the other shrunken estimator if an inverted gamma prior is such information is not available. This paper considers the simplest case, when there used. These priors are, therefore, utilized to derive the are two subpopulations with exponential densities Bayes estimators of reliability function R(t) and its mixed in proportion p and (1 - p). It is further assumed Bayesian confidence interval when the failure data arise from a mixture of two exponential distributions that items which fail can be classified and can be attributed to the appropriate subpopulation. Thus the with known mixing proportion p. The exponential distribution is a widely exploited pdf of the mixture of two exponentials is given by model in a variety of statistical procedures. Among h(x;01,02) = pf(x;O1)+(1-p)f(x; 02), (2) 217

218

M. PANDEYand S. K. UPADHYAY

where f(x; 01) and f(x; 02) are specified in (1). 0 l and 02 are the two scale parameters. The reliability function R(t) in model (2), which is the probability of failure free operation for at least time t, is given by

where q = ( l - p ) and the mixing proportion p is assumed to be known. The LF can be further written as

In pn~(1--p)n~ L(x;OI'O2'P) - ~ , i n 2 0~'0~~

g(t) = p exp [ - (t/O0] + (1 - p) exp [ - (t/02)] . (3) The maximum likelihood estimator (mle) 0~ of 0~ (i = l, 2), and hence of R(t), can easily be obtained. Mixture models are, however, rather difficult to handle statistically. In particular, formal methods of estimation often run into problems, primarily because there are usually many unknown parameters in the model and the LF is flat in certain regions. It is, therefore, assumed for simplicity that mixing proportion p is known. There is a great deal in the literature on mixtures of exponential distributions rS-ll]. As already discussed, we have derived in this paper the Bayes estimators of 0i (i = 1,2) and R(t) for model (2) by choosing prior densities with means at the known prior values, say 0° (i = 1, 2). We have also derived the conditions when the proposed estimators of 0~ are better than their corresponding mle. The estimator of R(t) is also obtained by substituting the Bayes estimators of 0~ in equation (3). The two reliability estimators are then compared with the corresponding mle on the basis of a Monte Carlo study of 500 samples. Lastly, the Bayesian confidence interval for R(t) is obtained using the marginal posteriors of 0~ (i = 1,2). The performance of the proposed interval is also studied by obtaining the estimates of probability of inclusion (i.e. the ratio of the number of samples for which the true value of R(t) is included in the proposed limits to the total number of samples). It is seen that the proposed estimators are better in such cases and the confidence limits also provide high values of estimates of probabilities of inclusion (PI). 2. E S T I M A T I O N O F P A R A M E T E R S

Let a random sample of size n be drawn from a population with pdf (2). Thus the data consists of n failure times grouped according to the subpopulations

{(x. ..... Xl,.),

(x2~..... x2,0},

In L(X; 01, 02,

P) -- I n l I/'/2 0]10~ 2 exp

~x 01

nl

n2

£1-i=1

and

22-i=1

nl

n2

are the arithmetic means. Let the prior distributions of 01 and independent and given by

o(o,)

1

{al"~ c~+l

=

02

[- ( a l ) ]

exp L-

be

(5)

and

("2Y2+1 a2rc2\02J × exp [ - (~22)], a~, ci > 1; i = 1,2,

(6)

respectively [10]. 2.1. Scale parameters Thus the joint posterior of 01 and

02

is given by

L(x)g(Ox)g(02) p ( 0 1 ' 02) =

~

co

fofoL(X)g(OOg(O2)dOldO2 where L(x) stands for L(x; 01, 02, p). Substituting the values of L(x), g(O~) (i = 1, 2) in p(O 1, 02) and solving we get

1

1

p(ol,o2)= 0~.,+-~,+1) 0~.2+c,+1) +, 221 '"2+c'' r(n2 + c2)

(al+n121) (',+q) r(.l+cO

[_ (el +.12l

exp --

\

01

az +n2£2~]~ +

~-2

,}JJ'

(7)

Integrating out 0 2 and 01 from (7), the marginal posteriors of 0i (i = 1, 2) are

×exp[_(a,+",2,)]

n2 "~

where

1 (a i q'- "i2i) (n'+ c,) p(Oi) = 01,,+c,+1) F(ni+ci)

pn~q,,2 nl

'/222

(4)

X

where it is assumed that nl, n2, (ha + n : = n) are the observed frequencies in the sample of the units belonging to the subpopulations with pdf's f(x; 01) and f(x; 02), respectively. Assuming a complete sample, the likelihood function (LF) is given by

e x p { - { nt£1

x

02

/]J

(8)

Hence, the Bayes estimates of 0 1 and 0 2, i.e. the means

Bayesian inference of two exponentials

R rn = P exp [ -- (t/O1)] + (1 -- p) exp [ - (t/02)]. (16)

of the marginal posteriors of 01 and 0 2, are given by ~t

al + nix1

nl + c l - l '

(9)

~2 = a2 + n2.x2 n2+c2--1

Let the prior values 0 ° of 0 i be available and let the priors be so chosen that the means of the prior distributions of 0 i are equal to these prior values, i.e. 0o _

ai

ci-l'

i = 1,2.

Also, since R(t) is a function of (0~, 02) only, the Bayes estimator of R(t) can be directly obtained with the help of the joint posterior of (01, 02) given in (7). Thus the Bayes estimator of R(t) is

~(t) =

for0

al + n l X 1

(10)

~1 = k l x l +(1 - k O 0 °, )

R(t)p(Ol, 02) dO1 dO2,

which on solving reduces to

Thus, by substituting the values of a~ from (10) in (9) we get

~2

219

..

.f

(hi+Ca)

a2+n2ff 2 ",~(n2+c9

+(1--p)~,a2+n2£2+t )

, (17)

where

k2£2 +(l_k2)O0, ~

(11)

ai=(ci--1)O °,

i = 1,2.

where

k i .--

The corresponding posterior variance of R(t) is

ni n i + c i - 1, i = 1,2.

var (R(t)/x) = E(R2(t)/x)- E(R(t)/x) 2

Now, the unbiased estimator of 0 i, which is also the mle, is given by 0i=xi,

i = 1,2.

= p2q~(xx)+ (1--p)2q~(X2), where

(12)

Thus it is seen that the Bayes estimators obtained in ( l l ) reduce to the Thompson shrunken estimator if we choose the prior distributions with means equal to the prior values. The mean square errors (MSE's) of ~i (i = 1, 2) can be obtained as O2 MSE(~) = k 2V' + (1 - k2)(O° - 032,

(13)

ni

(18)

a~ + n i • xi

")(n,+c3

~(Xi) = k.a i + 2t + n i • 2i,/

( ai+n.ffi ~2(n,+q, - ~,a i + t +~i "--xi,]

i = 1,2.

'

Also, the mle of R(t) is given by /~(t) = p exp [ - (t/£1)] +(1 - p) exp [ - (t/x2). (19) Thus we compare the proposed reliability estimators [(16), (17)] with respect to the mle given in (19). The relative efficiencies are defined as

whereas the variance of 01 is var(0i) = 0~2

(14) E 1 = ref(Rrn,/~(t))

ni

MSE(/~(t)) MSE(RTH)'

Thus ~i is more efficient than 0 i if E 2 = ref(/~(t),/~(t)) - MSE(/~(t)) MSE(/~(t))'

MSE(~,) < var(0i),

(20)

where ref(zl, z2) means relative efficiency of zl with respect to z 2.

which leads to the condition that

2.3. Bayesian confidence interval of R(t) for

i=1,2.

(15)

The expression for reliability can be written as

0o Thus (15) gives the range for ~-/ when the Bayes estimators derived in (11) are more efficient than £~, i = 1,2. 2.2. Reliability function Substituting the values of ~ (i = 1, 2) from equation (11) in (3), an estimator of reliability function may be defined as

R(t) = pRt(t)+(1-p)R2(t),

(21)

where

gi(t ) = exp[--(t/O,)],

i --- 1,2.

The confidence interval of R(t) is obtained by finding the posterior limits for Rl(t ) and R2(t ) separately. Let 11 and ul be two 100(1-~)~o posterior limits of R 1(0 such that

P{I1 < exp[--(t/O0] < ul} = (l--a).

220

M. PANDEYand S. K. UPADHYAY

F o r equal tail posterior limits

Table 1. Values of E 1, E 2 and PI when O°/Oi (i = 1, 2) vary simultaneously O0/Oi

(i = 1,2)

E1

E2

PI

0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

1.038 1.156 1.247 1.135 1.102 0.905 0.883 0.711 0.573 0.481

1.046 1.191 1.346 1.340 1.333 1.155 1.128 0.917 0.749 0.617

0.986 0.996 0.996 0.996 0.996 0.996 0.998 0.988 0.990 0.976

and >~

t

(22)

N o w the first e q u a t i o n of (22) can be simplified as (23)

feS f(y) dy = ~/2, 1

where ~bl=

2(al +nl2l)ln(1/ll) t

Table2. Values of El, E 2 and PI when 01°/01 varies and 0o/02 is fixed at 1.00

and

1 y(nl+q-1) f(y) = F(nl + c l ) 2(,,+cl) e x p [ - - ( y / 2 ) ] . f(y) is the density function of a g a m m a distribution with parameters 2 a n d (nl+cl) [or, equivalently, a chi-square distribution with 2 ( n l + q ) freedom]. Similarly, the second term of (22) gives

degrees of

f~

2 f(Y) dy = ct/2,

(24)

010/01

E1

E2

PI

0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

1.122 1.156 1.196 1.194 1.177 1.216 1.159 1.155 1.037 0.961

1.156 1.191 1.249 1.294 1.317 1.363 1.341 1.371 1.260 1.199

0.990 0.996 0.996 0.990 0.996 0.996 0.994 0.998 0.998 0.998

where ~b2=

Table 3. Values of El, E 2 and PI when 00/02 varies and 0°/01 is fixed at 1.00

2(a 1 + n l ~ l ) I n (l/u1) t

The two equations (23) a n d (24) can be solved for ~b1 a n d ~b2 a n d hence 11 a n d u~, the two limits for R~(t), can be obtained. Again, if it is assumed t h a t 12 a n d u 2 are the two 100(1 -ct)~o posterior limits of R2(t), then

P{12 < exp [-(t/02) ] < u2} = ( 1 - e ) . Thus, the two limits 12 a n d u2 can be determined exactly as 11 a n d u r Hence, the required limits for R(t) [ e q u a t i o n (21)] are RL 1 = pl 1 +(1 -p)12,

RU 1 = pu 1 + ( 1 --p)U 2.

(25)

In other words, RL 1 a n d RU 1 are such t h a t

P { R L 1 < R(t) < RU1} = ( l - e ) . Thus, RL~ a n d R U t give the two limits of the confidence interval of R(t). 3. NUMERICAL RESULTS

A M o n t e Carlo study of 500 samples, from two exponential distributions mixed in p r o p o r t i o n p a n d ( 1 - p ) [cf. e q u a t i o n (2)], was conducted for different values of O°/Oi [0.50(0.50)5.00] in order to study the relative efficiencies E 1 a n d E 2. Two cases have been considered. In one case b o t h O°/Oi (i = 1,2) were

0o/02

E1

E2

PI

0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

1.086 1.156 1.231 1.247 1.245 1.311 1.237 1.196 1.096 0.964

1.113 1.191 1.293 1.368 1.419 1.504 1.468 1.467 1.361 1.225

0.990 0.996 0.996 0.992 0.996 0.996 0.996 0.996 0.996 0.996

varied simultaneously (see Table 1), whereas in the other case one was fixed a n d the other varied (Tables 2 a n d 3). O t h e r values chosen were c 1 = 2.00, c 2 = 2.00, n l = 10, n 2 = 15, t = 3.00, p = 0.40, 01 = 2.00 a n d 02 = 3.00. The ratio 0°/0i was chosen such t h a t a few of the values lie within the range 1 + x f O / n i + 2 / (c i - 1)) a n d a few outside it. The a b o v e M o n t e Carlo study was also extended to o b t a i n the estimates of probability of inclusion in order to study the performance of the proposed Bayesian confidence interval of the reliability function R(t). The estimates of probabilities of inclusion (PI) are shown in Tables 1, 2 a n d 3 for 9 5 ~ posterior limits of R(t). The average values of the posterior variance of R(t) [ e q u a t i o n (18)] for all the 500 samples was found to be

Bayesian inference of two exponentials of the order of 10-2, Thus it is observed from the Tables 1, 2 and 3 that the proposed estimators of reliability [(16), (17)] and the corresponding Bayesian confidence interval are quite good. The first four values of the ratio 0°/0i (i = 1, 2) in the Tables fall within the limits of equation (15). It can also be seen in Table 1 that if we vary both O°/Oi (i = 1,2) simultaneously, the efficiencies become less than unity for values of 0°/0i greater than the upper limit of the range (2.4 approx.) from (15). However, if we fix one of 0°/0i at about 1.00 and vary the other, even outside the range, the efficiencies do not fall below unity (Tables 2 and 3) for O°/Oi up to 5.00. However, E i decreases for increasing values of O°/Oi (i = 1,2) in Tables 2 and 3 and these efficiencies may fall below unity if we increase O°/Oi further. As far as the proposed interval estimator of R(t) is concerned, the PI's are very high for all the considered cases (Tables 1, 2 and 3), showing that the performance of the proposed interval is very satisfactory.

2. 3. 4. 5. 6. 7.

8.

9. 10.

REFERENCES

11. 1. J. R. Thompson, Some shrinkage techniques for

221

estimating the mean, J. Am. statist. Ass. 63, 113-123 (1968). J.S. Mehta and R. Srinivasan, Estimation of the mean by shrinkage to a point, J. Am. statist. Ass. 66, 86-90 (1971). B. N. Pandey, On shrinkage estimation of normal population variance, Commun. Statist. A8(4), 359-365 (1979). H. H. Lemmer, From ordinary to Bayesian shrinkage estimators, S. Aft. statist. J. 15, 57-72 (1981). M. Pandey, A Bayesian approach to shrinkage estimation of Weibull scale parameters, S. Afr. statist. J. (submitted for publication). M. Pandey and S. K. Upadhyay, Bayes shrinkage estimators of Weibull parameters, IEEE Trans. Reliab. 34, 491-494 (1985). M. Pandey and S. K. Upadhyay, Bayesian shrinkage estimation of reliability from censored sample with exponential failure model, S. Afr. statist. J. 19, 21-33 (1985). W. Mendenhall and R. J. Hader, Estimation of parameters of mixed exponentially distributed failure time distributions from censored failure data, Biometrika 45, 504-520 (1958). G. M. Tallis and R. Light, The use of fractional moments for estimating the parameters of a mixed exponential distribution, Technometrics 10, 161-175 (1968). S. K. Sinha and B. K. Kale, Life Testin 9 and Reliability Estimation, Wiley, New Delhi (1980). J.F. Lawless, Statistical Models and Methods for Lifetime Data, Wiley, New York (1982).