- Email: [email protected]
On maximum likelihood estimation for the two parameter Weibull distribution
Microelectron. Reliab., Vol. 36, No. 5, pp. 595-603, 1996
~ )
Copyright © 1996 Published by Elsevier Science Ltd Printed in Great Britain. All rights reserved 0026-2714/96 $15.00+ .00
Pergamon
0026-2714(95)00171-9 ON MAXIMUM LIKELIHOOD ESTIMATION FOR THE TWO PARAMETER WEIBULL DISTRIBUTION A.J.WATKINS Statistics
and OperationalResearch GroupEBMS, University of Wales, Swansea SA2 8PP, U.K.
(Receivedfor publication 22 August 1995)
Abstract This paper examines recent results presented on maximum likelihood estimation for the two parameter Weibutl distribution. In particular, we seek to explain some recently reported values for estimator bias when the data for analysis contains both times to failure and censored times in operation; our discussion centres on the generation of sample data sets. We conclude that, under appropriate conditions, estimators are asymptotically unbiased, with relatively low bias in small to moderate samples. We then present the results of some further experiments which suggest that the previously reported values for estimator bias can be attributed to the method of generating sample data sets in simulation experiments.
1. I n t r o d u c t i o n This paper examines recent results presented by Wang and Keats [3] on estimation for the two parameter Weibull distribution; for simplicity, we adopt their notation. Given the central role of likelihood and likelihood-based estimators in statistics in general, and in the analysis of quantitative reliability data in particular, we focus here on the reported values for bias in maximum likelihood estimators of the two parameter Weibull distribution when the data for analysis contains both times to failure and censored times in operation. General results based on the principles of likelihood (see, for example, Cox and Hinkley, [1]) mean that the maximum likelihood estimators of the shape and scale parameters /~ and 0 based on an independent and identically distributed sample of size n should be asymptotically unbiased, with variances attaining the appropriate 595
596
A. J. Watkins Cramer-Rao lower bound: expressions for the asymptotic bias in /~ and 0 are both
O(n 1).
Thus, for large samples, it is impossible to out-perform maximum likeli-
hood estimators in terms of estimator accuracy in any systematic way. However, we emphasise that these results are asymptotic; it is known that the bias observed for estimates calculated in small to moderate samples need not be insignificant. This small-sample behaviour, together with other issues such as the computational effort required to calculate estimators, has prompted research into alternative methods for estimating the parameters of the Weibull distribution. For instance, some statistical packages calculate estimates by applying least squares methods to hazard plots, while Seki and Yokoyama [2] have proposed a percentile-based technique, which has been further explored by Wang and Keats [3]. It is natural to assess the relative merits of such alternative techniques by comparisons with maximum likelihood estimators; the general lack of analytical progress then means that comparisons are based on simulation experiments. We note that some experiments seem to indicate that several alternative methods may provide acceptable results in small samples; however, we do not pursue these comparisons here. We are, however, concerned with the biases reported by Wang and Keats in the final columns of their Tables 3 and 4 in [3]. The structure of this remainder of this paper is as follows: we first consider their results in detail, and raise some points of concern. We then outline our attempts to reproduce these results; this leads to a discussion on the generation of sample data sets comprising both times to failure and censored times in operation. We next show that, under appropriate conditions, the maximum likelihood estimators have relatively low bias in small to moderate samples, and we confirm that these estimators are asymptotically unbiased. We finally present the results of
Sample data
bias in ~
bias in
complete
0.0730
0.9964
censored - case 1
1.0463
72.6282
censored - c a s e 2
-0.3046
80.2096
censored - c a s e 3
0.1319
64.0827
Table 2.1: Reported biases in maximum likelihood estimators for n = 20, fl -- 1,0 = 100 from Wang and Keats
Two parameter Weibull distribution some further experiments which suggest that the values for estimator bias reported by Wang and Keats in Tables 3 and 4 in [3] can be attributed to the method of generating sample data sets in simulation experiments; we argue that this simulation method is inherently biased.
2. On R e p o r t e d Values Of Estimator Bias We consider the biases reported by Wang and Keats in the final columns of Tables 3 and 4 in [3]. As these Tables are based on three cases with n = 20, it is not reasonable to expect the asymptotic arguments to hold fully, and we should therefore be prepared to see some evidence of non-zero bias. However, there is a striking difference between the magnitude of these entries and their counterparts in Tables 1 and 2, based on
complete samples in which all data are times to failure. For instance, with n --- 20,/3 = 1,0 = 100, Wang and Keats report the biases reproduced in Table 2.1, and we are immediately struck by the large biases reported for 0; however, the values reported for ~ also give rise for concern, since this parameter characteristics the shape of the Weibull distribution, and is often interpreted in terms of failure patterns such as infant mortality and wear-out. The overall implication of the values in Table 2.1 is that there are serious problems with the method of maximum likelihood, particularly in the three cases involving some form of censored data.
3. On D a t a Simulation We now discuss our attempts to reproduce the values in Table 2.1, and take this opportunity to separate two issues which are often presented together. First, we note that if U is a uniform random variate on [0, 1], then V = - log e U follows the standard negative exponential distribution, with probability density function
e x p ( - v ) , v > 0,
and W = OV a/t~ = 0 [ - log e U] x/~ follows a Weibull distribution with probability density function
zw - e
exp
w ___ O;
597
598
A. J. Watkins t h a t is, the two p a r a m e t e r Weibull distribution with shape ~ a n d scale 0. Thus, given a mechanism for generating pseudo-random uniform variates, we can easily obtain pseudo-random variates from a Weibull distribution with specified shape a n d scale; this allows us to generate a complete sample t l, t2, ..., t~. T h e second issue is t h a t of generating a sample in which some observations correspond to items censored in service; we now start with a complete sample tl, t2, ..., t~, consider how items with these times to failure would be affected by a specified censoring regime, and outline the consequent changes to the ti.
• Type I censoring: the trial finishes at some specified time - to, say. Here, any item for which ti <_ t¢ is deemed to have failed at t h a t time, while any item with
ti > tc is now regarded as censored, with observed time in service to.
• Type H censoring, the trial finishes after a specified n u m b e r of failures - n l , say - have been observed. In this case, we may rank the sample in ascending order, thus defining the order statistics t(l), t(2), ..., t(~). Then, the first n s order statistics are the observed failures, and, as the last failure is deemed to occur at t(~I) , we m a y regard the remaining n - nf items as censored, each with observed time in service t(~1).
• Random censoring: each item is tested for some r a n d o m time, a n d thus has a chance of being withdrawn from service before failure. This scenario is usually modelled by defining a second distribution for times under test, and, for each item, generating a variate zi from this distribution together with the Weibull variate ti. If ti <_ zi, then the item is observed to fail at ti; otherwise, the item is regarded as censored, with observed time in service zi.
We emphasise t h a t these three procedures each involve the replacement of some of the original times ti by other quantities which reflect the censoring regime of interest.
4. On O b s e r v e d Values Of E s t i m a t o r Bias T h e second censored case in Wang and Keats [3] is based on censoring the last eight observations; we therefore consider our second case above with n = 20, n f = 12, 0 -100, and the same range of shape parameters used in [3]. Our results, also based on
Two parameter Weibull distribution 10000 replications for each value of/3, are summarised in Table 4.1, and this shows a very different picture fi'om that reported in Table 2.1; essentially, the biases reported here for estimators from samples subjected to Type II censoring are of the same order as those for estimators from complete samples. To confirm that the estimators are asymptotically unbiased, we may repeat the above experiment with larger values of n - each time taking nf = -~; for brevity, we focus on fl = 1. Our results are summarised in Table 4.2, which provides the required confirmation that the maximum likelihood estimators of the shape and scale parameters in the two parameter Weibull distribution are asymptotically unbiased.
Complete samples bias in fl
Type II censored samples
bias in
bias in fl
bias in
0.5
0.03827
6.81136
0.08209
3.12848
1.0
0.07341
0.82887
0.16262
-2.89283
1.5
0.11599
-0.10295
0.24168
-2.65132
2.0
0.14882
-0.47383
0.33014
-2.51337
3.0
0.22415
-0.34424
0.50827
-1.90043
4.0
0.29573
-0.25974
0.67247
-1.56839
Table 4.1: Observed biases in maximum likelihood estimators for n = 20, 0 = 100 for complete samples and Type II censoring with nf = 12: each entry is based on 10000 replications
Complete samples bias in fl
bias in
Type II censored samples bias in/3
bias in
20
0 . 0 7 3 4 1 0.82887
0.16262
-2.89283
50
0.02984
0.57103
0.06177
-1.22464
100
0.01353
0.15249
0.02936
-0.63007
200
0.00686
0.10771
0.01519
-0.19109
500
0.00298
0.05253
0.00421
-0.09632
1000
0.00125
0.03138
0.00264
-0.05248
2000
0.00057
0.01838
0.00191
-0.02559
Table 4.2: Observed biases in maximum likelihood estimators for/3 = 1,0 = 100 for complete samples and Type II censoring for various n: each entry is based on 10000 replications
599
600
A.J. Watkins These findings, which are consistent with intuition based on the general principles for maximum likelihood estimators, mean that we need to consider again the differences between the observed biases in Table 4.1 and those reported in Wang and Keats [3].
5. On Variations In Censoring D a t a Unfortunately, Wang and Keats [3] provide only sparse details of their experimental procedures; however, after some investigation, we now seek to show that the differences between our results and those reported in [3] are due to the fact Wang and Keats do not seem to subject their simulated Weibull variates to any replacement.
That is,
Wang and Keats [3] seem to regard some of the generated t~ as censored, without any change to ti. This assertion is supported by the following experiment, in which we rank the sample t x , t 2 , ..., t~ in ascending order, define the first n ] order statistics as failures, but now regard the last n - n l order statistics as censored: the interested reader will note that this change to our previous experimental procedure is possible with only a very small change in coding. With n = 20, n: = 12,/3 = 1 and 0 = 100, we observe the biases reported in the first row of Table 5.1, these values are, of course, consistent with those reported by Wang and Keats [3] fur their second censored case. We then considered the situation in which the first n - n : order statistics are regarded as censored, and the remaining n : order statistics as failures. With the same parameter values as above, this variation gives rise to the entries in the second row of Table 5.1, and are consistent with those reported for the first censored case by Wang
Censored data
O b s e r v e d bias
R e p o r t e d bias for t h e c e n s o r i n g p a t t e r n s in W a n g a n d K e a t s
/3
0
Pattern
¢)
t03 ),...,t(20)
-0.28861
82.02330
2
-0.3046
80.2096
t(1), ..., t(s)
1.06021
72.02772
1
1.0463
72.6282
0.13859
63.24656
3
0.1319
64.0827
to),t(a),t(~),t(7),
t04 ), t0s ), t(ls), t(20) Table 5.1: Observed biases for three variations in which values for censored items are not transformed: each entry is based on 10000 replications.
Two parameter Weibull distribution
601
and Keats [3]. Finally, we consider the variation in which t0), t(3), t(5), t(7), t04 ), t0s ), t0s ) and t(20) are regarded as censored, and the remaining n ! order statistics as failures. With the same parameter values as above, this variation gives rise to the entries for the final case in Table 5.1, and are consistent with those reported for the third censored case by Wang and Keats [3]. Further interpretation of the values in Table 5.1 is also possible. • The first censored case in Wang and Keats [3], in which the first
n
-
nf
order
statistics are regarded as censored rather than failure times, is the simplest situation. Intuitively, this is akin to pretending that the early failures in a sample have not occurred, but that we have been able to withdraw the appropriate items from the trial just before failure. Since early failures are instrumental in determining failure pattern, the absence of early failures here will effectively preclude us fl'om detecting infant mortality, with a concomitant increase in the value of the shape parameter: the large positive bias in/3 is consistent with this reasoning. • In the second censored case in Wang and Keats [3], the first n / o r d e r statistics are defined as failures, but the final n - n / o r d e r statistics are regarded as censored times in service. This is akin to pretending that the final failures in a sample are still to occur, but that we have been able prolong the trial as necessary, and withdrawing these items just before failure. Thus, the presence of large values deemed to be censored means that the defined failures will be regarded as occurring at relatively early times, and will result in the detection of infant mortality, with a concomitant decrease in the value of the shape parameter: the negative bias in/~ is consistent with this reasoning.
Pattern
from (5.1)
bias in theoretical
observed
1
2.06021
184.36
84.36
72.02772
2
0.71139
179.82
79.82
82.02330
3
1.13859
165.76
65.76
63.24656
Table 5.2: Approximate theoretical biases in 0 for the three patte,ms studied by Wang and Keats.
A. J. Watkins
602
• The final censored case in Wang and Keats [3] is more balanced, with failures intermingled with those values deemed to be censored; this balance explains the fact that this case has the lowest absolute bias in ~ and 0.
For each of three cases, the positive biases in ~) largely stem from the relation
\ n: /
\
in which F is the usual gamma function.
-~-:
I
(5.1)
Replacing ~ by its observed average in
the three variations above leads to the approximate theoretical values for 0 shown in Table 5.2. In all cases, the approximate theoretical bias in 0 based on (5.1) is quite close to that observed in simulation experiments: for the second and third cases, the agreement is very good. We remark that there are two mains contributory factors to these biases: these are the multiplier ~-s and the factor F(1 + )3-1/~).
6. S u m m a r y
In this paper, we have sought to explain some recently reported values for bias in maximum likelihood estimators of the shape and scale in the two parameter Weibull distribution for cases when the data for analysis contains both times to failure and censored times in service. We started by contrasting these reported values with their counterparts based on complete samples, in which all item are observed to failure. We then documented our attempts to reproduce the reported values, outlined our experimental procedures,and summarised the results from our simulation experiments. These experiments showed that biases for estimates calculated for samples subject to Type II censoring were of the same order as those for estimates from complete samples, and that these biases disappeared as the sample size increased. We then demonstrated that the reported values for bias were based on a different treatment of censored values, and we argued that this treatment was inherently biased. The main conclusions of our work are
• care is required in the generation of samples in which some values are to be regarded as censored, and
Two parameter Weibull distribution • under appropriate circumstances, the bias in maximum likelihood estimators is relatively low, even in small samples, and will disappear as the sample size increases. We have, to facilitate comparison with the values reported in Wang and Keats [3], focused our discussion on estimator bias: however, it is also possible to consider other aspects of estimator behaviour, such as variance and mean square error. We have also noted that it is also possible to consider estimation by techniques other than maximum likelihood; we repeat that, for large samples, it is impossible to out-perform maximum likelihood estimators in any systematic way, but that other techniques show promise when used with small samples. There is considerable scope for further work in this area.
References
[1] D. R. Cox and D. V. Hinkley. Theoretical Statistics.
Chapman and Hall, New
York, 1974. [2] T. Seki and S. Yokoyama. Simple and robust estimation of the Weibull parameters.
Microelectronics and Reliability, 33:45-52, 1993. [3] F. K. Wang and J. B. Keats. Improved percentile estimation for the two parameter Weibull distribution. Mieroelectronics and Reliability, 35:883-892, 1995.
603
~ )
Copyright © 1996 Published by Elsevier Science Ltd Printed in Great Britain. All rights reserved 0026-2714/96 $15.00+ .00
Pergamon
0026-2714(95)00171-9 ON MAXIMUM LIKELIHOOD ESTIMATION FOR THE TWO PARAMETER WEIBULL DISTRIBUTION A.J.WATKINS Statistics
and OperationalResearch GroupEBMS, University of Wales, Swansea SA2 8PP, U.K.
(Receivedfor publication 22 August 1995)
Abstract This paper examines recent results presented on maximum likelihood estimation for the two parameter Weibutl distribution. In particular, we seek to explain some recently reported values for estimator bias when the data for analysis contains both times to failure and censored times in operation; our discussion centres on the generation of sample data sets. We conclude that, under appropriate conditions, estimators are asymptotically unbiased, with relatively low bias in small to moderate samples. We then present the results of some further experiments which suggest that the previously reported values for estimator bias can be attributed to the method of generating sample data sets in simulation experiments.
1. I n t r o d u c t i o n This paper examines recent results presented by Wang and Keats [3] on estimation for the two parameter Weibull distribution; for simplicity, we adopt their notation. Given the central role of likelihood and likelihood-based estimators in statistics in general, and in the analysis of quantitative reliability data in particular, we focus here on the reported values for bias in maximum likelihood estimators of the two parameter Weibull distribution when the data for analysis contains both times to failure and censored times in operation. General results based on the principles of likelihood (see, for example, Cox and Hinkley, [1]) mean that the maximum likelihood estimators of the shape and scale parameters /~ and 0 based on an independent and identically distributed sample of size n should be asymptotically unbiased, with variances attaining the appropriate 595
596
A. J. Watkins Cramer-Rao lower bound: expressions for the asymptotic bias in /~ and 0 are both
O(n 1).
Thus, for large samples, it is impossible to out-perform maximum likeli-
hood estimators in terms of estimator accuracy in any systematic way. However, we emphasise that these results are asymptotic; it is known that the bias observed for estimates calculated in small to moderate samples need not be insignificant. This small-sample behaviour, together with other issues such as the computational effort required to calculate estimators, has prompted research into alternative methods for estimating the parameters of the Weibull distribution. For instance, some statistical packages calculate estimates by applying least squares methods to hazard plots, while Seki and Yokoyama [2] have proposed a percentile-based technique, which has been further explored by Wang and Keats [3]. It is natural to assess the relative merits of such alternative techniques by comparisons with maximum likelihood estimators; the general lack of analytical progress then means that comparisons are based on simulation experiments. We note that some experiments seem to indicate that several alternative methods may provide acceptable results in small samples; however, we do not pursue these comparisons here. We are, however, concerned with the biases reported by Wang and Keats in the final columns of their Tables 3 and 4 in [3]. The structure of this remainder of this paper is as follows: we first consider their results in detail, and raise some points of concern. We then outline our attempts to reproduce these results; this leads to a discussion on the generation of sample data sets comprising both times to failure and censored times in operation. We next show that, under appropriate conditions, the maximum likelihood estimators have relatively low bias in small to moderate samples, and we confirm that these estimators are asymptotically unbiased. We finally present the results of
Sample data
bias in ~
bias in
complete
0.0730
0.9964
censored - case 1
1.0463
72.6282
censored - c a s e 2
-0.3046
80.2096
censored - c a s e 3
0.1319
64.0827
Table 2.1: Reported biases in maximum likelihood estimators for n = 20, fl -- 1,0 = 100 from Wang and Keats
Two parameter Weibull distribution some further experiments which suggest that the values for estimator bias reported by Wang and Keats in Tables 3 and 4 in [3] can be attributed to the method of generating sample data sets in simulation experiments; we argue that this simulation method is inherently biased.
2. On R e p o r t e d Values Of Estimator Bias We consider the biases reported by Wang and Keats in the final columns of Tables 3 and 4 in [3]. As these Tables are based on three cases with n = 20, it is not reasonable to expect the asymptotic arguments to hold fully, and we should therefore be prepared to see some evidence of non-zero bias. However, there is a striking difference between the magnitude of these entries and their counterparts in Tables 1 and 2, based on
complete samples in which all data are times to failure. For instance, with n --- 20,/3 = 1,0 = 100, Wang and Keats report the biases reproduced in Table 2.1, and we are immediately struck by the large biases reported for 0; however, the values reported for ~ also give rise for concern, since this parameter characteristics the shape of the Weibull distribution, and is often interpreted in terms of failure patterns such as infant mortality and wear-out. The overall implication of the values in Table 2.1 is that there are serious problems with the method of maximum likelihood, particularly in the three cases involving some form of censored data.
3. On D a t a Simulation We now discuss our attempts to reproduce the values in Table 2.1, and take this opportunity to separate two issues which are often presented together. First, we note that if U is a uniform random variate on [0, 1], then V = - log e U follows the standard negative exponential distribution, with probability density function
e x p ( - v ) , v > 0,
and W = OV a/t~ = 0 [ - log e U] x/~ follows a Weibull distribution with probability density function
zw - e
exp
w ___ O;
597
598
A. J. Watkins t h a t is, the two p a r a m e t e r Weibull distribution with shape ~ a n d scale 0. Thus, given a mechanism for generating pseudo-random uniform variates, we can easily obtain pseudo-random variates from a Weibull distribution with specified shape a n d scale; this allows us to generate a complete sample t l, t2, ..., t~. T h e second issue is t h a t of generating a sample in which some observations correspond to items censored in service; we now start with a complete sample tl, t2, ..., t~, consider how items with these times to failure would be affected by a specified censoring regime, and outline the consequent changes to the ti.
• Type I censoring: the trial finishes at some specified time - to, say. Here, any item for which ti <_ t¢ is deemed to have failed at t h a t time, while any item with
ti > tc is now regarded as censored, with observed time in service to.
• Type H censoring, the trial finishes after a specified n u m b e r of failures - n l , say - have been observed. In this case, we may rank the sample in ascending order, thus defining the order statistics t(l), t(2), ..., t(~). Then, the first n s order statistics are the observed failures, and, as the last failure is deemed to occur at t(~I) , we m a y regard the remaining n - nf items as censored, each with observed time in service t(~1).
• Random censoring: each item is tested for some r a n d o m time, a n d thus has a chance of being withdrawn from service before failure. This scenario is usually modelled by defining a second distribution for times under test, and, for each item, generating a variate zi from this distribution together with the Weibull variate ti. If ti <_ zi, then the item is observed to fail at ti; otherwise, the item is regarded as censored, with observed time in service zi.
We emphasise t h a t these three procedures each involve the replacement of some of the original times ti by other quantities which reflect the censoring regime of interest.
4. On O b s e r v e d Values Of E s t i m a t o r Bias T h e second censored case in Wang and Keats [3] is based on censoring the last eight observations; we therefore consider our second case above with n = 20, n f = 12, 0 -100, and the same range of shape parameters used in [3]. Our results, also based on
Two parameter Weibull distribution 10000 replications for each value of/3, are summarised in Table 4.1, and this shows a very different picture fi'om that reported in Table 2.1; essentially, the biases reported here for estimators from samples subjected to Type II censoring are of the same order as those for estimators from complete samples. To confirm that the estimators are asymptotically unbiased, we may repeat the above experiment with larger values of n - each time taking nf = -~; for brevity, we focus on fl = 1. Our results are summarised in Table 4.2, which provides the required confirmation that the maximum likelihood estimators of the shape and scale parameters in the two parameter Weibull distribution are asymptotically unbiased.
Complete samples bias in fl
Type II censored samples
bias in
bias in fl
bias in
0.5
0.03827
6.81136
0.08209
3.12848
1.0
0.07341
0.82887
0.16262
-2.89283
1.5
0.11599
-0.10295
0.24168
-2.65132
2.0
0.14882
-0.47383
0.33014
-2.51337
3.0
0.22415
-0.34424
0.50827
-1.90043
4.0
0.29573
-0.25974
0.67247
-1.56839
Table 4.1: Observed biases in maximum likelihood estimators for n = 20, 0 = 100 for complete samples and Type II censoring with nf = 12: each entry is based on 10000 replications
Complete samples bias in fl
bias in
Type II censored samples bias in/3
bias in
20
0 . 0 7 3 4 1 0.82887
0.16262
-2.89283
50
0.02984
0.57103
0.06177
-1.22464
100
0.01353
0.15249
0.02936
-0.63007
200
0.00686
0.10771
0.01519
-0.19109
500
0.00298
0.05253
0.00421
-0.09632
1000
0.00125
0.03138
0.00264
-0.05248
2000
0.00057
0.01838
0.00191
-0.02559
Table 4.2: Observed biases in maximum likelihood estimators for/3 = 1,0 = 100 for complete samples and Type II censoring for various n: each entry is based on 10000 replications
599
600
A.J. Watkins These findings, which are consistent with intuition based on the general principles for maximum likelihood estimators, mean that we need to consider again the differences between the observed biases in Table 4.1 and those reported in Wang and Keats [3].
5. On Variations In Censoring D a t a Unfortunately, Wang and Keats [3] provide only sparse details of their experimental procedures; however, after some investigation, we now seek to show that the differences between our results and those reported in [3] are due to the fact Wang and Keats do not seem to subject their simulated Weibull variates to any replacement.
That is,
Wang and Keats [3] seem to regard some of the generated t~ as censored, without any change to ti. This assertion is supported by the following experiment, in which we rank the sample t x , t 2 , ..., t~ in ascending order, define the first n ] order statistics as failures, but now regard the last n - n l order statistics as censored: the interested reader will note that this change to our previous experimental procedure is possible with only a very small change in coding. With n = 20, n: = 12,/3 = 1 and 0 = 100, we observe the biases reported in the first row of Table 5.1, these values are, of course, consistent with those reported by Wang and Keats [3] fur their second censored case. We then considered the situation in which the first n - n : order statistics are regarded as censored, and the remaining n : order statistics as failures. With the same parameter values as above, this variation gives rise to the entries in the second row of Table 5.1, and are consistent with those reported for the first censored case by Wang
Censored data
O b s e r v e d bias
R e p o r t e d bias for t h e c e n s o r i n g p a t t e r n s in W a n g a n d K e a t s
/3
0
Pattern
¢)
t03 ),...,t(20)
-0.28861
82.02330
2
-0.3046
80.2096
t(1), ..., t(s)
1.06021
72.02772
1
1.0463
72.6282
0.13859
63.24656
3
0.1319
64.0827
to),t(a),t(~),t(7),
t04 ), t0s ), t(ls), t(20) Table 5.1: Observed biases for three variations in which values for censored items are not transformed: each entry is based on 10000 replications.
Two parameter Weibull distribution
601
and Keats [3]. Finally, we consider the variation in which t0), t(3), t(5), t(7), t04 ), t0s ), t0s ) and t(20) are regarded as censored, and the remaining n ! order statistics as failures. With the same parameter values as above, this variation gives rise to the entries for the final case in Table 5.1, and are consistent with those reported for the third censored case by Wang and Keats [3]. Further interpretation of the values in Table 5.1 is also possible. • The first censored case in Wang and Keats [3], in which the first
n
-
nf
order
statistics are regarded as censored rather than failure times, is the simplest situation. Intuitively, this is akin to pretending that the early failures in a sample have not occurred, but that we have been able to withdraw the appropriate items from the trial just before failure. Since early failures are instrumental in determining failure pattern, the absence of early failures here will effectively preclude us fl'om detecting infant mortality, with a concomitant increase in the value of the shape parameter: the large positive bias in/3 is consistent with this reasoning. • In the second censored case in Wang and Keats [3], the first n / o r d e r statistics are defined as failures, but the final n - n / o r d e r statistics are regarded as censored times in service. This is akin to pretending that the final failures in a sample are still to occur, but that we have been able prolong the trial as necessary, and withdrawing these items just before failure. Thus, the presence of large values deemed to be censored means that the defined failures will be regarded as occurring at relatively early times, and will result in the detection of infant mortality, with a concomitant decrease in the value of the shape parameter: the negative bias in/~ is consistent with this reasoning.
Pattern
from (5.1)
bias in theoretical
observed
1
2.06021
184.36
84.36
72.02772
2
0.71139
179.82
79.82
82.02330
3
1.13859
165.76
65.76
63.24656
Table 5.2: Approximate theoretical biases in 0 for the three patte,ms studied by Wang and Keats.
A. J. Watkins
602
• The final censored case in Wang and Keats [3] is more balanced, with failures intermingled with those values deemed to be censored; this balance explains the fact that this case has the lowest absolute bias in ~ and 0.
For each of three cases, the positive biases in ~) largely stem from the relation
\ n: /
\
in which F is the usual gamma function.
-~-:
I
(5.1)
Replacing ~ by its observed average in
the three variations above leads to the approximate theoretical values for 0 shown in Table 5.2. In all cases, the approximate theoretical bias in 0 based on (5.1) is quite close to that observed in simulation experiments: for the second and third cases, the agreement is very good. We remark that there are two mains contributory factors to these biases: these are the multiplier ~-s and the factor F(1 + )3-1/~).
6. S u m m a r y
In this paper, we have sought to explain some recently reported values for bias in maximum likelihood estimators of the shape and scale in the two parameter Weibull distribution for cases when the data for analysis contains both times to failure and censored times in service. We started by contrasting these reported values with their counterparts based on complete samples, in which all item are observed to failure. We then documented our attempts to reproduce the reported values, outlined our experimental procedures,and summarised the results from our simulation experiments. These experiments showed that biases for estimates calculated for samples subject to Type II censoring were of the same order as those for estimates from complete samples, and that these biases disappeared as the sample size increased. We then demonstrated that the reported values for bias were based on a different treatment of censored values, and we argued that this treatment was inherently biased. The main conclusions of our work are
• care is required in the generation of samples in which some values are to be regarded as censored, and
Two parameter Weibull distribution • under appropriate circumstances, the bias in maximum likelihood estimators is relatively low, even in small samples, and will disappear as the sample size increases. We have, to facilitate comparison with the values reported in Wang and Keats [3], focused our discussion on estimator bias: however, it is also possible to consider other aspects of estimator behaviour, such as variance and mean square error. We have also noted that it is also possible to consider estimation by techniques other than maximum likelihood; we repeat that, for large samples, it is impossible to out-perform maximum likelihood estimators in any systematic way, but that other techniques show promise when used with small samples. There is considerable scope for further work in this area.
References
[1] D. R. Cox and D. V. Hinkley. Theoretical Statistics.
Chapman and Hall, New
York, 1974. [2] T. Seki and S. Yokoyama. Simple and robust estimation of the Weibull parameters.
Microelectronics and Reliability, 33:45-52, 1993. [3] F. K. Wang and J. B. Keats. Improved percentile estimation for the two parameter Weibull distribution. Mieroelectronics and Reliability, 35:883-892, 1995.
603