- Email: [email protected]
Transient solutions of a software reliability model with imperfect debugging and error generation
Microelectron. Reliab., Vol. 32, No. 4, pp. 475-478, 1992. Printed in Great Britain.
0026-2714/9255.00+ .00 Pergamon Press plc
TRANSIENT SOLUTIONS OF A SOFTWARE RELIABILITY MODEL WITH IMPERFECT D E B U G G I N G A N D ERROR GENERATION P. K. KAPUR Department of Operational Research, University of Delhi, Delhi-110 007, India K. D. SHARMAand R. B. GARG Department of Computer Science, University of Delhi, Delhi-110 007, India (Received for publication 17 December 1990) Abstract--In this paper we obtain transient solutions of a software reliability model under the assumption that the failure rate is proportional to the remaining errors in the software under imperfect debugging and error generation. The maximum number of errors in the software is assumed to be finite. We obtain the transient probabilities for the remaining errors, mean number of errors remaining, reliability of the software, expected number of failures etc. A method to estimate model parameters is given. Finally, a numerical example is presented.
INTRODUCTION As no software system can be made error free, it is of p a r a m o u n t importance to find the reliability of a software system and estimate the error content in the software. Several Software Reliability Growth Models (SRGMs) exist in the literature to describe the above phenomenon. Jelinski and Moranda [1] were probably the first to introduce such a model. Similar models were later considered by Shooman [2], Musa [3] and Littlewood [4]. Goel and Okumoto [5] described an error detection or debugging phenomenon based on the Non-Homogeneous Poisson Protess (NHPP), now termed as an exponential SRGM. Recently, more S R G M s have been proposed, namely modified exponential [6] S-shaped [7], inflection Sshaped [8], flexible S R G M [9] etc. Two review articles by Shanthikumar [10] and Yamada and Osaki [11] summarize some of these models. Alternatively, release policies/stopping rules have been discussed for such S R G M s [12-17, 24]. The underlying assumption in all these models is that no new error is generated while removing an error. However, some attempts have been made to incorporate the concept of imperfect debugging and error generation by K u o [18] and Kremer [19]. They have assumed that the error generation may lead the software to having an infinite number of faults, starting from a finite number, and as a result the models are easily described by Kendall's [20] birth and death process, which is also given in Chiang [21]. This assumption may only be theoretically true as it requires the error generation rate to be several times more than the error removal rate. However, if that is the case in reality, it is advisable to a b a n d o n the testing process and hire a new test team. At best, the generation rate may be 475
higher than the removal rate in the initial stages of testing, thereby also justifying a finite maximum number of faults that can exist in the software. In this paper, we describe a software reliability model, under the assumption that the failure rate at any time is proportional to the n u m b e r of remaining errors, debugging is imperfect and the error generation phenomenon will never lead the software to having infinite errors. We obtain the transient probabilities of errors in the system and obtain the following software reliability measures: mean number of remaining errors; mean number of failures; software reliability. A method to estimate the model parameters is provided. Since no live data were available, numerical results are given at the end assuming the values of model parameters.
ASSUMPTIONS 1. The software system has a finite number of faults n and each time a failure occurs, the fault causing that failure is immediately connected. 2. The failure rate of the software is equally affected by the errors remaining in the software. 3. The maximum number of faults in the software never exceeds a finite limit N > n. 4. At any time, the failure rate of the software is proportional to the number of faults present in the software at that time. 5. When a failure occurs, an instantaneous repair effort starts and the following may occur: (a) the fault content is reduced by one with probability P0; (b) the fault content remains unchanged with probability Pl; (c) the fault content is increased by one with probability P2, where Po+P~ +P2 = 1 and P0 ~'Pl ~'P2.
476
P. K. KAPURet al.
NOTATION n initial fault content N maximum fault content q,(t) pr{k faults in the software at time t} failure rate per remaining software error or proportionality constant. MODEL ANALYSlS The following difference-differential equations may easily be written
From Ref. [22] it may be observed that the eigenvalues of D are real and distinct, and from Ref. [23] we observe that D is positive definite. Thus all its eigenvalues are positive. Let 0k (1 ~
I A i = s l i (s +Ok), k=l
so that
qo(t) = ~Poql (t)
IA~+j(s)l
q.(s) =
ql (t) = - (~ -- c~pl)ql (t) + 2~poq2(t)
O <~m <<.N.
N
s 1-I (s + Oh)
qk(t) = --(k~ - k~pl)qk(t )
kffil
+ ( k - 1)~p2qk_ l ( t ) + (k + 1)c~poqk+ l(t),
Using partial fractions, we have
2 <~k < ~ N - 1
q0(s) =--a° +
qN( t ) = --(Net -- Nxpl )qN( t )
s
(1)
+ ( N - l)~p2qN_ l(t),
qk(O) = 0
N amk
for k # n (0 ~
AQ = I, + ~
where
IAt(o)l
ao=
u
lie, j-I
and
(2)
where A is an (N + 1 × N + 1) matrix of coefficients, Q is an (N + 1 × 1) column vector in qk(s) and I,+ is an (N + 1 × 1) column vector having unity in the (n + 1)th position and zero elsewhere. From equation (2), using Cramer's rule, qk(S) are explicitly determined as Ihk+l(s)l qk(s) = - ,
(4)
qm(s)-~ ~ , k . l S +Ok
To study the operating characteristics of the model, we need to evaluate qk(t). We therefore solve the system of equations represented by (1). Taking the Laplace transform of (1) and denoting Laplace transform o f f ( t ) by f ( s ) , the system of equations in (1) reduces to
IAI
(3)
and
with initial conditions q,(O) = 1,
aok k . t S +Ok
IA.+,(-0DI u I-I
a~=
oh
jfl #k
Taking inverse Laplace transforms in (3) and (4) we get lat(0)l IAl(--Ok)lexp(--Ok t) q0( t ) = ~ N
~
1-Io
O <~ k <<.N,
k.,
okl-I(O -Ok)
k=l
j=l
and
where Ak+l(S) is obtained from A by replacing its (k + 1)th column by I,+~. Now, IAI may be expressed as slD(s)l, where ID(s)[ is the determinant of a real, symmetric tri-diagonal matrix D(s) of order N with negative off-diagonal elements. Specifically,
0~
,k
q m ( t ) = - ~ [Am+t(--Ok)lexp(--Ok(t) N
k-1
OkI-I
(0: -- 0 k)
j=l #k
l <~m <<.N.
s +e-ep~ s + 2~ - 2otpt I I
D(s) =
I I I I
I t
- ~ / ( N - 2)(N - 1)~2p0p2
s + ( N - 1)(~ --~Pl) -~/(N
]D(s)l is a polynomial of degree N in s and its roots are the negative of the eigenvalues of matrix D obtained by replacing s with zero.
- l )N~2poP2
- ~/(N - 1)N~ 2poP2 s + N(~ - ~p~)
As 0.. (1 ~
Transient solutions of a software reliability model
477
Table 2. Mean number of faults remaining at time t
which is also evident from (1). Using expressions for transient probabilities, one may define several software reliability measures:
Time
Mean number of faults
0 1 2 3 4 5 6 7 8
5.00 2.61 1.36 0.71 0.37 0.19 0.10 0.05 0.03
1. The probability of a perfect program at time t is given by qo(t). 2. The mean number of faults remaining in the software at time t is given as N
~, mqm(t). m=0
3. S-expected number of failures in time t is given by Table 3. Expected number of failures in time t
• ~ mqm(x),tx. m=0
Time
Expected number of failures
0 1 2 3 4 5 6 7 8
0.00 3.68 5.60 6.60 7.12 7.39 7.54 7.61 7.65
4. The software reliability is defined as N
R(xlt) = ~
pr{k faults survive time t}
k=0
• pr{failure-free operation of the software between (t, t + x ) l k faults survive time t} N
= ~. qk(t)'
(exp(-~x)) k.
Table 4. Unconditional reliability in (t, t + x)
k=0
To
Other measures may be defined similarly. ESTIMATION OF PARAMETERS Letting N--+oo, our model agrees with Refs [18] and [19]. All the model parameters except N can thus be estimated as in Ref. [19]. To estimate N, we proceed as follows. The expected number of failures in time t in Ref. [19] is given by n ( 1 - e x p ( - ( p o - p 2 ) ~ t ) ) / ( p o - P 2 ) , and in our case it is equal to
From
1
2
3
4
5
6
0 1 2 3 4 5
0.01
0.00 0.15
0.00 0.06 0.42
0.00 0.05 0.30 0.65
0.00 0.04 0.26 0.56 0.80
0.00 0.04 0.25 0.53 0.75 0.89
these parameters, we have derived the above mentioned measures and they are shown in Tables 1--4. F r o m these tables, it can be observed that the mean number of faults remaining decreases with time while the expected number of failures increases with time. Also, as x increases for a given t, the reliability R(xlt) decreases.
m~ ~2 q.,(x) dx: m~O
equating the two expressions N may be estimated. NUMERICAL EXAMPLE
CONCLUSION
Since no live data were available, we could not estimate the model parameters. We therefore assume n = 5, • = 1, P0 = 0.75, p~ = 0.15 and P2 = 0.10. With
Looking at the numerical results presented in this paper one gets some idea about the release time of the
Table 1. Probability of faults remaining at time t l
Probability
0
1
2
3
4
5
6
7
qo(t)
0 0 0 0 0 1 0 0 0
0.036 0.158 0.289 0.284 0.163 0.056 0.012 0.002 0
0.245 0.358 0.245 0.107 0.034 0.009 0.002 0 0
0.511 0.325 0.120 0.034 0.008 0.1)02 0 0 0
0.715 0.217 0.054 0.012 0.002 0 0 0 0
0.842 0.128 0.024 0.005 0.001 0 0 0 0
0.915 0.071 0.012 0.002 0 0 0 0 0
0.955 0.038 0.006 0.001 0 0 0 0 0
qt (t) q2(t)
q3(t) q4(t)
qs(t) q6(t) qT(t) qs(t)
qg(t)
0
0
0
0
0
0
0
0
qto(t)
0
0
0
0
0
0
0
0
478
P.K. KAPUR et al.
software system. For example, Table 2 gives the mean number of errors remaining in the software at different units of time. As the initial error content in the software is assumed to be 5 and if we are interested in deciding the time to a specific software purity level (i.e. the time beyond which the number of errors remaining in the software is ~
REFERENCES
1. Z. Jelinski and P. B. Moranda, Software Reliability Research, Statistical Computer Performance Evaluation, pp. 468-484. Academic Press, New York (1972). 2. M. L. Shooman, Software reliability: analysis and prediction, Report AGARD AG224, Integrity in Electronic Flight Control Systems, pp. 1-17 (1977). 3. J. D. Musa, Theory of software reliability and its application, IEEE Trans. Software Engng SE-I, 312-327 (1975). 4. B. Littlewood, Stochastic reliability growth: a model for fault removal in computer-programs and hardwaredesign, IEEE Trans. Reliab. R-30, 313-320. (1981). 5. A. L. Goel and K. Okumoto, Time dependent error detection rate model for software reliability and other performance measures. IEEE Trans. Reliab. R-28(3), 206-211 (1979). 6. S. Yamada and S. Osaki, Non-homogeneous error detection rate models for software reliability growth, in Stochastic Models in Reliability Theory, pp. 120-143. Springer-Verlag, Berlin (1984). 7. S. Yamada, M. Ohba and S. Osaki, S-shaped reliability growth modelling for software error detection, IEEE Trans. Reliab. R-32, 475-478 (1983). 8. M. Ohba, Software reliability analysis models, IBM J. Res. Dev. 28(4), 428-443 (1984). 9. S. Bittanti, P. Bolzern, E. Pedrotti, M. Pozzi and R.
Scattolini, A flexible modelling approach for software reliability growth, In Software Reliability Modelling and Identification (edited by G. Goos and J. Hartmanis). Springer-Verlag, Berlin (1988). 10. J. G. Shanthikumar, Software reliability models: a review, Microelectron. Reliab. 23(5), 903-943 (1983). 11. S. Yamada and S. Osaki, Reliability growth models for hardware and software systems based on nonhomogeneous Poisson processes: a survey, Microelectron. Reliab. 23(1), 91-112 (1983). 12. P. A. Caspi and E. F. Kouka, Stopping rules for a debugging process based on different software reliability models. Int. Conf. on Fau#-tolerant Computing, 14, Kissimmee, pp. 114-119 (1984). 13. E. H. Forman and N. D. Singpurwalla, An empirical stopping rule for debugging and testing computer software. Am. statist. Ass. 72, 750-757 (1977). 14. P. K. Kaput and R. B. Garg, Optimum release policy for inflection S-shaped software reliability growth model, Microelectron. Reliab. 31(1), 43-46 (1991). 15. P. K. Kaput and R. B. Garg, Optimal software policies for software reliability growth models under imperfect debugging, R.A.LR.O. (in press). 16. P. K. Kapur and V. K. Bhalla, Optimal release policies for a flexible software reliability growth model, Reliab. Engng Syst. Safety (in press). 17. S. Yamada and S. Osaki, Optimal software release policies with simultaneous cost and reliability requirements, Eur. J. Op. Res. 31, 46-51 (1987). 18. W. Kuo, Software reliability estimation: a realisation of competing risk, Microelectron. Reliab. 23(2), 249-260 (1983). 19. W. Kremer, Birth-death and bug counting, IEEE Trans. Reliab. R-3I(1), 37-47 (1983). 20. D. G. Kendall, Generalised birth and death process, Ann. math. Statist. 19, 1-15 (1948). 21. C. L. Chiang, Introduction to Stochastic Processes in Biostatistics. Wiley, New York. 22. D. M. Young and R. T. Gregory, A Survey of Numerical Mathematics. Addison-Wesley, Reading, MA (1973). 23. B. Keller, Numerical Methods for Two-point Boundaryvalue Problems. Blaisdell, MA. 24. S. M. Ross, Software reliability: the stopping rule problem, 1EEE Trans. Software Engng SE-I1, 1472-1476 (1985).
0026-2714/9255.00+ .00 Pergamon Press plc
TRANSIENT SOLUTIONS OF A SOFTWARE RELIABILITY MODEL WITH IMPERFECT D E B U G G I N G A N D ERROR GENERATION P. K. KAPUR Department of Operational Research, University of Delhi, Delhi-110 007, India K. D. SHARMAand R. B. GARG Department of Computer Science, University of Delhi, Delhi-110 007, India (Received for publication 17 December 1990) Abstract--In this paper we obtain transient solutions of a software reliability model under the assumption that the failure rate is proportional to the remaining errors in the software under imperfect debugging and error generation. The maximum number of errors in the software is assumed to be finite. We obtain the transient probabilities for the remaining errors, mean number of errors remaining, reliability of the software, expected number of failures etc. A method to estimate model parameters is given. Finally, a numerical example is presented.
INTRODUCTION As no software system can be made error free, it is of p a r a m o u n t importance to find the reliability of a software system and estimate the error content in the software. Several Software Reliability Growth Models (SRGMs) exist in the literature to describe the above phenomenon. Jelinski and Moranda [1] were probably the first to introduce such a model. Similar models were later considered by Shooman [2], Musa [3] and Littlewood [4]. Goel and Okumoto [5] described an error detection or debugging phenomenon based on the Non-Homogeneous Poisson Protess (NHPP), now termed as an exponential SRGM. Recently, more S R G M s have been proposed, namely modified exponential [6] S-shaped [7], inflection Sshaped [8], flexible S R G M [9] etc. Two review articles by Shanthikumar [10] and Yamada and Osaki [11] summarize some of these models. Alternatively, release policies/stopping rules have been discussed for such S R G M s [12-17, 24]. The underlying assumption in all these models is that no new error is generated while removing an error. However, some attempts have been made to incorporate the concept of imperfect debugging and error generation by K u o [18] and Kremer [19]. They have assumed that the error generation may lead the software to having an infinite number of faults, starting from a finite number, and as a result the models are easily described by Kendall's [20] birth and death process, which is also given in Chiang [21]. This assumption may only be theoretically true as it requires the error generation rate to be several times more than the error removal rate. However, if that is the case in reality, it is advisable to a b a n d o n the testing process and hire a new test team. At best, the generation rate may be 475
higher than the removal rate in the initial stages of testing, thereby also justifying a finite maximum number of faults that can exist in the software. In this paper, we describe a software reliability model, under the assumption that the failure rate at any time is proportional to the n u m b e r of remaining errors, debugging is imperfect and the error generation phenomenon will never lead the software to having infinite errors. We obtain the transient probabilities of errors in the system and obtain the following software reliability measures: mean number of remaining errors; mean number of failures; software reliability. A method to estimate the model parameters is provided. Since no live data were available, numerical results are given at the end assuming the values of model parameters.
ASSUMPTIONS 1. The software system has a finite number of faults n and each time a failure occurs, the fault causing that failure is immediately connected. 2. The failure rate of the software is equally affected by the errors remaining in the software. 3. The maximum number of faults in the software never exceeds a finite limit N > n. 4. At any time, the failure rate of the software is proportional to the number of faults present in the software at that time. 5. When a failure occurs, an instantaneous repair effort starts and the following may occur: (a) the fault content is reduced by one with probability P0; (b) the fault content remains unchanged with probability Pl; (c) the fault content is increased by one with probability P2, where Po+P~ +P2 = 1 and P0 ~'Pl ~'P2.
476
P. K. KAPURet al.
NOTATION n initial fault content N maximum fault content q,(t) pr{k faults in the software at time t} failure rate per remaining software error or proportionality constant. MODEL ANALYSlS The following difference-differential equations may easily be written
From Ref. [22] it may be observed that the eigenvalues of D are real and distinct, and from Ref. [23] we observe that D is positive definite. Thus all its eigenvalues are positive. Let 0k (1 ~
I A i = s l i (s +Ok), k=l
so that
qo(t) = ~Poql (t)
IA~+j(s)l
q.(s) =
ql (t) = - (~ -- c~pl)ql (t) + 2~poq2(t)
O <~m <<.N.
N
s 1-I (s + Oh)
qk(t) = --(k~ - k~pl)qk(t )
kffil
+ ( k - 1)~p2qk_ l ( t ) + (k + 1)c~poqk+ l(t),
Using partial fractions, we have
2 <~k < ~ N - 1
q0(s) =--a° +
qN( t ) = --(Net -- Nxpl )qN( t )
s
(1)
+ ( N - l)~p2qN_ l(t),
qk(O) = 0
N amk
for k # n (0 ~
AQ = I, + ~
where
IAt(o)l
ao=
u
lie, j-I
and
(2)
where A is an (N + 1 × N + 1) matrix of coefficients, Q is an (N + 1 × 1) column vector in qk(s) and I,+ is an (N + 1 × 1) column vector having unity in the (n + 1)th position and zero elsewhere. From equation (2), using Cramer's rule, qk(S) are explicitly determined as Ihk+l(s)l qk(s) = - ,
(4)
qm(s)-~ ~ , k . l S +Ok
To study the operating characteristics of the model, we need to evaluate qk(t). We therefore solve the system of equations represented by (1). Taking the Laplace transform of (1) and denoting Laplace transform o f f ( t ) by f ( s ) , the system of equations in (1) reduces to
IAI
(3)
and
with initial conditions q,(O) = 1,
aok k . t S +Ok
IA.+,(-0DI u I-I
a~=
oh
jfl #k
Taking inverse Laplace transforms in (3) and (4) we get lat(0)l IAl(--Ok)lexp(--Ok t) q0( t ) = ~ N
~
1-Io
O <~ k <<.N,
k.,
okl-I(O -Ok)
k=l
j=l
and
where Ak+l(S) is obtained from A by replacing its (k + 1)th column by I,+~. Now, IAI may be expressed as slD(s)l, where ID(s)[ is the determinant of a real, symmetric tri-diagonal matrix D(s) of order N with negative off-diagonal elements. Specifically,
0~
,k
q m ( t ) = - ~ [Am+t(--Ok)lexp(--Ok(t) N
k-1
OkI-I
(0: -- 0 k)
j=l #k
l <~m <<.N.
s +e-ep~ s + 2~ - 2otpt I I
D(s) =
I I I I
I t
- ~ / ( N - 2)(N - 1)~2p0p2
s + ( N - 1)(~ --~Pl) -~/(N
]D(s)l is a polynomial of degree N in s and its roots are the negative of the eigenvalues of matrix D obtained by replacing s with zero.
- l )N~2poP2
- ~/(N - 1)N~ 2poP2 s + N(~ - ~p~)
As 0.. (1 ~
Transient solutions of a software reliability model
477
Table 2. Mean number of faults remaining at time t
which is also evident from (1). Using expressions for transient probabilities, one may define several software reliability measures:
Time
Mean number of faults
0 1 2 3 4 5 6 7 8
5.00 2.61 1.36 0.71 0.37 0.19 0.10 0.05 0.03
1. The probability of a perfect program at time t is given by qo(t). 2. The mean number of faults remaining in the software at time t is given as N
~, mqm(t). m=0
3. S-expected number of failures in time t is given by Table 3. Expected number of failures in time t
• ~ mqm(x),tx. m=0
Time
Expected number of failures
0 1 2 3 4 5 6 7 8
0.00 3.68 5.60 6.60 7.12 7.39 7.54 7.61 7.65
4. The software reliability is defined as N
R(xlt) = ~
pr{k faults survive time t}
k=0
• pr{failure-free operation of the software between (t, t + x ) l k faults survive time t} N
= ~. qk(t)'
(exp(-~x)) k.
Table 4. Unconditional reliability in (t, t + x)
k=0
To
Other measures may be defined similarly. ESTIMATION OF PARAMETERS Letting N--+oo, our model agrees with Refs [18] and [19]. All the model parameters except N can thus be estimated as in Ref. [19]. To estimate N, we proceed as follows. The expected number of failures in time t in Ref. [19] is given by n ( 1 - e x p ( - ( p o - p 2 ) ~ t ) ) / ( p o - P 2 ) , and in our case it is equal to
From
1
2
3
4
5
6
0 1 2 3 4 5
0.01
0.00 0.15
0.00 0.06 0.42
0.00 0.05 0.30 0.65
0.00 0.04 0.26 0.56 0.80
0.00 0.04 0.25 0.53 0.75 0.89
these parameters, we have derived the above mentioned measures and they are shown in Tables 1--4. F r o m these tables, it can be observed that the mean number of faults remaining decreases with time while the expected number of failures increases with time. Also, as x increases for a given t, the reliability R(xlt) decreases.
m~ ~2 q.,(x) dx: m~O
equating the two expressions N may be estimated. NUMERICAL EXAMPLE
CONCLUSION
Since no live data were available, we could not estimate the model parameters. We therefore assume n = 5, • = 1, P0 = 0.75, p~ = 0.15 and P2 = 0.10. With
Looking at the numerical results presented in this paper one gets some idea about the release time of the
Table 1. Probability of faults remaining at time t l
Probability
0
1
2
3
4
5
6
7
qo(t)
0 0 0 0 0 1 0 0 0
0.036 0.158 0.289 0.284 0.163 0.056 0.012 0.002 0
0.245 0.358 0.245 0.107 0.034 0.009 0.002 0 0
0.511 0.325 0.120 0.034 0.008 0.1)02 0 0 0
0.715 0.217 0.054 0.012 0.002 0 0 0 0
0.842 0.128 0.024 0.005 0.001 0 0 0 0
0.915 0.071 0.012 0.002 0 0 0 0 0
0.955 0.038 0.006 0.001 0 0 0 0 0
qt (t) q2(t)
q3(t) q4(t)
qs(t) q6(t) qT(t) qs(t)
qg(t)
0
0
0
0
0
0
0
0
qto(t)
0
0
0
0
0
0
0
0
478
P.K. KAPUR et al.
software system. For example, Table 2 gives the mean number of errors remaining in the software at different units of time. As the initial error content in the software is assumed to be 5 and if we are interested in deciding the time to a specific software purity level (i.e. the time beyond which the number of errors remaining in the software is ~
REFERENCES
1. Z. Jelinski and P. B. Moranda, Software Reliability Research, Statistical Computer Performance Evaluation, pp. 468-484. Academic Press, New York (1972). 2. M. L. Shooman, Software reliability: analysis and prediction, Report AGARD AG224, Integrity in Electronic Flight Control Systems, pp. 1-17 (1977). 3. J. D. Musa, Theory of software reliability and its application, IEEE Trans. Software Engng SE-I, 312-327 (1975). 4. B. Littlewood, Stochastic reliability growth: a model for fault removal in computer-programs and hardwaredesign, IEEE Trans. Reliab. R-30, 313-320. (1981). 5. A. L. Goel and K. Okumoto, Time dependent error detection rate model for software reliability and other performance measures. IEEE Trans. Reliab. R-28(3), 206-211 (1979). 6. S. Yamada and S. Osaki, Non-homogeneous error detection rate models for software reliability growth, in Stochastic Models in Reliability Theory, pp. 120-143. Springer-Verlag, Berlin (1984). 7. S. Yamada, M. Ohba and S. Osaki, S-shaped reliability growth modelling for software error detection, IEEE Trans. Reliab. R-32, 475-478 (1983). 8. M. Ohba, Software reliability analysis models, IBM J. Res. Dev. 28(4), 428-443 (1984). 9. S. Bittanti, P. Bolzern, E. Pedrotti, M. Pozzi and R.
Scattolini, A flexible modelling approach for software reliability growth, In Software Reliability Modelling and Identification (edited by G. Goos and J. Hartmanis). Springer-Verlag, Berlin (1988). 10. J. G. Shanthikumar, Software reliability models: a review, Microelectron. Reliab. 23(5), 903-943 (1983). 11. S. Yamada and S. Osaki, Reliability growth models for hardware and software systems based on nonhomogeneous Poisson processes: a survey, Microelectron. Reliab. 23(1), 91-112 (1983). 12. P. A. Caspi and E. F. Kouka, Stopping rules for a debugging process based on different software reliability models. Int. Conf. on Fau#-tolerant Computing, 14, Kissimmee, pp. 114-119 (1984). 13. E. H. Forman and N. D. Singpurwalla, An empirical stopping rule for debugging and testing computer software. Am. statist. Ass. 72, 750-757 (1977). 14. P. K. Kaput and R. B. Garg, Optimum release policy for inflection S-shaped software reliability growth model, Microelectron. Reliab. 31(1), 43-46 (1991). 15. P. K. Kaput and R. B. Garg, Optimal software policies for software reliability growth models under imperfect debugging, R.A.LR.O. (in press). 16. P. K. Kapur and V. K. Bhalla, Optimal release policies for a flexible software reliability growth model, Reliab. Engng Syst. Safety (in press). 17. S. Yamada and S. Osaki, Optimal software release policies with simultaneous cost and reliability requirements, Eur. J. Op. Res. 31, 46-51 (1987). 18. W. Kuo, Software reliability estimation: a realisation of competing risk, Microelectron. Reliab. 23(2), 249-260 (1983). 19. W. Kremer, Birth-death and bug counting, IEEE Trans. Reliab. R-3I(1), 37-47 (1983). 20. D. G. Kendall, Generalised birth and death process, Ann. math. Statist. 19, 1-15 (1948). 21. C. L. Chiang, Introduction to Stochastic Processes in Biostatistics. Wiley, New York. 22. D. M. Young and R. T. Gregory, A Survey of Numerical Mathematics. Addison-Wesley, Reading, MA (1973). 23. B. Keller, Numerical Methods for Two-point Boundaryvalue Problems. Blaisdell, MA. 24. S. M. Ross, Software reliability: the stopping rule problem, 1EEE Trans. Software Engng SE-I1, 1472-1476 (1985).