On Modelling Reliability Growth for Software

Copyright© IFAC Identification and System Parameter Estimation , Beijing, PRC 1988

ON MODELLING RELIABILITY GROWTH FOR SOFTWARE M. Xie and B. Bergman Dil'ision of Quality Technology, Department of Mechanical Engineering, Linkoping Institute of Technology, 5-581 83 , Linkoping, Sweden

Abstract . sciences

An essential parameter of is

the

many

important

increased use of software and the unavailability practical

systems

in

engineering

reliability of software. In social sciences there is also an

problems.

There

are

of

it

usually

causes

many

several models suggested in studying software

reliability. However, one of the widely used assumptions

is

that

all

faults

contribute the same amount to the failure probability . In this paper we present some new models in which this assumption is not used . A software tion

process

is

modelled

by

a

Markov

process

with

decreases as the number of faults detected increases .

fault

detec-

jump intensity which

The

intensity

function

may for example be a power type function of the number of the remaining faults . A sampling argument suggests that a second order power function may be

a

good

approximation of the reality . We also present some numerical example on estimation model parameters with the maximum likelihood function. Keywords.

Modelling;

stochas t ic system;

software reliability;

reliability theory;

In

INTRODUCTION In the last several years, much research has

been

is

the

most

important

product.

quality

parameter

Section

2

new

models

are

described.

Some

modification of the study of the

new

shortly

3 some numerical

presented.

In

Section

models

are

examples are given.

carried out in studying software reliability which software

parameter estimation;

failure intensity.

of

It is now recognized that many

practical problems have arisen due

to

A GENERAL MODEL AND SOME SPECIAL CASES

unreliable

software, e.g . from the use of software containing faults. Many software reliability models have been

One

introduced,

reliability

see

e . g . Goel (1985) and Ramamoorthy

and Bastani (1982) . Some of them have been

of

the

most model

sively

studied, like that of the Jelinski-Moranda

(1972)

model,

assumptions are

the

model,

the

Littlewood-Verrall Goel-Okumoto

and

Bendell

(1973)

However,

software

made.

models, some essential

It

is

assumed

also

Musa

likely to cause a failure of the software .

Faults

assumed

ins tan-

of

are

detected time

important to construct new models

which

to

be

removed

taneously. No new faults are introduced during the

the

between failures

probably

of

The

have these properties.

the test . It may be shown that the times

failure

are

rate

exponentially

least

reliability

under

a

distributed.

remains constant until another

fault is detected and removed. software

This

is

also the main assumption of this paper.

tion of the Jelinski-Moranda model (the JM-model).

we

One

removing the i:th

the

unrealistic assumptions made for the

567

at

Generally

use A(i) to denote the failure intensity after

simply have

JM-model is not used for our new models.

true

random testing condition which is

models which embody a generalization and modificaof

all

independent of others and are equally

models are both simple and realistic. Hence, it is

In this paper we study some

that

faults

none

are

reliability

non-

and MelIor (1986) where many

other models are studied.

the JM-model. For

(1979)

homogeneous Poisson process model . See (1987)

software

discussed

perhaps

this model which is perhaps the simplest one among

inten-

all

Bayesian

widely is

fault.

For

the

JM-model,

we

568

M. Xie and B. Bergman

(1)

early

detected

faults should correspond to large

failure rates. whence the decrease of A(i) Here

AO

is

a proportional constant and N is the

should

exceed that predicted by a linear function.

initial number of faults. Another From the assumptions made for the JM-model, we see that some are

more

essential

than

others.

possibility

to

model A(i) which is very

simple is

For

example, it is quite certain that some faults will contribute others.

more

This

to

the

software

failure

than

is an assumption which has received

In (3) the term -1 is added

much criticism from software reliability analysts.

failure

Other

removed.

assumptions

tant

since

are believed to be less impor-

they

are

true

under

some

to

ensure

that

minor

modifications. If new faults are few compared with

Recently Xie (1987) used a shock model in

the existing faults, and if we do

ing

second,

third

not

the

intensity equals zero when all faults are

count

the

failure etc. due to a detected but

modell-

software failures. Each data used in the test

may be treated as a shock to the software.

Denote

not removed fault, the JM-model may be used.

by

the number of data in the input space which

Under

assumption, the failure intensity is a function of

~

causes software a

random

software

testing condition, the

failure intensity of the testing process stant

is

con-

unless the software is changed, for example

through the removal of a software fault. Hence is

quite

reasonable

failure.

the total number of

Under

the

some

remaining

reasonable

data

causing

software failure, i.e.

it

to assume that the time be-

(4)

tween failures is exponentially distributed with a parameter

which

depends on the number of the fa-

ults removed. We assume that all

faults

detected

The failure intensity decreases

decreases

as

this

number

due to the removing of faults. Assuming

are removed immediately and that no new faults may be introduced, in agreement with the assumptions

Mk to be a random variable and using a Size-biased

made

sampling

for

the

JM-model.

It follows that Ti' the

time between the i:th and (i+1):th failure of software,

is

exponentially

the

theory.

we

distributed with pa-

rameter A(i). Since A(i) is obviously a decreasing

For example. if Mi is

function

then

of

may approximate the failure

intensity A(i).

i, such a model will be called a DFI

a

geometrically

about twice as large as the average

(Decreasing Failure Intensity) model.

distributed.

randomly chosen fault has a size which is size

of

the

remaining faults. The failure intensity A(i) after Let

N(t)

ults

denote the cumulative number of the fa-

detected

assumption,

until

the

time

failure

t.

Then

process

Markov process. The advantage of models

is

by

(N(t),t~O)

the

DFI

be

is a

Markov

removing the i:th fault is proportional to the the total

easily

solved.

see

a

N

equations

A (i)

function

A(i)

is

a

the

power

failure

N M. - 2· J

~

j=i

0:

M. / (N-i+1) J

N-~-1 ' A(i_1) N-l.+1

(5)

N-i-1 N-i ~ N-2 N-i+1'N-i+2' ... ·N_1 · N· A(O).

type function of the Hence, we have that A(i) ~ constant· (N-i)2

(2)

where N, A ' and a are constants. The value of a O is expected to be greater than 1 which is the case reason

~

N-i-1 N N-i+1' ~ Mj j=i

intensity

number of the remaining faults. i.e.

for the JM-model. The

0:

j=i

e.g. Xie (1986) for

simple power type DFI-model studied by Xie

(1986) it is assumed that

mathematical

term this is

details. For

size of the (N-i) faults minus two times of

the average size of these faults . In

that (Pi(t)=P(N(t)=i)} which satisfies

the so-called Kolmogorov's differential may

our

for

this

is

that

which

(6)

corresponds to (2) with a=2. As pointed out

before a value of a>l is very reasonable since

~.

On Modelling Reliability Growth for Software TABLE 1

the number of data causing failure due to the k:th

Some Empirical Test Data from Littlewood (1980)

fault, are not equal for all k.

SOME NUMERICAL EXAMPLES

For any DFI-function A(i), generally

can

be

the

estimated

model by

parameters

maximizing

likelihood function for the given that

model.

the

Suppose

the software system is tested until the n:th

fault is detected and corrected and that obtained where

the

t.

is

1

failures

collection the

during

of

i:th the

test

time

test.

569

have

data (ti,iSn)

interval Then

we

100 successive execution times between failures: 3, 3D, 113, 81, 115, 9, 2, 91, 112, 15, 138, 50, 77, 24, 108, 88, 670, 120, 26, 114, 325, 55, 242, 68, 422, 180, 10, 1146, 600, 15. 36. 4. o. 8. 227, 65. 176, 58. 457, 300. 97. 263. 452, 255, 197, 193, 6, 79, 816. 1351, 148, 21. 233. 134, 357, 193. 236. 31. 369. 748, O. 232. 330, 365. 1222. 543, 10, 16, 529, 379. 44. 129. 810. 290. 300, 529. 281. 160, 828. 1011. 445. 296. 1755. 1064, 1783, 860, 983, 707, 33, 868, 724. 2323. 2930. 1461. 843. 12, 261. 1800. 865. 1435

between

the likelihood

For Model I no reasonable solution has been since

found

the likelihood function increases slowly as

N and a increases . However, for any fixed a we can find the maximum likelihood estimates of N and Al '

function is

It may thus be of interest to present the

numeri-

cal results gi ven in Table 2. Note that the larger L(n,A(') )

the value of

the

a,

more

conservative

is

the

predicted failure intensity for the next failure . and hence the log-likelihood function is Simply TABLE 2

ML-Estimates of N and Al in Model I for Different Fixed

l(n,A(')) = log L(n,A( ' ))

a

after 50 Observations

n-l

=

L {In(A(i)) - A(i)t.} . j=O 1

(8)

For

DFI-functions AI(i) and All (i) in Model I and

11,

l(n,A('))

numerically.

is

most

Recall

conveniently

that

AI(i)

and

(9)

(10) the

scale

a

or

p

through

the

condition

"k

log

58 82 107 131 156 181 230 280 410 530 1040 1540 2040 2540

2

3

4

5 6 8 10 15 20 40 60 80 100

i

~

-308.571 -308 . 363 -308.251 -308 . 187 -308 . 145 -308.117 -308.081 -308.058 -308.027 -308 . 011 -307.988 -307.980 -307.976 -307·973

.~a

~(n+l)

0.0110 0.0124 0 . 0131 0.0136 0.0139 0.0141 0 . 0144 0.0146 0.0146 0.0149 0 . 0150 0 . 0151 0 . 0151 0.0151

0.00152 0.00189 0.00198 0.00199 0.00201 0 . 00202 0 . 00203 0.00204 0.00208 0.00206 0.00209 0.00208 0.00208 0.00208

1

parameters Ak' k=l,2, can be

determined for given N and

a -a'

1

maximized All (i) are

defined by

However,

~

a

In Table 3 below we give the ML-estimates of N, A2 and p for model 11. The estimates are quite

l(n , A( ' ))

0,

(11)

TABLE 3

ML-Estimates of (N,A ,@) in Model 11 2

which gives for Model I, n-l

n.{ j=O (N-i )a . t .}

n

-1

1

For Model 11 we have,

The

above

n-l peN-i) }-1 . L (e .t . { j=O 1 procedure

(13)

is applied to the test data

available to us . Here we give only examples the

data

(1980) .

(in

Table

1)

~2

~

log

570 540 370 420 490 640 600 650 600 460 320 120 140

3· 59E-13 l.08E-ll 5·72E-09 1 .92E-08 8.40E-09 2.01E-I0 3. 49E-09 6.48E-09 1.1OE-08 7 . 58E-08 3.84E-06 1 . 1OE-03 4.75E-04

0 . 043 0 . 039 0.040 0 .032 0.029 0.028 0 . 025 0 . 022 0.023 0 . 026 0.025 0.020 0 . 023

-239·325 -272 . 216 -307.964 -340 . 465 -374.403 -409.592 -443.604 -477.784 -514.386 -554.842 -592.605 -635.266 -674.490

i

~(n+1)

(12)

L

n·

~

taken

from

using

Littlewood

40 45 50 55 60 65 70 75 80 85

90 95 100

0.002838 0 . 002616 0.002072 0.002274 0 . 002188 0.001979 0.001980 0.002016 0.001727 0.001300 0 . 001202 0 . 000714 0.000717

M. Xie and B. Bergman

570 reasonable.

but

still they are unstable. This is

especially true for the estimates of N.

reliability. models

All

existing

i.e. the reliability does For

the

sake

of comparisons we give a numerical

software

reliability

agree with each other on the first aspect. increase.

However

the

rate of this growth may be better modelled.

table similar to Table 3 for the model I with 0=2. Note that the estimates are more stable than those

The

given in Table 3. It should be

both simplicity

the

value

0=2

pointed

out

that

is perhaps the most realistic one

among all reasonable values of o.

DFI

parameter

0

and

realism

into

account.

The

in (2) may be approximately equal to

the rate of the size of a random detected fault to the

TABLE 4

Markov model studied in this paper takes

ML-Estimates of (N.A11 in Model I with 0=2

average size of all faults. Hence. if not all

faults are as likely to be detected as others. the value

of

0

should exceed unity. This means that

the rate of the reliability growth is faster

than

modelled by the JM-model. ~

n 40 45 50 55 60 65 70 75 80 85 90 95 100

log

77 88 82 100 110 113 127 140 140 129 134 128 134

i

~

0.0126 0.0117 0.0124 0.0109 0.0103 0.0102 0.0094 0.0089 0.0089 0 .0095 0·0092 0 . 0096 0 . 0091

1

.~o

~(n+1)

-239.881 -272.723 -308.363 -341 .028 -374.982 -410.144 -444.256 -478.484 -515.059 -555.166 -593.118 -635 . 480 -674.942

From the results. it

may

be

REFERENCES

0.00292 0.00280 0.00189 0.00220 0.00213 0.00184 0.00190 0.00192 0.00164 0.00110 0.00099 0.00064 0.00059 expected

State of the Bendell. A.• and P. MelIor (1986). art report: Software reliability . Infotech Limited. UK. Goel. A.L. (1985). Software reliability models: assumptions. limitations. and applicability. IEEE Trans. Software Eng.. SE-ll. 1411-1423. Goel. A.L .• and K. Okumoto (1979) . Time dependent error detection rate model for software reliability and other performance measures. IEEE Trans . ReI . • R-28. 206-211.

that

the

estimate of A(n+1) after observing the nth failure is both reasonable and stable. This is

an

impor-

tant future.

Littlewood. B. (1987) . Software Reliability: Achievement and Assessment. Blackwell Scien. Publications. Oxford.

DISCUSSION The are many software reliability models suggested and studied . Some unrealistic assumptions are made for

each

of

them. This is why there are so many

different models . In addition to the study of types

of

models.

new

another possibility is to con-

struct models through removing

or

replacing

the

most unrealistic assumptions made for the existing models. The advantage is that it is easy to see if the new model is better than the old one . There are many parameters which are interesting in studying the reliability

of

a

software

system.

e.g. mean time to next failure. number of software faults remaining in iability

growth

the

is

software reliability. crease

the

system.

etc.

first

order

The

reliability

The

rel-

effect of does

in-

when faults are removed . The second aspect

which should be studied as a consequence of removing

a

fault

Jelinski. Z.. and P.B. Moranda (1972) . Software reliability research. In W. Freiber~er (Ed.). Statistical Computer Performance Evaluation. Academic Press. New York. pp. 465-484.

is

the

rate

of

increase

of the

Littlewood. B .• and J.L. Verrall (1973). A Bayesian reliability growth model for computer software. J. Royal Statist. Soc .• Series C. 22. 332-346. Musa. J.D .• A. Iannino. and K. Okumoto (1987). Software reliability: Measurement. prediction. application. McGraw-Hill. New York. Ramamoorthy. C.V .• and F.B. Bastani (1982). Software reliability - status and perspectives. IEEE Trans. Software Eng .• SE-4. 104-120. Xie. M. (1986). Some general software reliability models and their applications. LiTH-IKP-R426. Link~ping Institute Technology. Sweden. Xie.

A shock model for software M. (1987). failures. Microel. ReI .• ll. 4. 717-724.