An Approach to the Concurrent Software Testing

Copyriglll Blldape~,_

©

IFAC 9,h Triennial \\"orld Congl-ess I-Iullgar\'. 19H-I

AN APPROACH TO THE CONCURRENT SOFTWARE TESTING K. Kuchcinski and B. Wiszn iewski IIISlilll/(' of ComplllPr ScinlcP. DpparlmPlI1 of Elpclrollics. Po/ilpchllika Gdallska. Gdallsk. Polaml

Abstract . An attempt to provide some definitional framework for the concurrent system testing is made here. A model of a concurrent system is introduced and the main problem concerning test reproducibility is specified. Authors show that such tests exist and provide a basis for concurrent software verification strategies. They make some brief comments on it, proposing some extensions of the path testing strategy. Some problems for further investigation conclude the paper.

Keywords. Petri Nets, reproducible tests, concurrent computations.

INTRODUCTION

(synchronous experiment ) for the concurren t system implemented in a special way. In this paper, however, an attempt is made to find the gene r a l formula for the concurrent system test reproducibility to enable their effective debugging.

Concurrency has been applied to the most real-time control systems in a natural way for a long time. Many systems based on it were successfully implemented by engineers and several theories developed by computer scientists. There is not, hOl.,ever, sound mathematical basis for such systems development and verification. Since the early 1960s many different models of concurrent computations have been described by Keller (1976 ) , Kasai (1982) and others . These models are used to prove some properties of the algorithms to consider a number of problems which arise in concurrent computations like deadlock, starvation etc. There are, however, models and related notations \.,hich are introduced to verify or prove concurrent programs (Keller, 1976 ) . As is often, the methods for formal verification and/or concurrent programs proving become very complex . There are many definitions, lemmas, theorems and a long list of results. However very important they are many problems arise applying them to the program testing. Since nobody can say that a program being tested is correct, except some trivial examples, we cannot debug it comple tely . In practice, some criteria are applied to conclude a testing experiment. That is why the whole experiment plannin g plays a key role in achieving "save" soft~yare. It is mo r e comp1ex to do that for concurrent programs than for sequential ones, for which we must solve only the test representativity problem (see Wiszniewski, 1983 ) here we deal also with the test reproducibility which must be solved first. Our aim is to propose in the paper an approach to that ;>roblem . \,e begin from the introduction of a concurrent system model based on Petri Nets. Having one defined, we introduce a scheduling function to describe how the concurrent system operates in its evironment. Finally, we define reproducible and partially reproducible tests and prove their existence.

CONCURRENT SYSTEM TESTING To consider concurrent systems testing it is necessary to introduce some definitional framewo r k . We have the fo l lowing model:

De f i ni t i on: By a concurrent system, written as CS we shall mean a quadruple given as CS= (P,A,M,C ) , where: o P is a set of places, A is a set of actions and p" A=0, 11 is a relation given as Mep x A x P combining places with actions, and C is a set of marked places, called b r ief l y the o initial configuration. Denote the set of places preceding an act i on a E A by I (a ) and following one by Ola ). By v l pl we mean here the number of markers in p €. P. Each action a € A is specified by two sets of memory locat ions: a domain D~ ) and a range R(a ) and has attached an unary predicate q . I,e say that an action a E; A may be performed in a ~i ven CS iff:

lV H

I (al )

lv l p)

:> d

lV pe I ( a ) -0 ( a )l l v'

I p)

v l p)

lvpE.O ( a ) - I la l) ( v'

( p)

v Ip)

(V peO

( a)1)

for e E. D I a I,

I

l al) ( v' ( p)

This approach is not new. There are papers on concurrent systems testing. In a couple of them, Brinch-Hanscn (1973, 1978 ) shows that one can construct a reproducible test for a monitor and proposes a testing method for a concurrent system kernel . Akkoyunlu ( 1978 ) uses a reproducible test

v l pl}

qa (X cont l e II =true, and

X cont (e 1-X cont l e e E D (a )

-1) +1)

I

e E R (a )

where cont (e) denotes the contents of a r elated memory location e, _ denotes a relation called briefly a transfer and v' I p is the new value assigned to v ( p ).

1

2733

2734

K. Kuchcinski and B. Wiszniewski

However general this model is, it sometimes suffices to limit it to marked graphs or state machines {Lauer, 1979). In particular, if we represent by our model a set of state machines, with exactly one marked place lv I p)~l) each, we have a model of cooperating sequential processes. Every state m~chine chosen from that set represents a sequentlal program. To complete our model we define configurations and states.

Definition: i\ A concurrent system configuration is a subset C ~P, where p€C implies vlp);)l, and ii) A state ration C is exactly one and I(a) •

in a given concurrent system configua subset S C:C for which there exists action a e.A-which may be performed S.

If we consider for example, cooperating sequential processes, we may thought a vector of values representing a set of program counters as a configuration, with each vector component as a state. Finally, to express the semantics of a concurrent system we introduce a s chedu~ing f unction SF defined as: SF: SxCxE -S, where S is a set of states in the system, C is a set of its configurations and E is a set of discernible events. This function determines, for each triple r:presenting a current state, a system configuratl0n and an occurin g event, what is the state to be entered. States and configurations are internal to the whole CS structure while events are external; they may occur at any moment, that is for configurations and states, both chosen arbitrarily. The next state is determined by each action accordingly to the set of rules specified before. Since we have defined concurrent systems, we consider their testing. Let call a path, from a state S CC to S c: C , in CS the nonempty ~tring ofOactigns al~a~, ~ •. ,a such, that there lS one sequence of conliguratigns CQ' Cl ' .•. , C _ ' C and for all i=l, ..• ,n I(a.). S. CC. n 1 C'R :: C and Ora.) Where s. C:C. 'where,-1- 1-1' o ') .1 . 1 1 ' C Ca' Before speclt y lng sets of data for CS n we deflne the follo"in g :

=

=

Definition : A set T of input data for which there is an exactly one path performed in a concurrent system is a r ep roducib~e test. Test reproducibility must be a basic feature for sets used in concurrent systems testing. For sequential programs we have that, once a domain of a path performed by one is identified, all program executions for data chosen from that domain concern only that particular path. Such set is a reproducible test for that program. An existence of reproducible test s i s not, however, obvious for concurrent systems defined here. They are event-driven systems, i.e. scheduling function does depend on occurin g e ve nts, not only internal transformations of data. We want to show, that there exists a strate gy, "ithin which it is possible to build such tests. Before doin o it we consider concurrent systems for which ~he ~et E is empty, that is there are no occuring events at all. Such a system is called here a quasi Concu:rent System or bri e fly QCS. A scheduling functIon SF has henceforth only two arguments and is given as: SF : SxC -S. Any QCS is a set of cooperating se q3ential pro grams. A schedulin g in QCS systems is in fact a simple switching of programs: for a given confi guration and a state of a currently executed pro gram a decision is taken which program is to be c ontinued next. There are'

two possibilities: to continue a given program or to leave one and continue another. In the latter case we say that the s witch is done. For QCS we have the following lemma:

Lemma 1 : There does exist for each QCS at least one set of tes t data ,.hich is a reproducib le tes t. ~ro of ( sketch) : We will show that for every QCS, If only a chosen final state sec is entered from a given initial one S C ~ere does exist such a set of test data fo~ whiSh an only one path is performed in that QCS. Assume, however, there are several and collect them all in one program, ~alled here a program of activat ion s. This program IS constructed here from conditional and transfer operations. It operates on the whole set of memory locations and a single abstract variable, given as an ordered pair in the following way: for an input value and the current values of associated predica~esQq , the decision is taken what is the next data t¥ansformation to be performed. After that, new data are computed - among them a new value for is obtained. Decisions and transformations are performed until the value ~Sn'~R:> is finall~ reached. Since SF is a func ,on In our model, ln other words, dec~sions 7 determIne exactly one data transformation each time, and the program of activations input domain contains a single value ' we have that all paths in this program a~e rnfeasible exept one pa:ticular path. We call it he r e an active path ; th,S concept was originally introduced in less formal way by authors (1982). Therefore, a given QCS run for a set of data identified by all values involved in this path's computations, performs only one path. Thus, by our definition, such a set of data is a reproducible test and exists for each path performed in QCS."

e-

A result obtained here may be applied to problems raised originally by Brinch-Hansen (1978), concerned on the reproducible testing of monitors. Since each set of cooperating monitor functions is, by our definition, a certain QCS, the question whether there exists a r e producible test for a given monitor, may be answered positively. Sequences of monitor function calls, which must be applied in certain order to verify one, are in fact identical with a sequential program testin g and are used only to transform some internal data, not specified explicitly in one's input domain. We extend this result now to those concurrent systems, which are event-driven: that is states computed by a scheduling function depend on occuring events. Let define the following:

De f inition : A set T of input data for which there is a set of paths a w a w ... w la, performed in a concur1 1 2 2 rent system, Where n'5 C~ a. e A, w. finite and wi E A" , i=l ..• n, is callea a partially r eproduc1..b~e test. This definition is necessary to consider an eventdriven system, where events may occur at any moment. Thus, several distinct paths may be performed in a given system for a single test, each one corresponding to a certain event. Notice that each w can transform data computed by further actions i a (i < j) in that path. We say that side-effects i occur. Let define them formally:

Definition If for the given set of paths a 1w a 2w2 ···w _la 1 defined previousl y , the followin g conaitioR hords:

Concurrent Software Testing

F

U

D (a .))

i
, where

2735

producer

J

i=l, ... , n-l

we say that this set is free of side- effects . This definition does not, however, cover all sideeffects in the system and is introduced for the following lemma.

consumer

Lemma 2: There does exist at least one partially reproduc ible set of data for each CS which has at least one set of paths free of side-effects. Proof (sket ch ): We will show, that for every CS , if only a chosen final state sec is entered , ~ere does exist from a given initial one S C a set of test data, for wh~ch a Qset of paths a lw 1a 2w2 :·. w _ 1a u ' where ait A, wi finite and n w.e A , 1=1 ..• n, 1S performed. Le t construct, as w~ did it in a "roof of Lemma 1, a program of activations, collecting all paths from that set and consi der how the y are perfol~d. However there are conditional and transfer operations, some of them are executed in a sl i ghtly different manner and depend not only on a c urrent state , CS configuration and ottored data but also on occuring events. That is, in our sequential program of activations, a decision which data transformation is to be performed next, is taken not only for sto r ed data and a value of an abstract variable < S ,C> but also for the value of another abstract one. This variable is Boolean and set true if a given event has occured. Hhe n a chosen data transformation is perfo rmed, new data are computed - among them a new value for . Decisions and transformations are performed until the value is finally r eached. Since SF i s a functioR, exact l y one transformation is determined each time; if a decision preceding it does not need any Boolean variable, i. e . does not depend on an occuring event, only stored data determine that transformation. In another case , an occuring event determines one from several which may be performed. The former transformations are relat ed to actions a.€ A, i=l .•. n in the mentioned set of paths, wfiile the latter co rrespon d to some sets of actions performed for certain events. Since our set of performed paths is free of side-effects no data may be spoiled by actions executed for previously occuring even t s. Therefore, a given CS run for a set of data, identified by all values involved in computations of the related program of activations, performs all paths from our set . Thus, such a test is partially reproducible and exists for each CS with at least one set of paths free of side -effects . •

e-

As we see, concurrent systems which are event-driven may behave for the same set of data (partially r eproducib l e ) in seve ral different ways . Strings of actions, performed in CS for ouch data, contain some chosen act i ons a.€ A, performed in a given order , interleaved ho~ever, with certain substrings wi E A*. Let conside r for example a simple CS consist in g of two sequential processes: producer and consume r sharing a buffer and an event-driven alar m process given in fi g . I:

alarm

Fig. 1. An examp le concurrent sys t em Let discuss informally producer's actions. Action a I provides a portion of data to be appended to tRe buffer by its further actions. To share a buffer two actions are needed. They determine two following states: non-waitinB \P03) an d waiting (P 04 )' and have attach~d two predicates: Q01.given as count er=O and Q B1ven as counter=l. Act10ns 04 aOS and a append a portion to the shared buffer, 06 wfi11e a feeds back all producer actions. Consu07 mer act10ns are complementary . To define the scheduling f un ction assume that EllEO means that the event has or has not occured respectively. Since the initia l configuration in our CS is Co (POl , "12' P211, the scheduling is as follows: P06' (P06,P 12,P21) , EO -P1 2 P16' (P0 1,P16,P21) , EO -POI

(1\ (2)

P0l' \P07 , P 16,P2 1) , Eo -P16

(3)

P17' (P 06,P 17,P21) , EO -P06 Po I' ( PO l '!'17'P 21 ) , EO - P 17

(S)

(4 )

(6 ) Pl2 ' (P 0l'''12,P2 d, EO- P07 arbitrary \7 ) where Si ,C Sl, Cj ,E I - P2 1' j for given ; chosen C (8) where S. P21 ,C j ,EO~Si' j 1 Decisions (1) and ( 2 ) con cern the waiting state determination for pr oducer and c onsumer respectively, whi l e (3\ and (4) concern the waiting state exit . pr oduJ'er and co" suoner conditions after completion of their buffer ope ration s are given by Is) and (6) respectively. Finally (71 and (81 are taken if an event occurs . Remaining decisions preserve the actual state and may be described as: S. ,C., Eo- S., whe r e S. arbitrary and C. diffefent from (1) .. ~(6). Consider first QCS, i~e. no events occur. If produce r starts the who le QCS operations we have the sequence of actions determining reproducible set of test data given in fig. 2 .

2736

K. Kuchcinski and B. Wiszniewski

a

Ol

a

a

03

05

a

a

07

nl a 04 a 13 a 15 a 06 a 07 a 17 all a 14 a Ol a 03 a 05 a 16 a 17 all a 07

t

t

f

preserve 5tate

( L,)

(1 )

f

t

(5)

(2)

t (3)

t

(6)

a'J 1 a 03 a 21 a 05 a 07 a 21 a 21 a 21 a 0 1. ••

t t

(7) (8)

t t

f

t

(7) (7) (7) (8) Fig. 2. Strings of performed actions

To complete our example, let consider events.

CONCLUSIONS

Again, if pr oducer starts the whole CS operations we have sequences of actions determining partially

reproducible set of test data like this given in fig. 2. Notice, that if the range of alarm actions is not disjoint with domains of pr oducer and cons umer actions (see our definition) the scheduling may be disturbed. PRACTICAL ASPECTS OF TEST DATA SELECTION We discuss now in some detail practical aspects of a concurrent system testing, where to detect and localize a bug it is necessary to identify data, for which it reacts uniquely. For sequential programs it is rather obvious to adopt a path testing strategy: once input data, for which a given path in one is performe d, are identified, its computa-

tions may be verified. Our results help to extend this concept to concurrent systems: once input data, for a given active path performed in one, are identified, its operations may be verified. We may do it exactly as we do for sequential programs. In authors opinion, there are many testing situations in which, if one is conce rned on some particular

actions, other substrings may not be considered. This approach was succesfully applied by authors to the concurrent system d~scribed by them (1983) This idea may be explored twofold by:

i)

creating a testing environment, where all

events are determined, in other words attached to certain decisions in the related program of activations, or

ii)

monitoring stored data for each action a.E A, performed by related paths to check theif consistency.

Concurrent systems perform, finitely or infinitely, many active paths as well as sequential programs do. To test a system the test data set should cover possibly all of them. Such tests are called here re liable ones. Reliable tests select those paths in a system which represent some others, making the whole verification process effective. Owing to our result, we may adopt successfully to concurrent software strategies developed for sequential program testing. Particularly, path computations provide an efficient base for some effective procedures like that one descibed by ~iszniewski (1983). In general we do not intend to fo1101, here test data reliability aspects in the verification of a concurrent system nor~o present any particular strategy for their selection either. Related problems are, as so far not investigated enough,

although Akkoyunlu (1978) has made an interesting step toward them. He has proposed some restrictions to be imposed upon the whole concurrent system implementation in order to eliminate time-dependent errors. Each synchronous

ex~eriment,

or as we call

it here, an experiment with reproducible sets of data, carried for a system built in a way de fined by him is equivalent to the asynchronou s one . Those restricted methodologies, however, do not cover the whole class of concurrent systems (recall again one described by authors before, 1983). Anyway, they deal with the major problem: how to detect/avoid time dependent bugs.

In our quest for providing a firm basis for concurrent systems testing we have introduced a model called a concurrent system (CS). We have specified the main problem of a test data set reproducibility (or partial reproducibility) and then showed that such tests exist. Having them defined, we have attempted to provide some necessary insight into the real world of the concurrent systems verification, making some brief comments on related strategies. Our model covers wide class to concurrent systems with shared resources. It may be a processor shared by cooperating sequential processes as well as multiprocessor system sharing a memory unit. However, we discuss here concurrent software testing, our

results concern system development too. Side-effects may be eliminated not only by testing but also by design methodologies adopted in a project (e. g. firewalls). This paper is general in several ways. First, there are some properties and implementatory details of the scheduling function, that we have not touched on here. The user, who knows our results may adopt his own testing strategy; of course, there may be another approaches to this problem, not based on the path testing strategy, e.g. data flow analysis. Finally we raise here some open questions, that could be studied . One is the classification of the bugs which may be detected. Next, is the test reliability. Another two are, whether there exist effective procedures for test data generation and how they are effective. All those results, if obtained, could provide a way to improve the whole debugging process of concurrent systems making them powerful and reliable.

REFERENCES Akkoyunlu,E.A., A.J.Bernstein, F.B.Schneider and A.Silberschatz (1978). Conditions for the equivalence of synchronous and asynchronous systems. IEEE Trans. on Software Eng., SE-4, 507-516. Brinch-Hansen P. (1973). Testing a mUltiprogramming system. Software - Practice and Experience, 3, 145-150. Brinch-Hansen P. (1978). Reproducible testing of monitors. Software - Practice and Experience, 8, 72 1- 72 9 • Kasai T. (1982). Homomorphisms between models of parallel computations, J.Comput. System Sci., 25, 285-331. Ke11er R.M., (1976). Formal verification of parallel programs. Comm. ACM, 19, 371-284. Kuchciciski K., B .Ihszniewski (1982). Verification methodology for non-preemptive concurrent systems. Proc. of the 5th International Conference FTSD'82. Katowice, Poland. ZG Polo Sl. 1983.

Concurrent Software Testing Kuchcinski K., B .l-/iszniewski (1983). A man-machine intercommunication facility in well structured concurrent systems. Proc. of the 12th IFAC/IFIP Workshop on Real-Time Programming, Hertford, England. Pergamon· Press 1983. Lauer P.E., H.W.Shields and E.Best (1979). Formal theory of the basic COSY notation. Technical Report, no. 143, Univ. of Newcastle upon Tyne, England. Wiszniewski B. (1983).

An

automatic verification

of sequential program computations. Proe.

of the 22nd International Symp. on llini/l1icrocomputers and Their Applications lIIMI' 83. Lugano, Switzerland, Acta Press 1983.

IWC5-N*

2737