- Email: [email protected]
Generating test cases from class vectors
The Journal of Systems and Software 66 (2003) 35–46 www.elsevier.com/locate/jss
Generating test cases from class vectors Karl R.P.H. Leung a, Wai Wong b, Joseph Kee-Yin NG
b,*
a
b
Compuware Software Testing Laboratory, Department of Information and Communications Technology, Hong Kong Institute of Vocational Education (Tsing Yi), Hong Kong Department of Computer Science, Hong Kong Baptist University, R702 224, Waterloo Road, Kowloon, Hong Kong Received 27 August 2001; received in revised form 3 January 2002; accepted 15 April 2002
Abstract Specifications are the primary source for obtaining test cases in software testing. Specification based testing is becoming more and more important when black box components and COTS are widely used in software development. An important issue in system testing is to identify all the legitimate input. One of these systematic approaches is deriving test cases from classification tree. This approach partitions the input domain into classifications which are further partitioned into classes. Test cases are combinations of classes. Relations between classification and classes are identified and are used to construct the classification tree. From the classification tree, combination table is constructed. Test cases are derived from the combination table. However, human decisions are required in determining whether test cases derived from the combination table are legitimate. This problem is incurred by the limitation of the expressive power of classification trees which cannot express the relations among classes precisely. We propose an enhancement by expressing the relations among classes and the relations among classifications directly in vectors. We call this new approach Class Vectors. This paper presents the class vector approach with formal definitions of basic concepts and entities. We find that the expressive power of class vectors is higher than classification trees, that is, this approach can express the information given in the specification in a better way and derive all legitimate test cases with minimal human decisions. Furthermore, a method of generating legitimate test cases using class vectors is described. This method is derived from the formal semantics of class vector, hence it is theoretical sound. Finally, we discussed that the proposed method requires the least amount of human decisions, can be highly automatic and has good usability. Ó 2002 Elsevier Science Inc. All rights reserved.
1. Introduction Specifications are one of the major references for deriving test cases. Specification based test case generation is becoming more and more important together with the increase in the size of software systems and the widely use of components in software development. All the legitimate inputs form the basic testing oracle of a system. One important issue in system testing is to identify all the legitimate inputs from the specification effectively. Each of these legitimate inputs is a test case of the system. This paper reports our approach of using class vectors in generating test cases. * Corresponding author. Tel.: +852-2339-7864; fax: +852-23397892. E-mail addresses: [email protected] (K.R.P.H. Leung), [email protected] (W. Wong), [email protected] (J. Kee-Yin NG).
One of the software testing methods is functional methods which include, for instance, functional partitioning and cause-effect graphing (Myers, 1979). The ‘‘category-partition method’’, on the other hand, is one of the promising approach which gives good support for functional testing (Ostrand and Balcer, 1988). The classification tree (Grochtmann and Grimm, 1993, Grochtmann et al., 1995) is a further development of the category-partition approach for deriving test cases. A method is proposed by Chen et al. (Chen and Poon, 1996a, 1997) for deriving classification trees from specifications. Test cases can then be generated from these classification trees. However, some of the test cases generated are illegitimate, therefore, it requires manual selection to filter out the unwanted cases. Although Chen et al. proposed some improvements for re-structuring classification trees (Chen and Poon, 1996b; Chen et al., 1999), the problems cannot be solved completely. It is because the intrinsic restrictions of classification
0164-1212/03/$ - see front matter Ó 2002 Elsevier Science Inc. All rights reserved. doi:10.1016/ S0164-1212(02)00061-4
36
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
tree make it unable to express some of the information from specifications. Hence, there is room for improvement. A classification tree is capable of expressing only three kinds of relations––root-classification, classification-classes, and class-classification. The relations between two classes or among three or more classes are very important in finding legitimate test cases. They cannot be expressed directly in a classification tree construct. In recognising this, we attempted to find an alternative structure to represent the class and classification relationship. We chose class graph as our underlining structure. This study has been reported in Leung and Wong (2000b). We then noticed that the test cases are vectors spanning the input space. After identifying the relations among the classes and classifications, the test cases can be generated directly from the relations without the need of building up an intermediate structure, such as graphs or trees. A preliminary study of using vectors as studying tool has been reported in Leung and Wong (2000a). In this paper, we report our more delineated study of class vectors. With reference to the formal theoretical framework of class vector, we developed a method to generate class vectors. This method is highly automatic and is theoretically sound. It requires the minimal amount of human decisions, has good usability, and has high degree of automation. The rest of this paper is organized as follows. In Section 2, we briefly review the classification tree method (CT) (Grochtmann and Grimm, 1993; Grochtmann et al., 1995), and present a slightly modified version of the case study used in (Chen and Poon, 1997). We then formalize a number of basic notions, such as class and classifications. With reference to this formalization, we define class vectors and relations among classes and classifications in Section 4. We propose a method for deriving test cases by using class vectors in Section 5. This is followed by a discussion in Section 6 comparing the conceptual and practical aspects of our approach with the CT. We then conclude our work in Section 7.
2. The classification tree method The CT (Grochtmann and Grimm, 1993; Grochtmann et al., 1995) is a black box testing technique. The basic idea of this method is to divide the input domain of a test object under different aspects and then combine the different partitions to form test cases (Chen and Poon, 1996a). Each aspect of the input domain is known as a classification. Each classification is further divided into a disjoint set of classes. Each class represents a range of input values to the test object.
This method, as described in Chen and Poon (1997), consists of four steps which are summarized as follows. 1. Identify all the classifications and their classes from the specifications. 2. Construct the classification tree with the classifications and classes. 3. Construct the combination table from the classification tree. 4. Select all the feasible combinations of classes from the combination table. Thus, each selected combination of classes constitute one test case. 2.1. Case Study To illustrate how to apply the CT, we present a case study here. The example is about reward points handling of credit cards which was used in Chen and Poon (1997) albeit with a number of minor changes. This example will also be used in later sections of this paper. The ABC Bank issues credit cards to approved customers. These customers will earn certain ‘reward points’ for every dollar charged to their cards. The number of reward points earned by each customer then determines the type of benefit (e.g. free air-line tickets or free shopping vouchers) which the customer is entitled to. For each purchase, the program shall accept the transaction details together with the various information of the card. Thereafter, validation of these details is performed in order to determine whether this purchase is approved. If the purchase is approved, the program will calculate the number of reward points. There are three classes of credit cards, namely platinum, gold and classic. The ABC Bank offers two types of credit card, namely corporate and personal. Each personal card holder holds a principle card. Personal card holders can apply for additional cards. The credit limit for classic cards are $1000. The credit limit for gold cards and some platinum cards are $4000. The credit limit for the rest of the platinum cards are $8000. Card holders can redeem air-tickets or products with their reward points. Air-tickets can only be redeemed from airlines CX or DG. All classes of tickets, including first, business or economy class can be redeemed from CX. Only first or business class air-tickets can be redeemed from DG. Products can be redeemed from suppliers RedA or TDC. The minimum transaction amount should be greater than $0. The cumulative balance is the total purchase amount of all outstanding transactions. The amount of cumulative balance should be greater than $0 but no greater than the approved credit limit of the card. From the above specification, we can identify the classifications and classes. These classifications are summarized in Fig. 1. From these classifications, a classification tree and a combination table can be constructed
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
37
Each test case is obtained by selecting a class under each top-level classification. In the case where there are more classifications below the top-level one, selecting a class will select all its ancestor classes. The ancestor classes are those classes that are on the path from the selected class to the root of the tree. For example, in Row 1 of the table, ‘first’ is selected which means the ancestor classes ‘CX’ and ‘air-ticket’ are also selected. Therefore, the test case representing by Row 1 is: ðplatinum; corporate; L3; air-ticket; CX; first; A1 and C1Þ: Fig. 1. Classifications and its associated classes.
following the method proposed in Chen and Poon (1997). The upper portion of Fig. 2 shows the classification tree. In the classification tree, the small circle at the top is the general root node which represent the whole input domain of the program. The classifications directly under the general root node are called top level classifications. In a classification tree, a class links to the classifications which the classes of these classifications have some relations with the class. Hence classifications and classes are in alternate levels. Because of this structure, the leaves of a classification tree, called the terminal nodes, are always classes. For example, the class air-ticket links to the classification airline. It is because the classes CX and DG of the classification air-ticket are the airlines which air tickets can be redeemed according to the specification. The lower portion of Fig. 2 where a number of horizontal lines labelled with numbers is a part of the combination table. Test cases can be constructed from this table. Each horizontal line in the table represents a possible test case. There are three cases shown in Fig. 2.
Illegitimate test cases can be derived from the combination table. For example, Row 3 represents such a case. In this case, a classic card has a cumulative balance in C2 which is not permitted by the specification. Although combinations of this type may be interested test cases of exception handling, it is an illegitimate input according to the user specification. Generating test cases for testing exception handling is beyond the scope of this study. Illegitimate test cases in classification trees have to be eliminated by test engineers on a case by case bases. For this simple example, test engineers may have to examine a total of 1296 test cases in order to eliminate the invalid cases. 2.2. Analysis of classification trees As shown in Fig. 2, a classification tree embraces three kinds of relations among the classes and classifications: Root-classification: This kind of relation is represented as the edges from the root of the tree to the top level classification. This relation has little significance to
Fig. 2. The CT and a torso of the combination table.
38
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
the generation of test cases. It simply binds all the top level classifications into a tree. Classification-classes: This kind of relations is represented as the edges from a classification to a class. Since each class must belong to certain classification, these edges simply make the this relation explicit. They also help to structure the tree into an alternating layers of classes and classification. Class-classification: This kind of relations links a class to a classification. They indicate that the class has certain relation with the classes in the classification that is linked. With this rigid tree structure, the resulting combination table, hence the number of generated test cases become large. This includes a number of illegitimate test cases which require manual filtering. The reasons for this are: 1. The structure of the classification trees makes it unable to express the relation between classes directly. Each test case is a sequence of classes one from each classification. Therefore, it is important to identify the combinations of classes which are legitimate test cases from those combinations that are not legitimate. This calls for the ability to represent the relations between classes directly. 2. To overcome the above problem, some classifications are duplicated in a classification tree, for example, the air-ticket type in Fig. 2. This makes the structure more complex. 3. There are still some relations that cannot be expressed by duplicating classifications. For example, in Fig. 2, the classifications card class, credit limit, purchase balance and cumulative balance are related to each other as specified in the specification. (The specification states the the cumulative balance cannot be larger than the credit limit.) The key of these shortcomings is that the structure of classification trees restricts the effectiveness of their expression. To overcome this, we propose to get rid of the rigid tree structure, and use a more flexible approach instead. We choose to use vectors because they are closer to the goal of finding the test cases than classification trees. We also take a more formal approach in which we first define a number of basic notions, such as classes, classifications, class vectors and legitimate inputs. Then, we develop our method of finding test cases based on these formal definitions.
3. Basic definitions of classes and classifications In this section, we analyze the notions of classifications, classes and legitimate test cases and provide formal definitions for these notions.
3.1. Variables and values Let X be an input variable and sðXÞ be a set of values X can take, i.e., the type of X. sðXÞ can be partitioned into a number of disjoint subsets, i.e., [ sðXÞ ¼ sk ðXÞ k
where 8kl k 6¼ l ) sk ðXÞ \ sl ðXÞ ¼ ; Definition 1 (Value of an input variable). A value function Val is defined to return the value of a variable we are now interested in, i.e., ValðXÞ is the value of X. Thus, ValðXÞ will be in sðXÞ and also ValðXÞ 2 sk ðXÞ for some k: Consider a program having n input variables ðX1 ; . . . Xi ; . . . ; Xn Þ denoted by ~ X, the values of these variables form an input value vector ! Valð~ XÞ ¼ ðValðX1 Þ; . . . ; ValðXi Þ; . . . ; ValðXn ÞÞ 3.2. Classification and class Definition 2 (classification and class). Let X be a set such that [ X ¼ fsk ðXÞjsðXÞ ¼ sk ðXÞg k
where 8kl k 6¼ l ) sk ðXÞ \ sl ðXÞ ¼ ;. The X is called a classification while the disjoint subsets are called the classes. Later in this paper, we will also denote a class by xk when we do not need to refer to the input variable X explicitly. Definition 3 (class function). A function ‘class’ on X is defined to return the class in which the value of X lies, i.e., classðXÞ ¼ sk ðXÞ if and only if ValðXÞ 2 sk ðXÞ Suppose there are d classifications X1 ; . . . ; Xi ; . . . ; Xd where ðd P 1Þ. For each Xi , there is an associated set of classes fxki : 1 6 k 6 cj g where cj is the number of classes in the jth classification. The function ‘class’ may be suffixed with a classification to indicate which classification we are referring to, i.e., classY ðXÞ. Extending Definition 3, we can define a function ‘classes’ on the input vector ~ X to return a vector of classes in the form of a d-tuple of classes.
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Definition 4 (classes function). XÞ; . . . ; class Xd ð~ XÞÞ classesð~ XÞ ¼ ðxk11 ; . . . ; xkdd Þ ¼ ðclass X1 ð~ In the case study presented in Section 2.1, 10 classifications have been identified. Thus, d ¼ 10 and, given an input vector ~ X, the function ‘classes’ applying to the input vector will return a 10-tuple of classes ðxk11 ; . . . ; xk1010 Þ. 3.3. Legitimate input Suppose a program having n input variables ðX1 ; . . . Xi ; . . . ; Xn Þ and from its specification, d classifications ðX1 ; . . . Xi ; . . . Xd Þ are identified. The value of a variable Xi will be in some class ski ðXi Þ. The specification will further specify that, giving ValðXi Þ 2 ski ðXi Þ, the value of another variable, say Xj , must be in certain class, say slj ðXj Þ. Thus, we define that an input vector is a legitimate input if and only if, for all i, Xi is in the allowed class according to the specification. In our credit card case study, if in an input vector the value of the variable corresponding to the classification card class equals ‘Classic’ and the value of the variable corresponding to credit limit equals 4000, then this input vector is not legitimate. It is because according to the specification, this combination of input values is not allowed. Definition 5 (legitimate input). A predicate legitimate is defined. It holds with respect to a specification S, a set of classifications X and an input vector ~ X if and only if the value of ~ X is a legitimate input according to S. This is denoted as S
L ðX ; ~ XÞ Consider a program having n input variables ~ X ¼ ðX1 ; . . . ; Xi ; . . . Xn Þ, according to its specification S, d classifications ðX1 ; . . . ; Xi ; . . . ; Xd Þ are identified. Since each classification Xi associates with a set of classes fx1i ; . . . ; xci i g, there are d sets of classes fx11 ; . . . ; xk11 ; . . . ; xc11 g .. .
39
classes ð~ XÞ. The input is legitimate if and only if R holds on these classes. Therefore, XÞ Rðclassesð~ XÞÞ LS ðX ; ~ For our case study, if the input vector ~ X has the values described earlier in this section, then Rðclassesð~ XÞ does not hold. It should be noticed that the relation R partition the set of d-tuples into two disjoint subsets; one represents the legitimate inputs, the other non-legitimate inputs. 3.4. Class vectors We call the d-tuples of classes ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ as described in previous section class vectors. In subsequent sections, we will use class vectors in working out the test cases. Here, we first define an auxiliary function on class vectors to facilitate our discussions. Definition 6 (elem function). The function elem on a class vector V returns the set of classes that appear in the dtuple ðx1 ; . . . xi ; . . . xn Þ, i.e., elemðV Þ fx1 ; . . . xi ; . . . ; xn g: 3.4.1. Class sub-vectors In order to build test cases from class vectors, we define here the concept of sub-vectors. Definition 7 (sub-vector). A class vector V is a sub-vector of another class vector U if and only if elemðV Þ elemðU Þ. Let V1 ¼ ðxi ; . . . ; xj Þ and V2 ¼ ðxm ; . . . ; xn Þ. We define a predicate link as follows. Definition 8 (link). linkðV1 ; V2 Þ if and only if xj ¼ xm , i.e. the last class in the tuple of V1 is the same of the first class in the tuple of V2 . We define an operator chain to form a new class subvector by chaining up two linkable class sub-vectors.
fx1i ; . . . ; xki i ; . . . ; xci i g .. .
Definition 9 (chain function). chainðV1 ; V2 Þ ¼ ðxi ; . . . ; xj ; xðmþ1Þ ; . . . ; xn Þ if and only if linkðV1 ; V2 Þ.
fx1d ; . . . ; xkdd ; . . . ; xcdd g
With the notion of class sub-vector, we define an auxiliary function legitimate over a set of classifications which return a set of class sub-vectors that are formed by the given set of classifications.
From the specification S, a relation R on the d-tuple ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ can be identified. This relation holds, i.e., Rðxk11 ; . . . ; xki i ; . . . ; xkdd Þ evaluates to TRUE, if and only if the d-tuple ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ satisfies the specification S. ! When considering whether a particular input Valð~ XÞ is a legitimate input, we can evaluate the relation R on
Definition 10 (legitimate). Let X ¼ fX1 ; . . . ; Xm g be a set of classifications. legitimateðX Þ ¼ fv1 ; . . . ; vn g where vi ¼ ðx1 ; . . . ; xm Þ; 8j; k : 1 6 j 6 n; 1 6 k 6 m; vj is a class sub-vector of a class vector and xk 2 Xk .
40
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
4. Relation Our class vector approach only rely on one principle relation, the relation between classes. We call this relation co-exist which is discussed in Section 4.1. Based on the co-exist relation, we define independent relation between classifications. This relation is defined in Section 4.2. This independent relations can help to speed up the process when deriving test cases.
Definition 13 (independent classifications). Two classifications X1 and X2 are independent to each other if and only if 8i xi1 2 X1 , xi1 co-exists with xj2 for all j such that xj2 2 X2 . This means that if, for any class in the classification X1 , it co-exists with any class in X2 , then the two classifications are independent. This relation of independence can be extended to any number of classifications.
4.1. Co-exist relation The basic relation in class vector is the co-exist relation between two classes. This relation describes the values of two classes that can co-exist in a legitimate input. Definition 11 (co-exist of two classes). Two classes xki i k and xj j are considered co-exist if and only if they appear in a class vector ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ which is in LS ðX ; ~ XÞ. The co-exist relation between two classes can be extended to more than two classes, thus we have the following definition. Definition 12 (co-exist of more than two classes). Given a set of classes C ¼ fx1 ; . . . xi ; . . . ; xn g; they are co-exist if and only if there exists a class vector V ¼ ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ such that V 2 LS ðX ; ~ XÞ and for all xi 2 C, xi is in the set of classes elem(list(V)). By definition, all classes in a classification are disjoint, therefore, they are not co-exist. In our case study, the class ‘Classic’ in the classification card class cannot co-exist with the class 4000 in the classification credit limit. On the other hand, it can co-exist with the class 1000 in credit limit. Corollary 1. By definition of co-exist relation, if a class appears in a class vector representing a legitimate input, this class co-exist with itself.
Definition 14 (independence of more than two classifications). p classifications X1 ; . . . ; Xp are independent if and only if any p classes which are selected one from each classification are co-exist. From the two definitions above, we come up with the following two corollaries. Corollary 2. An input domain is composed by one or more independent partitions. Corollary 3. The cartesian product of the sub-vectors of all the independent partitions constructs all the class vectors of the input domain, and hence, all the test cases of the input domain. 4.3. Basic class sub-vectors With the co-exist relation, we are interested in the minimal number of classes that are required to determine whether a class is in a legitimate input. We call these class sub-vectors basic class sub-vectors. A basic class sub-vector is defined as follows. Definition 15 (basic class sub-vector). For a given set of classifications X ¼ fX1 ; . . . ; Xn g, a class sub-vector ðx1 ; . . . ; xn Þ formed from taking one class from each classification in X is a basic class sub-vector if either of the following conditions is satisfied: 1. The class sub-vector ðx1 ; . . . ; xn Þ is a legitimate input. 2. X is the smallest set of classifications satisfying the inequality Y cardðlegitimateðX ÞÞ < cardðxÞ x2X
and ðx1 ; . . . ; xn Þ 2 legitimateðX Þ 4.2. Independent Relation One very important relation between classifications is independence. If every class in a classification can coexist with every class in another classification, then we say these two classifications are independent. For example, the classifications customer type and card class are independent since every type of customers may has any class of credit cards.
For example, in our credit card case study, only the class CX is required to determine that class economy is in an legitimate input, i.e. the (CX, economy) form a class sub-vector. But for the class corporate, there is no need to have another class to determine whether it is in a legitimate input. It itself forms a basic class sub-vector. On the contrary, the class sub-vector (air-ticket, CX, economy) is not a basic class sub-vector because neither
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
one of the classes in this tuple need all the other two classes to determine whether this class is in a legitimate input. It should be noticed that a basic class sub-vector may carry no physical meaning. However this is the smallest amount of information to be managed at each time to determine whether a class is in a legitimate input. Let us look at another example. Let A, B and C are classifications. Let {a1, a2}, {b1, b2} and {c1, c2} are classes of A, B and C, respectively. Let (a1, b1, c1), (a1, b2, c2), (a2, b1, c2) and (a2, b2, c1) be the legitimate inputs, then the smallest number of classes in a class sub-vector to determine whether the classes form a class sub-vector is 3.
5. Class vector generation method Based on the definitions of class vector and the Corollaries 2 and 3, we derived a method to generate all the class vectors of an input domain. This method consists of the following five steps. 5.1. Step 1: Identifying classifications and classes This step is similar to the first step of the CT as described in Section 2. In this step, we analyze the input domain using the information given in the specification. The aim of this step is to identify all the classifications that cover the entire input domain, and to partition each of the classifications into a number of classes. This process is guided by the specification which specifies what are the possible ranges of input values. For example, in our case study ten classifications and their associated classes are identified. They are listed in Fig. 1. 5.2. Step 2: Identifying independent partition of classifications By making use of the independence relation among classifications and Corollary 2, we can partition the set of classifications C into a number of disjoint subsets. This is an important property because this partitioning reduces the number of classifications to be handled together in the following step, thus making the tasks in the next step simpler and faster. To illustrate this step, we partition the classifications of our credit card example into the following three subsets: fcustomer type; sub-customer typeg
5.3. Step 3: Input basic class sub-vectors The task of this step is to fill in a basic class sub-vector table for each partition of classifications. The table consists of a square matrix of cells and an optional constraint column. The rows and the columns of the matrix are labelled by the classes of all classifications of the partition. The order of the classes is insignificant. For convenience, the class of the same classification should be put next to each other. The basic class sub-vectors are identified by a number. For example, the class vectors of the example described in Section 4.3 can be identified as in the left side of Fig. 3. The right side of this figure shows the basic class sub-vector table. The table is filled from the first row. For each basic class sub-vector, its number is put in the cells that corresponding to its constituent classes. For example, a ‘1’ is placed in the cells b1 and c1 of row a1 for the first subvector. It should be noticed that since the class sub-vectors express the co-exist relation between the classes and this relation is symmetric, the matrix is symmetric. Therefore, only the upper-right half of the matrix will be filled. Sometimes, a basic class sub-vector consists of only one class. For example, the corporate in the classification partition Eq. (1) is of this type. The number identifying this sub-vector, namely ‘3’, is placed in the cell of the interaction of row and column both labelled by corporate in Fig. 4. More than one number may be placed in the same cell. For instance, if the following two sub-vectors are identified and numbered 1 and 2, respectively: ða1; b1; c1Þ
ða1; b1; c2Þ
then, the numbers 1 and 2 will be placed in the cell of row a1 column b1. The basic class sub-vectors of our credit card case study are captured in Figs. 4–6. The optional constraint column is placed at the righthand side of the table. Its function is to record the constraints that will be used in the next step. The constraints are written as predicates. They describe the relationship between the classes and sub-vectors. Let us look at a more complicated specification. Let A, B and C be classifications. Let {a1, a2}, {b1, b2} and
ð1Þ
fcard class; credit limit; purchase balance; cumulative balanceg fredeem type; airline; air-ticket type; supplierg
ð2Þ ð3Þ
41
Fig. 3. A simple basic class sub-vector table.
42
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Fig. 4. Basic class sub-vector table of partition classification 1.
{c1, c2} be the classes of A, B and C, respectively. The legitimate inputs of this specification are listed in the left side of Fig. 7. It is easy to see that when class a1 exists, class b1 and class c2 are mutually exclusive to each other. But when a1 does not exist, (b1, c2) is a legitimate input. In this situation, the constraints table plays its role. It is used to record the constraints, mutual exclusion in this example, in the form of a predicate. The predicate b1 eor c2 in the constraint column of row a1. This means that with the existence of a1, b1 and c2 are mutually exclusive with each other. Incidentally, the constraints part is used to record any special case by expressing the case in the form of logical expressions of classes. The basic class sub-vector table of this example is shown in right side of Fig. 7. 5.4. Step 4: Generating class sub-vectors After inputting the basic class sub-vectors, class subvectors of each partition of classifications can be gen-
Fig. 7. Basic class sub-vector table with constraints part.
erated using algorithms. We have designed an algorithm for exemplification which is listed in Fig. 8. Algorithms of this kind is implementable and hence no human effort is required for generating class sub-vectors of a partition of classifications. In the algorithm, the variables result1, tmp1, tmp2 are sets of class vectors. The variable row holds the current row number, and the variable tmpv holds a class subvector. Line 2 initializes the variables to empty set. The variable result1 is used to hold the class sub-vectors obtained from the basic class sub-vector table. The variables tmp1 and tmp2 are working variables to hold the class sub-vectors obtained from and to be removed from result1, respectively. Line 3 initializes the variable row to the number of classes in the table. The repeat loop between line 4 and line 24 is the control structure for processing all the rows in the table. The for loop between line 5 and line 18 is the control structure for ensuring that all the basic class sub-vectors
Fig. 5. Basic class sub-vector table of partition classification 2.
Fig. 6. Basic class sub-vector table of partition classification 3.
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
43
Fig. 8. An algorithm for generating class sub-vectors of a partition of classification.
in a row is processed. The for loop between line 6 and line 17 is the control structure for ensuring that all the class sub-vectors in the variable result1 is processed. The nested if between line 7 and line 16 is used to check if an basic class sub-vector bcsv in the table is linkable with a class sub-vector sv in the set result1. If it is linkable on either ends, they are chained by the chain function to form a new class sub-vector and the result is put in the variable tmpv. Then, this new class sub-vector is checked against the constraints in the procedure constraintProcess(). If the classes bcsv and sv is not linked in either end, the bcsv is added to the tmp1 instead. In the procedure constraintProcess() between line 26 and 32, if the new class sub-vector satisfy all the constraints in the constraint table, this new class sub-vector
is added to the set tmp1. The sv is added to the tmp2 for removal from result1. Otherwise, the bcsv is added to the tmp1. In line 19, the class sub-vectors stored in tmp2, i.e. the class sub-vectors originally in result1 which have chained with at least one basic class sub-vector, are removed from result1. In line 20, the new class sub-vectors obtained in the two for loops are added to result1. In lines 21 and 22, the two working variables tmp1 and tmp2 are cleared to empty set. In line 23, subtraction of row is used to move the process to the next row. Applying the algorithm to the basic class sub-vector table in Fig. 7, after the first and second iterations, result1 is holding the class sub-vectors fða1; b1Þ; ða1; c2Þg. In the third iteration, the value of bcsv is (b1,c2). Since
44
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
(a1, b2, c2) does not satisfy the constraint a1 ^ ðb1 eor c2Þ, the bcsv, i.e. the class sub-vector (b2, c2) is added to the tmp1 instead of the tmpv, (a1, b1, c2). Then, after all the iterations, result1 is holding (a1, b1), (a1, c2), (b1, c2). Applying the algorithm in Fig. 8 to the first partition of our case study generates the following partial class vectors: {corporate}, {personal, principle}, {personal, additional}, Similarly, for the second and third partitions, we obtain the following two sets of vectors: {platinum, L3, A3, C3}, {platinum, L2, A2, C2}, {gold, L2, A2, C2}. {air-ticket, CX, first}, {air-ticket, CX, business}, {air-ticket, CX, economy}, {air-ticket, DG, first}, {air-ticket, DG, business}, {product, RedA}, {product, TDC}. 5.5. Step 5: Generating class vectors Having generated these three sets of partial class vectors, based on Corollary 3, all the class vectors of the input domain can be obtained from the Cartesian Product of the sub-vectors in all the independent partitions of classifications. Some of the class vectors derived are as follows: {corporate, platinum, L3, A3, C3, air-ticket, CX, first}, {corporate, gold, L2, A2, C2, product, TDC}, {personal, principle, platinum, L2, A2, C2, air-ticket, DG, business}.
On the other hand, class vectors have many advantages over classification tree. The information obtained from specifications are better captured in the class vectors. For example, the relations between class ‘platinum’ and class ‘A3’ can be captured in the class vector. There is no relation between class ‘A4’ and any class in the classification credit limit can also be captured by class vectors. Class vectors possess more expressive power than classification trees and hence can describe more complicated relations between classifications. For example, all co-exist relations between any number of classes can be expressive directly by class vector. This expressive power is beyond the capability of classification tree. The concepts of class vector is simpler than classification tree. This is because there are three kinds of relations namely, the root-classification, the classification-class and class-classification in classification tree. But none of these three kinds of relations are natural to end users. On the other hand, there is only one basic coexist relation in class vector. The independent relation in class vector is derived from the co-exist relation. Furthermore, these two relations are natural to end users. Hence, conceptually, class vector is simpler that classification tree. This helps to reduce errors made by test engineers and facilitate users in validating the test cases. Since information from specifications are captured in a more precise manner, legitimate test cases can be generated directly from the class vectors with much less intensive human decisions to select the legitimate inputs from the combination table of classification tree. In particular, the test case generation of class vector can be semi-automated while classification tree requires much human decision makings. The only human input in the class vector approach is the co-exist relations among classes. But, for the classification tree method, in addition to the input of co-exist relations among classes, it also requires other human effort to pick the right cases from the combination table. 6.2. Practical aspects
6. Discussion 6.1. Conceptual aspects The class vector method is an enhancement of the classification tree method. This method focuses on the relations among classes and used them to derive test cases. Abstraction of the root-classification, classification-class and class-classification relations is left out. These relations do not directly capture the relations among classes as stated in the specifications. Furthermore, these three relations can lead to many illegitimate test cases which require extra manual effort to remove.
Both class vector and CTs make use of the relations between classifications and class. However, class vector method uses these relations in a finer manner. Since test cases are derived from combinations of classes, classifications are used in two areas in the deriving of test cases in class vector method. First, dependent classifications are grouped together so as to partition the problem into smaller problems. Second, since classes within a classification are mutually exclusive, classifications are used to help to eliminate human decisions. The class vector method proposed in this paper to derive test cases is better than the CT in the following areas:
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Amount of human decision making: In the class vector approach, human decisions are required to decide the basic class sub-vectors. This is the smallest number of classes that are required to be considered in each decision and that cannot be shortened. After this step, all legitimate test cases are generated automatically without any additional human decision. On the other hand, in the classification tree approach, manual decisions are required to identify the co-exist relations between every two classes. Then a classification tree is developed with respect to these results. Many illegitimate test cases are derived from the classification tree. Extra human decision are required to select those legitimate test cases from all the test cases generated. One should notice that the length of these test cases is equal to the maximum length of the test cases. Hence, the amount of effort would be more than that of the class vector approach. Usability: The approach of class vector is incremental and eliminating. Class vectors are constructed by adding one class at a time. Those combinations of class vectors which may lead to illegitimate input are eliminated in the processing. The approach of CT cannot eliminate all the illegitimate cases in the process. Furthermore, the construction of test cases is not incremental. It is a ’’big bang’’ approach. Consequently, it will be easier for users to manage the test cases deriving by class vector approach than CT. The usability of class vector method is better than classification tree method. Degree of automation: In case of the CT, human efforts are required to 1. 2. 3. 4.
input the co-exist relations between every two classes, partition the problem, derive the classification tree, select the legitimate input from the results of the classification tree. In class vector method, human efforts are required to
45
class are formalized by set and partitions of set, respectively. Class vectors are defined as legitimate combinations of classes, i.e. the legitimate inputs of the system. These are also all the test cases of the system. With reference to these formalization and definition, different relations among classifications and classes are defined. By making us of different relations among classifications and classes, a method is proposed to derive class vectors which are legitimate inputs. From the process of derivation, it is easy to see that this method is sound, i.e. all class vectors will be generated by this method. Hence, the confident of class vector method is higher than the CT. 6.3. Limitation Our class vector test cases generation method is a kind of partition testing approach. The limitation of our method, similar to the limitation of partition testing method, is limited by the issue that whether the input domain of the system to be tested can be partitioned. In general, there are few systems which cannot be partitioned. The only issue is how to partition the input domains. However, the method for partitioning input domains is beyond the scope of this paper.
7. Conclusion By choosing vectors to represent the information obtained from specifications, we enhance the CT to class vector method. The vector structure provides better expressive power than tree, therefore, class vector method can capture the relations between classes in a better way. With this improvement, all the legitimate test cases stated in the specification, can be derived from class vector much more efficiently than the classification tree counterpart and at the same time involves fewer human decisions. Furthermore, the method proposed in this paper has good usability, high degree of automation and is theoretically sound.
1. partition the problem and 2. input to the basic class sub-vectors table. Acknowledgement Programs can be developed to guide the users and to record the basic class sub-vectors. Afterwards, class subvectors and class vectors can be generated automatically by programs. The only human efforts being required in class vector method are the efforts of formalizing a specification. This procedure cannot be automated. Hence, class vector approach needs only the minimum amount of human effort. Theoretical foundation: This class vector test case generation method is derived from the formal model of class vector. The informal notion of classification and
This work was partially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (project no. CityU 1118/ 99E).
References Chen, T.Y., Poon, P.L., 1996a. Classification-hierarchy table: a methodology for constructing the classification tree. In: Proceedings of
46
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
the Australian Software Engineering Conference (ASWEC’96). IEEE Computer Society Press, pp. 93–104. Chen, T.Y., Poon, P.L., 1996b. Improving the quality of classification tree via restructuring. In: Proceedings of the 3rd Asia-Pacific Software Engineering Conference (APSEC’96). IEEE Computer Society Press, pp. 83–92. Chen, T.Y., Poon, P.L., 1997. Construction of classification trees via the classification-hierarchy table. Information and Software Technology 39, 889–896. Chen, T.Y., Poon, P.L., Tse, T.H., 1999. A new restructuring algorithm for the classification-tree method. In: Tilley, S., Verner, J. (Eds.), Proceedings of the 9th International Workshop on Software Technology and Engineering Practice (STEP’99). IEEE Computer Society Press, pp. 105–114. Grochtmann, M., Wegener, J., Grimm, K., 1995. Test case design using classification trees and the classification-tree Editor CTE. In: Proceedings of the 8th International Software Quality Week, pp. 1–11. Grochtmann, M., Grimm, K., 1993. Classification trees for partition testing. Software Testing, Verification and Reliability 3, 63–82. Leung, K.R.P.H., Wong, W., 2000a. Deriving test cases using class vectors. In: Proceedings of the 7th Asia-Pacific Software Engineering Conference (APSEC 2000). IEEE Computer Society Press, Singapore, pp. 146–153. Leung, K.R.P.H., Wong, W., 2000b. Generating test cases from class graphs. In: Proceedings of the 1st Asia-Pacific Conference on Quality Software. IEEE Computer Society Press, Hong Kong, pp. 285–293. Myers, G.J., 1979. In: The Art of Software Testing. Wiley, New York, p. 1979. Ostrand, T., Balcer, M., 1988. The category-partitioning method for specifying and generating functional tests. Communication of the ACM 31 (6), 676–686. Karl R.P.H. Leung received a M.Phil. and a Ph.D. from The University of Hong Kong in the years 1992 and 1997, respectively. Dr. Leung is currently a Principal Lecturer in the Department of Information and Communications Technology at the Hong Kong Institute of Vocational Education. His research areas include: Domain Modelling, Mission Critical Software Engineering Methodology, Secure Work-
flow Systems, GSM-based Location Estimation, QoS of MPEG Streaming. Dr. Leung is a Senior Member of the IEEE and IEEE Computer Society, and has been an Executive Committee member (1990–1993), General Secretary (1994), Vice-Chair (1995–1997) and Chair (1998– 1999), Past Chair since 2000, of the IEEE Hong Kong Section Computer Chapter. The Chapter won the IEEE Most Outstanding Computer Society Chapter Award in 1998. Dr. Leung is also a C.Eng., an Ir., a Chartered Information Systems Engineer, a RPE (Information), and a member of ACM, BCS, ACS, HKIE and HKCS. Wai Wong received his B.Sc. in Computer Engineering, his M.Sc. and Ph.D. in Engineering from the University of Warwick, UK in the year of 1986, 1989 and 1993, respectively. Dr. Wong is currently an assistant professor in the Department of Computer Science at Hong Kong Baptist University. His current research interests includes software engineering, formal specification and verification, multimedia and its applications in education, Chinese language processing. Dr. Wong is a member of the IEEE and Computer Society. He has been an EXCO member of the IEEE Hong Kong Section since 1995. He is the vice chairman of the IEEE Hong Kong section Computer Chapter for the year 2002. Joseph Kee-Yin NG received a B.Sc. in Mathematics and Computer Science, a M.Sc. in Computer Science, and a Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign in the years 1986, 1988, and 1993, respectively. Dr. Ng is currently an associate professor in the Department of Computer Science at Hong Kong Baptist University. His current research interests includes Real-Time Networks, Multimedia Communications, ATM Delay Analysis, High-Speed Network Simulation, and Distributed Systems Performance Evaluation. He served as the General and Program Co-Chair for The International Computer Congress 1999 (ICC’99), the Program Co-Chair for The Sixth International Conference on Real-Time Computing Systems and Applications (RTCSA’99) and the General Co-Chair for The 1999 International Computer Science Conference (ICSC’99). Dr. Ng is an associate editor of Real-Time Systems: The International Journal of Time-Critical Computing Systems by Kluwer Academic Publishers. He is a senior member of the IEEE and has been a member of the IEEE Computer Society since 1991. Dr. Ng has been an exco-member (1993–95), General Secretary (1995–1997), Vice-Chair (1997–1999), Chair (1999–2001) and is the immediate past Chairman of the IEEE, Hong Kong Section, Computer Chapter. He is also a member of the IEEE Communication Society, and ACM.
Generating test cases from class vectors Karl R.P.H. Leung a, Wai Wong b, Joseph Kee-Yin NG
b,*
a
b
Compuware Software Testing Laboratory, Department of Information and Communications Technology, Hong Kong Institute of Vocational Education (Tsing Yi), Hong Kong Department of Computer Science, Hong Kong Baptist University, R702 224, Waterloo Road, Kowloon, Hong Kong Received 27 August 2001; received in revised form 3 January 2002; accepted 15 April 2002
Abstract Specifications are the primary source for obtaining test cases in software testing. Specification based testing is becoming more and more important when black box components and COTS are widely used in software development. An important issue in system testing is to identify all the legitimate input. One of these systematic approaches is deriving test cases from classification tree. This approach partitions the input domain into classifications which are further partitioned into classes. Test cases are combinations of classes. Relations between classification and classes are identified and are used to construct the classification tree. From the classification tree, combination table is constructed. Test cases are derived from the combination table. However, human decisions are required in determining whether test cases derived from the combination table are legitimate. This problem is incurred by the limitation of the expressive power of classification trees which cannot express the relations among classes precisely. We propose an enhancement by expressing the relations among classes and the relations among classifications directly in vectors. We call this new approach Class Vectors. This paper presents the class vector approach with formal definitions of basic concepts and entities. We find that the expressive power of class vectors is higher than classification trees, that is, this approach can express the information given in the specification in a better way and derive all legitimate test cases with minimal human decisions. Furthermore, a method of generating legitimate test cases using class vectors is described. This method is derived from the formal semantics of class vector, hence it is theoretical sound. Finally, we discussed that the proposed method requires the least amount of human decisions, can be highly automatic and has good usability. Ó 2002 Elsevier Science Inc. All rights reserved.
1. Introduction Specifications are one of the major references for deriving test cases. Specification based test case generation is becoming more and more important together with the increase in the size of software systems and the widely use of components in software development. All the legitimate inputs form the basic testing oracle of a system. One important issue in system testing is to identify all the legitimate inputs from the specification effectively. Each of these legitimate inputs is a test case of the system. This paper reports our approach of using class vectors in generating test cases. * Corresponding author. Tel.: +852-2339-7864; fax: +852-23397892. E-mail addresses: [email protected] (K.R.P.H. Leung), [email protected] (W. Wong), [email protected] (J. Kee-Yin NG).
One of the software testing methods is functional methods which include, for instance, functional partitioning and cause-effect graphing (Myers, 1979). The ‘‘category-partition method’’, on the other hand, is one of the promising approach which gives good support for functional testing (Ostrand and Balcer, 1988). The classification tree (Grochtmann and Grimm, 1993, Grochtmann et al., 1995) is a further development of the category-partition approach for deriving test cases. A method is proposed by Chen et al. (Chen and Poon, 1996a, 1997) for deriving classification trees from specifications. Test cases can then be generated from these classification trees. However, some of the test cases generated are illegitimate, therefore, it requires manual selection to filter out the unwanted cases. Although Chen et al. proposed some improvements for re-structuring classification trees (Chen and Poon, 1996b; Chen et al., 1999), the problems cannot be solved completely. It is because the intrinsic restrictions of classification
0164-1212/03/$ - see front matter Ó 2002 Elsevier Science Inc. All rights reserved. doi:10.1016/ S0164-1212(02)00061-4
36
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
tree make it unable to express some of the information from specifications. Hence, there is room for improvement. A classification tree is capable of expressing only three kinds of relations––root-classification, classification-classes, and class-classification. The relations between two classes or among three or more classes are very important in finding legitimate test cases. They cannot be expressed directly in a classification tree construct. In recognising this, we attempted to find an alternative structure to represent the class and classification relationship. We chose class graph as our underlining structure. This study has been reported in Leung and Wong (2000b). We then noticed that the test cases are vectors spanning the input space. After identifying the relations among the classes and classifications, the test cases can be generated directly from the relations without the need of building up an intermediate structure, such as graphs or trees. A preliminary study of using vectors as studying tool has been reported in Leung and Wong (2000a). In this paper, we report our more delineated study of class vectors. With reference to the formal theoretical framework of class vector, we developed a method to generate class vectors. This method is highly automatic and is theoretically sound. It requires the minimal amount of human decisions, has good usability, and has high degree of automation. The rest of this paper is organized as follows. In Section 2, we briefly review the classification tree method (CT) (Grochtmann and Grimm, 1993; Grochtmann et al., 1995), and present a slightly modified version of the case study used in (Chen and Poon, 1997). We then formalize a number of basic notions, such as class and classifications. With reference to this formalization, we define class vectors and relations among classes and classifications in Section 4. We propose a method for deriving test cases by using class vectors in Section 5. This is followed by a discussion in Section 6 comparing the conceptual and practical aspects of our approach with the CT. We then conclude our work in Section 7.
2. The classification tree method The CT (Grochtmann and Grimm, 1993; Grochtmann et al., 1995) is a black box testing technique. The basic idea of this method is to divide the input domain of a test object under different aspects and then combine the different partitions to form test cases (Chen and Poon, 1996a). Each aspect of the input domain is known as a classification. Each classification is further divided into a disjoint set of classes. Each class represents a range of input values to the test object.
This method, as described in Chen and Poon (1997), consists of four steps which are summarized as follows. 1. Identify all the classifications and their classes from the specifications. 2. Construct the classification tree with the classifications and classes. 3. Construct the combination table from the classification tree. 4. Select all the feasible combinations of classes from the combination table. Thus, each selected combination of classes constitute one test case. 2.1. Case Study To illustrate how to apply the CT, we present a case study here. The example is about reward points handling of credit cards which was used in Chen and Poon (1997) albeit with a number of minor changes. This example will also be used in later sections of this paper. The ABC Bank issues credit cards to approved customers. These customers will earn certain ‘reward points’ for every dollar charged to their cards. The number of reward points earned by each customer then determines the type of benefit (e.g. free air-line tickets or free shopping vouchers) which the customer is entitled to. For each purchase, the program shall accept the transaction details together with the various information of the card. Thereafter, validation of these details is performed in order to determine whether this purchase is approved. If the purchase is approved, the program will calculate the number of reward points. There are three classes of credit cards, namely platinum, gold and classic. The ABC Bank offers two types of credit card, namely corporate and personal. Each personal card holder holds a principle card. Personal card holders can apply for additional cards. The credit limit for classic cards are $1000. The credit limit for gold cards and some platinum cards are $4000. The credit limit for the rest of the platinum cards are $8000. Card holders can redeem air-tickets or products with their reward points. Air-tickets can only be redeemed from airlines CX or DG. All classes of tickets, including first, business or economy class can be redeemed from CX. Only first or business class air-tickets can be redeemed from DG. Products can be redeemed from suppliers RedA or TDC. The minimum transaction amount should be greater than $0. The cumulative balance is the total purchase amount of all outstanding transactions. The amount of cumulative balance should be greater than $0 but no greater than the approved credit limit of the card. From the above specification, we can identify the classifications and classes. These classifications are summarized in Fig. 1. From these classifications, a classification tree and a combination table can be constructed
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
37
Each test case is obtained by selecting a class under each top-level classification. In the case where there are more classifications below the top-level one, selecting a class will select all its ancestor classes. The ancestor classes are those classes that are on the path from the selected class to the root of the tree. For example, in Row 1 of the table, ‘first’ is selected which means the ancestor classes ‘CX’ and ‘air-ticket’ are also selected. Therefore, the test case representing by Row 1 is: ðplatinum; corporate; L3; air-ticket; CX; first; A1 and C1Þ: Fig. 1. Classifications and its associated classes.
following the method proposed in Chen and Poon (1997). The upper portion of Fig. 2 shows the classification tree. In the classification tree, the small circle at the top is the general root node which represent the whole input domain of the program. The classifications directly under the general root node are called top level classifications. In a classification tree, a class links to the classifications which the classes of these classifications have some relations with the class. Hence classifications and classes are in alternate levels. Because of this structure, the leaves of a classification tree, called the terminal nodes, are always classes. For example, the class air-ticket links to the classification airline. It is because the classes CX and DG of the classification air-ticket are the airlines which air tickets can be redeemed according to the specification. The lower portion of Fig. 2 where a number of horizontal lines labelled with numbers is a part of the combination table. Test cases can be constructed from this table. Each horizontal line in the table represents a possible test case. There are three cases shown in Fig. 2.
Illegitimate test cases can be derived from the combination table. For example, Row 3 represents such a case. In this case, a classic card has a cumulative balance in C2 which is not permitted by the specification. Although combinations of this type may be interested test cases of exception handling, it is an illegitimate input according to the user specification. Generating test cases for testing exception handling is beyond the scope of this study. Illegitimate test cases in classification trees have to be eliminated by test engineers on a case by case bases. For this simple example, test engineers may have to examine a total of 1296 test cases in order to eliminate the invalid cases. 2.2. Analysis of classification trees As shown in Fig. 2, a classification tree embraces three kinds of relations among the classes and classifications: Root-classification: This kind of relation is represented as the edges from the root of the tree to the top level classification. This relation has little significance to
Fig. 2. The CT and a torso of the combination table.
38
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
the generation of test cases. It simply binds all the top level classifications into a tree. Classification-classes: This kind of relations is represented as the edges from a classification to a class. Since each class must belong to certain classification, these edges simply make the this relation explicit. They also help to structure the tree into an alternating layers of classes and classification. Class-classification: This kind of relations links a class to a classification. They indicate that the class has certain relation with the classes in the classification that is linked. With this rigid tree structure, the resulting combination table, hence the number of generated test cases become large. This includes a number of illegitimate test cases which require manual filtering. The reasons for this are: 1. The structure of the classification trees makes it unable to express the relation between classes directly. Each test case is a sequence of classes one from each classification. Therefore, it is important to identify the combinations of classes which are legitimate test cases from those combinations that are not legitimate. This calls for the ability to represent the relations between classes directly. 2. To overcome the above problem, some classifications are duplicated in a classification tree, for example, the air-ticket type in Fig. 2. This makes the structure more complex. 3. There are still some relations that cannot be expressed by duplicating classifications. For example, in Fig. 2, the classifications card class, credit limit, purchase balance and cumulative balance are related to each other as specified in the specification. (The specification states the the cumulative balance cannot be larger than the credit limit.) The key of these shortcomings is that the structure of classification trees restricts the effectiveness of their expression. To overcome this, we propose to get rid of the rigid tree structure, and use a more flexible approach instead. We choose to use vectors because they are closer to the goal of finding the test cases than classification trees. We also take a more formal approach in which we first define a number of basic notions, such as classes, classifications, class vectors and legitimate inputs. Then, we develop our method of finding test cases based on these formal definitions.
3. Basic definitions of classes and classifications In this section, we analyze the notions of classifications, classes and legitimate test cases and provide formal definitions for these notions.
3.1. Variables and values Let X be an input variable and sðXÞ be a set of values X can take, i.e., the type of X. sðXÞ can be partitioned into a number of disjoint subsets, i.e., [ sðXÞ ¼ sk ðXÞ k
where 8kl k 6¼ l ) sk ðXÞ \ sl ðXÞ ¼ ; Definition 1 (Value of an input variable). A value function Val is defined to return the value of a variable we are now interested in, i.e., ValðXÞ is the value of X. Thus, ValðXÞ will be in sðXÞ and also ValðXÞ 2 sk ðXÞ for some k: Consider a program having n input variables ðX1 ; . . . Xi ; . . . ; Xn Þ denoted by ~ X, the values of these variables form an input value vector ! Valð~ XÞ ¼ ðValðX1 Þ; . . . ; ValðXi Þ; . . . ; ValðXn ÞÞ 3.2. Classification and class Definition 2 (classification and class). Let X be a set such that [ X ¼ fsk ðXÞjsðXÞ ¼ sk ðXÞg k
where 8kl k 6¼ l ) sk ðXÞ \ sl ðXÞ ¼ ;. The X is called a classification while the disjoint subsets are called the classes. Later in this paper, we will also denote a class by xk when we do not need to refer to the input variable X explicitly. Definition 3 (class function). A function ‘class’ on X is defined to return the class in which the value of X lies, i.e., classðXÞ ¼ sk ðXÞ if and only if ValðXÞ 2 sk ðXÞ Suppose there are d classifications X1 ; . . . ; Xi ; . . . ; Xd where ðd P 1Þ. For each Xi , there is an associated set of classes fxki : 1 6 k 6 cj g where cj is the number of classes in the jth classification. The function ‘class’ may be suffixed with a classification to indicate which classification we are referring to, i.e., classY ðXÞ. Extending Definition 3, we can define a function ‘classes’ on the input vector ~ X to return a vector of classes in the form of a d-tuple of classes.
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Definition 4 (classes function). XÞ; . . . ; class Xd ð~ XÞÞ classesð~ XÞ ¼ ðxk11 ; . . . ; xkdd Þ ¼ ðclass X1 ð~ In the case study presented in Section 2.1, 10 classifications have been identified. Thus, d ¼ 10 and, given an input vector ~ X, the function ‘classes’ applying to the input vector will return a 10-tuple of classes ðxk11 ; . . . ; xk1010 Þ. 3.3. Legitimate input Suppose a program having n input variables ðX1 ; . . . Xi ; . . . ; Xn Þ and from its specification, d classifications ðX1 ; . . . Xi ; . . . Xd Þ are identified. The value of a variable Xi will be in some class ski ðXi Þ. The specification will further specify that, giving ValðXi Þ 2 ski ðXi Þ, the value of another variable, say Xj , must be in certain class, say slj ðXj Þ. Thus, we define that an input vector is a legitimate input if and only if, for all i, Xi is in the allowed class according to the specification. In our credit card case study, if in an input vector the value of the variable corresponding to the classification card class equals ‘Classic’ and the value of the variable corresponding to credit limit equals 4000, then this input vector is not legitimate. It is because according to the specification, this combination of input values is not allowed. Definition 5 (legitimate input). A predicate legitimate is defined. It holds with respect to a specification S, a set of classifications X and an input vector ~ X if and only if the value of ~ X is a legitimate input according to S. This is denoted as S
L ðX ; ~ XÞ Consider a program having n input variables ~ X ¼ ðX1 ; . . . ; Xi ; . . . Xn Þ, according to its specification S, d classifications ðX1 ; . . . ; Xi ; . . . ; Xd Þ are identified. Since each classification Xi associates with a set of classes fx1i ; . . . ; xci i g, there are d sets of classes fx11 ; . . . ; xk11 ; . . . ; xc11 g .. .
39
classes ð~ XÞ. The input is legitimate if and only if R holds on these classes. Therefore, XÞ Rðclassesð~ XÞÞ LS ðX ; ~ For our case study, if the input vector ~ X has the values described earlier in this section, then Rðclassesð~ XÞ does not hold. It should be noticed that the relation R partition the set of d-tuples into two disjoint subsets; one represents the legitimate inputs, the other non-legitimate inputs. 3.4. Class vectors We call the d-tuples of classes ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ as described in previous section class vectors. In subsequent sections, we will use class vectors in working out the test cases. Here, we first define an auxiliary function on class vectors to facilitate our discussions. Definition 6 (elem function). The function elem on a class vector V returns the set of classes that appear in the dtuple ðx1 ; . . . xi ; . . . xn Þ, i.e., elemðV Þ fx1 ; . . . xi ; . . . ; xn g: 3.4.1. Class sub-vectors In order to build test cases from class vectors, we define here the concept of sub-vectors. Definition 7 (sub-vector). A class vector V is a sub-vector of another class vector U if and only if elemðV Þ elemðU Þ. Let V1 ¼ ðxi ; . . . ; xj Þ and V2 ¼ ðxm ; . . . ; xn Þ. We define a predicate link as follows. Definition 8 (link). linkðV1 ; V2 Þ if and only if xj ¼ xm , i.e. the last class in the tuple of V1 is the same of the first class in the tuple of V2 . We define an operator chain to form a new class subvector by chaining up two linkable class sub-vectors.
fx1i ; . . . ; xki i ; . . . ; xci i g .. .
Definition 9 (chain function). chainðV1 ; V2 Þ ¼ ðxi ; . . . ; xj ; xðmþ1Þ ; . . . ; xn Þ if and only if linkðV1 ; V2 Þ.
fx1d ; . . . ; xkdd ; . . . ; xcdd g
With the notion of class sub-vector, we define an auxiliary function legitimate over a set of classifications which return a set of class sub-vectors that are formed by the given set of classifications.
From the specification S, a relation R on the d-tuple ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ can be identified. This relation holds, i.e., Rðxk11 ; . . . ; xki i ; . . . ; xkdd Þ evaluates to TRUE, if and only if the d-tuple ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ satisfies the specification S. ! When considering whether a particular input Valð~ XÞ is a legitimate input, we can evaluate the relation R on
Definition 10 (legitimate). Let X ¼ fX1 ; . . . ; Xm g be a set of classifications. legitimateðX Þ ¼ fv1 ; . . . ; vn g where vi ¼ ðx1 ; . . . ; xm Þ; 8j; k : 1 6 j 6 n; 1 6 k 6 m; vj is a class sub-vector of a class vector and xk 2 Xk .
40
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
4. Relation Our class vector approach only rely on one principle relation, the relation between classes. We call this relation co-exist which is discussed in Section 4.1. Based on the co-exist relation, we define independent relation between classifications. This relation is defined in Section 4.2. This independent relations can help to speed up the process when deriving test cases.
Definition 13 (independent classifications). Two classifications X1 and X2 are independent to each other if and only if 8i xi1 2 X1 , xi1 co-exists with xj2 for all j such that xj2 2 X2 . This means that if, for any class in the classification X1 , it co-exists with any class in X2 , then the two classifications are independent. This relation of independence can be extended to any number of classifications.
4.1. Co-exist relation The basic relation in class vector is the co-exist relation between two classes. This relation describes the values of two classes that can co-exist in a legitimate input. Definition 11 (co-exist of two classes). Two classes xki i k and xj j are considered co-exist if and only if they appear in a class vector ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ which is in LS ðX ; ~ XÞ. The co-exist relation between two classes can be extended to more than two classes, thus we have the following definition. Definition 12 (co-exist of more than two classes). Given a set of classes C ¼ fx1 ; . . . xi ; . . . ; xn g; they are co-exist if and only if there exists a class vector V ¼ ðxk11 ; . . . ; xki i ; . . . ; xkdd Þ such that V 2 LS ðX ; ~ XÞ and for all xi 2 C, xi is in the set of classes elem(list(V)). By definition, all classes in a classification are disjoint, therefore, they are not co-exist. In our case study, the class ‘Classic’ in the classification card class cannot co-exist with the class 4000 in the classification credit limit. On the other hand, it can co-exist with the class 1000 in credit limit. Corollary 1. By definition of co-exist relation, if a class appears in a class vector representing a legitimate input, this class co-exist with itself.
Definition 14 (independence of more than two classifications). p classifications X1 ; . . . ; Xp are independent if and only if any p classes which are selected one from each classification are co-exist. From the two definitions above, we come up with the following two corollaries. Corollary 2. An input domain is composed by one or more independent partitions. Corollary 3. The cartesian product of the sub-vectors of all the independent partitions constructs all the class vectors of the input domain, and hence, all the test cases of the input domain. 4.3. Basic class sub-vectors With the co-exist relation, we are interested in the minimal number of classes that are required to determine whether a class is in a legitimate input. We call these class sub-vectors basic class sub-vectors. A basic class sub-vector is defined as follows. Definition 15 (basic class sub-vector). For a given set of classifications X ¼ fX1 ; . . . ; Xn g, a class sub-vector ðx1 ; . . . ; xn Þ formed from taking one class from each classification in X is a basic class sub-vector if either of the following conditions is satisfied: 1. The class sub-vector ðx1 ; . . . ; xn Þ is a legitimate input. 2. X is the smallest set of classifications satisfying the inequality Y cardðlegitimateðX ÞÞ < cardðxÞ x2X
and ðx1 ; . . . ; xn Þ 2 legitimateðX Þ 4.2. Independent Relation One very important relation between classifications is independence. If every class in a classification can coexist with every class in another classification, then we say these two classifications are independent. For example, the classifications customer type and card class are independent since every type of customers may has any class of credit cards.
For example, in our credit card case study, only the class CX is required to determine that class economy is in an legitimate input, i.e. the (CX, economy) form a class sub-vector. But for the class corporate, there is no need to have another class to determine whether it is in a legitimate input. It itself forms a basic class sub-vector. On the contrary, the class sub-vector (air-ticket, CX, economy) is not a basic class sub-vector because neither
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
one of the classes in this tuple need all the other two classes to determine whether this class is in a legitimate input. It should be noticed that a basic class sub-vector may carry no physical meaning. However this is the smallest amount of information to be managed at each time to determine whether a class is in a legitimate input. Let us look at another example. Let A, B and C are classifications. Let {a1, a2}, {b1, b2} and {c1, c2} are classes of A, B and C, respectively. Let (a1, b1, c1), (a1, b2, c2), (a2, b1, c2) and (a2, b2, c1) be the legitimate inputs, then the smallest number of classes in a class sub-vector to determine whether the classes form a class sub-vector is 3.
5. Class vector generation method Based on the definitions of class vector and the Corollaries 2 and 3, we derived a method to generate all the class vectors of an input domain. This method consists of the following five steps. 5.1. Step 1: Identifying classifications and classes This step is similar to the first step of the CT as described in Section 2. In this step, we analyze the input domain using the information given in the specification. The aim of this step is to identify all the classifications that cover the entire input domain, and to partition each of the classifications into a number of classes. This process is guided by the specification which specifies what are the possible ranges of input values. For example, in our case study ten classifications and their associated classes are identified. They are listed in Fig. 1. 5.2. Step 2: Identifying independent partition of classifications By making use of the independence relation among classifications and Corollary 2, we can partition the set of classifications C into a number of disjoint subsets. This is an important property because this partitioning reduces the number of classifications to be handled together in the following step, thus making the tasks in the next step simpler and faster. To illustrate this step, we partition the classifications of our credit card example into the following three subsets: fcustomer type; sub-customer typeg
5.3. Step 3: Input basic class sub-vectors The task of this step is to fill in a basic class sub-vector table for each partition of classifications. The table consists of a square matrix of cells and an optional constraint column. The rows and the columns of the matrix are labelled by the classes of all classifications of the partition. The order of the classes is insignificant. For convenience, the class of the same classification should be put next to each other. The basic class sub-vectors are identified by a number. For example, the class vectors of the example described in Section 4.3 can be identified as in the left side of Fig. 3. The right side of this figure shows the basic class sub-vector table. The table is filled from the first row. For each basic class sub-vector, its number is put in the cells that corresponding to its constituent classes. For example, a ‘1’ is placed in the cells b1 and c1 of row a1 for the first subvector. It should be noticed that since the class sub-vectors express the co-exist relation between the classes and this relation is symmetric, the matrix is symmetric. Therefore, only the upper-right half of the matrix will be filled. Sometimes, a basic class sub-vector consists of only one class. For example, the corporate in the classification partition Eq. (1) is of this type. The number identifying this sub-vector, namely ‘3’, is placed in the cell of the interaction of row and column both labelled by corporate in Fig. 4. More than one number may be placed in the same cell. For instance, if the following two sub-vectors are identified and numbered 1 and 2, respectively: ða1; b1; c1Þ
ða1; b1; c2Þ
then, the numbers 1 and 2 will be placed in the cell of row a1 column b1. The basic class sub-vectors of our credit card case study are captured in Figs. 4–6. The optional constraint column is placed at the righthand side of the table. Its function is to record the constraints that will be used in the next step. The constraints are written as predicates. They describe the relationship between the classes and sub-vectors. Let us look at a more complicated specification. Let A, B and C be classifications. Let {a1, a2}, {b1, b2} and
ð1Þ
fcard class; credit limit; purchase balance; cumulative balanceg fredeem type; airline; air-ticket type; supplierg
ð2Þ ð3Þ
41
Fig. 3. A simple basic class sub-vector table.
42
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Fig. 4. Basic class sub-vector table of partition classification 1.
{c1, c2} be the classes of A, B and C, respectively. The legitimate inputs of this specification are listed in the left side of Fig. 7. It is easy to see that when class a1 exists, class b1 and class c2 are mutually exclusive to each other. But when a1 does not exist, (b1, c2) is a legitimate input. In this situation, the constraints table plays its role. It is used to record the constraints, mutual exclusion in this example, in the form of a predicate. The predicate b1 eor c2 in the constraint column of row a1. This means that with the existence of a1, b1 and c2 are mutually exclusive with each other. Incidentally, the constraints part is used to record any special case by expressing the case in the form of logical expressions of classes. The basic class sub-vector table of this example is shown in right side of Fig. 7. 5.4. Step 4: Generating class sub-vectors After inputting the basic class sub-vectors, class subvectors of each partition of classifications can be gen-
Fig. 7. Basic class sub-vector table with constraints part.
erated using algorithms. We have designed an algorithm for exemplification which is listed in Fig. 8. Algorithms of this kind is implementable and hence no human effort is required for generating class sub-vectors of a partition of classifications. In the algorithm, the variables result1, tmp1, tmp2 are sets of class vectors. The variable row holds the current row number, and the variable tmpv holds a class subvector. Line 2 initializes the variables to empty set. The variable result1 is used to hold the class sub-vectors obtained from the basic class sub-vector table. The variables tmp1 and tmp2 are working variables to hold the class sub-vectors obtained from and to be removed from result1, respectively. Line 3 initializes the variable row to the number of classes in the table. The repeat loop between line 4 and line 24 is the control structure for processing all the rows in the table. The for loop between line 5 and line 18 is the control structure for ensuring that all the basic class sub-vectors
Fig. 5. Basic class sub-vector table of partition classification 2.
Fig. 6. Basic class sub-vector table of partition classification 3.
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
43
Fig. 8. An algorithm for generating class sub-vectors of a partition of classification.
in a row is processed. The for loop between line 6 and line 17 is the control structure for ensuring that all the class sub-vectors in the variable result1 is processed. The nested if between line 7 and line 16 is used to check if an basic class sub-vector bcsv in the table is linkable with a class sub-vector sv in the set result1. If it is linkable on either ends, they are chained by the chain function to form a new class sub-vector and the result is put in the variable tmpv. Then, this new class sub-vector is checked against the constraints in the procedure constraintProcess(). If the classes bcsv and sv is not linked in either end, the bcsv is added to the tmp1 instead. In the procedure constraintProcess() between line 26 and 32, if the new class sub-vector satisfy all the constraints in the constraint table, this new class sub-vector
is added to the set tmp1. The sv is added to the tmp2 for removal from result1. Otherwise, the bcsv is added to the tmp1. In line 19, the class sub-vectors stored in tmp2, i.e. the class sub-vectors originally in result1 which have chained with at least one basic class sub-vector, are removed from result1. In line 20, the new class sub-vectors obtained in the two for loops are added to result1. In lines 21 and 22, the two working variables tmp1 and tmp2 are cleared to empty set. In line 23, subtraction of row is used to move the process to the next row. Applying the algorithm to the basic class sub-vector table in Fig. 7, after the first and second iterations, result1 is holding the class sub-vectors fða1; b1Þ; ða1; c2Þg. In the third iteration, the value of bcsv is (b1,c2). Since
44
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
(a1, b2, c2) does not satisfy the constraint a1 ^ ðb1 eor c2Þ, the bcsv, i.e. the class sub-vector (b2, c2) is added to the tmp1 instead of the tmpv, (a1, b1, c2). Then, after all the iterations, result1 is holding (a1, b1), (a1, c2), (b1, c2). Applying the algorithm in Fig. 8 to the first partition of our case study generates the following partial class vectors: {corporate}, {personal, principle}, {personal, additional}, Similarly, for the second and third partitions, we obtain the following two sets of vectors: {platinum, L3, A3, C3}, {platinum, L2, A2, C2}, {gold, L2, A2, C2}. {air-ticket, CX, first}, {air-ticket, CX, business}, {air-ticket, CX, economy}, {air-ticket, DG, first}, {air-ticket, DG, business}, {product, RedA}, {product, TDC}. 5.5. Step 5: Generating class vectors Having generated these three sets of partial class vectors, based on Corollary 3, all the class vectors of the input domain can be obtained from the Cartesian Product of the sub-vectors in all the independent partitions of classifications. Some of the class vectors derived are as follows: {corporate, platinum, L3, A3, C3, air-ticket, CX, first}, {corporate, gold, L2, A2, C2, product, TDC}, {personal, principle, platinum, L2, A2, C2, air-ticket, DG, business}.
On the other hand, class vectors have many advantages over classification tree. The information obtained from specifications are better captured in the class vectors. For example, the relations between class ‘platinum’ and class ‘A3’ can be captured in the class vector. There is no relation between class ‘A4’ and any class in the classification credit limit can also be captured by class vectors. Class vectors possess more expressive power than classification trees and hence can describe more complicated relations between classifications. For example, all co-exist relations between any number of classes can be expressive directly by class vector. This expressive power is beyond the capability of classification tree. The concepts of class vector is simpler than classification tree. This is because there are three kinds of relations namely, the root-classification, the classification-class and class-classification in classification tree. But none of these three kinds of relations are natural to end users. On the other hand, there is only one basic coexist relation in class vector. The independent relation in class vector is derived from the co-exist relation. Furthermore, these two relations are natural to end users. Hence, conceptually, class vector is simpler that classification tree. This helps to reduce errors made by test engineers and facilitate users in validating the test cases. Since information from specifications are captured in a more precise manner, legitimate test cases can be generated directly from the class vectors with much less intensive human decisions to select the legitimate inputs from the combination table of classification tree. In particular, the test case generation of class vector can be semi-automated while classification tree requires much human decision makings. The only human input in the class vector approach is the co-exist relations among classes. But, for the classification tree method, in addition to the input of co-exist relations among classes, it also requires other human effort to pick the right cases from the combination table. 6.2. Practical aspects
6. Discussion 6.1. Conceptual aspects The class vector method is an enhancement of the classification tree method. This method focuses on the relations among classes and used them to derive test cases. Abstraction of the root-classification, classification-class and class-classification relations is left out. These relations do not directly capture the relations among classes as stated in the specifications. Furthermore, these three relations can lead to many illegitimate test cases which require extra manual effort to remove.
Both class vector and CTs make use of the relations between classifications and class. However, class vector method uses these relations in a finer manner. Since test cases are derived from combinations of classes, classifications are used in two areas in the deriving of test cases in class vector method. First, dependent classifications are grouped together so as to partition the problem into smaller problems. Second, since classes within a classification are mutually exclusive, classifications are used to help to eliminate human decisions. The class vector method proposed in this paper to derive test cases is better than the CT in the following areas:
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
Amount of human decision making: In the class vector approach, human decisions are required to decide the basic class sub-vectors. This is the smallest number of classes that are required to be considered in each decision and that cannot be shortened. After this step, all legitimate test cases are generated automatically without any additional human decision. On the other hand, in the classification tree approach, manual decisions are required to identify the co-exist relations between every two classes. Then a classification tree is developed with respect to these results. Many illegitimate test cases are derived from the classification tree. Extra human decision are required to select those legitimate test cases from all the test cases generated. One should notice that the length of these test cases is equal to the maximum length of the test cases. Hence, the amount of effort would be more than that of the class vector approach. Usability: The approach of class vector is incremental and eliminating. Class vectors are constructed by adding one class at a time. Those combinations of class vectors which may lead to illegitimate input are eliminated in the processing. The approach of CT cannot eliminate all the illegitimate cases in the process. Furthermore, the construction of test cases is not incremental. It is a ’’big bang’’ approach. Consequently, it will be easier for users to manage the test cases deriving by class vector approach than CT. The usability of class vector method is better than classification tree method. Degree of automation: In case of the CT, human efforts are required to 1. 2. 3. 4.
input the co-exist relations between every two classes, partition the problem, derive the classification tree, select the legitimate input from the results of the classification tree. In class vector method, human efforts are required to
45
class are formalized by set and partitions of set, respectively. Class vectors are defined as legitimate combinations of classes, i.e. the legitimate inputs of the system. These are also all the test cases of the system. With reference to these formalization and definition, different relations among classifications and classes are defined. By making us of different relations among classifications and classes, a method is proposed to derive class vectors which are legitimate inputs. From the process of derivation, it is easy to see that this method is sound, i.e. all class vectors will be generated by this method. Hence, the confident of class vector method is higher than the CT. 6.3. Limitation Our class vector test cases generation method is a kind of partition testing approach. The limitation of our method, similar to the limitation of partition testing method, is limited by the issue that whether the input domain of the system to be tested can be partitioned. In general, there are few systems which cannot be partitioned. The only issue is how to partition the input domains. However, the method for partitioning input domains is beyond the scope of this paper.
7. Conclusion By choosing vectors to represent the information obtained from specifications, we enhance the CT to class vector method. The vector structure provides better expressive power than tree, therefore, class vector method can capture the relations between classes in a better way. With this improvement, all the legitimate test cases stated in the specification, can be derived from class vector much more efficiently than the classification tree counterpart and at the same time involves fewer human decisions. Furthermore, the method proposed in this paper has good usability, high degree of automation and is theoretically sound.
1. partition the problem and 2. input to the basic class sub-vectors table. Acknowledgement Programs can be developed to guide the users and to record the basic class sub-vectors. Afterwards, class subvectors and class vectors can be generated automatically by programs. The only human efforts being required in class vector method are the efforts of formalizing a specification. This procedure cannot be automated. Hence, class vector approach needs only the minimum amount of human effort. Theoretical foundation: This class vector test case generation method is derived from the formal model of class vector. The informal notion of classification and
This work was partially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (project no. CityU 1118/ 99E).
References Chen, T.Y., Poon, P.L., 1996a. Classification-hierarchy table: a methodology for constructing the classification tree. In: Proceedings of
46
K.R.P.H. Leung et al. / The Journal of Systems and Software 66 (2003) 35–46
the Australian Software Engineering Conference (ASWEC’96). IEEE Computer Society Press, pp. 93–104. Chen, T.Y., Poon, P.L., 1996b. Improving the quality of classification tree via restructuring. In: Proceedings of the 3rd Asia-Pacific Software Engineering Conference (APSEC’96). IEEE Computer Society Press, pp. 83–92. Chen, T.Y., Poon, P.L., 1997. Construction of classification trees via the classification-hierarchy table. Information and Software Technology 39, 889–896. Chen, T.Y., Poon, P.L., Tse, T.H., 1999. A new restructuring algorithm for the classification-tree method. In: Tilley, S., Verner, J. (Eds.), Proceedings of the 9th International Workshop on Software Technology and Engineering Practice (STEP’99). IEEE Computer Society Press, pp. 105–114. Grochtmann, M., Wegener, J., Grimm, K., 1995. Test case design using classification trees and the classification-tree Editor CTE. In: Proceedings of the 8th International Software Quality Week, pp. 1–11. Grochtmann, M., Grimm, K., 1993. Classification trees for partition testing. Software Testing, Verification and Reliability 3, 63–82. Leung, K.R.P.H., Wong, W., 2000a. Deriving test cases using class vectors. In: Proceedings of the 7th Asia-Pacific Software Engineering Conference (APSEC 2000). IEEE Computer Society Press, Singapore, pp. 146–153. Leung, K.R.P.H., Wong, W., 2000b. Generating test cases from class graphs. In: Proceedings of the 1st Asia-Pacific Conference on Quality Software. IEEE Computer Society Press, Hong Kong, pp. 285–293. Myers, G.J., 1979. In: The Art of Software Testing. Wiley, New York, p. 1979. Ostrand, T., Balcer, M., 1988. The category-partitioning method for specifying and generating functional tests. Communication of the ACM 31 (6), 676–686. Karl R.P.H. Leung received a M.Phil. and a Ph.D. from The University of Hong Kong in the years 1992 and 1997, respectively. Dr. Leung is currently a Principal Lecturer in the Department of Information and Communications Technology at the Hong Kong Institute of Vocational Education. His research areas include: Domain Modelling, Mission Critical Software Engineering Methodology, Secure Work-
flow Systems, GSM-based Location Estimation, QoS of MPEG Streaming. Dr. Leung is a Senior Member of the IEEE and IEEE Computer Society, and has been an Executive Committee member (1990–1993), General Secretary (1994), Vice-Chair (1995–1997) and Chair (1998– 1999), Past Chair since 2000, of the IEEE Hong Kong Section Computer Chapter. The Chapter won the IEEE Most Outstanding Computer Society Chapter Award in 1998. Dr. Leung is also a C.Eng., an Ir., a Chartered Information Systems Engineer, a RPE (Information), and a member of ACM, BCS, ACS, HKIE and HKCS. Wai Wong received his B.Sc. in Computer Engineering, his M.Sc. and Ph.D. in Engineering from the University of Warwick, UK in the year of 1986, 1989 and 1993, respectively. Dr. Wong is currently an assistant professor in the Department of Computer Science at Hong Kong Baptist University. His current research interests includes software engineering, formal specification and verification, multimedia and its applications in education, Chinese language processing. Dr. Wong is a member of the IEEE and Computer Society. He has been an EXCO member of the IEEE Hong Kong Section since 1995. He is the vice chairman of the IEEE Hong Kong section Computer Chapter for the year 2002. Joseph Kee-Yin NG received a B.Sc. in Mathematics and Computer Science, a M.Sc. in Computer Science, and a Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign in the years 1986, 1988, and 1993, respectively. Dr. Ng is currently an associate professor in the Department of Computer Science at Hong Kong Baptist University. His current research interests includes Real-Time Networks, Multimedia Communications, ATM Delay Analysis, High-Speed Network Simulation, and Distributed Systems Performance Evaluation. He served as the General and Program Co-Chair for The International Computer Congress 1999 (ICC’99), the Program Co-Chair for The Sixth International Conference on Real-Time Computing Systems and Applications (RTCSA’99) and the General Co-Chair for The 1999 International Computer Science Conference (ICSC’99). Dr. Ng is an associate editor of Real-Time Systems: The International Journal of Time-Critical Computing Systems by Kluwer Academic Publishers. He is a senior member of the IEEE and has been a member of the IEEE Computer Society since 1991. Dr. Ng has been an exco-member (1993–95), General Secretary (1995–1997), Vice-Chair (1997–1999), Chair (1999–2001) and is the immediate past Chairman of the IEEE, Hong Kong Section, Computer Chapter. He is also a member of the IEEE Communication Society, and ACM.