- Email: [email protected]
Algebraic Techniques for Covering Arrays and related Structures
Available online at www.sciencedirect.com
Electronic Notes in Discrete Mathematics 70 (2018) 49–54
www.elsevier.com/locate/endm
Algebraic Techniques for Covering Arrays and related Structures Bernhard Garn 1 Dimitris E. Simos 2 SBA Research, Vienna A-1040, Austria
Abstract In this paper, we extend an existing algebraic modelling technique for covering arrays by considering additional properties which are required when these structures are applied in practice in a branch of software testing called combinatorial testing. Corresponding to these properties, we give semantically equivalent systems of multivariate polynomial equations. Keywords: covering arrays, Groebner bases, modelling, combinatorial testing
1
Introduction
Discrete structures called covering arrays (CAs) appearing in design theory have recently found an application in a branch of software testing called combinatorial testing (CT) [7]. Denote the natural numbers starting from one with N = {1, 2, 3, . . .}. Let t, k, v ∈ N ∧ t ≤ k, then a covering array for the configuration (t, k, v) can be viewed as an N × k array over a finite alphabet A of cardinality v, such that in any N × t subarray all possible t-tuples arising from this alphabet A appear at least once as rows in the selected subarray [3], [5]. The number t is called the strength of the CA, and the resulting condition 1 2
Email: [email protected] Email: [email protected]
https://doi.org/10.1016/j.endm.2018.11.008 1571-0653/© 2018 Elsevier B.V. All rights reserved.
50
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
is usually referred to as the t-way coverage condition. In this paper, we are only concerned with strength two CAs over the binary alphabet. In [5], an approach for an algebraic modelling of CAs was presented using commutative algebra, linear algebra and symbolic computation techniques for solving the resulting multivariate polynomial equation systems. In CT, constraints are usually given either as logic expressions or as a list of forbidden tuples. In heuristic extension strategies for CA construction, depending on the nature of the logic expressions, different kind of solvers can be used to create or check a (possibly only partial) candidate test case for any violations of the given constraints. When constraints are given as a list of forbidden tuples, an exhaustive search of the list of forbidden tuples can be used to establish that a given (possibly only partial) candidate test case does not violate any constraints. Multiple criteria have been used to decide, under each setting, which constraint handling method will be used. Due to the high general complexity of Gr¨obner base computation, heuristic approaches following the previously outlined strategies often outperform purely algebraic approaches for real-world problem sizes. We build upon our previous work [5] and present algebraic modellings for CAs and related structures, which arise when CAs are used in CT and have to satisfy additional conditions. An example for such an additional condition is the important applied notion of constraints on parameter values (e.g., let pi be a possible value of parameter Pi for i = 1, 2, and consider the implication P1 = p1 → P2 = p2 ), expressing necessary semantic relationships of the application domain. We link such constraints to semantically equivalent algebraic expressions. We have attained an algebraic method to enforce the appearance, as well as the non-appearance of certain binary two-tuples. This way, we can equivalently express constraints given in the form of a list of forbidden tuples in an algebraic setting. This paper is structured as follows. In Section 1, we also briefly comment on how constraints are handled in heuristic construction techniques for CAs. In Section 2, we extend the algebraic approach to enforce the appearance or non-appearance of certain binary two-tuples. In Section 3, we show how this new algebraic technique can be used to model problems regarding CAs coming from the application domain of CT, and conclude the paper in Section 4.
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
2
51
Membership Equations for covering Tuples in CAs
Theorem 2.1 Assume that P is a ring, a, b ∈ P , and that (P, a, b) has the pairwise binary tuple distinguishing property (cf. [5], Definition 4), t1 , t2 is a tuple with ti ∈ {0, 1}, i = 1, 2, and T = { 0, 0 , 1, 0 , 0, 1 , 1, 1 }, and let S be a subset of T , i.e. S ⊆ T . Denote with ϕ : T −→ {0, 1, a, b} the bijection derived in [5], Lemma 1, which is given by ϕ( 0, 0 ) = 0, ϕ( 1, 0 ) = a, ϕ( 0, 1 ) = b, ϕ( 1, 1 ) = a + b. (1) Then, the following statements (i), (ii) and (iii) are equivalent 3 . (i) (ii)
(iii)
t1 , t2 ∈ S;
(2)
0 ∈ { a, b t1 , t2 − ϕ(τ ) | τ ∈ S};
(3)
a, b t1 , t2 − ϕ(τ ) = 0.
(4)
τ ∈S
Proof. First, assume that S = ∅. The equivalence of (i) and (ii) follows from the fact that the function ϕ is a bijection. The equivalence of (ii) and (iii) follows from the defining property of an integral domain. Second, assume that S is empty. Then, the claimed equivalences trivially hold. 2 Remark 2.2 An algebraic distinguisher for 1-tuples; i.e., for individual elements, under the same conditions and with similar properties as given in Theorem 2.1 for 2-tuples, can be constructed (cf. [5], Section 3.1). Definition 2.3 For k, N ∈ N with 2 ≤ k, let C = (2, k, 2) be a configuration and M a compatible (cf. [5], Section 2.1) N × k matrix. We call 4 5 6 the function Γ : [N ] × Ik −→ P (T ) an interaction-membership function. This function is interpreted as assigning, to each unordered selection of two different 3
For matrices L, M , denote with M the transpose of M and with LM matrix multiplication when defined. 4 For n ∈ N, let [n] = {1, . . . , n}. 5 For k ∈ N, let Ik = {(i, j) | 1 ≤ i < j ≤ k} ⊆ [k] × [k]. 6 For a set S, let P (S) denote its power set.
52
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
columns per row, a set of binary 2-tuples that are allowed to appear at this position. Such a function may specify contradicting conditions. Remark 2.4 Note that the non-appearance of certain value pairs at a position can be specified by considering their complement with respect to the set T and encoding this condition within an interaction-membership function. Corollary 2.5 For k, N ∈ N with 2 ≤ k, let C = (2, k, 2) be a configuration, M a compatible N × k matrix and Γ an interaction − membershipf unction. Then, there exists a system of multivariate polynomial equations E such that M fulfills the conditions given by Γ if and only if its elements satisfy E. We call these equations membership-equations. Proof. By Theorem 2.1, the condition expressed by every element in ran (Γ) (where for a function f , with ran (f ) we denote its range) holds if and only if the respective values satisfy the corresponding equation (4). Denote by E the set of all these equations, and the statement follows. 2 Corollary 2.6 For 2 ≤ k ∈ N, let C = (2, k, 2) be a configuration, M a compatible candidate matrix (cf. [5], Section 3.3) containing X-variables (cf. [5], Section 3.4) and Γ an interaction-membership function. Then, the following statements are equivalent. (i) The existence of a matrix, obtained from M by replacing all X-variables in M with a binary value, fulfilling all the conditions given by Γ. (ii) The non-emptiness of a variety corresponding to a system of multivariate polynomial equations derived from M consisting of the resulting binary conditions (cf. [5], Section 3.1) and the set E of membership-equations. Moreover, the multivariate polynomial equation system given above in (ii) can be solved using the theory of Gr¨ obner Bases. Proof. Follows from [5], Section 3.5; Corollary 2.5 and [2].
3
2
Possible Applications
Relaxed coverage requirements: CAs can be equivalently described with a set cover formulation of the coverage criteria [6]. Similarly, an efficient way of storing and querying t-tuple value selections has been developed [8], to which we will refer to as the tuple-enumerator. By definition, CAs require full t-way coverage. However, it is possible to only consider a proper nonempty subset of the tuple-enumerator. Such a proper nonempty subset might be incomplete regarding the unordered pairwise column selections, or incomplete regarding
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
53
the t-tuples that have to appear in an unordered t-selection of columns, or both. Variants of this concept have been investigated in the literature [10]. Furthermore, a comparison of [9] and [5] reveals that in case of a binary alphabet, techniques from [5] can be used to reason about covering arrays on graphs in the sense of [9] (see Definition 3). Non-appearance of interactions: In the binary alphabet case, for combinatorial objects termed CAFES in [4], Definition 2, the extended algebraic modelling presented in Section 2 is capable of equivalently expressing the properties of these objects. The appearance of interactions can be enforced according to [5]. The non-appearance of an interaction can be modelled algebraically according to Corollaries 2.5 and 2.6 and Remark 2.2. Constraints: We assume, for the scope of the paper, that constraints will be given as a list of forbidden pairwise interactions [11]. Such a CA with constraints problem can be equivalently formulated as a CAFE problem. Therefore, it can be treated with the methods presented in this paper. In [1], the authors distinguish between soft and hard constraints. A hard constraint is an for the application domain illegal interaction that may not appear in any test, whereas a soft constraint is an interaction that should be avoided or is neutral. Hard constraints can be treated with the methods presented in this paper. When an interaction that arises from a soft constraint is neither explicitly enforced to appear via a coverage equation (cf. [5], Definition 6) nor explicitly excluded by a membership-equation, then, depending on the used algebraic solver, this interaction may or may not appear in the constructed test suite.
4
Conclusion
In this paper, we have presented an algebraic modelling for enforcing memberships in pre-defined sets of certain pairwise interactions, providing an algebraic way of enforcing the appearance and non-appearance of certain valuepairs. Together with the results from [5], the new extended algebraic modelling makes it possible to argue about the mapping of CAs and related structures which appear in CT into a formal algebraic setting. Acknowledgements. This research was carried out in the context of the Austrian COMET K1 program and publicly funded by the Austrian Research Promotion Agency (FFG) and the Vienna Business Agency (WAW).
54
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
References [1] Ren´ee C Bryce and Charles J Colbourn. Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information and Software Technology, 48(10):960–970, 2006. [2] Bruno Buchberger. Bruno buchberger’s phd thesis 1965: An algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. J. Symb. Comput., 41:475–511, March 2006. [3] Charles J Colbourn and Jeffrey H Dinitz. Handbook of combinatorial designs. CRC press, 2006. [4] Peter Danziger, Eric Mendelsohn, Lucia Moura, and Brett Stevens. Covering arrays avoiding forbidden edges. Theoretical Computer Science, 410(52):5403– 5414, 2009. [5] Bernhard Garn and Dimitris E Simos. Algebraic modelling of covering arrays. In Special Sessions in Applications of Computer Algebra, pages 149– 170. Springer, 2015. [6] Ludwig Kampel, Bernhard Garn, and Dimitris E Simos. Covering arrays via set covers. Electronic Notes in Discrete Mathematics, 65:11–16, 2018. [7] D Richard Kuhn, Raghu N Kacker, and Yu Lei. Introduction to combinatorial testing. CRC press, 2013. [8] Yu Lei, Raghu Kacker, D Richard Kuhn, Vadim Okun, and James Lawrence. Ipog: A general strategy for t-way software testing. In Engineering of ComputerBased Systems, 2007. ECBS’07. 14th Annual IEEE International Conference and Workshops on the, pages 549–556. IEEE, 2007. [9] Karen Meagher and Brett Stevens. Covering arrays on graphs. Journal of Combinatorial Theory, Series B, 95(1):134–151, 2005. [10] Kaushik Sarkar, Charles J Colbourn, Annalisa De Bonis, and Ugo Vaccaro. Partial covering arrays: algorithms and asymptotics. In International Workshop on Combinatorial Algorithms, pages 437–448. Springer, 2016. [11] Linbin Yu, Feng Duan, Yu Lei, Raghu N Kacker, and D Richard Kuhn. Constraint handling in combinatorial test generation using forbidden tuples. In Software Testing, Verification and Validation Workshops (ICSTW), 2015 IEEE Eighth International Conference on, pages 1–9. IEEE, 2015.
Electronic Notes in Discrete Mathematics 70 (2018) 49–54
www.elsevier.com/locate/endm
Algebraic Techniques for Covering Arrays and related Structures Bernhard Garn 1 Dimitris E. Simos 2 SBA Research, Vienna A-1040, Austria
Abstract In this paper, we extend an existing algebraic modelling technique for covering arrays by considering additional properties which are required when these structures are applied in practice in a branch of software testing called combinatorial testing. Corresponding to these properties, we give semantically equivalent systems of multivariate polynomial equations. Keywords: covering arrays, Groebner bases, modelling, combinatorial testing
1
Introduction
Discrete structures called covering arrays (CAs) appearing in design theory have recently found an application in a branch of software testing called combinatorial testing (CT) [7]. Denote the natural numbers starting from one with N = {1, 2, 3, . . .}. Let t, k, v ∈ N ∧ t ≤ k, then a covering array for the configuration (t, k, v) can be viewed as an N × k array over a finite alphabet A of cardinality v, such that in any N × t subarray all possible t-tuples arising from this alphabet A appear at least once as rows in the selected subarray [3], [5]. The number t is called the strength of the CA, and the resulting condition 1 2
Email: [email protected] Email: [email protected]
https://doi.org/10.1016/j.endm.2018.11.008 1571-0653/© 2018 Elsevier B.V. All rights reserved.
50
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
is usually referred to as the t-way coverage condition. In this paper, we are only concerned with strength two CAs over the binary alphabet. In [5], an approach for an algebraic modelling of CAs was presented using commutative algebra, linear algebra and symbolic computation techniques for solving the resulting multivariate polynomial equation systems. In CT, constraints are usually given either as logic expressions or as a list of forbidden tuples. In heuristic extension strategies for CA construction, depending on the nature of the logic expressions, different kind of solvers can be used to create or check a (possibly only partial) candidate test case for any violations of the given constraints. When constraints are given as a list of forbidden tuples, an exhaustive search of the list of forbidden tuples can be used to establish that a given (possibly only partial) candidate test case does not violate any constraints. Multiple criteria have been used to decide, under each setting, which constraint handling method will be used. Due to the high general complexity of Gr¨obner base computation, heuristic approaches following the previously outlined strategies often outperform purely algebraic approaches for real-world problem sizes. We build upon our previous work [5] and present algebraic modellings for CAs and related structures, which arise when CAs are used in CT and have to satisfy additional conditions. An example for such an additional condition is the important applied notion of constraints on parameter values (e.g., let pi be a possible value of parameter Pi for i = 1, 2, and consider the implication P1 = p1 → P2 = p2 ), expressing necessary semantic relationships of the application domain. We link such constraints to semantically equivalent algebraic expressions. We have attained an algebraic method to enforce the appearance, as well as the non-appearance of certain binary two-tuples. This way, we can equivalently express constraints given in the form of a list of forbidden tuples in an algebraic setting. This paper is structured as follows. In Section 1, we also briefly comment on how constraints are handled in heuristic construction techniques for CAs. In Section 2, we extend the algebraic approach to enforce the appearance or non-appearance of certain binary two-tuples. In Section 3, we show how this new algebraic technique can be used to model problems regarding CAs coming from the application domain of CT, and conclude the paper in Section 4.
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
2
51
Membership Equations for covering Tuples in CAs
Theorem 2.1 Assume that P is a ring, a, b ∈ P , and that (P, a, b) has the pairwise binary tuple distinguishing property (cf. [5], Definition 4), t1 , t2 is a tuple with ti ∈ {0, 1}, i = 1, 2, and T = { 0, 0 , 1, 0 , 0, 1 , 1, 1 }, and let S be a subset of T , i.e. S ⊆ T . Denote with ϕ : T −→ {0, 1, a, b} the bijection derived in [5], Lemma 1, which is given by ϕ( 0, 0 ) = 0, ϕ( 1, 0 ) = a, ϕ( 0, 1 ) = b, ϕ( 1, 1 ) = a + b. (1) Then, the following statements (i), (ii) and (iii) are equivalent 3 . (i) (ii)
(iii)
t1 , t2 ∈ S;
(2)
0 ∈ { a, b t1 , t2 − ϕ(τ ) | τ ∈ S};
(3)
a, b t1 , t2 − ϕ(τ ) = 0.
(4)
τ ∈S
Proof. First, assume that S = ∅. The equivalence of (i) and (ii) follows from the fact that the function ϕ is a bijection. The equivalence of (ii) and (iii) follows from the defining property of an integral domain. Second, assume that S is empty. Then, the claimed equivalences trivially hold. 2 Remark 2.2 An algebraic distinguisher for 1-tuples; i.e., for individual elements, under the same conditions and with similar properties as given in Theorem 2.1 for 2-tuples, can be constructed (cf. [5], Section 3.1). Definition 2.3 For k, N ∈ N with 2 ≤ k, let C = (2, k, 2) be a configuration and M a compatible (cf. [5], Section 2.1) N × k matrix. We call 4 5 6 the function Γ : [N ] × Ik −→ P (T ) an interaction-membership function. This function is interpreted as assigning, to each unordered selection of two different 3
For matrices L, M , denote with M the transpose of M and with LM matrix multiplication when defined. 4 For n ∈ N, let [n] = {1, . . . , n}. 5 For k ∈ N, let Ik = {(i, j) | 1 ≤ i < j ≤ k} ⊆ [k] × [k]. 6 For a set S, let P (S) denote its power set.
52
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
columns per row, a set of binary 2-tuples that are allowed to appear at this position. Such a function may specify contradicting conditions. Remark 2.4 Note that the non-appearance of certain value pairs at a position can be specified by considering their complement with respect to the set T and encoding this condition within an interaction-membership function. Corollary 2.5 For k, N ∈ N with 2 ≤ k, let C = (2, k, 2) be a configuration, M a compatible N × k matrix and Γ an interaction − membershipf unction. Then, there exists a system of multivariate polynomial equations E such that M fulfills the conditions given by Γ if and only if its elements satisfy E. We call these equations membership-equations. Proof. By Theorem 2.1, the condition expressed by every element in ran (Γ) (where for a function f , with ran (f ) we denote its range) holds if and only if the respective values satisfy the corresponding equation (4). Denote by E the set of all these equations, and the statement follows. 2 Corollary 2.6 For 2 ≤ k ∈ N, let C = (2, k, 2) be a configuration, M a compatible candidate matrix (cf. [5], Section 3.3) containing X-variables (cf. [5], Section 3.4) and Γ an interaction-membership function. Then, the following statements are equivalent. (i) The existence of a matrix, obtained from M by replacing all X-variables in M with a binary value, fulfilling all the conditions given by Γ. (ii) The non-emptiness of a variety corresponding to a system of multivariate polynomial equations derived from M consisting of the resulting binary conditions (cf. [5], Section 3.1) and the set E of membership-equations. Moreover, the multivariate polynomial equation system given above in (ii) can be solved using the theory of Gr¨ obner Bases. Proof. Follows from [5], Section 3.5; Corollary 2.5 and [2].
3
2
Possible Applications
Relaxed coverage requirements: CAs can be equivalently described with a set cover formulation of the coverage criteria [6]. Similarly, an efficient way of storing and querying t-tuple value selections has been developed [8], to which we will refer to as the tuple-enumerator. By definition, CAs require full t-way coverage. However, it is possible to only consider a proper nonempty subset of the tuple-enumerator. Such a proper nonempty subset might be incomplete regarding the unordered pairwise column selections, or incomplete regarding
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
53
the t-tuples that have to appear in an unordered t-selection of columns, or both. Variants of this concept have been investigated in the literature [10]. Furthermore, a comparison of [9] and [5] reveals that in case of a binary alphabet, techniques from [5] can be used to reason about covering arrays on graphs in the sense of [9] (see Definition 3). Non-appearance of interactions: In the binary alphabet case, for combinatorial objects termed CAFES in [4], Definition 2, the extended algebraic modelling presented in Section 2 is capable of equivalently expressing the properties of these objects. The appearance of interactions can be enforced according to [5]. The non-appearance of an interaction can be modelled algebraically according to Corollaries 2.5 and 2.6 and Remark 2.2. Constraints: We assume, for the scope of the paper, that constraints will be given as a list of forbidden pairwise interactions [11]. Such a CA with constraints problem can be equivalently formulated as a CAFE problem. Therefore, it can be treated with the methods presented in this paper. In [1], the authors distinguish between soft and hard constraints. A hard constraint is an for the application domain illegal interaction that may not appear in any test, whereas a soft constraint is an interaction that should be avoided or is neutral. Hard constraints can be treated with the methods presented in this paper. When an interaction that arises from a soft constraint is neither explicitly enforced to appear via a coverage equation (cf. [5], Definition 6) nor explicitly excluded by a membership-equation, then, depending on the used algebraic solver, this interaction may or may not appear in the constructed test suite.
4
Conclusion
In this paper, we have presented an algebraic modelling for enforcing memberships in pre-defined sets of certain pairwise interactions, providing an algebraic way of enforcing the appearance and non-appearance of certain valuepairs. Together with the results from [5], the new extended algebraic modelling makes it possible to argue about the mapping of CAs and related structures which appear in CT into a formal algebraic setting. Acknowledgements. This research was carried out in the context of the Austrian COMET K1 program and publicly funded by the Austrian Research Promotion Agency (FFG) and the Vienna Business Agency (WAW).
54
B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 49–54
References [1] Ren´ee C Bryce and Charles J Colbourn. Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information and Software Technology, 48(10):960–970, 2006. [2] Bruno Buchberger. Bruno buchberger’s phd thesis 1965: An algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. J. Symb. Comput., 41:475–511, March 2006. [3] Charles J Colbourn and Jeffrey H Dinitz. Handbook of combinatorial designs. CRC press, 2006. [4] Peter Danziger, Eric Mendelsohn, Lucia Moura, and Brett Stevens. Covering arrays avoiding forbidden edges. Theoretical Computer Science, 410(52):5403– 5414, 2009. [5] Bernhard Garn and Dimitris E Simos. Algebraic modelling of covering arrays. In Special Sessions in Applications of Computer Algebra, pages 149– 170. Springer, 2015. [6] Ludwig Kampel, Bernhard Garn, and Dimitris E Simos. Covering arrays via set covers. Electronic Notes in Discrete Mathematics, 65:11–16, 2018. [7] D Richard Kuhn, Raghu N Kacker, and Yu Lei. Introduction to combinatorial testing. CRC press, 2013. [8] Yu Lei, Raghu Kacker, D Richard Kuhn, Vadim Okun, and James Lawrence. Ipog: A general strategy for t-way software testing. In Engineering of ComputerBased Systems, 2007. ECBS’07. 14th Annual IEEE International Conference and Workshops on the, pages 549–556. IEEE, 2007. [9] Karen Meagher and Brett Stevens. Covering arrays on graphs. Journal of Combinatorial Theory, Series B, 95(1):134–151, 2005. [10] Kaushik Sarkar, Charles J Colbourn, Annalisa De Bonis, and Ugo Vaccaro. Partial covering arrays: algorithms and asymptotics. In International Workshop on Combinatorial Algorithms, pages 437–448. Springer, 2016. [11] Linbin Yu, Feng Duan, Yu Lei, Raghu N Kacker, and D Richard Kuhn. Constraint handling in combinatorial test generation using forbidden tuples. In Software Testing, Verification and Validation Workshops (ICSTW), 2015 IEEE Eighth International Conference on, pages 1–9. IEEE, 2015.