Weighted t-way Sequences

Weighted t-way Sequences

Available online at www.sciencedirect.com Electronic Notes in Discrete Mathematics 70 (2018) 43–48 www.elsevier.com/locate/endm Weighted t-way Sequ...

201KB Sizes 0 Downloads 54 Views

Available online at www.sciencedirect.com

Electronic Notes in Discrete Mathematics 70 (2018) 43–48

www.elsevier.com/locate/endm

Weighted t-way Sequences Bernhard Garn 1 Dimitris E. Simos 2 SBA Research, Vienna A-1040, Austria

Abstract We define the notion of weighted t-way sequences, which is built upon sequence covering arrays. The integration of a weight-based modelling formalism together with partitions of positive integers increases the expressiveness of the generated sequences considerably, and makes them applicable as abstract test sequences for real-world sequence testing problems. Applicability of this concept to real-world testing scenarios is investigated. Keywords: sequence covering array, covering array, partitions, combinatorial testing

1

Introduction

In this paper, we introduce the concept of combinatorial weighted t-way sequences, which can be seen as an extension of the notion of sequence covering arrays (SCAs). This more general definition is motivated both from the theory and application sides. Combinatorial methods based on permutations have been applied to develop methodologies for event sequence testing in a branch of software testing called combinatorial testing [3], leading to the notion of SCAs. In general event models for the real world, neither the appearance of all events nor the total lengths of the considered sequences can 1 2

Email: [email protected] Email: [email protected]

https://doi.org/10.1016/j.endm.2018.11.007 1571-0653/© 2018 Elsevier B.V. All rights reserved.

44

B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 43–48

be fixed a priori in general, making a permutation model not universally applicable. For example, if a sequence codifies occurring weather phenomena per day, the multiple appearance of an event (e.g., rain) interleaved with another event (e.g., sunshine) is a correct sequence. Also, in modelling team sport movement strategies, only a subset of all possible moves will appear in a single attack (sequence). Furthermore, in the modern pervasive computing landscape, the appearance, disappearance, and re-appearance of devices – not only in a classical client-server model, but also in the growing IoT landscape – is of utmost importance with regard to functionality and is also at the core of the computing services provided today regarding security and security testing. To address this issue, we have developed an extension of SCAs, which we call weighted t-way sequences. The integration of a weight-based modelling (based on partitions of positive integers) increases the expressiveness of the generated sequences considerably, while on the application side gives the power to incorporate current knowledge into the weights and therefore to tune or optimize generated test sequences for the application at hand. Moreover, our weight-based approach immediately gives reasons why certain sequences will be considered and provides a genuine justification of resulting sequence lengths as well as possible multiple occurrences of the same event. In particular, this paper makes the following contributions: (i) Discusses applicability of sequence covering arrays; (ii) Defines weighted t-way sequences; (iii) Provides examples how weighted t-way sequences can be applied. This paper is structured as follows. In Section 2, we briefly review SCAs and point to some of their issues in the Introduction. In Section 3, we gather necessary properties of partitions of positive integers. In Section 4, we define weighted t-way sequences and explain how resulting sequence test sets are constructed. We discuss applications of weighted t-way sequences in Section 5, and conclude the paper in Section 6.

2

Sequence Covering Arrays

Definition 2.1 [cf. [2]] A sequence covering array, SCA (N, S, t), is a N × s matrix, where the entries are from a finite set S of s symbols, such that every t-way permutation of symbols from S occurs in at least one row and each row is a permutation of the s symbols. The t symbols in the permutation are not required to be adjacent. That is, for every t-way arrangement of symbols x1 , x2 , · · · , xt , the regular expression . ∗ x1 . ∗ x2 · · · . ∗ xt .∗ matches at least one

B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 43–48

45

row in the array. The definition of SCAs was motivated by an applied testing scenario where a list of peripherals had to be connected to a computing device, and it was suspected that the order in which they are connected may give rise to errors when these peripherals then (try to) cooperate with each other. Since all peripherals had to be connected for full operational functionality (note that it only makes sense to connect them once), it follows naturally that in this case permutations where employed for the abstract modelling. In particular, this implies that in each test sequence, each event appears exactly once and that the length of each test sequence is constant, and equal to the total number of events.

3

Partitions of positive Integers

Definition 3.1 [cf. [1]] A partition of a positive integer n is  a finite nonincreasing sequence of positive integers λ1 , λ2 , . . . , λr such that ri=1 λi = n. The λi are called the parts of the partition. The partition function p (n) is the number of partitions of n. The function plistall (n) returns all possible partitions of n. As the order in which the parts of a partition appear is not   relevant, fρ f1 f2 we denote a partition as λ = (λ1 , λ2 , . . . , λr ) = λi1 , λi2 , . . . , λiρ . The exponential notation uses pairwise different λij , 1 ≤ j ≤ ρ, making explicit the number of times a particular integer occurs as a part. The number of parts  occurring in a partition including multiplicities is denoted as |λ| = r = ρi=1 fi .

4

Weighted t-way Sequences

Definition 4.1 We denote the positive integers with N. Let E be a nonempty finite set those elements are called events, and let ω be a positive integer-valued weight function defined on the set of events; i.e. ω : E −→ N. Let S be the set of all nonempty finite sequences over the alphabet E. Based on the weight function ω defined on the set of events E, we define a weight function for elements in S, which will, par abus de notation, also be denoted with ω. For μ ∈ N and given nonempty finite sequence s = (s1 , s2 , . . . , sμ ) = (si )i=1,...,μ ∈ S of length μ, we define the weight of s as ω(s) =

μ  i=1

w(si ).

(1)

46

B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 43–48

For given R ∈ N  let ∅ = B ⊆ {1, 2,. . . , R}, and for β ∈ B consider the set f plistall (β). Let λ = λfi11 , λfi22 , . . . , λiρρ ∈ plistall (β) and ζ ∈ {1, . . . , ρ}. The part λiζ now encodes a specific weight, and we consider all events which get assigned this weight; i.e., we consider the set 3 ω −1 [{λiζ }] ⊆ E.

(2)

Starting from the partition λ, we now build a Cartesian product of sets of events that correspond to the appearing parts in λ via their weight as shown in (2): Cλ = ω −1 [{λi1 }] × · · · × ω −1 [{λi1 }] × · · · × ω −1 [{λiρ }] × · · · × ω −1 [{λiρ }] . (3)     f1 times

fρ times

For t ∈ N and C ∈ Cλ , artificially consider all of the elements appearing in the coordinates of C as pairwise different and forming a set T (C); then it is possible to generate 4 a SCA S(T (C), t) of strength t for the alphabet T (C) in the sense of Definition 2.1. Let t ∈ N be the desired t-way coverage, then the complete weighted t-way sequences test set is defined as 5 :



T = S(T (C), t) ⊆ S. (4) β∈B λ∈plistall (β) C∈Cλ

Remark 4.2 Instead of considering all possible partitions of a positive integer, one might only consider a proper nonempty subset. For example, one could restrict the parts that are allowed to appear in the partitions to make sure that the set defined in relation (2) is nonempty; or allow only parts which are greater or smaller than a given threshold, or restrict the number of appearing summands. Any such condition can be interpreted as a condition on the events that are allowed to appear in the sequence (via their weight) or on the length of the sequence. Proposition 4.3 We list below some observations about the generated test set T in relation (4), which immediately follow from the construction process described above: 3

Let A, B be sets and f : A −→ B a function. For C ⊆ B, we denote the preimage of the subset C of B under the function f by f −1 [C] = {a ∈ A : f (a) ∈ C}. 4 In this paper, we regard the problem of practical generation of a SCA for given parameters t ∈ N and alphabet size μ ∈ N as solved and are not concerned with its efficiency. 5 In relation (4) the appearing SCAs are interpreted as sets of sequences derived from the rows of the SCAs, where the respective matrices are regarded as sets of rows.

B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 43–48

47



The set B determines those cumulative weights of sequences, which will be considered in the test sequence generation process, and thus makes it possible to limit the testing to exactly those cumulative weights of interest.



Individual test sequence lengths might be nonuniform across all partitions. This is due to the fact that various partitions may have a differing number of parts.



Events may appear multiple times in test sequences, or not at all. Multiple appearances happen, when a partition contains a part multiple times. However, from the point of generated t-SCAs, these events are regarded as different.



The determination of the weight function ω and the set B is application domain specific and can be adapted in an iterative testing setting.



The construction given above also works if the value t for the t-way coverage is nonuniform and depends on β, λ and Cλ .

Remark 4.4 It is possible to extend this approach, mutatis mutandis, to a setting where the weight of an event e ∈ E depends on the considered partition and cumulative weight, and where the weight of a finite nonempty sequence s ∈ S is a positive-valued function of the elements of the sequence and not necessarily their sum. These extensions might make it easier to tune the weights in an application domain, however, the connection to partitions of positive integers is then lost.

5

Applications

Security testing of software tries to identify and then exploit vulnerabilities in software. One common approach is submitting non-conformant input data to the system under test, for example by submitting the same data or command twice, skipping steps in negotiations during a handshake procedure, or requesting access to data without prior authentication. For security testing of implementations of the Transport Layer Security (TLS) protocol family, these kinds of attacks can naturally be modelled using weighted t-way sequences, where the individual weights could be based on an analysis of the severities of TLS vulnerabilities that have occurred in the past taken from a bug database. The idea is that the combination of impact and t-way permutations of events in the generated sequence test sets might expose new vulnerabilities. Another application domain for weighted t-way sequences are event models for the damages caused by natural disasters, for example by a tsunami hitting a coastal metropolitan city. Events codify damages, for example the (partial)

48

B. Garn, D.E. Simos / Electronic Notes in Discrete Mathematics 70 (2018) 43–48

breakage of an outer first embankment made of sand bags. The weight of such an event is defined as the number of resources (e.g., sand bags) that are necessary to repair the breakage. Strategies of emergency responders (firefighters, emergency medical services, etc.) can then be evaluated against the generated scenarios.

6

Conclusion and future work

In this paper, we have presented an approach for event sequence testing based on weighted t-way sequences. Our approach is motivated from impact analysis or physical requirements associated with events and operates on top of SCAs. Weights are derived application specific and can be tuned in an iterative testing process. Our extended approach offers some advantages over a permutation-based model for event sequence test generation. We plan to report on detailed practical evaluations of this concept in dedicated future work. Acknowledgements We thank our colleague Kristoffer Kleine for pointing out problems with SCAs for TLS testing. The research presented in the paper has been funded in part by the Austrian Research Promotion Agency (FFG) under grant 851205 (Security ProtocoL Interaction Testing in Practice - SPLIT) and carried out in the context of the Austrian COMET K1 program and partly publicly funded by the Austrian Research Promotion Agency (FFG) and the Vienna Business Agency (WAW). Moreover, this work was performed partly under the following financial assistance award 70NANB18H207 from U.S. Department of Commerce, National Institute of Standards and Technology.

References [1] George E Andrews. The theory of partitions. Number 2. Cambridge university press, 1998. [2] D Richard Kuhn, James M Higdon, James F Lawrence, Raghu N Kacker, and Yu Lei. Combinatorial methods for event sequence testing. In Software Testing, Verification and Validation (ICST), 2012 IEEE Fifth International Conference on, pages 601–609. IEEE, 2012. [3] D Richard Kuhn, Raghu N Kacker, and Yu Lei. Introduction to combinatorial testing. CRC press, 2013.