Choosing columns from the 12-run Plackett–Burman design

Available online at www.sciencedirect.com

Statistics & Probability Letters 67 (2004) 193 – 201

Choosing columns from the 12-run Plackett–Burman design A. Millera;∗ , R.R. Sitterb;1 a

b

Department of Statistics, University of Auckland, Auckland, New Zealand Department of Statistics and Actuarial Science, Simon Fraser University, Burnaby, Canada BC, V5A 1S6 Received October 2003

Abstract There are two non-isomorphic choices of 5 columns and of 6 columns from the 12-run Plackett–Burman design which are essentially identical in terms of E(s2 ) and extended word length pattern. Surprisingly, in each case one choice is superior to the other for exploring models with main e2ects and a small number of 2-factor interactions. For 6 columns the di2erence between choices is substantial. c 2004 Elsevier B.V. All rights reserved.
Keywords: Complex aliasing; Estimation capacity; Resolvability; Search designs

1. Introduction Plackett–Burman designs are most often used to estimate main e2ects under the assumption that all interactions are negligible (Plackett and Burman, 1946). However, Hamada and Wu (1992) demonstrated that Plackett–Burman designs could also be used to investigate a small number of two-factor interactions (29’s) in addition to main e2ects provided that the number of signi9cant main e2ects is not too large. This article focuses on the 12-run Plackett–Burman design (PB12), see Table 1, which can accommodate up to 11 factors. If the number of factors to be investigated, k, is less than 11 then k of the 11 columns are selected. Sun et al. (2003) show that for orthogonal 12 × k designs there exists only one isomorphic class for k = 4 and 7 6 k 6 11 and exactly two isomorphic classes for k = 5 and 6. In particular for k = 4 and 7 6 k 6 11, any orthogonal 12 × k design must be isomorphic to the design created by taking the 9rst k columns from the design in Table 1. For k = 5, taking columns 1 through 5 yields a design which is non-isomorphic to the design obtained by ∗

Corresponding author. Tel.: +011-649-373-7599x85053; fax: +011-649-373-7018. E-mail address: [email protected] (A. Miller). 1 Sitter was supported by a grant from the Natural Sciences and Engineering Council of Canada.

c 2004 Elsevier B.V. All rights reserved. 0167-7152/$ - see front matter  doi:10.1016/j.spl.2004.01.006

194

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

Table 1 The 12-run Plackett–Burman design 1

2

3

4

5

6

7

8

9

10

11

1 −1 1 1 1 −1 1 −1 −1 1 −1 −1

1 −1 −1 1 1 1 −1 1 −1 −1 1 −1

1 −1 −1 −1 1 1 1 −1 1 −1 −1 1

1 1 −1 −1 −1 1 1 1 −1 1 −1 −1

1 −1 1 −1 −1 −1 1 1 1 −1 1 −1

1 1 1 −1 1 −1 −1 1 −1 −1 −1 1

1 1 1 1 −1 1 −1 −1 1 −1 −1 −1

1 −1 −1 1 −1 −1 −1 1 1 1 −1 1

1 1 −1 −1 1 −1 −1 −1 1 1 1 −1

1 −1 1 −1 −1 1 −1 −1 −1 1 1 1

1 1 −1 1 −1 −1 1 −1 −1 −1 1 1

taking columns 7 through 11—we will refer to these designs as PB125a and PB125b , respectively— and thus any other orthogonal 12 × 5 design must be isomorphic to one of these two designs. For k = 6 the two non-orthogonal designs, denoted as PB126a and PB126b , can be obtained by taking the columns not in PB125a and PB125b , respectively. As there are two non-isomorphic choices for k = 5 and 6, the question of whether it makes any di2erence as to which design is used arises. Clearly, it will not make any di2erence for main e2ects only applications as the PB12 has each main e2ect orthogonal to all other main e2ects. Intuitively, it would be expected that the choice of columns would have minimal impact for applications where the experimenter wants to investigate 29’s in addition to main e2ects since the PB12 design has a great deal of symmetry with respect to the way 29’s are related to main e2ects and to other 29’s. This intuition is supported by standard criteria, such as E(s2 )=N 2 , which indicate that there is little di2erence between the non-isomorphic choices. Thus it seems that the choice of columns should be of little consequence. This paper considers the properties of the PB125a , PB125b , PB126a and PB126b designs with respect to their suitability for search applications—speci9cally we are interested in the ability of these designs to correctly identify a small number of active 29’s. Surprisingly, our investigations show that the choice of columns does have a clear impact in this regard for both k = 5 and 6 and that for k = 6 the impact is substantial. In related work, Wang and Wu (1995) considered the eHciency with which the PB125a , PB125b , PB126a and PB126b designs estimate models that contain some 29’s in addition to main e2ects. They found di2erences between the PB125a and PB125b designs and between the PB126a and PB126b designs in terms of D-eHciency but nothing as dramatic as the di2erences we found by considering the suitability of these designs for search applications. Note that a design that can estimate each of two candidate models with high eHciency will not necessarily be able to distinguish between the models. For example, a 24−1 design with de9ning relation I = 1234 can be used to estimate both the model containing all main e2ects and the 12 interaction and the model containing all main e2ects and the 34 interaction with full eHciency but it is clearly unable to distinguish between them.

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

195

This article is structured in the following manner: Section 2 describes two criteria that are useful in assessing the overall aliasing in non-regular fractional factorial designs and applies them to the PB125a , PB125b , PB126a and PB126b designs. Section 3 discusses criteria that are directly related to the suitability of a design for search applications. Section 4 discusses the search problem in more detail and introduces the concept of minimal dependent subsets. Concluding remarks are made in Section 5. 2. E(s2 )=N 2 and extended word length pattern A number of criteria have been proposed to characterize aliasing in non-regular fractional factorial designs. Booth and Cox (1962) proposed E(s2 ) which is the mean of the squared o2-diagonal elements of XT X as a means of comparing supersaturated designs. Wu (1993) suggested modifying this criterion to E(s2 )=N 2 (where N is the number of runs) as this represents the averaged squared correlation over all pairs of columns in the design matrix. Since we are interested in 29’s in addition to main e2ects we apply this criterion to the matrix that contains all main e2ects and all 29’s. The value of E(s2 )=N 2 is 0.048 for both PB125a and PB125b , and is 0.056 for both PB126a and PB126b (see Table 2). Thus the E(s2 )=N 2 criterion does not distinguish between PB125a and PB125b or between PB126a and PB126b . A regular fractional factorial two-level design is often characterized by its de9ning contrast subgroup (see Wu and Hamada, 2000, pp. 157–159) which is the set of all e2ects that are aliased with the overall mean—note that each e2ect is either completely aliased with the overall mean or is orthogonal to it. It is well known that the aliasing in such a design is completely determined by its de9ning contrast subgroup. The e2ects in the de9ning contrast subgroup are referred to as “words” and the number of factors involved in each e2ect as its “word length”. The word length pattern of a design is a sequence of numbers (A3 , A4 ; : : :) where Aj indicates the number of words of length j in the de9ning contrast subgroup. As short words cause more severe aliasing than long words, the word length pattern provides a convenient method of assessing the aliasing in a design and of comparing the amount of aliasing in di2erent designs. Recent work by Deng and Tang (1999), and by Li et al. (2003) have extended these ideas to non-regular fractional factorial two-level designs. For these designs, e2ects in the de9ning contrast subgroup may be partially aliased with the overall mean. This complication is dealt with by extending the de9nition of word length to allow for fractional lengths. Let E represent an e2ect in the de9ning Table 2 E(s2 )=N 2 and EWLP Design

E(s2 )=N 2

EWLP

PB125a PB125b PB126a PB126b

0.048 0.048 0.056 0.056

(0; 0; 0)2 (0; 0; 0)2 (0; 0; 0)2 (0; 0; 0)2

Note: EWLP lists words of length t, t + 13 , and t +

2 3

(0; 0; 10)3 (0; 0; 10)3 (0; 0; 20)3 (0; 0; 20)3

for t = 2; 3; : : :.

(0; 0; 5)4 (0; 0; 0)5 (0; 0; 5)4 (0; 1; 0)5 (0; 0; 15)4 (0; 1; 0)5 (0; 0; 0)6 (0; 0; 15)4 (0; 0; 0)5 (0; 1; 0)6

196

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

contrast subgroup and let E represent the number of factors involved in E. Assume that the levels of E are designated as ±1 and let mE represent the mean level of E over the runs in the design. Note that E is orthogonal to the overall mean if mE = 0, E is completely aliased with the overall mean if mE = ±1, and E is partially aliased with the overall mean if 0 ¡ |mE | ¡ 1. The word length of E is de9ned as E + 1 − |mE |. For example, if E is a three-factor interaction that is completely aliased with the overall mean (mE = ±1) it has word length 3 but if it is only partially aliased with mE = ± 13 it has word length 3.67. For designs created by taking subsets of columns from the PB12 design all e2ects in the de9ning contrast subgroup have |mE | = 13 or 23 . Following the notation in Li et al. (2003), the sequence (A2:00 ; A2:33 ; A2:67 ), (A3:00 ; A3:33 ; A3:67 ); : : : will be referred to as the extended word length pattern (EWLP). Table 2 presents the EWLP’s for PB125a , PB125b , PB126a and PB126b . The PB125a and PB125b designs both have ten words of length 3.67 and 9ve words of length 4.67 in their de9ning contrast subgroups. However, PB125b has one additional word of length 5.33 and thus, judging by their EWLP’s, PB125a is slightly more desirable. The PB126a and PB126b designs both have 20 words of length 3.67 and 15 words of length 4.67 in their de9ning contrast subgroups. In addition, the PB126a design has one word of length 5.33 whereas the PB126b design has one word of length 6.33. Thus the PB126b design is judged as slightly more desirable. We say slightly, as the di2erences in the EWLP occur only in words of length greater than 5, which would normally be considered of little consequence if one is willing to assume interactions of order greater than 2 are negligible. 3. Estimation capacity and resolvability In this section, two criteria, estimation capacity and resolvability, are used to gain greater insight into the abilities of the PB125a , PB125b , PB126a and PB126b designs to consider 29’s. The framework for “search designs” proposed by Srivastava (1975) is adopted. Suppose that the factorial e2ects can be divided into three categories: (i) those that are assumed negligible, (ii) those for which an estimate is required, and (iii) the remaining e2ects most of which are negligible but a few of which may be non-negligible. The goal for a search application is to estimate all the e2ects in (ii) and to identify (and estimate) the non-negligible e2ects from (iii) under the assumption that the e2ects from (i) are negligible. For this paper we de9ne the groups as follows: (i) interactions involving three or more factors, (ii) main e2ects and (iii) 29’s. Thus a search design must provide estimates for all of the main e2ects and in addition be able to identify active 29’s—the larger the number of active 29’s that a design can correctly identify (with high probability) the more suitable it is for search applications. Clearly in this framework, a necessary condition for being able to correctly identify the active 29’s is that the model that contains all of the main e2ects plus the active 29’s must be estimable. Thus to be able to identify up to h active 29’s all models that contain up to h 29’s in addition to the main e2ects should be estimable—note that this is a necessary condition but, as will be explained shortly, it is not suHcient. A form of the estimation capacity sequence (see Sun, 1996) is used to evaluate the PB125a , PB125b , PB126a and PB126b designs in this regard. For a given design, consider all models that contain all of the main e2ects plus i of the 29’s and de9ne ECi as the proportion of those models that are estimable. The sequence EC1 , EC2 ; : : : ; will be referred to as the estimation capacity sequence for that design. The estimation capacity sequences for the PB125a ,

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

197

Table 3 Estimation capacity sequences Design

EC1

EC2

EC3

EC4

EC5

EC6

EC7

PB125a PB125b PB126a PB126b

1.000 1.000 1.000 1.000

1.000 1.000 1.000 0.857

1.000 1.000 1.000 0.593

0.952 0.929 0.945 0.297

0.762 0.643 0.689 0.081

0.000 0.000 0.000 0.000

0.000 0.000 0.000 0.000

PB125b , PB126a and PB126b designs are given in Table 3. For both PB125a and PB125b it is evident that all models that contain one, two, or three 29’s in addition to the 9ve main e2ects are estimable (EC1 = EC2 = EC3 = 1). PB125a out performs PB125b with respect to EC4 (0.952 to 0.929) and EC5 (0.762 to 0.643). Thus PB125a has a small advantage over PB125b in terms of estimation capacity. The di2erence between PB126a and PB126b is much more dramatic. The PB126a design allows all models that contain three or less 29’s to be estimated. On the other hand, the PB126b allows all one 29 models to be estimated but it can estimate less than 86% of two 29 models and less than 60% of three 29 models. Thus, in terms of estimation capacity the PB126a design is far superior to the PB126b design. It is interesting to recall that EWLP suggested that the PB126b design was slightly superior. In order to be able to distinguish between two competing models a stronger condition than simply requiring that both models are estimable is needed. Srivastava (1975) looks at this problem. Following Srivastava’s approach, suppose that h of the 29’s are active and consider the set of models that consist of all main e2ects and h of the 29’s—call this the h-candidate set. Now suppose that the true model is a member of the h-candidate set, then a design that can always identify the true model from the rest of this set assuming that there is no error in the observations, is said to be strongly resolvable with resolving power h—we adopt the notation rp = h. Srivastava (1975) shows that a necessary and suHcient condition for rp = h is that every model consisting of all of the main e2ects and 2h of the 29’s must be estimable. Thus from Table 3 it is easy to deduce that designs PB125a , PB125b and PB126a all have resolving power 1 whereas PB126b has resolving power 0. 4. E(RSS) and minimal dependent subsets One diHculty with Srivastava’s de9nition of strong resolvability is that it depends on there being no error in the observations which, of course, is never realized in practice. However, it is reasonably straightforward to show that Srivastava’s condition for strong resolvability stated at the end of the preceding section also implies the following: Theorem 1. For a design that is strongly resolvable with resolving power rp , if the true model is a member of an h-candidate set where h 6 rp then its expected residual sum of squares, E(RSS), is strictly less than that of any other h-candidate model. Proof. See Appendix.

198

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

Table 4 Minimal dependent sets Design

PB125a PB125b PB126a PB126b

Number of 29’s in the minimal dependent set 1

2

3

4

5

6

0 0 0 0

0 0 0 15

0 0 0 0

10 15 75 0

0 0 110 0

80 15 680 0

The essence of the proof is that if the response vector y has mean vector  and covariance matrix I2 then a candidate model with model matrix X will have E(RSS) =  (I − P) + 2 × rank(I − P) where P = X(X X)−1 X . All models from the same h-candidate set will have the same rank(I − P) and thus any di2erence in E(RSS) will be due to di2erences in  (I − P). Clearly  (I − P) ¿ 0 and is equal to zero for the true model. In addition,  (I − P) will equal 0 for an alternative model if and only if  is in the model space of that model. The condition that all of the models that contain 2h interactions are estimable insures that if h interactions are active then  cannot occur in the model space of any h-candidate model other than the true model. A certain amount of care should be taken when interpreting the resolving power of a design. First, a design with rp ¡ h may in fact be able to correctly identify a true model that contains h category (iii) e2ects in many situations, and second, two designs that have the same resolving power are not necessarily equally well suited for search applications. To illustrate these points we consider the PB125a and PB125b designs. In the previous section it was deduced that both these designs have rp = 1 since each design had EC2 = 1 (implying that rp ¿ 1) and EC4 ¡ 1 (implying that rp ¡ 2). Since it is the dependent sets that determine the resolving power of a design, we now look at these more closely. First, de9ne a minimal dependent set as a set of 29’s such that the model that contains all of the main e2ects and this set of 29’s is not estimable but if any one of the 29’s is removed the resulting model is estimable. In Table 4, the number of minimal dependent sets for the PB125a and PB125b designs are enumerated by size. For both designs the smallest minimal subsets are of size four. Each minimal subset of size four identi9es three pairs of 2-candidate models where it may be diHcult to distinguish between the models in each pair. For example, the PB125a design created by taking the 9rst 9ve columns of Table 1 has {12; 13; 24; 35} as one of its minimal dependent sets. This signi9es that under certain circumstances it will be diHcult to distinguish between the following sets of interactions: (a) {12; 13} from {24; 35}, (b) {12; 24} from {13; 35} and (c) {12; 35} from {13; 24}. To understand what is meant by “under certain circumstances” consider pair (a) and suppose that {12; 13} are active. Since all models under consideration contain the overall mean and all 9ve main e2ects, we are speci9cally interested in the additional information provided by the 29’s. Let Pme represent the projection matrix onto the model space for the main e2ects model and, as an example, de9ne 12∗ =(I −Pme )12 where 12 denotes the column vector for the 12 interaction. If interactions 12 and 13 are active then the component of  not explained by the main e2ects model, ∗ =(I−Pme ), is an element of the column space of [12∗ 13∗ ]—call this S1 and denote the column space of [24∗ 35∗ ] as S2 . The following identity is well known: dim(S1 ∩ S2 ) = dim(S1 ) + dim(S2 ) − dim(S1 + S2 ).

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

199

The following substitutions can be made: (a) dim(S1 ) = rank([12∗ 13∗ ]) = 2, since EC2 = 1 and similarly dim(S2 ) = 2, and (b) dim(S1 + S2 ) = rank([12∗ 13∗ 24∗ 35∗ ]) = 3, since {12; 13; 24; 35} is a minimal dependent set. It follows that dim(S1 ∩ S2 ) = 1. Thus if the e2ects of 12 and 13 have values such that ∗ is in the one-dimensional subspace of S1 de9ned by S1 ∩ S2 then the E(RSS) will be just as small for the model containing {24; 35} as for the true model otherwise the E(RSS) will be smaller for the true model. In practice, any time ∗ falls close to S1 ∩ S2 it will be diHcult to distinguish the true model {12; 13} from the {24; 35} model. Note that the same situation would arise if {24; 35} were the true model. Thus it will be diHcult to distinguish between these two models if ∗ falls close to S1 ∩ S2 . For the PB125a design there are ten minimal dependent sets of size 4. Each of the ten 29’s occur in four of these sets. More importantly 9ve of the 45 possible pairs of 29’s—{12; 35}, {13; 24}, {14; 25}, {15; 34} and {23; 45}—occur in four minimal dependent sets and all other 40 pairs occur in one each. Thus if the true model is one of the 9rst 9ve it may be diHcult to distinguish it from four other models (these being the other four listed pairs) but if it is one of the other 40 then it may be diHcult to distinguish from one other model. For the PB125b design there are 15 minimal dependent sets of size 4, each 29 occurs in 6 of these sets, and each pair of 29’s occur in the same set twice. Thus for any true model in the 2-candidate set, it may be diHcult to distinguish that model from two others. Thus both the PB125a and PB125b designs should be e2ective at identifying one active 29. If there are two active 29’s they should be able to at least narrow the set of possibilities down and often will be able to identify the correct model. Overall, we would recommend the PB125a design over the PB125b as it has fewer minimal dependent sets of size 4 but in practice the performance of these designs should not vary greatly. Now consider the PB126a and PB126b designs. The minimal dependent sets for these designs are enumerated in Table 4. The PB126a design is similar to the PB125a and PB125b designs in that its smallest minimal dependent subsets are of size 4. Thus it should be e2ective at identifying one active 29 and in many cases it will also be e2ective at identifying two active 29’s but may have diHculty distinguishing between certain sets of 2-candidate models. For the PB126a design there are 75 minimal dependent sets of size 4 and a total of 105 possible pairs of 29’s. Each of these pairs occurs in from 3 to 6 of the minimal dependent sets. The PB126b design has 15 minimal dependent sets of size 2. To explore the consequences of these sets we consider a speci9c example. For the PB126b design formed by columns 1 through 6 of Table 1, one of these sets consists of {12; 35}. Let S1 ≡ colspace(12∗ ) and S2 ≡ colspace(35∗ ). Clearly dim(S1 ) = dim(S2 ) = 1. As {12; 35} is a dependent set dim(S1 + S2 ) = 1. Thus dim(S1 ) ∩ dim(S2 ) = 1. It follows that S1 and S2 are the same vector space and thus the RSS for the model containing 12 will always be exactly the same as that for the model containing 35—note that this statement concerns the actual RSS and not just the E(RSS). The situation is very similar to what occurs for resolution IV 2k −p designs in that there are sets of 29’s that are completely aliased with each other. For the PB126b design there are 9ve such sets consisting of three 29’s each. If we adopt the usual practice of using “=” to denote “completely aliased with” these strings are 12∗ =35∗ =46∗ , 13∗ = 24∗ = 56∗ , 14∗ = 25∗ = 36∗ , 15∗ = 26∗ = 34∗ and 16∗ = 23∗ = 45∗ . The “∗ ” symbol is used to indicate that the complete aliasing only applies given that all main e2ects are in the model. Thus a practitioner using the PB126b design may be able to identify that one or two of these strings contain active 29’s but to isolate exactly which 29 was responsible in each string would require additional runs. Thus the PB126a design is superior to the PB126b design in that it should always be e2ective

200

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

at identifying one active 29 and often will be able to identify two active 29’s. On the other hand, in either situation the PB126b design will be able to narrow down the set of possibilities but will never be able to isolate the active 29’s. 5. Concluding remarks For search applications, the key attribute of a design is its ability to discriminate between competing models. For nonregular fractional factorial designs this ability is related to the aliasing between the candidate e2ects. The results presented in this paper, especially those pertaining to the PB126a and PB126b designs, indicate that this relationship is not reliably captured by standard measures such as E(s2 )=N 2 and the extended word length pattern. Considering the minimal dependent subsets gives a clearer picture of the suitability of a design for search applications. Appendix Proof of Theorem 1. Assume Srivastava’s framework as outlined in Section 3 and that all group (ii) e2ects and exactly h group (iii) e2ects are active. Further assume that Srivastava’s condition for strong resolvability with rp = h is satis9ed which implies that the model matrix for any g-candidate model where g 6 2h is of full rank. If a random vector y has E(y) =  and COV(y) = I2 then E(y Ay) =  A + 2 × trace(A) for any symmetric matrix A—see Schott (1996, p. 391). Thus if A = I − P where P = X(X X)−1 X for a model matrix X, then for that model E(RSS) =  (I − P) + 2 × rank(I − P). Now let XT be the model matrix for the true model and let  = XT  where  = (1 ; 2 ; : : :). Note that j = 0 for all j since, by de9nition, the true model contains only active e2ects. Clearly  ∈ colspace(XT ) and thus  (I − PT ) = 0 which means that for the true model E(RSS) = 2 × rank(I − PT ). Let XA represent the model matrix for any h-candidate model other than the true model—call this the alternative model. For this model E(RSS) =  (I − PA ) + 2 × rank(I − PA ). Srivastava’s condition insures that rank(I − PA ) = rank(I − PT ). Thus the E(RSS) for this model will be strictly greater than that for the true model if and only if  (I − PA ) ¿ 0 which, in turn, is true if and only if  ∈ colspace(XA ). We now show that XT  cannot be in colspace(XA ) unless some elements of  are 0 which is not allowed for . Consider ∗ such that XT ∗ ∈ colspace(XT ) ∩ colspace(XA ). Srivastava’s condition insures that the dimension of colspace(XT ) ∩ colspace(XA ) equals m + c where m is the number of group (ii) e2ects and c is the number of group (iii) e2ects that are common to both the true model and the alternative model (0 6 c ¡ h). Now let XC be the model matrix that contains all e2ects that are common to both XT and XA . Note that XC contains m + c columns and is of full rank. Therefore the columns of XC form a basis for colspace(XT ) ∩ colspace(XA ). Thus ∗ must have a zero for each element that corresponds to a column of XT that is not a column of XC . Note that if Srivastava’s condition is not true then for some alternative models the dimension of colspace(XT ) ∩ colspace(XA ) will be greater than m + c and it will be possible to 9nd XT ∗ ∈ colspace(XT ) ∩ colspace(XA ) where ∗ has no zero elements.

A. Miller, R.R. Sitter / Statistics & Probability Letters 67 (2004) 193 – 201

201

References Booth, K.H.V., Cox, D.R., 1962. Some systematic supersaturated designs. Technometrics 4, 489–495. Deng, L., Tang, B., 1999. Generalized resolution and minimum aberration criteria for Plackett–Burman and other nonregular factorial designs. Statist. Sin. 9, 1071–1082. Hamada, M., Wu, C.F.J., 1992. Analysis of designed experiments with complex aliasing. J. Quality Technol. 24, 130–137. Li, W., Lin, D.K.J., Ye, K., 2003. Optimal Foldover Plans for Non-Regular Designs. Technometrics 45, 347–351. Plackett, R.L., Burman, J.P., 1946. The design of optimum multifactor experiments. Biometrika 33, 305–325. Schott, J.R., 1996. Matrix Analysis for Statistics. Wiley, New York. Srivastava, J.N., 1975. Designs for searching non-negligible e2ects. In: Srivastava, J.N. (Ed.), A Survey of Statistical Design and Linear Models. North-Holland, Amsterdam. Sun, D.X., 1996. Estimation Capacity and Related Topics in Experimental Design. Unpublished Ph.D. Thesis, University of Waterloo. Sun, D.X., Li, W., Ye, K.Q., 2003. An algorithm for sequentially constructing non-isomorphic orthogonal designs and its applications. Unpublished manuscript. Wang, J.C., Wu, C.F.J., 1995. A hidden projection property of Plackett–Burman and related designs. Statist. Sin. 5, 235–250. Wu, C.F.J., 1993. Construction of supersaturated designs through partially aliased interactions. Biometrika 80, 661–669. Wu, C.F.J., Hamada, M., 2000. Experiments Planning, Analysis, and Parametric Design Optimization. Wiley, New York.