- Email: [email protected]
Construction of three-level supersaturated design
Journal of Statistical Planning and Inference 81 (1999) 183–193
www.elsevier.com/locate/jspi
Construction of three-level supersaturated design Shu Yamada ∗ , Yoshiko T. Ikebe, Hiroki Hashiguchi, Naoto Niki Department of Management Science, Science University of Tokyo, Kagurazaka 1-3, Shinjuku, Tokyo 162-8601, Japan Received 4 December 1997; accepted 22 December 1998
Abstract Supersaturated design is one type of fractional factorial design where the number of columns is greater than the number of rows. This type of design would be useful when costs of experiments are expensive and the number of factors is large, and there is a limitation on the number of runs. This paper presents some theorems on three-level supersaturated design and their application to construction. The construction methods proposed in this paper can be regarded as an extension c 1999 Elsevier Science B.V. of the methods developed for two-level supersaturated designs. All rights reserved. MSC: primary 62K15; secondary 05B20 Keywords: 2 statistic; Design of experiments; Measure of dependency; Orthogonal; Permutation of rows
1. Introduction In experimental studies, when costs of experiments are expensive and the number of factors is large, ordinary techniques are not applicable, due to the limitation on the number of experimental runs. One approach in such a situation is application of supersaturated designs under the ‘e ect sparsity’ assumption, that implies only a few dominant factors actually a ect the response. Supersaturated design is a fractional factorial design in which the number of factors is greater than the number of experimental runs. Some practical applications of supersaturated design are shown in Lin (1993, 1995) and Wu (1993). Supersaturated design was introduced by Satterthwaite (1959), who derived a random balance supersaturated design. One of the earliest systematic construction of supersaturated designs is that due to Booth and Cox (1962). In their construction, a number of initial columns are generated, then additional columns are added sequentially, while ∗
Corresponding author. Tel.: +81-3-3260-4271, ext 3336; fax: +81-3-3235-6479. E-mail address: [email protected] (S. Yamada)
c 1999 Elsevier Science B.V. All rights reserved. 0378-3758/99/$ - see front matter PII: S 0 3 7 8 - 3 7 5 8 ( 9 9 ) 0 0 0 0 7 - 5
184
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
avoiding the creation of large values of squared inner products between two columns of the design. By employing computer search, seven supersaturated designs have been obtained in this manner. Recently, Lin (1993) has proposed a method of construction for supersaturated designs via half fractions of Plackett and Burman (1946) design. Wu (1993) has proposed supersaturated designs by adding the cross products of two columns in the Plackett and Burman design. Iida (1994) has obtained some mathematical background on the addition of the cross products in the Plackett and Burman design. Lin (1995) has examined the maximum number of columns that can be accommodated when the degree of nonorthogonality is speciÿed by computer search. Nguyen (1996) described a method of constructing supersaturated designs from balanced incomplete block designs. Tang and Wu (1997) have shown a method for constructing supersaturated designs while considering the average squared inner products. Li and Wu (1997) have developed columnwise-pairwise algorithms to construct supersaturated designs. Yamada and Lin (1997) have given a new class of supersaturated design including an orthogonal base. As regarding two-level supersaturated design, there are some construction methods as follows: (a) Computer search: Booth and Cox (1962), Lin (1991), Lin (1995), Li and Wu (1997); (b) Exploration from previous two-level designs, e.g., half fraction of Plackett and Burman design: Lin (1993), Wu (1993), Iida (1994), Yamada and Lin (1997); (c) Sequential generation of columns by permutation of rows on the initial matrix: Tang and Wu (1997). As opposed to most of the existing literature, which considers two-level supersaturated designs, Yamada and Lin (1998) have shown a new class of three-level supersaturated design, i.e., the number of columns are greater than the number of rows and each column consists of three levels, such as 1, 2 and 3. The criteria on the dependency between two columns are derived by an analogy from hypothesis testing statistic on the two-way contingency table. Furthermore, a construction method which can be regarded as type (b) is shown. They give three-level supersaturated designs with 24, 36 and 48 rows. This paper ÿrst discusses some theorems on three-level supersaturated design similar to those on the previous two-level supersaturated designs. A method for construction of three-level supersaturated design is described base on the theorems, where the construction method can be regarded as an extension of Tang and Wu’s (1997) approach to three-level supersaturated design. Constructions based on the theorems and evaluation of the constructed designs are demonstrated.
2. Theorems on three-level supersaturated design Let N be a multiple of three and DN be the set of N -dimensional vectors consisting of equal numbers of 1s, 2s and 3s. For d1 ; d2 ; : : : ; dK ∈ DN , the N × K matrix D = [d1 ; d2 ; : : : ; dK ]
(1)
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
185
is called a three-level design. The usual deÿnition of two-level supersaturated design is that the number of the columns is greater than or equal to the number of rows. It implies that it is impossible to estimate all factor e ects under this relationship between the numbers of rows and columns (for example Tang and Wu (1997)). In case of three-level design, it is impossible to estimate the all factor e ects if 2K − 1 ¿ N . We call D a three-level supersaturated design if 2K − 1 ¿ N under consideration of the impossibility of estimation. 1 Yamada and Lin (1998) have deÿned a measure for dependency between two columns d ; e ∈ DN by 2 (d ; e) =
(nab − N=9)2 ; N=9 a;b=1;2;3 P
(2)
where nab is the number of rows whose values are [a; b] in the N × 2 matrix [d ; e]. Two vectors d; e ∈ DN are said to be independent if 2 (d ; e) = 0. Furthermore, they have deÿned the design criteria of dependency of a whole design D = [d1 ; : : : ; dK ] by P K 2 (di ; dj ) ; (3) ave(D) = 2 2 16i¡j6K max(D) = 2
max
16i¡j6K
2 (di ; dj ):
(4)
The construction of three-level supersaturated design can be described as the selection of vectors from the set DN by a reasonable rule. The following lemma can be regarded as a special case for the theorem on the multi-level orthogonal design, proven by Niki et al. (1997), and it is useful to construct initial matrices in three-level supersaturated designs. Lemma 1. 1. If N is a power of three; then the maximum number of mutually independent vectors in DN is (N − 1)=2. 2. Let d1 ; : : : ; d(N −1)=2 be mutually independent vectors; then it holds that (N P −1)=2 i=1
2 (di ; d) = 2N
(5)
for any d ∈ DN . By using this lemma, we may prove the next three theorems. Theorem 1. Let N be a power of three and let D1 be an N × L matrix consisting of L = (N − 1)=2 mutually independent column vectors. Suppose that the matrices D2 ; D3 ; : : : ; DM are generated by permutations of rows on D1 ; and consider the N × (ML) matrix: e = [D1 D2 · · · DM ]: D 1
This deÿnition of three-level supersaturated design is due to a useful comment given by a referee.
(6)
186
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Then the average 2 values on the design D˜ is e = ave(D) 2
4N (M − 1) ; M (N − 1) − 2
(7)
regardless of the permutation rule. Proof. Write e = [D1 D2 · · · DM ] = [d11 ; : : : ; d1L ; d21 ; : : : ; d2L ; : : : ; dM 1 ; : : : ; dML ] D
(8)
then, from the independence of the d’s and Lemma 1, the sum of 2 values of all e is pairs between two columns in D M P P P P ML e = 2 (dmi ; dmj ) + 2 (dmi ; dm0 j ) ave(D) 2 0 2 m=1 16i¡j6L 16m¡m 6M 16i; j6L P P 2 (dmi ; dm0 j ) (9) = 16m¡m0 6M 16i; j6L
MN (M − 1)(N − 1) ; 2 which completes the proof. =
(10)
Theorem 2. Let N be a power of three. Then; there exists no three-level supersaturated design which contains (N − 1)=2 mutually independent columns and satisÿes max(D) ¡ 2
4N : N −1
(11)
Proof. Suppose that there exists a three-level supersaturated design which contains L = (N − 1)=2 mutually independent columns and satisÿes Eq. (11), say D = [d1 ; : : : ; dL ; dL+1 ; : : : ; dK ]. Without loss of generality, assume that d1 ; : : : ; dL are mutually independent. Since max 2 (D) is less than 4N=(N − 1), the sum of the 2 values between any d ∈ DN and di is L P i=1
2 (di ; d) ¡
N − 1 4N = 2N: 2 N −1
(12)
The right-hand side and Eq. (5) contradict each other and the theorem is proved. Theorem 3. Given a three-level (supersaturated) design D = [d1 ; d2 ; : : : ; dk ] with N rows; let + (D) be the matrix obtained from D by transforming all 1s to 2s; all 2s to 3s and all 3s to 1s; similarly; let − (D) be the matrix obtained from D by transforming all 1s to 3s; all 2s to 1s and all 3s to 2s (note that all columns of + (D) and − (D) belong to DN ). Next; by using these matrices; construct the three-level e by supersaturated designs D∗ and D D D D ; (13) D∗ = D + (D) − (D)
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
1 e D= 2 3
D D D
D − (D) ; + (D)
D + (D) − (D)
187
(14)
where 1; 2 and 3 are column vectors of 1s; 2s and 3s; respectively. Then the maximum values on these designs are
max(D ) = max N; 2 max(D) ;
(15)
e = 3 max(D): max(D)
(16)
∗
2
2
2
2
Proof. We put D ∗ D = D
1 e =2 D 3
D + (D) D D D
D ∗ ∗ ∗ ∗ = [d11 ; : : : ; d1k ; : : : ; d31 ; : : : d3k ]; − (D)
D + (D) − (D)
D − (D) = [de0 ; de11 ; : : : ; de1k ; : : : ; de31 ; : : : ; de3k ]: + (D)
(17)
(18)
If m 6= m0 , then the relations ∗ ; dm∗0 i ) = N 2 (dmi
(m 6= m0 );
∗ ∗ ; dmj )62 2 (di ; dj ) 2 (dmi ∗ ; dm∗0 j )62 2 (di ; dj ) 2 (dmi
(19)
(i 6= j);
(20)
(m 6= m0 ; i 6= j)
(21)
can be derived straightforwardly by calculating the 2 value proving Eq. (15). Furthermore, 2 (de0 ; demi ) = 0; 2 (demi ; dem0 i ) = 0
(22) (m 6= m0 );
(23)
2 (demi ; demj )63 2 (di ; dj )
(i 6= j);
(24)
2 (demi ; dem0 j )63 2 (di ; dj )
(m 6= m0 ; i 6= j)
(25)
can be derived in a similar way, verifying Eq. (16). Remark. We now summarize the relationship between the theorems on two-level and three-level supersaturated designs. As regarding two-level supersaturated designs, Tang and Wu (1997) have proposed the following construction algorithm:
188
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
1. An N -dimensional orthogonal base, consisting of a set of N − 1 vectors, is selected from the set of two-level equal occurrence vectors, where a two-level equal occurrence vector is a vector in which the number of occurrences at one level is equal to the number of occurrences at another level. The base is used as an initial matrix. 2. Some designs are generated sequentially by permutations of the rows of the initial matrix. This algorithm is justiÿed by the fact that the criterion for the average squared inner product does not depend on the permutation rule. Hence, by specifying the average squared inner product to a given level, two-level supersaturated designs may be derived by this algorithm. Theorem 1 can be regarded as an extension of the theorem on two-level supersaturated designs, obtained by Tang and Wu (1997), to three-level. Furthermore, Yamada and Lin (1997) has obtained a method to generate two-level supersaturated designs with N = 2n rows and K = 2k + 1 columns from a two-level supersaturated design with n rows and k columns, when the maximum squared inner product on the generated design is equal to a pre-speciÿed level. Theorems 2 and 3 can be regarded as an extension of Yamada and Lin (1997). The theorems shown in this paper imply that three-level supersaturated designs can be constructed in a similar way to the constructions of two-level supersaturated designs.
3. Construction of three-level supersaturated designs 3.1. Example: N = 9 Theorem 1 ensures that three-level supersaturated designs can be generated by permutation of an initial three-level design while keeping ave 2 (D) at a pre-speciÿed level. Furthermore, Theorem 2 indicates that there does not exist a three-level supersaturated design such that max 2 (D)64 for N = 9, where it includes four mutually independent columns. Table 1 shows all possible variations of 2 values for N = 9 and D. This table was obtained by a brute force examination of all possible combinations of two three-level columns. A three-level supersaturated design which satisÿes max 2 (D) = 6 Table 1 Variations of 2 values N
Variations
9 18
0, 4, 6, 10, 18 0, 2, 3, 5, 6, 8, 9, 11, 12, 14, 17, 18 20, 21, 26, 36 0, 1 13 ; 2, 3 13 ; 4, 5 13 ; 6, 7 13 ; 8, 9 13
27
10, 11 13 ; 12, 13 13 ; 14, 15 13 ; 16, 17 13 ; 18, 19 13 20, 21 13 ; 23 13 ; 24, 26, 27 13 ; 29 13 ; 30, 33 13 ; 35 13 ; 38, 43 13 ; 54
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
189
Table 2 A permutation rule to construct the design with N = 9 and K = 20: max 2 (D) = 6 D1
D2
D3
D4
D5
1 2 3 4 5 6 7 8 9
1 7 9 8 6 5 4 3 2
1 9 6 7 8 5 3 4 2
1 6 7 8 9 3 5 4 2
1 7 9 3 8 6 5 4 2
is explored since 1 1 1 2 D1 = 2 2 3 3 3
2 = 6 is the next to 4 in Table 1. Speciÿcally, the 9 × 4 matrix: 1 1 1 2 2 2 3 3 3 1 2 3 (26) 2 3 1 3 1 2 1 3 2 2 1 3 3
2
1
was utilized as an initial matrix for the construction. The matrices D2 ; D3 ; : : : were generated sequentially by permutations of the rows on the initial matrix, where a permuted matrix Dj is added if the maximum 2 value with each previous column is less than or equal to 6. Finally a three-level supersaturated design D = [D1 ; D2 ; : : : ; DK ] which satisÿes max 2 (D) = 6 was constructed. Table 2 shows the permuted orders on the initial matrix. The table was derived by an examination of all permutations on the rows (9!=362 880). Speciÿcally, all columns are examined in some order, so, there is a possibility to explore other combinations of columns by other orders. For example, the second column in Table 2 indicates that D2 was constructed by the permutation as follows: 1st row to 1st row, 2nd row to 7th row, 3rd row to 9th row, : : :, 9th row to 2nd row. Furthermore, a three-level supersaturated design such that max 2 (D) = 10 was explored by a similar computer search which examines all permutations of rows. Table 3 shows the permutations on the initial matrix. A three-level supersaturated design with 9 rows and 220 columns is generated from this table. 3.2. Example: N = 18 and 27 Theorem 3 was applied to construct a three-level supersaturated design because there are too many permutations (18! ≈ 6 × 1015 ) to employ a computer search similar to the case N = 9. A three-level supersaturated design with 18 rows and 60 columns was
190
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Table 3 A permutation rule to construct the design with N = 9 and K = 220: max 2 (D) = 10 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1 2 3 4 5 6 7 8 9
1 2 3 4 5 7 6 9 8
1 2 3 4 5 7 8 6 9
1 2 3 4 5 7 9 8 6
1 2 3 4 7 6 5 9 8
1 2 3 4 7 6 8 5 9
1 2 3 4 7 6 9 8 5
1 2 3 4 7 8 6 9 5
1 2 3 4 7 8 9 5 6
1 2 3 4 8 9 5 6 7
1 2 4 3 5 6 8 9 7
1 2 4 3 5 7 9 6 8
1 2 4 3 5 8 6 7 9
1 2 4 3 5 8 7 9 6
1 2 4 3 7 5 6 8 9
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1 2 4 3 7 5 8 9 6
1 2 4 3 7 8 9 6 5
1 2 4 3 7 9 5 6 8
1 2 4 3 8 6 7 5 9
1 2 4 3 8 7 5 9 6
1 2 4 5 3 6 7 8 9
1 2 4 5 3 8 9 7 6
1 2 4 5 3 9 6 7 8
1 2 4 5 3 9 8 6 7
1 2 4 5 6 3 8 7 9
1 2 4 5 6 7 3 8 9
1 2 4 5 6 7 8 9 3
1 2 4 5 6 7 9 3 8
1 2 4 5 8 6 3 9 7
1 2 4 5 8 6 9 7 3
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
1 2 4 5 8 9 7 3 6
1 2 4 5 9 6 7 3 8
1 2 4 6 7 3 5 8 9
1 2 4 6 7 3 9 5 8
1 2 4 6 8 7 3 5 9
1 2 4 6 9 3 7 8 5
1 2 4 6 9 7 5 3 8
1 2 4 6 9 8 3 7 5
1 2 4 7 6 5 9 8 3
1 2 4 7 6 9 3 8 5
1 2 4 7 6 9 8 5 3
1 2 4 7 9 3 5 6 8
1 2 4 7 9 5 6 3 8
1 2 4 7 9 5 8 6 3
1 2 4 8 3 5 6 9 7
46
47
48
49
50
51
52
53
54
55
1 2 4 8 7 9 3 6 5
1 2 4 8 7 9 5 3 6
1 2 4 8 7 9 6 5 3
1 2 4 9 3 6 8 7 5
1 2 4 9 3 7 5 8 6
1 2 4 9 3 7 6 5 8
1 2 4 9 3 8 7 5 6
1 2 4 9 6 3 5 8 7
1 2 4 9 6 8 3 5 7
1 2 4 9 8 3 6 7 5
generated by substituting the supersaturated design with 9 rows and 20 columns into Eq. (17). In this design, the maximum dependency on the generated supersaturated design is assured to be 12 because max 2 (D∗ ) = 12 by Theorem 3. A three-level supersaturated design with 27 rows and 61 columns was generated by Theorem 3 from a supersaturated design with 9 rows and 20 columns. In this case, the maximum dependency on the design is 18.
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
191
The supersaturated design with 9 rows and 220 columns whose permutations are shown in Table 3 may also be used to construct three-level supersaturated designs with 18 or 27 rows. In the generation of designs with 18 rows and 660 columns, Theorem 3 assures that max 2 (D∗ )=20, and in designs with 27 rows and 661 columns, e = 30: max 2 (D) 3.3. Evaluation of dependency in the design 1. There are 190 pairs of two columns in the three-level supersaturated design with 9 rows and 20 columns. The frequency of the 2 values is shown in Table 4. Table 4 also shows the frequency of 2 values on the three-level supersaturated design with 9 rows and 220 columns. By a comparison between the designs with 20 columns and 220 columns, the design with 220 columns is more ecient in terms of the number of columns although the maximum and average dependencies between columns is strong. In the design with 220 columns, the maximum value of 2 is 10 and the frequency of 2 (di ; dj ) = 10 to the total is around 10%, so the proportion may not be so high. This can be explained by the following reason. Let us consider d1 ; : : : ; d4 ; these are mutually independent columns. Eq. (5) implies that when a vector d has the relation 2 (d ; d1 ) = 10, the 2 values with the other columns are {4; 4; 0} from Table 1. 2. Consider the comparison on the designs where the numbers of rows are di erent, such as comparison between designs with 9 and 18 rows. Since the number of rows N a ects 2 , a direct comparison of the frequency would be inadequate. Eq. (5) shows that the sum of the 2 values of independent columns are 2N , so 2 =N would be a measure for the dependency between two columns standardized by N . Of course, this is an index for the comparison of the di erent size of designs, but it would be acceptable as an initial method for comparison. For example, 2 = 6 in the design with N = 9 rows corresponds to 2 = 12 in the design with N = 18 rows in terms of the standardized dependency. Table 5 shows the frequency of the standardized measure 2 =N for the designs with 9; 18 and 27 rows. 3. Fig. 1 demonstrates the proportion based on the frequency of 2 =N on the designs with 9; 18 and 27 rows. From this ÿgure and Table 5, the design with 27 rows is
Table 4 Frequency of 2 on the designs with N = 9 rows N
K
K 2
2 0
9
20
190
4
32 114 17 60 9 220 24090 3152 13972 13 58 Note: Upper: frequency, Lower: percentage (%).
6 44 23 4657 19
10
2309 10
max
ave
6
3.79
10
4.43
192
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Table 5 Frequency of the standardized dependency: 2 =N on the designs with N = 9, 18 and 27 rows N
K
K 2
2 =N 0/18
9
20
190
18
60
1770
27
61
1830
32 17 288 17 1128 62
2 2/18
456 26
3/18
264 15
4/18
456 25
5/18
8/18
456 26
114 60 114 6 114 6
9/18
12/18
60 4
44 23 132 7 132 7
Max.
Ave.
6
3.79
12
3.97
18
3.54
Note: Upper: frequency, Lower: percentage (%).
Fig. 1. Proportions of the frequencies on 2 =N in the designs with N = 9, 18 and 27 rows.
the most ecient in terms of the standardized dependency since the most of the pairs concentrate low level of 2 =N , e.g. 87% of the pairs appear up to 2 =N = 4=18. 4. Concluding remarks This paper shows some theorems on three-level supersaturated design and their applications to construction. Some three-level supersaturated designs with 9 rows are derived by the sequential addition of matrices constructed by permutations of rows on the initial matrix, because the average dependency between two columns does not depend on the permutation rule. Here the maximum dependencies on two columns in the designs are equal to some pre-speciÿed levels. In the cases of row sizes N = 18 and 27; it is impossible to apply similar computer search methods, so a construction method is demonstrated. Speciÿcally, the method generates three-level supersaturated designs with 18 and 27 rows from a design with 9
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
193
rows. Here, the maximum dependencies between two columns are also equal to some pre-speciÿed level. The construction methods can be regarded as an extension of the methods developed for two-level supersaturated designs. The existence of the correspondence between three-level and two-level designs would be one of the main results of this paper. However, this paper considers only the construction of designs. As regarding the data analysis, stepwise regression is usually applied to analyze data collected by two-level supersaturated design (Lin, 1993,1995). In the case of three-level supersaturated design, stepwise regression would be also acceptable as well as the case of two-level supersaturated design, where each of three-level columns is transformed to some columns constructed by dummy variables. According to the analysis, an estimate of the effect of a factor is independent to an estimate of the e ect of another factor when 2 value between two columns is equal to 0. On the other hand, the two estimates are completely dependent when 2 = 2N . Intuitively, the two estimates assigned to two di erent three-level columns would be nearly independent when the 2 value is close to 0. More detailed discussion on analysis of data collected by the proposed three-level supersaturated design is a problem for future research. Acknowledgements We express sincere thanks to the editor, associate editor and two anonymous referees, each of whom made valuable comments that helped us to improve this article. References Booth, K.H.V., Cox, D.R., 1962. Some systematic supersaturated designs. Technometrics 4, 489–495. Iida, T., 1994. A construction method of two level supersaturated design derived from L12. Jpn. J. Appl. Statist. 23, 147–153 (in Japanese). Li, W.W., Wu, C.F.J., 1997. Columnwise-pairwise algorithms with applications to the construction of supersaturated designs. Technometrics 39, 171–179. Lin, D.K.J., 1993. A new class of supersaturated designs. Technometrics 35, 28–31. Lin, D.K.J., 1995. Generating systematic supersaturated designs. Technometrics 37, 213–225. Niki, N., Hashiguchi, H., Yamada, S., Ikebe, Y.T., 1997. Three theorems on multi-level orthogonal design (in preparation). Nguyen, N.K., 1996. An algorithm approach to constructing supersaturated designs. Technometrics 38, 69–73. Plackett, R.L., Burman, J.P., 1946. The design of optimum multifactorial experiments. Biometrika 33, 303–325. Satterthwaite, F.E., 1959. Random balance experimentation (with discussions). Technometrics 1, 111–137. Tang, B., Wu, C.F.J., 1997. A method for constructing supersaturated designs and its E(s2 ) optimality. Canadian J. Statist. 25, 191–201. Wu, C.F.J., 1993. Construction of supersaturated designs through partially aliased interactions. Biometrika 80, 661–669. Yamada, S., Lin, D.K.J., 1997. Supersaturated design including an orthogonal base. Canadian J. Statist. 25, 203–213. Yamada, S., Lin, D.J.K., 1998. Three-level supersaturated design. Submitted for publication.
www.elsevier.com/locate/jspi
Construction of three-level supersaturated design Shu Yamada ∗ , Yoshiko T. Ikebe, Hiroki Hashiguchi, Naoto Niki Department of Management Science, Science University of Tokyo, Kagurazaka 1-3, Shinjuku, Tokyo 162-8601, Japan Received 4 December 1997; accepted 22 December 1998
Abstract Supersaturated design is one type of fractional factorial design where the number of columns is greater than the number of rows. This type of design would be useful when costs of experiments are expensive and the number of factors is large, and there is a limitation on the number of runs. This paper presents some theorems on three-level supersaturated design and their application to construction. The construction methods proposed in this paper can be regarded as an extension c 1999 Elsevier Science B.V. of the methods developed for two-level supersaturated designs. All rights reserved. MSC: primary 62K15; secondary 05B20 Keywords: 2 statistic; Design of experiments; Measure of dependency; Orthogonal; Permutation of rows
1. Introduction In experimental studies, when costs of experiments are expensive and the number of factors is large, ordinary techniques are not applicable, due to the limitation on the number of experimental runs. One approach in such a situation is application of supersaturated designs under the ‘e ect sparsity’ assumption, that implies only a few dominant factors actually a ect the response. Supersaturated design is a fractional factorial design in which the number of factors is greater than the number of experimental runs. Some practical applications of supersaturated design are shown in Lin (1993, 1995) and Wu (1993). Supersaturated design was introduced by Satterthwaite (1959), who derived a random balance supersaturated design. One of the earliest systematic construction of supersaturated designs is that due to Booth and Cox (1962). In their construction, a number of initial columns are generated, then additional columns are added sequentially, while ∗
Corresponding author. Tel.: +81-3-3260-4271, ext 3336; fax: +81-3-3235-6479. E-mail address: [email protected] (S. Yamada)
c 1999 Elsevier Science B.V. All rights reserved. 0378-3758/99/$ - see front matter PII: S 0 3 7 8 - 3 7 5 8 ( 9 9 ) 0 0 0 0 7 - 5
184
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
avoiding the creation of large values of squared inner products between two columns of the design. By employing computer search, seven supersaturated designs have been obtained in this manner. Recently, Lin (1993) has proposed a method of construction for supersaturated designs via half fractions of Plackett and Burman (1946) design. Wu (1993) has proposed supersaturated designs by adding the cross products of two columns in the Plackett and Burman design. Iida (1994) has obtained some mathematical background on the addition of the cross products in the Plackett and Burman design. Lin (1995) has examined the maximum number of columns that can be accommodated when the degree of nonorthogonality is speciÿed by computer search. Nguyen (1996) described a method of constructing supersaturated designs from balanced incomplete block designs. Tang and Wu (1997) have shown a method for constructing supersaturated designs while considering the average squared inner products. Li and Wu (1997) have developed columnwise-pairwise algorithms to construct supersaturated designs. Yamada and Lin (1997) have given a new class of supersaturated design including an orthogonal base. As regarding two-level supersaturated design, there are some construction methods as follows: (a) Computer search: Booth and Cox (1962), Lin (1991), Lin (1995), Li and Wu (1997); (b) Exploration from previous two-level designs, e.g., half fraction of Plackett and Burman design: Lin (1993), Wu (1993), Iida (1994), Yamada and Lin (1997); (c) Sequential generation of columns by permutation of rows on the initial matrix: Tang and Wu (1997). As opposed to most of the existing literature, which considers two-level supersaturated designs, Yamada and Lin (1998) have shown a new class of three-level supersaturated design, i.e., the number of columns are greater than the number of rows and each column consists of three levels, such as 1, 2 and 3. The criteria on the dependency between two columns are derived by an analogy from hypothesis testing statistic on the two-way contingency table. Furthermore, a construction method which can be regarded as type (b) is shown. They give three-level supersaturated designs with 24, 36 and 48 rows. This paper ÿrst discusses some theorems on three-level supersaturated design similar to those on the previous two-level supersaturated designs. A method for construction of three-level supersaturated design is described base on the theorems, where the construction method can be regarded as an extension of Tang and Wu’s (1997) approach to three-level supersaturated design. Constructions based on the theorems and evaluation of the constructed designs are demonstrated.
2. Theorems on three-level supersaturated design Let N be a multiple of three and DN be the set of N -dimensional vectors consisting of equal numbers of 1s, 2s and 3s. For d1 ; d2 ; : : : ; dK ∈ DN , the N × K matrix D = [d1 ; d2 ; : : : ; dK ]
(1)
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
185
is called a three-level design. The usual deÿnition of two-level supersaturated design is that the number of the columns is greater than or equal to the number of rows. It implies that it is impossible to estimate all factor e ects under this relationship between the numbers of rows and columns (for example Tang and Wu (1997)). In case of three-level design, it is impossible to estimate the all factor e ects if 2K − 1 ¿ N . We call D a three-level supersaturated design if 2K − 1 ¿ N under consideration of the impossibility of estimation. 1 Yamada and Lin (1998) have deÿned a measure for dependency between two columns d ; e ∈ DN by 2 (d ; e) =
(nab − N=9)2 ; N=9 a;b=1;2;3 P
(2)
where nab is the number of rows whose values are [a; b] in the N × 2 matrix [d ; e]. Two vectors d; e ∈ DN are said to be independent if 2 (d ; e) = 0. Furthermore, they have deÿned the design criteria of dependency of a whole design D = [d1 ; : : : ; dK ] by P K 2 (di ; dj ) ; (3) ave(D) = 2 2 16i¡j6K max(D) = 2
max
16i¡j6K
2 (di ; dj ):
(4)
The construction of three-level supersaturated design can be described as the selection of vectors from the set DN by a reasonable rule. The following lemma can be regarded as a special case for the theorem on the multi-level orthogonal design, proven by Niki et al. (1997), and it is useful to construct initial matrices in three-level supersaturated designs. Lemma 1. 1. If N is a power of three; then the maximum number of mutually independent vectors in DN is (N − 1)=2. 2. Let d1 ; : : : ; d(N −1)=2 be mutually independent vectors; then it holds that (N P −1)=2 i=1
2 (di ; d) = 2N
(5)
for any d ∈ DN . By using this lemma, we may prove the next three theorems. Theorem 1. Let N be a power of three and let D1 be an N × L matrix consisting of L = (N − 1)=2 mutually independent column vectors. Suppose that the matrices D2 ; D3 ; : : : ; DM are generated by permutations of rows on D1 ; and consider the N × (ML) matrix: e = [D1 D2 · · · DM ]: D 1
This deÿnition of three-level supersaturated design is due to a useful comment given by a referee.
(6)
186
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Then the average 2 values on the design D˜ is e = ave(D) 2
4N (M − 1) ; M (N − 1) − 2
(7)
regardless of the permutation rule. Proof. Write e = [D1 D2 · · · DM ] = [d11 ; : : : ; d1L ; d21 ; : : : ; d2L ; : : : ; dM 1 ; : : : ; dML ] D
(8)
then, from the independence of the d’s and Lemma 1, the sum of 2 values of all e is pairs between two columns in D M P P P P ML e = 2 (dmi ; dmj ) + 2 (dmi ; dm0 j ) ave(D) 2 0 2 m=1 16i¡j6L 16m¡m 6M 16i; j6L P P 2 (dmi ; dm0 j ) (9) = 16m¡m0 6M 16i; j6L
MN (M − 1)(N − 1) ; 2 which completes the proof. =
(10)
Theorem 2. Let N be a power of three. Then; there exists no three-level supersaturated design which contains (N − 1)=2 mutually independent columns and satisÿes max(D) ¡ 2
4N : N −1
(11)
Proof. Suppose that there exists a three-level supersaturated design which contains L = (N − 1)=2 mutually independent columns and satisÿes Eq. (11), say D = [d1 ; : : : ; dL ; dL+1 ; : : : ; dK ]. Without loss of generality, assume that d1 ; : : : ; dL are mutually independent. Since max 2 (D) is less than 4N=(N − 1), the sum of the 2 values between any d ∈ DN and di is L P i=1
2 (di ; d) ¡
N − 1 4N = 2N: 2 N −1
(12)
The right-hand side and Eq. (5) contradict each other and the theorem is proved. Theorem 3. Given a three-level (supersaturated) design D = [d1 ; d2 ; : : : ; dk ] with N rows; let + (D) be the matrix obtained from D by transforming all 1s to 2s; all 2s to 3s and all 3s to 1s; similarly; let − (D) be the matrix obtained from D by transforming all 1s to 3s; all 2s to 1s and all 3s to 2s (note that all columns of + (D) and − (D) belong to DN ). Next; by using these matrices; construct the three-level e by supersaturated designs D∗ and D D D D ; (13) D∗ = D + (D) − (D)
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
1 e D= 2 3
D D D
D − (D) ; + (D)
D + (D) − (D)
187
(14)
where 1; 2 and 3 are column vectors of 1s; 2s and 3s; respectively. Then the maximum values on these designs are
max(D ) = max N; 2 max(D) ;
(15)
e = 3 max(D): max(D)
(16)
∗
2
2
2
2
Proof. We put D ∗ D = D
1 e =2 D 3
D + (D) D D D
D ∗ ∗ ∗ ∗ = [d11 ; : : : ; d1k ; : : : ; d31 ; : : : d3k ]; − (D)
D + (D) − (D)
D − (D) = [de0 ; de11 ; : : : ; de1k ; : : : ; de31 ; : : : ; de3k ]: + (D)
(17)
(18)
If m 6= m0 , then the relations ∗ ; dm∗0 i ) = N 2 (dmi
(m 6= m0 );
∗ ∗ ; dmj )62 2 (di ; dj ) 2 (dmi ∗ ; dm∗0 j )62 2 (di ; dj ) 2 (dmi
(19)
(i 6= j);
(20)
(m 6= m0 ; i 6= j)
(21)
can be derived straightforwardly by calculating the 2 value proving Eq. (15). Furthermore, 2 (de0 ; demi ) = 0; 2 (demi ; dem0 i ) = 0
(22) (m 6= m0 );
(23)
2 (demi ; demj )63 2 (di ; dj )
(i 6= j);
(24)
2 (demi ; dem0 j )63 2 (di ; dj )
(m 6= m0 ; i 6= j)
(25)
can be derived in a similar way, verifying Eq. (16). Remark. We now summarize the relationship between the theorems on two-level and three-level supersaturated designs. As regarding two-level supersaturated designs, Tang and Wu (1997) have proposed the following construction algorithm:
188
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
1. An N -dimensional orthogonal base, consisting of a set of N − 1 vectors, is selected from the set of two-level equal occurrence vectors, where a two-level equal occurrence vector is a vector in which the number of occurrences at one level is equal to the number of occurrences at another level. The base is used as an initial matrix. 2. Some designs are generated sequentially by permutations of the rows of the initial matrix. This algorithm is justiÿed by the fact that the criterion for the average squared inner product does not depend on the permutation rule. Hence, by specifying the average squared inner product to a given level, two-level supersaturated designs may be derived by this algorithm. Theorem 1 can be regarded as an extension of the theorem on two-level supersaturated designs, obtained by Tang and Wu (1997), to three-level. Furthermore, Yamada and Lin (1997) has obtained a method to generate two-level supersaturated designs with N = 2n rows and K = 2k + 1 columns from a two-level supersaturated design with n rows and k columns, when the maximum squared inner product on the generated design is equal to a pre-speciÿed level. Theorems 2 and 3 can be regarded as an extension of Yamada and Lin (1997). The theorems shown in this paper imply that three-level supersaturated designs can be constructed in a similar way to the constructions of two-level supersaturated designs.
3. Construction of three-level supersaturated designs 3.1. Example: N = 9 Theorem 1 ensures that three-level supersaturated designs can be generated by permutation of an initial three-level design while keeping ave 2 (D) at a pre-speciÿed level. Furthermore, Theorem 2 indicates that there does not exist a three-level supersaturated design such that max 2 (D)64 for N = 9, where it includes four mutually independent columns. Table 1 shows all possible variations of 2 values for N = 9 and D. This table was obtained by a brute force examination of all possible combinations of two three-level columns. A three-level supersaturated design which satisÿes max 2 (D) = 6 Table 1 Variations of 2 values N
Variations
9 18
0, 4, 6, 10, 18 0, 2, 3, 5, 6, 8, 9, 11, 12, 14, 17, 18 20, 21, 26, 36 0, 1 13 ; 2, 3 13 ; 4, 5 13 ; 6, 7 13 ; 8, 9 13
27
10, 11 13 ; 12, 13 13 ; 14, 15 13 ; 16, 17 13 ; 18, 19 13 20, 21 13 ; 23 13 ; 24, 26, 27 13 ; 29 13 ; 30, 33 13 ; 35 13 ; 38, 43 13 ; 54
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
189
Table 2 A permutation rule to construct the design with N = 9 and K = 20: max 2 (D) = 6 D1
D2
D3
D4
D5
1 2 3 4 5 6 7 8 9
1 7 9 8 6 5 4 3 2
1 9 6 7 8 5 3 4 2
1 6 7 8 9 3 5 4 2
1 7 9 3 8 6 5 4 2
is explored since 1 1 1 2 D1 = 2 2 3 3 3
2 = 6 is the next to 4 in Table 1. Speciÿcally, the 9 × 4 matrix: 1 1 1 2 2 2 3 3 3 1 2 3 (26) 2 3 1 3 1 2 1 3 2 2 1 3 3
2
1
was utilized as an initial matrix for the construction. The matrices D2 ; D3 ; : : : were generated sequentially by permutations of the rows on the initial matrix, where a permuted matrix Dj is added if the maximum 2 value with each previous column is less than or equal to 6. Finally a three-level supersaturated design D = [D1 ; D2 ; : : : ; DK ] which satisÿes max 2 (D) = 6 was constructed. Table 2 shows the permuted orders on the initial matrix. The table was derived by an examination of all permutations on the rows (9!=362 880). Speciÿcally, all columns are examined in some order, so, there is a possibility to explore other combinations of columns by other orders. For example, the second column in Table 2 indicates that D2 was constructed by the permutation as follows: 1st row to 1st row, 2nd row to 7th row, 3rd row to 9th row, : : :, 9th row to 2nd row. Furthermore, a three-level supersaturated design such that max 2 (D) = 10 was explored by a similar computer search which examines all permutations of rows. Table 3 shows the permutations on the initial matrix. A three-level supersaturated design with 9 rows and 220 columns is generated from this table. 3.2. Example: N = 18 and 27 Theorem 3 was applied to construct a three-level supersaturated design because there are too many permutations (18! ≈ 6 × 1015 ) to employ a computer search similar to the case N = 9. A three-level supersaturated design with 18 rows and 60 columns was
190
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Table 3 A permutation rule to construct the design with N = 9 and K = 220: max 2 (D) = 10 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1 2 3 4 5 6 7 8 9
1 2 3 4 5 7 6 9 8
1 2 3 4 5 7 8 6 9
1 2 3 4 5 7 9 8 6
1 2 3 4 7 6 5 9 8
1 2 3 4 7 6 8 5 9
1 2 3 4 7 6 9 8 5
1 2 3 4 7 8 6 9 5
1 2 3 4 7 8 9 5 6
1 2 3 4 8 9 5 6 7
1 2 4 3 5 6 8 9 7
1 2 4 3 5 7 9 6 8
1 2 4 3 5 8 6 7 9
1 2 4 3 5 8 7 9 6
1 2 4 3 7 5 6 8 9
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1 2 4 3 7 5 8 9 6
1 2 4 3 7 8 9 6 5
1 2 4 3 7 9 5 6 8
1 2 4 3 8 6 7 5 9
1 2 4 3 8 7 5 9 6
1 2 4 5 3 6 7 8 9
1 2 4 5 3 8 9 7 6
1 2 4 5 3 9 6 7 8
1 2 4 5 3 9 8 6 7
1 2 4 5 6 3 8 7 9
1 2 4 5 6 7 3 8 9
1 2 4 5 6 7 8 9 3
1 2 4 5 6 7 9 3 8
1 2 4 5 8 6 3 9 7
1 2 4 5 8 6 9 7 3
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
1 2 4 5 8 9 7 3 6
1 2 4 5 9 6 7 3 8
1 2 4 6 7 3 5 8 9
1 2 4 6 7 3 9 5 8
1 2 4 6 8 7 3 5 9
1 2 4 6 9 3 7 8 5
1 2 4 6 9 7 5 3 8
1 2 4 6 9 8 3 7 5
1 2 4 7 6 5 9 8 3
1 2 4 7 6 9 3 8 5
1 2 4 7 6 9 8 5 3
1 2 4 7 9 3 5 6 8
1 2 4 7 9 5 6 3 8
1 2 4 7 9 5 8 6 3
1 2 4 8 3 5 6 9 7
46
47
48
49
50
51
52
53
54
55
1 2 4 8 7 9 3 6 5
1 2 4 8 7 9 5 3 6
1 2 4 8 7 9 6 5 3
1 2 4 9 3 6 8 7 5
1 2 4 9 3 7 5 8 6
1 2 4 9 3 7 6 5 8
1 2 4 9 3 8 7 5 6
1 2 4 9 6 3 5 8 7
1 2 4 9 6 8 3 5 7
1 2 4 9 8 3 6 7 5
generated by substituting the supersaturated design with 9 rows and 20 columns into Eq. (17). In this design, the maximum dependency on the generated supersaturated design is assured to be 12 because max 2 (D∗ ) = 12 by Theorem 3. A three-level supersaturated design with 27 rows and 61 columns was generated by Theorem 3 from a supersaturated design with 9 rows and 20 columns. In this case, the maximum dependency on the design is 18.
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
191
The supersaturated design with 9 rows and 220 columns whose permutations are shown in Table 3 may also be used to construct three-level supersaturated designs with 18 or 27 rows. In the generation of designs with 18 rows and 660 columns, Theorem 3 assures that max 2 (D∗ )=20, and in designs with 27 rows and 661 columns, e = 30: max 2 (D) 3.3. Evaluation of dependency in the design 1. There are 190 pairs of two columns in the three-level supersaturated design with 9 rows and 20 columns. The frequency of the 2 values is shown in Table 4. Table 4 also shows the frequency of 2 values on the three-level supersaturated design with 9 rows and 220 columns. By a comparison between the designs with 20 columns and 220 columns, the design with 220 columns is more ecient in terms of the number of columns although the maximum and average dependencies between columns is strong. In the design with 220 columns, the maximum value of 2 is 10 and the frequency of 2 (di ; dj ) = 10 to the total is around 10%, so the proportion may not be so high. This can be explained by the following reason. Let us consider d1 ; : : : ; d4 ; these are mutually independent columns. Eq. (5) implies that when a vector d has the relation 2 (d ; d1 ) = 10, the 2 values with the other columns are {4; 4; 0} from Table 1. 2. Consider the comparison on the designs where the numbers of rows are di erent, such as comparison between designs with 9 and 18 rows. Since the number of rows N a ects 2 , a direct comparison of the frequency would be inadequate. Eq. (5) shows that the sum of the 2 values of independent columns are 2N , so 2 =N would be a measure for the dependency between two columns standardized by N . Of course, this is an index for the comparison of the di erent size of designs, but it would be acceptable as an initial method for comparison. For example, 2 = 6 in the design with N = 9 rows corresponds to 2 = 12 in the design with N = 18 rows in terms of the standardized dependency. Table 5 shows the frequency of the standardized measure 2 =N for the designs with 9; 18 and 27 rows. 3. Fig. 1 demonstrates the proportion based on the frequency of 2 =N on the designs with 9; 18 and 27 rows. From this ÿgure and Table 5, the design with 27 rows is
Table 4 Frequency of 2 on the designs with N = 9 rows N
K
K 2
2 0
9
20
190
4
32 114 17 60 9 220 24090 3152 13972 13 58 Note: Upper: frequency, Lower: percentage (%).
6 44 23 4657 19
10
2309 10
max
ave
6
3.79
10
4.43
192
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
Table 5 Frequency of the standardized dependency: 2 =N on the designs with N = 9, 18 and 27 rows N
K
K 2
2 =N 0/18
9
20
190
18
60
1770
27
61
1830
32 17 288 17 1128 62
2 2/18
456 26
3/18
264 15
4/18
456 25
5/18
8/18
456 26
114 60 114 6 114 6
9/18
12/18
60 4
44 23 132 7 132 7
Max.
Ave.
6
3.79
12
3.97
18
3.54
Note: Upper: frequency, Lower: percentage (%).
Fig. 1. Proportions of the frequencies on 2 =N in the designs with N = 9, 18 and 27 rows.
the most ecient in terms of the standardized dependency since the most of the pairs concentrate low level of 2 =N , e.g. 87% of the pairs appear up to 2 =N = 4=18. 4. Concluding remarks This paper shows some theorems on three-level supersaturated design and their applications to construction. Some three-level supersaturated designs with 9 rows are derived by the sequential addition of matrices constructed by permutations of rows on the initial matrix, because the average dependency between two columns does not depend on the permutation rule. Here the maximum dependencies on two columns in the designs are equal to some pre-speciÿed levels. In the cases of row sizes N = 18 and 27; it is impossible to apply similar computer search methods, so a construction method is demonstrated. Speciÿcally, the method generates three-level supersaturated designs with 18 and 27 rows from a design with 9
S. Yamada et al. / Journal of Statistical Planning and Inference 81 (1999) 183–193
193
rows. Here, the maximum dependencies between two columns are also equal to some pre-speciÿed level. The construction methods can be regarded as an extension of the methods developed for two-level supersaturated designs. The existence of the correspondence between three-level and two-level designs would be one of the main results of this paper. However, this paper considers only the construction of designs. As regarding the data analysis, stepwise regression is usually applied to analyze data collected by two-level supersaturated design (Lin, 1993,1995). In the case of three-level supersaturated design, stepwise regression would be also acceptable as well as the case of two-level supersaturated design, where each of three-level columns is transformed to some columns constructed by dummy variables. According to the analysis, an estimate of the effect of a factor is independent to an estimate of the e ect of another factor when 2 value between two columns is equal to 0. On the other hand, the two estimates are completely dependent when 2 = 2N . Intuitively, the two estimates assigned to two di erent three-level columns would be nearly independent when the 2 value is close to 0. More detailed discussion on analysis of data collected by the proposed three-level supersaturated design is a problem for future research. Acknowledgements We express sincere thanks to the editor, associate editor and two anonymous referees, each of whom made valuable comments that helped us to improve this article. References Booth, K.H.V., Cox, D.R., 1962. Some systematic supersaturated designs. Technometrics 4, 489–495. Iida, T., 1994. A construction method of two level supersaturated design derived from L12. Jpn. J. Appl. Statist. 23, 147–153 (in Japanese). Li, W.W., Wu, C.F.J., 1997. Columnwise-pairwise algorithms with applications to the construction of supersaturated designs. Technometrics 39, 171–179. Lin, D.K.J., 1993. A new class of supersaturated designs. Technometrics 35, 28–31. Lin, D.K.J., 1995. Generating systematic supersaturated designs. Technometrics 37, 213–225. Niki, N., Hashiguchi, H., Yamada, S., Ikebe, Y.T., 1997. Three theorems on multi-level orthogonal design (in preparation). Nguyen, N.K., 1996. An algorithm approach to constructing supersaturated designs. Technometrics 38, 69–73. Plackett, R.L., Burman, J.P., 1946. The design of optimum multifactorial experiments. Biometrika 33, 303–325. Satterthwaite, F.E., 1959. Random balance experimentation (with discussions). Technometrics 1, 111–137. Tang, B., Wu, C.F.J., 1997. A method for constructing supersaturated designs and its E(s2 ) optimality. Canadian J. Statist. 25, 191–201. Wu, C.F.J., 1993. Construction of supersaturated designs through partially aliased interactions. Biometrika 80, 661–669. Yamada, S., Lin, D.K.J., 1997. Supersaturated design including an orthogonal base. Canadian J. Statist. 25, 203–213. Yamada, S., Lin, D.J.K., 1998. Three-level supersaturated design. Submitted for publication.