- Email: [email protected]
D-optimum designs in multi-factor models with heteroscedastic errors
Journal of Statistical Planning and Inference 128 (2005) 623 – 631
www.elsevier.com/locate/jspi
D-optimum designs in multi-factor models with heteroscedastic errors C. Rodr)*guez∗ , I. Ortiz Department of Statistics and Applied Mathematics, Fac. Ciencias Experimentales, La Canada de San Urbano s/n, University of Almer!a, Almeria 04120, Spain Received 23 December 2001; accepted 1 December 2003
Abstract This paper considers the construction of D-optimum designs for Kronecker product and additive regression models when the errors are heteroscedastic. Su3cient conditions are given so that D-optimum designs for the multi-factor models can be built from D-optimum designs for their sub-models with a single factor. A robustness study is included to investigate how design e3ciencies change when the e3ciency functions are miss-speci6ed. c 2004 Elsevier B.V. All rights reserved. MSC: 62K05 Keywords: D-optimum designs; E3ciency function; Information matrices; Kronecker product models; Additive models; Product designs
1. Introduction Let f(x)T = (f1 (x); : : : ; fm (x)) denote m linearly independent regression functions de6ned on some compact subset X of Rk and let T = (1 ; : : : ; m ) denote a vector of parameters. We consider the usual linear regression model Y = f(x)T + ;
x ∈ X;
m
(1)
in which for each x in X, a random response variable with mean j=1 fj (x)j and 2 variance = (x) can be observed. The di?erent observations are assumed to be uncorrelated. The e3ciency function, (x), is a positive real-valued continuous function de6ned on X. A design will be regarded as a probability measure on X, and the ∗
Corresponding author. Tel.: +34-950015810; fax: +34-820015788. E-mail addresses: [email protected] (C. Rodr)*guez), [email protected] (I. Ortiz).
c 2004 Elsevier B.V. All rights reserved. 0378-3758/$ - see front matter doi:10.1016/j.jspi.2003.12.013
624
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
information contained in is measured by its information matrix M () =
(x)f(x)f(x)T (d x): X
Optimum experimental designs typically minimize some convex function of the inverse information matrix. If the aim is to estimate the parameters in the model, a reasonable criterion is D-optimality, which minimizes the generalized variance of the parameter estimators, or equivalently maximizes the determinant of the information matrix. Most experiments are described by multi-factor models. The construction of optimum designs is more complicated for these models than for models with a single factor. Therefore, it is interesting to obtain optimum designs for multi-factor models in terms of optimum designs for their one-dimensional components. Schwabe (1996) makes a review of optimal designs for multi-factor models with homoscedastic errors. The heteroscedastic case has been less deeply studied. Wong (1994) considers G-optimality and Montepiedra and Wong (2001) compute designs for D-optimality. This paper investigates the search for D-optimum designs for two classes of multi-factor models with heteroscedastic errors: The Kronecker product and additive models. In Section 2, it is shown how the results for homoscedastic product models can be extended to the heteroscedastic case. In Section 3, D-optimum designs for additive models with a particular heteroscedastic structure are studied. Su3cient conditions for the product of the optimum designs for its one-factor sub-models to be an optimum design for the multi-factor model are given. Results for orthogonal additive models are also shown. In Section 4, an e3ciency study is carried out to check whether the product of D-optimum designs for polynomial one-factor models is robust for an incorrectly assumed model or e3ciency function. Furthermore, the behavior of D-optimum designs for a multi-factor model when the structure of the regression model changes is analyzed. The paper ends with a concluding remark in Section 5. 2. Product models Consider the n one-factor heteroscedastic models T
E(Yxk ) = fk (xk ) k =
pk
fk; ik (xk )k; ik ;
xk ∈ Xk
(2)
ik =1
with Var(Yxk ) = k2 = k (xk ), k = 1; : : : ; n and n ¿ 2. The resulting n-factor Kronecker product model with interactions is E(Yx1 ;:::;x n ) =
p1 i1 =1
···
pn
f1; i1 (x1 ) · : : : · fn; in (x n )i1 ;:::;in
(3)
in =1
with Var(Yx1 ;:::;x n ) = 2 =( 1 (x1 ) · : : : · n (x n )), (x1 ; : : : ; x n ) ∈ X1 × · · · × Xn . This product model has p1 ·: : :·pn parameters, its vector of regression functions is the Kronecker product of the vectors of the regression functions associated with the respective factors and the heteroscedastic structure is determined by the
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
625
e3ciency function
(x1 ; : : : ; x n ) = 1 (x1 ) · : : : · n (x n ):
(4)
Let k be a design on Xk , k = 1; : : : ; n. The product design, , on X1 × · · · × Xn , is de6ned by the product measure (A1 ; : : : ; An ) = 1 (A1 ) · : : : · n (An ), where Ak ⊆ Xk is measurable w.r.t. k , k = 1; : : : ; n. As in the homoscedastic case, the information matrix and hence, the covariance matrix for a product design factorize into their marginal counterparts. This fact as well as the next theorem can be provedby replacing the regression functions fk (xk ) used in Schwabe (1996), Section 4, by k (xk )fk (xk ), k = 1; : : : ; n. Theorem 1. If ∗k , k = 1; : : : ; n, are D-optimum designs in the marginal model (2), then the product design ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the n-factor model (3). Example 2. Consider the two-factor model E(Yx1 ;x2 )=0 +1 x1 +2 x2 +3 x1 x2 +4 x22 + 5 x1 x22 , with e3ciency function (x1 ; x2 )=x1 +1 e−x1 (1−x2 )+1 (1+x2 )+1 , x1 in (0; ∞), x2 in (−1; 1) and ; ; ¿−1. This model can be expressed as Model I ⊗ Model II, where Models I and II are 6rst and second degree one-factor polynomial models, respectively, with e3ciency functions 1 (x1 ) = x1 +1 e−x1 and 2 (x2 ) = (1 − x2 )+1 (1 + x2 )+1 . The D-optimum designs, ∗1 and ∗2 , for Models I and II are equally supported at the roots of the Laguerre and Jacobi polynomials L2( ) (x1 ) and P3(; ) (x2 ), respectively (see Fedorov (1972, p. 89)). Therefore, the product design ∗1 ⊗ ∗2 is D-optimum for the above two-factor model. 3. Additive models This section deals with the construction of D-optimum designs for additive models from the D-optimum designs for the corresponding marginal models. We will use the following lemma, which is obtained as an extension of the results in Schwabe (1996, Section 3), to heteroscedastic models. We will consider model (1) where f(x) and are partitioned as f(x)T = (f0 (x)T | f1 (x)T ); T
T = (0T | 1T );
with fi (x) = (fi1 (x); : : : ; fipi (x)) and
iT
(5)
= (i1 ; : : : ; ipi ), i = 0; 1.
Lemma 3. Let M0 () be the information matrix of the sub-model E(Yx ) = f0 (x)T 0 with Var(Yx ) = 02 = (x). Let H be the matrix
f0 f1T d and J = f1 f1T d − − T H M0 () H . Then, for model (1) (i) det{M ()} = det{M0 ()}det{J }. (ii) The following matrix is a generalized inverse of M (), for any generalized inverse of M0 () and J M0 ()− + M0 ()− HJ − H T M0 ()− −M0 ()− HJ − : −J − H T M0 ()− J−
626
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
(iii) If 1 is estimable under , then Cov1 () = J −1 . 3.1. Additive models with constant term In this subsection, we consider heteroscedastic models explicitly including a constant term, i.e., pk fk; ik (xk )k; ik ; xk ∈ Xk : (6) E(Yxk ) = 0 + fk (xk )T k = 0 + ik =1
The n-factor additive model n fk (xk )T k E(Yx1 ;:::;x n ) = 0 +
(7)
k=1
has 1 + p1 + · · · + pn parameters, and the e3ciency function de6ned in Eq. (4) will be used. Theorem 4. Let ∗k be a D-optimum design in the marginal model given by (6). If for every k = 1; : : : ; n, ∗k is supported on the set of points which maximize the e;ciency function k (xk ), i.e. if
j d∗j
j∗ = (8) 16j=k6n
1≤j=k6n
Xj
with k∗ = maxxk ∈Xk k (xk ), then ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the additive model given by (7). Proof. Let Mk (k ) be the information matrix of a design k in the marginal model (6), k = 1; : : : ; n. Applying Lemma 3 we 6nd that pk n
j dj det{Mk (k )} det{M (1 ⊗ · · · ⊗ n )} = k=1
6
n k=1
16j=k6n
Xj
pk
j∗ det{Mk (∗k )}
16j=k6n
= det{M (∗1 ⊗ · · · ⊗ ∗n )}: Thus, ∗1 ⊗ · · · ⊗ ∗n is the best product design for the additive model (7). Now, by the Kiefer–Wolfowitz equivalence theorem, the D-optimality of the best product design can be extended to the entire class of competing designs. Example 5. Consider the response surface of 6rst order E(Yx1 ; x2 ) = 0 + 1 x1 + 2 x2 , x1 ; x2 in [−1; 1], with (x1 ; x2 )=(1+x12 )(1+x22 ). This model can be expressed as the sum of two 6rst-order polynomial regression models with e3ciency function (x) = 1 + x2 . For these single models the D-optimum design, ∗ , is equally supported at ±1 and
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
627
maxx∈[−1; 1] (x) = (±1) = 2. Thus, the design ∗ ⊗ ∗ is D-optimum for the two-factor model. Note that Theorem 4 is valid for an additive e3ciency function as (x1 ; : : : ; x n ) = ( 1 (x1 )−1 + · · · + n (x n )−1 )−1 if ∗k is D-optimum in the heteroscedastic model and in the corresponding homoscedastic model simultaneously. Moreover, Theorem 4 can be formulated for any Dk -optimum design in the marginal model (6), since every Dk -optimum design satisfying (8) is D-optimum as well. 3.2. Orthogonal additive model Throughout this subsection we consider additive models where a constant term is not necessarily involved, but an orthogonality assumption on the marginal regression functions is assumed. We consider n heteroscedastic models de6ned by (2) and the resulting additive n-factor model, n fk (xk )T k (9) E(Yx1 ;:::;x n ) = k=1
with e3ciency function (4). We assume that at most one of the marginal models has a constant term, in which case the 6rst model is the model with intercept. The additive model has p1 + · · · + pn unknown parameters. Theorem 6. Let ∗k , k = 1; : : : ; n, be D-optimum designs in the marginal model given by (2), and assume that the following two conditions hold: (10) (i)
k fk d∗k = 0; k = 2; : : : ; n; (ii)maxxk ∈Xk k (xk ) = k (xk∗ ) for every xk∗ in the support of ∗k :
(11)
Then, ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the additive model (9). Proof. If k fk d∗k = 0 for k = 2; : : : ; n, then the information matrix of ∗1 ⊗ · · · ⊗ ∗n is partitioned as
j d∗j Mk (∗k ) : M (∗1 ⊗ · · · ⊗ ∗n ) = diag 16j=k6n
Xj
For every design , det{M ()} 6 det{M (∗1 ⊗ · · · ⊗ ∗n )} for the additive model (9) and hence, ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the class of all designs. Example 7. Let Models I and II be as in Example 5, but considering Model II with no intercept. This model is considered when it is known that the model passes though the origin (see Huang et al. (1995) for moredetails). The design equally supported at ±1, ∗ , is D-optimum for both models, and (x)x d∗ = 0. Thus, the design ∗ ⊗ ∗ is D-optimum for Model I⊕ Model II.
628
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
4. E"ciency study In this section we will study the behavior of D-optimum designs when conditions under which they were constructed change. We will use the D-e3ciency of a design with respect to a D-optimum design ∗ , 1=m
D − e? = (det{M ()}=det{M (∗ )})
;
where m is the number of unknown parameters. 4.1. Product models Let ∗k be a D-optimum design in the marginal model (2). By Theorem 1, the D-e3ciency of a product design = 1 ⊗ · · · ⊗ n in the product model (3), with respect to the D-optimum design ∗ = ∗1 ⊗ · · · ⊗ ∗n , is the product of the D-e3ciencies of the marginal designs in the corresponding marginal models. At the beginning of the experimentation, the exact expression of the regression model, including the variance of the response variable, may be partially unknown. Then, it is interesting to study whether or not the e3ciency of a D-optimum design for a given model is good when it is used to estimate another model with some di?erences. For example, working with multi-factor heteroscedastic models that are expressed as the product of several heteroscedastic polynomial models, we can study the e?ects of changing the degrees of the sub-models or changing the e3ciency functions. For a two-factor model that is the product of two heteroscedastic polynomial models of degrees s1 and s2 , respectively, certain characteristics from the following two classes of e3ciency functions will be discussed: (i) (x)=(1−x)+1 (1+x)+1 with , ¿−1 and x in (−1; 1), and (ii) (x) = x$+1 e−x with $ ¿ − 1 and x in (0; ∞). D-optimum designs for these heteroscedastic models can be seen in Fedorov (1972, p. 89) and Dette (1990). In order to estimate the sub-model of degree si we use the D-optimum design for a model of degree ri with the same e3ciency function, (ri ¿ si ), that will be denoted by ∗ri ;i , i = 1; 2. We are interested in studying the D-e3ciencies of these optimum designs (Dette and Wong (1995) studied one-factor homoscedastic models). We have observed that when and increase, D-e3ciencies increase as well. However, for e3ciency function (ii), e3ciencies also increase when $ increases, but they are lower than in case (i). Besides, D-e3ciencies decrease as the distance between ri and si increases, and for ri − si 6xed, D-e3ciencies increase as si increases (these results are in accordance with those in Dette and Wong (1995)). Even if ri = si + 1, the e3ciencies in the two-factor models are low. For instance, for homoscedastic models 0:667 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:88. For = = 0:5, 0:635 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:878 and for 1 2 1 2 $ = 1, 0:576 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:856 (1 ≤ ri 6 7). 1 2 Another interesting issue is to study the D-e3ciency of an optimum design for a heteroscedastic polynomial model of degree r when it is used to estimate a homoscedastic or another heteroscedastic polynomial model of the same degree. In general, there is a big loss of e3ciency if the e3ciency function is not correctly speci6ed. Consider for example the one-factor polynomial model of degree r (1 6 r 6 7). D-optimum designs
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
629
for models with e3ciency function (x) = x2 exp(−x) with respect to models with
(x) = x exp(−x) have e3ciencies between 0:78 and 0.88. We observe that, when more than one regression and/or e3ciency functions are miss-speci6ed, there is a great loss of e3ciency. D-optimum designs in polynomial multi-factor models show a reasonable performance only when the degrees of the polynomial sub-models are large, ri is near to si and there are small variations in the e3ciency functions. For these reasons, the model must be exactly de6ned before carrying out the design of the experiment. 4.2. Changes in the structure of the model In this section we analyze the e?ect of changing the structure of a multi-factor model. We work with product and additive of the
nmodels, and with e3ciency functions n product or sum type, i.e. (x1 ; : : : ; x n ) = i=1 i (xi ) or (x1 ; : : : ; x n )−1 = i=1 i (xi )−1 , respectively. We denote by PP a product model with product e3ciency and by SP and SS, additive models with product or sum e3ciency functions, respectively. We will denote by ∗Model the D-optimum designs for these models within the class of all competing designs. We have implemented the necessary software to calculate these designs, when the preceding theorems are not applicable, using the Gauss package. We have considered the e3ciency functions (i) and (ii) in the previous subsection. In each case we have started o? studying D-optimum designs for one and two-factor polynomial models of 6rst degree, in some design spaces. For the sake of simplicity, in this section we will only include details for the e3ciency function (x) = 1 − x2 , x ∈ [ − b; b] and 0 ¡ b ¡ 1. A design equally supported at ±a will be denoted by a . 2 The D-optimum √ design for the model √ E(Yx ) = 0 + 1 x, (x) = 1 − x and x ∈ [ − b; b] √ is b if b 6 1= 3 and 1= 3 if b ¿ 1= 3. √ If b 6 12 , then ∗PP = ∗SP = ∗SS = b ⊗ b . If 12 ¡ b 6 1= 3, then the D-optimum √ ∗ design for Model SP changes to ∗SP = 1=2 ⊗ 1=2 . But if 1= 3 ¡ b 6 2=5, then PP ∗ changes with respect to previous cases, being PP = 1=√3 ⊗ 1=√3 . Finally, if b ¿ 2=5 then ∗SS = √2=5 ⊗ √2=5 . Table 1 contains the D-e3ciencies of Model PP when it is used to estimate Models SP and SS, and those of Model SP with respect to Model SS. The two-factor models are built from two one-factor polynomial models of degrees 1, 2 or 3. We observe that e3ciencies decrease as b increases, and for every value of b, the e3ciencies show
Table 1 E3ciencies of Model PP with respect to Models SP and SS, and of Model SP with respect to Model SS √ b = 1=2 b = 1= 3 b = 2=5 Degree 1 Degree 2 Degree 3
1,1,1 0.994, 0.999, 0.998 0.994, 0.999, 0.997
0.957, 1, 0.929 0.987, 0.998, 0.994 0.988, 0.998, 0.993
0.957, 0.984, 0.914 0.977, 0.996, 0.988 0.979, 0.996, 0.986
630
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
a similar behavior for any degree. In any case, we can conclude that the product of D-optimum designs is a good design for the three-studied models. For this e3ciency function, if a model is built from two one-factor models of 6rst degree and the design space is a symmetric interval, then the D-optimum designs for Models PP, SP and SS are the product of two one-factor designs that are equally supported at two points. Nevertheless, for other e3ciency functions the number of support points can change depending on the model. For example, if (x) = xe−x , x ∈ (0; ∞), the D-optimum design for an one-factor model of 6rst-degree is supported at the ze√ ros of the Laguerre polynomial L(0) 2). For the model with two (x) (points 2 ± 2 factors, built from two 6rst-order one-factor models, we have that ∗PP is equally sup√ √ ported at the four points (2 ± 2; 2 ± 2) and ∗SP is equally supported at the three points (0.61,0.61), (0.93,3.46) and (3.46,0.93). Design ∗SS is supported at (0.51,0.51), (0.51,3.70), (3.70,3.70) and (3.70,0.51), with weights 0.306, 0.265, 0.163 and 0.265, respectively. Table 1 also shows that the D-optimum design for Model PP has better e3ciencies for Model SS than for Model SP. Besides, the design for Model SP is good for Model SS. These results hold for the two analyzed classes of e3ciency functions.
5. Concluding remarks For product models with heteroscedastic structure, the product of D-optimum designs in the marginal models is D-optimum. Therefore, the result for the homoscedastic case can be applied to the heteroscedastic case without additional conditions, either on the factors or the e3ciency functions. For additive models with constant term and product heteroscedastic structure, the product of D-optimum designs in the marginal models is D-optimum if conditions (8) are satis6ed. For orthogonal additive models, the preceding result is valid if the more restrictive conditions (10) and (11) hold. This work has focused on additive models with product e3ciency functions for two main reasons. The e3ciency study shows that the product of D-optimum designs for heteroscedastic polynomial sub-models has a very good behavior if it is used to estimate a multi-factor additive model with a sum heteroscedastic structure. Mathematical calculus for these last models is more complicated. Moreover, under the conditions of Theorems 4 and 6, the D-optimum design for Model PP coincides with that for Model SP. On the other hand, in practice, these models can be used if the multi-factor model has an exponential e3ciency function such that the exponent is the sum of one-factor functions, as in Kunert and Lehmkuhl (1998).
Acknowledgements The authors would like to thank the anonymous referees and the editor for their helpful comments and suggestions which have improved the content of this paper.
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
631
This work has been partially supported by Junta de Andaluc)*a, research group FQM244. References Dette, H., 1990. A generalization of D- and D1 -optimal designs in polynomial regression. Ann. Statist. 18, 1784–1804. Dette, H., Wong, W.K., 1995. On G-e3ciency calculation for polynomial models. Ann. Statist. 6, 2081–2101. Fedorov, V.V., 1972. Theory of Optimal Experiments. Academic Press, New York. Transl. Studden, W.J., Klimko, E.M. Huang, M.-N., Chang, F.-C., Wong, W.K., 1995. D-optimal designs for polynomial regression without an intercept. Statistica Sinica 5, 441–458. Kunert, J., Lehmkuhl, F., 1998. The generalized -method in Taguchi experiments. In: Atkinson, A.C., Pronzato, L., Wynn, H.P. (Eds.), MODA 5, Advances in Model-Oriented Data Analysis and Experimental Design, Physica-Verlag, Heidelberg, pp. 223–230. Montepiedra, G., Wong, W.K., 2001. A new design criterion when heteroscedasticity is ignored. Ann. Inst. Statist. Math. 53, 418–426. Schwabe, R., 1996. Optimum Designs for Multi-Factor Models, 1996. Lecture Notes in Statistic, Vol. 113. Springer, Berlin. Wong, W.K., 1994. G–optimal designs for multifactor experiments with heteroscedastic errors. J. Statist. Plann. Inference 40, 127–133.
www.elsevier.com/locate/jspi
D-optimum designs in multi-factor models with heteroscedastic errors C. Rodr)*guez∗ , I. Ortiz Department of Statistics and Applied Mathematics, Fac. Ciencias Experimentales, La Canada de San Urbano s/n, University of Almer!a, Almeria 04120, Spain Received 23 December 2001; accepted 1 December 2003
Abstract This paper considers the construction of D-optimum designs for Kronecker product and additive regression models when the errors are heteroscedastic. Su3cient conditions are given so that D-optimum designs for the multi-factor models can be built from D-optimum designs for their sub-models with a single factor. A robustness study is included to investigate how design e3ciencies change when the e3ciency functions are miss-speci6ed. c 2004 Elsevier B.V. All rights reserved. MSC: 62K05 Keywords: D-optimum designs; E3ciency function; Information matrices; Kronecker product models; Additive models; Product designs
1. Introduction Let f(x)T = (f1 (x); : : : ; fm (x)) denote m linearly independent regression functions de6ned on some compact subset X of Rk and let T = (1 ; : : : ; m ) denote a vector of parameters. We consider the usual linear regression model Y = f(x)T + ;
x ∈ X;
m
(1)
in which for each x in X, a random response variable with mean j=1 fj (x)j and 2 variance = (x) can be observed. The di?erent observations are assumed to be uncorrelated. The e3ciency function, (x), is a positive real-valued continuous function de6ned on X. A design will be regarded as a probability measure on X, and the ∗
Corresponding author. Tel.: +34-950015810; fax: +34-820015788. E-mail addresses: [email protected] (C. Rodr)*guez), [email protected] (I. Ortiz).
c 2004 Elsevier B.V. All rights reserved. 0378-3758/$ - see front matter doi:10.1016/j.jspi.2003.12.013
624
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
information contained in is measured by its information matrix M () =
(x)f(x)f(x)T (d x): X
Optimum experimental designs typically minimize some convex function of the inverse information matrix. If the aim is to estimate the parameters in the model, a reasonable criterion is D-optimality, which minimizes the generalized variance of the parameter estimators, or equivalently maximizes the determinant of the information matrix. Most experiments are described by multi-factor models. The construction of optimum designs is more complicated for these models than for models with a single factor. Therefore, it is interesting to obtain optimum designs for multi-factor models in terms of optimum designs for their one-dimensional components. Schwabe (1996) makes a review of optimal designs for multi-factor models with homoscedastic errors. The heteroscedastic case has been less deeply studied. Wong (1994) considers G-optimality and Montepiedra and Wong (2001) compute designs for D-optimality. This paper investigates the search for D-optimum designs for two classes of multi-factor models with heteroscedastic errors: The Kronecker product and additive models. In Section 2, it is shown how the results for homoscedastic product models can be extended to the heteroscedastic case. In Section 3, D-optimum designs for additive models with a particular heteroscedastic structure are studied. Su3cient conditions for the product of the optimum designs for its one-factor sub-models to be an optimum design for the multi-factor model are given. Results for orthogonal additive models are also shown. In Section 4, an e3ciency study is carried out to check whether the product of D-optimum designs for polynomial one-factor models is robust for an incorrectly assumed model or e3ciency function. Furthermore, the behavior of D-optimum designs for a multi-factor model when the structure of the regression model changes is analyzed. The paper ends with a concluding remark in Section 5. 2. Product models Consider the n one-factor heteroscedastic models T
E(Yxk ) = fk (xk ) k =
pk
fk; ik (xk )k; ik ;
xk ∈ Xk
(2)
ik =1
with Var(Yxk ) = k2 = k (xk ), k = 1; : : : ; n and n ¿ 2. The resulting n-factor Kronecker product model with interactions is E(Yx1 ;:::;x n ) =
p1 i1 =1
···
pn
f1; i1 (x1 ) · : : : · fn; in (x n )i1 ;:::;in
(3)
in =1
with Var(Yx1 ;:::;x n ) = 2 =( 1 (x1 ) · : : : · n (x n )), (x1 ; : : : ; x n ) ∈ X1 × · · · × Xn . This product model has p1 ·: : :·pn parameters, its vector of regression functions is the Kronecker product of the vectors of the regression functions associated with the respective factors and the heteroscedastic structure is determined by the
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
625
e3ciency function
(x1 ; : : : ; x n ) = 1 (x1 ) · : : : · n (x n ):
(4)
Let k be a design on Xk , k = 1; : : : ; n. The product design, , on X1 × · · · × Xn , is de6ned by the product measure (A1 ; : : : ; An ) = 1 (A1 ) · : : : · n (An ), where Ak ⊆ Xk is measurable w.r.t. k , k = 1; : : : ; n. As in the homoscedastic case, the information matrix and hence, the covariance matrix for a product design factorize into their marginal counterparts. This fact as well as the next theorem can be provedby replacing the regression functions fk (xk ) used in Schwabe (1996), Section 4, by k (xk )fk (xk ), k = 1; : : : ; n. Theorem 1. If ∗k , k = 1; : : : ; n, are D-optimum designs in the marginal model (2), then the product design ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the n-factor model (3). Example 2. Consider the two-factor model E(Yx1 ;x2 )=0 +1 x1 +2 x2 +3 x1 x2 +4 x22 + 5 x1 x22 , with e3ciency function (x1 ; x2 )=x1 +1 e−x1 (1−x2 )+1 (1+x2 )+1 , x1 in (0; ∞), x2 in (−1; 1) and ; ; ¿−1. This model can be expressed as Model I ⊗ Model II, where Models I and II are 6rst and second degree one-factor polynomial models, respectively, with e3ciency functions 1 (x1 ) = x1 +1 e−x1 and 2 (x2 ) = (1 − x2 )+1 (1 + x2 )+1 . The D-optimum designs, ∗1 and ∗2 , for Models I and II are equally supported at the roots of the Laguerre and Jacobi polynomials L2( ) (x1 ) and P3(; ) (x2 ), respectively (see Fedorov (1972, p. 89)). Therefore, the product design ∗1 ⊗ ∗2 is D-optimum for the above two-factor model. 3. Additive models This section deals with the construction of D-optimum designs for additive models from the D-optimum designs for the corresponding marginal models. We will use the following lemma, which is obtained as an extension of the results in Schwabe (1996, Section 3), to heteroscedastic models. We will consider model (1) where f(x) and are partitioned as f(x)T = (f0 (x)T | f1 (x)T ); T
T = (0T | 1T );
with fi (x) = (fi1 (x); : : : ; fipi (x)) and
iT
(5)
= (i1 ; : : : ; ipi ), i = 0; 1.
Lemma 3. Let M0 () be the information matrix of the sub-model E(Yx ) = f0 (x)T 0 with Var(Yx ) = 02 = (x). Let H be the matrix
f0 f1T d and J = f1 f1T d − − T H M0 () H . Then, for model (1) (i) det{M ()} = det{M0 ()}det{J }. (ii) The following matrix is a generalized inverse of M (), for any generalized inverse of M0 () and J M0 ()− + M0 ()− HJ − H T M0 ()− −M0 ()− HJ − : −J − H T M0 ()− J−
626
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
(iii) If 1 is estimable under , then Cov1 () = J −1 . 3.1. Additive models with constant term In this subsection, we consider heteroscedastic models explicitly including a constant term, i.e., pk fk; ik (xk )k; ik ; xk ∈ Xk : (6) E(Yxk ) = 0 + fk (xk )T k = 0 + ik =1
The n-factor additive model n fk (xk )T k E(Yx1 ;:::;x n ) = 0 +
(7)
k=1
has 1 + p1 + · · · + pn parameters, and the e3ciency function de6ned in Eq. (4) will be used. Theorem 4. Let ∗k be a D-optimum design in the marginal model given by (6). If for every k = 1; : : : ; n, ∗k is supported on the set of points which maximize the e;ciency function k (xk ), i.e. if
j d∗j
j∗ = (8) 16j=k6n
1≤j=k6n
Xj
with k∗ = maxxk ∈Xk k (xk ), then ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the additive model given by (7). Proof. Let Mk (k ) be the information matrix of a design k in the marginal model (6), k = 1; : : : ; n. Applying Lemma 3 we 6nd that pk n
j dj det{Mk (k )} det{M (1 ⊗ · · · ⊗ n )} = k=1
6
n k=1
16j=k6n
Xj
pk
j∗ det{Mk (∗k )}
16j=k6n
= det{M (∗1 ⊗ · · · ⊗ ∗n )}: Thus, ∗1 ⊗ · · · ⊗ ∗n is the best product design for the additive model (7). Now, by the Kiefer–Wolfowitz equivalence theorem, the D-optimality of the best product design can be extended to the entire class of competing designs. Example 5. Consider the response surface of 6rst order E(Yx1 ; x2 ) = 0 + 1 x1 + 2 x2 , x1 ; x2 in [−1; 1], with (x1 ; x2 )=(1+x12 )(1+x22 ). This model can be expressed as the sum of two 6rst-order polynomial regression models with e3ciency function (x) = 1 + x2 . For these single models the D-optimum design, ∗ , is equally supported at ±1 and
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
627
maxx∈[−1; 1] (x) = (±1) = 2. Thus, the design ∗ ⊗ ∗ is D-optimum for the two-factor model. Note that Theorem 4 is valid for an additive e3ciency function as (x1 ; : : : ; x n ) = ( 1 (x1 )−1 + · · · + n (x n )−1 )−1 if ∗k is D-optimum in the heteroscedastic model and in the corresponding homoscedastic model simultaneously. Moreover, Theorem 4 can be formulated for any Dk -optimum design in the marginal model (6), since every Dk -optimum design satisfying (8) is D-optimum as well. 3.2. Orthogonal additive model Throughout this subsection we consider additive models where a constant term is not necessarily involved, but an orthogonality assumption on the marginal regression functions is assumed. We consider n heteroscedastic models de6ned by (2) and the resulting additive n-factor model, n fk (xk )T k (9) E(Yx1 ;:::;x n ) = k=1
with e3ciency function (4). We assume that at most one of the marginal models has a constant term, in which case the 6rst model is the model with intercept. The additive model has p1 + · · · + pn unknown parameters. Theorem 6. Let ∗k , k = 1; : : : ; n, be D-optimum designs in the marginal model given by (2), and assume that the following two conditions hold: (10) (i)
k fk d∗k = 0; k = 2; : : : ; n; (ii)maxxk ∈Xk k (xk ) = k (xk∗ ) for every xk∗ in the support of ∗k :
(11)
Then, ∗ = ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the additive model (9). Proof. If k fk d∗k = 0 for k = 2; : : : ; n, then the information matrix of ∗1 ⊗ · · · ⊗ ∗n is partitioned as
j d∗j Mk (∗k ) : M (∗1 ⊗ · · · ⊗ ∗n ) = diag 16j=k6n
Xj
For every design , det{M ()} 6 det{M (∗1 ⊗ · · · ⊗ ∗n )} for the additive model (9) and hence, ∗1 ⊗ · · · ⊗ ∗n is D-optimum in the class of all designs. Example 7. Let Models I and II be as in Example 5, but considering Model II with no intercept. This model is considered when it is known that the model passes though the origin (see Huang et al. (1995) for moredetails). The design equally supported at ±1, ∗ , is D-optimum for both models, and (x)x d∗ = 0. Thus, the design ∗ ⊗ ∗ is D-optimum for Model I⊕ Model II.
628
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
4. E"ciency study In this section we will study the behavior of D-optimum designs when conditions under which they were constructed change. We will use the D-e3ciency of a design with respect to a D-optimum design ∗ , 1=m
D − e? = (det{M ()}=det{M (∗ )})
;
where m is the number of unknown parameters. 4.1. Product models Let ∗k be a D-optimum design in the marginal model (2). By Theorem 1, the D-e3ciency of a product design = 1 ⊗ · · · ⊗ n in the product model (3), with respect to the D-optimum design ∗ = ∗1 ⊗ · · · ⊗ ∗n , is the product of the D-e3ciencies of the marginal designs in the corresponding marginal models. At the beginning of the experimentation, the exact expression of the regression model, including the variance of the response variable, may be partially unknown. Then, it is interesting to study whether or not the e3ciency of a D-optimum design for a given model is good when it is used to estimate another model with some di?erences. For example, working with multi-factor heteroscedastic models that are expressed as the product of several heteroscedastic polynomial models, we can study the e?ects of changing the degrees of the sub-models or changing the e3ciency functions. For a two-factor model that is the product of two heteroscedastic polynomial models of degrees s1 and s2 , respectively, certain characteristics from the following two classes of e3ciency functions will be discussed: (i) (x)=(1−x)+1 (1+x)+1 with , ¿−1 and x in (−1; 1), and (ii) (x) = x$+1 e−x with $ ¿ − 1 and x in (0; ∞). D-optimum designs for these heteroscedastic models can be seen in Fedorov (1972, p. 89) and Dette (1990). In order to estimate the sub-model of degree si we use the D-optimum design for a model of degree ri with the same e3ciency function, (ri ¿ si ), that will be denoted by ∗ri ;i , i = 1; 2. We are interested in studying the D-e3ciencies of these optimum designs (Dette and Wong (1995) studied one-factor homoscedastic models). We have observed that when and increase, D-e3ciencies increase as well. However, for e3ciency function (ii), e3ciencies also increase when $ increases, but they are lower than in case (i). Besides, D-e3ciencies decrease as the distance between ri and si increases, and for ri − si 6xed, D-e3ciencies increase as si increases (these results are in accordance with those in Dette and Wong (1995)). Even if ri = si + 1, the e3ciencies in the two-factor models are low. For instance, for homoscedastic models 0:667 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:88. For = = 0:5, 0:635 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:878 and for 1 2 1 2 $ = 1, 0:576 6 D-e? ∗r ;1 ⊗∗r ;2 6 0:856 (1 ≤ ri 6 7). 1 2 Another interesting issue is to study the D-e3ciency of an optimum design for a heteroscedastic polynomial model of degree r when it is used to estimate a homoscedastic or another heteroscedastic polynomial model of the same degree. In general, there is a big loss of e3ciency if the e3ciency function is not correctly speci6ed. Consider for example the one-factor polynomial model of degree r (1 6 r 6 7). D-optimum designs
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
629
for models with e3ciency function (x) = x2 exp(−x) with respect to models with
(x) = x exp(−x) have e3ciencies between 0:78 and 0.88. We observe that, when more than one regression and/or e3ciency functions are miss-speci6ed, there is a great loss of e3ciency. D-optimum designs in polynomial multi-factor models show a reasonable performance only when the degrees of the polynomial sub-models are large, ri is near to si and there are small variations in the e3ciency functions. For these reasons, the model must be exactly de6ned before carrying out the design of the experiment. 4.2. Changes in the structure of the model In this section we analyze the e?ect of changing the structure of a multi-factor model. We work with product and additive of the
nmodels, and with e3ciency functions n product or sum type, i.e. (x1 ; : : : ; x n ) = i=1 i (xi ) or (x1 ; : : : ; x n )−1 = i=1 i (xi )−1 , respectively. We denote by PP a product model with product e3ciency and by SP and SS, additive models with product or sum e3ciency functions, respectively. We will denote by ∗Model the D-optimum designs for these models within the class of all competing designs. We have implemented the necessary software to calculate these designs, when the preceding theorems are not applicable, using the Gauss package. We have considered the e3ciency functions (i) and (ii) in the previous subsection. In each case we have started o? studying D-optimum designs for one and two-factor polynomial models of 6rst degree, in some design spaces. For the sake of simplicity, in this section we will only include details for the e3ciency function (x) = 1 − x2 , x ∈ [ − b; b] and 0 ¡ b ¡ 1. A design equally supported at ±a will be denoted by a . 2 The D-optimum √ design for the model √ E(Yx ) = 0 + 1 x, (x) = 1 − x and x ∈ [ − b; b] √ is b if b 6 1= 3 and 1= 3 if b ¿ 1= 3. √ If b 6 12 , then ∗PP = ∗SP = ∗SS = b ⊗ b . If 12 ¡ b 6 1= 3, then the D-optimum √ ∗ design for Model SP changes to ∗SP = 1=2 ⊗ 1=2 . But if 1= 3 ¡ b 6 2=5, then PP ∗ changes with respect to previous cases, being PP = 1=√3 ⊗ 1=√3 . Finally, if b ¿ 2=5 then ∗SS = √2=5 ⊗ √2=5 . Table 1 contains the D-e3ciencies of Model PP when it is used to estimate Models SP and SS, and those of Model SP with respect to Model SS. The two-factor models are built from two one-factor polynomial models of degrees 1, 2 or 3. We observe that e3ciencies decrease as b increases, and for every value of b, the e3ciencies show
Table 1 E3ciencies of Model PP with respect to Models SP and SS, and of Model SP with respect to Model SS √ b = 1=2 b = 1= 3 b = 2=5 Degree 1 Degree 2 Degree 3
1,1,1 0.994, 0.999, 0.998 0.994, 0.999, 0.997
0.957, 1, 0.929 0.987, 0.998, 0.994 0.988, 0.998, 0.993
0.957, 0.984, 0.914 0.977, 0.996, 0.988 0.979, 0.996, 0.986
630
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
a similar behavior for any degree. In any case, we can conclude that the product of D-optimum designs is a good design for the three-studied models. For this e3ciency function, if a model is built from two one-factor models of 6rst degree and the design space is a symmetric interval, then the D-optimum designs for Models PP, SP and SS are the product of two one-factor designs that are equally supported at two points. Nevertheless, for other e3ciency functions the number of support points can change depending on the model. For example, if (x) = xe−x , x ∈ (0; ∞), the D-optimum design for an one-factor model of 6rst-degree is supported at the ze√ ros of the Laguerre polynomial L(0) 2). For the model with two (x) (points 2 ± 2 factors, built from two 6rst-order one-factor models, we have that ∗PP is equally sup√ √ ported at the four points (2 ± 2; 2 ± 2) and ∗SP is equally supported at the three points (0.61,0.61), (0.93,3.46) and (3.46,0.93). Design ∗SS is supported at (0.51,0.51), (0.51,3.70), (3.70,3.70) and (3.70,0.51), with weights 0.306, 0.265, 0.163 and 0.265, respectively. Table 1 also shows that the D-optimum design for Model PP has better e3ciencies for Model SS than for Model SP. Besides, the design for Model SP is good for Model SS. These results hold for the two analyzed classes of e3ciency functions.
5. Concluding remarks For product models with heteroscedastic structure, the product of D-optimum designs in the marginal models is D-optimum. Therefore, the result for the homoscedastic case can be applied to the heteroscedastic case without additional conditions, either on the factors or the e3ciency functions. For additive models with constant term and product heteroscedastic structure, the product of D-optimum designs in the marginal models is D-optimum if conditions (8) are satis6ed. For orthogonal additive models, the preceding result is valid if the more restrictive conditions (10) and (11) hold. This work has focused on additive models with product e3ciency functions for two main reasons. The e3ciency study shows that the product of D-optimum designs for heteroscedastic polynomial sub-models has a very good behavior if it is used to estimate a multi-factor additive model with a sum heteroscedastic structure. Mathematical calculus for these last models is more complicated. Moreover, under the conditions of Theorems 4 and 6, the D-optimum design for Model PP coincides with that for Model SP. On the other hand, in practice, these models can be used if the multi-factor model has an exponential e3ciency function such that the exponent is the sum of one-factor functions, as in Kunert and Lehmkuhl (1998).
Acknowledgements The authors would like to thank the anonymous referees and the editor for their helpful comments and suggestions which have improved the content of this paper.
C. Rodr!guez, I. Ortiz / Journal of Statistical Planning and Inference 128 (2005) 623 – 631
631
This work has been partially supported by Junta de Andaluc)*a, research group FQM244. References Dette, H., 1990. A generalization of D- and D1 -optimal designs in polynomial regression. Ann. Statist. 18, 1784–1804. Dette, H., Wong, W.K., 1995. On G-e3ciency calculation for polynomial models. Ann. Statist. 6, 2081–2101. Fedorov, V.V., 1972. Theory of Optimal Experiments. Academic Press, New York. Transl. Studden, W.J., Klimko, E.M. Huang, M.-N., Chang, F.-C., Wong, W.K., 1995. D-optimal designs for polynomial regression without an intercept. Statistica Sinica 5, 441–458. Kunert, J., Lehmkuhl, F., 1998. The generalized -method in Taguchi experiments. In: Atkinson, A.C., Pronzato, L., Wynn, H.P. (Eds.), MODA 5, Advances in Model-Oriented Data Analysis and Experimental Design, Physica-Verlag, Heidelberg, pp. 223–230. Montepiedra, G., Wong, W.K., 2001. A new design criterion when heteroscedasticity is ignored. Ann. Inst. Statist. Math. 53, 418–426. Schwabe, R., 1996. Optimum Designs for Multi-Factor Models, 1996. Lecture Notes in Statistic, Vol. 113. Springer, Berlin. Wong, W.K., 1994. G–optimal designs for multifactor experiments with heteroscedastic errors. J. Statist. Plann. Inference 40, 127–133.