- Email: [email protected]
New optimal design criteria for regression models with asymmetric errors
Journal of Statistical Planning and Inference 149 (2014) 140–151
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
New optimal design criteria for regression models with asymmetric errors Lucy L. Gao, Julie Zhou n Department of Mathematics and Statistics, University of Victoria, Victoria, BC, Canada V8W 3R4
a r t i c l e in f o
abstract
Article history: Received 3 September 2013 Received in revised form 6 January 2014 Accepted 19 January 2014 Available online 23 January 2014
Optimal regression designs are usually constructed by minimizing some scalar functions of the covariance matrix of the ordinary least squares estimator. However, when the error distribution is not symmetric, the second-order least squares estimator is more efficient than the ordinary least squares estimator. Thus we propose new design criteria to construct optimal regression designs based on the second-order least squares estimator. Transformation invariance and symmetry properties of the new criteria are investigated, and sufficient conditions are derived to check for these properties of D-optimal designs. The results can be applied to both linear and nonlinear regression models. Several examples are given for polynomial, trigonometric and exponential regression models, and new designs are obtained. & 2014 Elsevier B.V. All rights reserved.
Keywords: A-optimal design D-optimal design Non-normal errors Polynomial regression Robust design Second-order least squares estimator Scale invariance Shift invariance Symmetric design Trigonometric regression
1. Introduction Consider a linear regression model yi ¼ z > ðxi Þθ þεi ;
i ¼ 1; ⋯; n;
ð1Þ
where yi is the ith response observed at xi ¼ ðx1i ; …; xpi Þ > of p independent variables x1 ; …; xp , zðxÞ A Rq is a vector of known functions of x; θ A Rq is a vector of unknown regression parameters, and the errors εi's are independent with mean 0 and variance s2. Optimal regression designs aim to choose optimal design points x1 ; …; xn such that we can get the most information about the unknown regression parameters or the regression response z > ðxÞθ. Optimal regression designs have been studied extensively in the literature and many optimal design criteria have been proposed and studied; see, for example, Fedorov (1972) and Pukelsheim (1993). The ordinary least squares estimator (OLSE) is usually used to estimate θ since it is the best linear unbiased estimator. The OLSE is given by θ^ OLS ¼ ðZ > ZÞ 1 Z > y; n
Corresponding author. Tel.: þ1 250 721 7470. E-mail address: [email protected] (J. Zhou).
0378-3758/$ - see front matter & 2014 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.jspi.2014.01.005
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
141
where y ¼ ðy1 ; …; yn Þ > , and the ith row of matrix Z is z > ðxi Þ. The covariance matrix of θ^ OLS is Covðθ^ OLS Þ ¼ s2 ðZ > ZÞ 1 . The commonly used A-optimal, D-optimal and E-optimal designs minimize the trace, the determinant and the largest eigenvalue of Covðθ^ OLS Þ respectively. Optimal regression designs have been derived for various regression models and design spaces, and they perform well under the ideal model assumptions for (1): (i) the response function is correct, (ii) the errors εi's are i.i.d. Nð0; s2 Þ. However, these assumptions are never true in practice. When there are possible violations of the model assumptions, robust regression designs against small departures have been explored. Many robust designs have been constructed to deal with small departures in the response function, in the equal variance of the errors, in the correlation structure of the errors, and combinations of those departures. For example, Huber (1981), Li and Notz (1982), Notz (1989) and Wiens (1992) investigated robust designs against small departures in the response function. Fang and Wiens (2000) studied robust designs against small departures in the response function and in the equal variance of the errors. Chen et al. (2008) worked on minimax designs for polynomial models with various types of heteroscedastic errors, and Wiens and Zhou (1999) and Zhou (2001) worked on minimax designs robust against small departures in the response function and in the correlation structure of the errors. However very little work has been done to study optimal/robust designs when there is a small departure from the assumption of normal distribution. Thus, in this paper we will explore optimal regression designs when the error distribution is asymmetric. If the errors εi's have non-normal distributions, there are more efficient estimators than the OLSE. When the exact distribution of the error term is known, the maximum likelihood estimator can be applied to estimate θ. However the exact distribution of the error term is often unknown in practice. In this situation, the second-order least squares estimator (SLSE) proposed in Wang and Leblanc (2008) is more efficient than the OLSE if the third moment of the error is nonzero, i.e., the error distribution is asymmetric. The SLSE does not depend on the exact distribution of the error term, so we propose to use the SLSE to construct optimal regression designs for asymmetric errors in this paper. In particular, for a given model we construct the optimal distribution of x, and the design points x1 ; …; xn are selected randomly from the optimal distribution. A-optimal and D-optimal designs are defined based on the SLSE, various properties including scale and shift invariance and symmetry of D-optimal designs are explored, and sufficient conditions are derived to check for these properties. In addition, optimal designs based on the SLSE and OLSE are compared. All results can be applied to both linear and nonlinear regression models. The paper is organized as follows. Section 2 reviews the SLSE and its asymptotic distribution. In Section 3, optimal designs based on the SLSE are defined and compared with optimal designs based on the OLSE. In Section 4, transformation invariance of D-optimal designs is studied, and a sufficient condition is obtained to check for transformation invariance. Section 5 investigates the symmetry of D-optimal designs. Section 6 presents examples of D-optimal designs. Concluding remarks are in Section 7. All proofs are given in the Appendix.
2. Second-order least squares estimator The SLSE was first introduced for nonlinear models with measurement errors in Wang (2003, 2004). Since then it has been studied and extended for regression models without measurement errors, for example, Wang and Leblanc (2008) for nonlinear models, Abarin and Wang (2009) for censored regression models, and Chen et al. (2012) for robust SLSE for linear models. The SLSE is asymptotically normally distributed and is more efficient than the OLSE if the third moment of the errors is nonzero. Here we review the SLSE and its related results in Wang and Leblanc (2008) for the following model: yi ¼ gðxi ; θÞ þ εi ;
i ¼ 1; …; n;
ð2Þ
where gðxi ; θÞ can be a linear or nonlinear function of θ A Rq , and the errors εi's are independent and have mean Eðε∣xÞ ¼ 0 and Eðε2 ∣xÞ ¼ s2 . Denote the parameter vector as γ ¼ ðθ > ; s2 Þ > . We assume that y and ϵ have finite fourth moments, where y ¼ ðy1 ; …; yn Þ > and ϵ ¼ ðε1 ; …; εn Þ > . The SLSE γ^ SLS for γ is defined as the measurable function that minimizes n
Q n ðγÞ ¼ ∑ ρi> ðγÞW i ρi ðγÞ; i¼1
where ρi ðγÞ ¼ ðyi gðxi ; θÞ; y2i g 2 ðxi ; θÞ s2 Þ > , and W i ¼ Wðxi Þ is a 2 2 positive semidefinite matrix which may depend on xi . Notice that parameters θ and s2 are estimated together in γ^ SLS , and W i can be any positive semidefinite matrix. Suppose the true parameter value of model (2) is γ 0 ¼ ðθ0> ; s20 Þ > , and design points x1 ; …; xn are randomly selected from a distribution ξ of x. Under some regularity conditions, the asymptotic covariance matrix of the SLSE is shown to be pffiffiffi Vðγ^ SLS Þ ¼ lim Covð nγ^ SLS Þ ¼ A 1 BA 1 ; n- þ 1
142
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
where > ∂ρ ðγ 0 Þ ∂ρðγ 0 Þ W ðx Þ ; > ∂γ ∂γ > ∂ρ ðγ 0 Þ ∂ρðγ0 Þ W ðxÞUW ðxÞ ; B¼E > ∂γ ∂γ A¼E
U ¼ UðxÞ ¼ E½ρðγ 0 Þρ > ðγ0 Þ∣x; and the expectation in A and B is taken with respect to the distribution of x. The regularity conditions include one assumption that A is nonsingular, and see Wang and Leblanc (2008, Assumption 6) for a detailed discussion. It is clear that the covariance matrix Vðγ^ SLS Þ depends on γ0 . Since the asymptotic covariance matrix also depends on the weighting matrix W, we can choose W to obtain the most efficient estimator. Similar to the weighted least squares estimation, it can be shown that > 1 ∂ρ ðγ 0 Þ 1 ∂ρ > ðγ 0 Þ U A 1 BA 1 ≽ E ; ∂γ ∂γ > where ≽ indicates that the difference of the left- and right-hand sides is positive semidefinite, and the lower bound is obtained when WðxÞ ¼ U 1 ðxÞ. It is shown in Wang and Leblanc (2008) that detðUÞ ¼ s20 ðμ4 s40 Þ μ23 , where μ3 ¼ Eðε3 ∣xÞ and μ4 ¼ Eðε4 ∣xÞ, and the optimal weighting matrix is given by W ðx Þ ¼ U 1 ðx Þ ¼
1 detðUÞ
μ4 þ 4μ3 gðx; θ0 Þ þ4s20 g 2 ðx; θ0 Þ s40
μ3 2s20 gðx; θ0 Þ
μ3 2s20 gðx; θ0 Þ
s20
! ;
ð3Þ
assuming detðUÞ a0. From the definition of U above, U is always positive semidefinite. When detðUÞ a0, i.e., detðUÞ ¼ s20 ðμ4 s40 Þ μ23 a0, U is positive definite, which implies that WðxÞ ¼ U 1 ðxÞ is also positive definite. Then the asymptotic covariance matrix of the most efficient SLSE is given by 0 1 μ3 Vðθ^ SLS Þ V s^ 2SLS G2 1 g 1 4 B C μ4 s0 B C C ¼ B μ3 ð4Þ C; 1 2 2 > @ A g V s ^ G Vð s ^ Þ 2 SLS SLS 1 4 μ4 s0 where
k1 V θ^ SLS ¼ s20 k1 G2 2 g 1 g 1> s0
!1 ;
ð5Þ
ðμ s4 Þðs2 k Þ 1 0 0 ; V s^ 2SLS ¼ 4 s20 k1 g 1> G2 1 g 1
ð6Þ
and g1 ¼ E
∂gðx; θ0 Þ ; ∂θ
G2 ¼ E
∂gðx; θ0 Þ ∂gðx; θ0 Þ ; ∂θ ∂θ >
k1 ¼
μ23 : μ4 s40
ð7Þ
The expectation in g 1 and G2 is taken with respect to the distribution of x. Under similar regularity conditions, the > asymptotic covariance matrix for the OLSE γ^ OLS ¼ ðθ^ OLS ; s^ 2OLS Þ > is as follows: 0 1 0 1 s20 G2 1 μ3 G2 1 g 1 Vðθ^ OLS Þ μ3 G2 1 g 1 A¼@ A: ð8Þ D¼@ μ3 g 1> G2 1 Vðs^ 2OLS Þ μ3 g 1> G2 1 μ4 s40 It is easily seen that the SLSE and the OLSE have the same covariance matrix when μ3 ¼ 0. When μ3 a 0, the SLSE for θ and s2 is more efficient than the OLSE, since (1) Vðs^ 2OLS Þ ZVðs^ 2SLS Þ, with equality holding if and only if g 1> G2 1 g 1 ¼ 1. (2) Vðθ^ OLS Þ Vðθ^ SLS Þ is nonnegative definite if g 1> G2 1 g 1 ¼ 1, and is positive definite if g 1> G2 1 g 1 o 1. Notice that 0 r g 1> G2 1 g 1 r1 for any model, from Wang and Leblanc (2008, The Proof of Theorem 4). The SLSE can be computed using the following two-stage procedure as suggested by Wang and Leblanc (2008): minimize Q n ðγÞ using W i ¼ I 2 for i ¼ 1; …; n to obtain the first-stage estimator γ~ SLS , then estimate the elements of U i 1 ¼ U 1 ðxi Þ in (3) 1 1 using γ~ SLS and μ^ 3 and μ^4 from the residuals to obtain U^ i . Finally, minimize Q n ðγÞ again using W i ¼ U^ i to obtain the SLSE γ^ SLS . However, this procedure can be improved by doing several iterations for the second stage.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
143
3. Optimal designs based on the SLSE In this section, we will define and explore D-optimal and A-optimal designs based on the SLSE. First we show the properties of the asymptotic covariance matrix of the SLSE in the next three lemmas. We have 0 rg 1> G2 1 g 1 r1 for all regression models. Lemma 1 gives one condition that g 1> G2 1 g 1 ¼ 1. Lemma 1. Let vðxÞ ¼ ∂gðx; θ0 Þ=∂θ. If the first element of vðxÞ is 1, i.e., vðxÞ½1 ¼ 1, then G2 1 g 1 ¼ ð1; 0; …; 0Þ > ; g 1> G2 1 ¼ ð1; 0; …; 0Þ, and g 1> G2 1 g 1 ¼ 1. The proof of Lemma 1 is in the Appendix. For linear models with an intercept term, it is obvious that vðxÞ½1 ¼ 1 and g 1> G2 1 g 1 ¼ 1. For other models, it may be true that g 1> G2 1 g 1 ¼ 1 for some distributions of x. The next two Lemmas present the results for the determinant and the trace of the covariance matrix of the SLSE, and their relationships with those of the OLSE. Lemma 2. For the optimal SLSE, we have s 2ðq 1Þ ðs2 k Þq 1 0 ¼ 0 det V θ^ OLS det V θ^ SLS 1 s20 k1 g 1> G2 g 1
ð9Þ
and detðC Þ ¼
s0 2q ðs20 k1 Þq þ 1 ðμ4 s40 Þ s20 k1 g 1> G2 1 g 1
det V θ^ OLS ;
where q is the dimension of vector θ. Lemma 3. For the optimal SLSE, we have s2 k ðs2 k Þk g > G 2 g 1 1 1 1 1 2 ¼ 0 2 trace V θ^ OLS þ 0 trace V θ^ SLS s0 s20 k1 g 1> G2 1 g 1
ð10Þ
and traceðC Þ ¼
2 > 2 4 s20 k1 ^ OLS þ ðs0 k1 Þðk1 g 1 G2 g 1 þ μ4 s0 Þ : trace V θ 1 2 > s20 s0 k1 g 1 G2 g 1
The proofs of Lemmas 2 and 3 are in the Appendix. With these results, we can define optimal designs based on the SLSE and investigate their properties. A-optimal and D-optimal designs based on the SLSE are defined to minimize traceðVðθ^ SLS ÞÞ and detðVðθ^ SLS ÞÞ respectively. From (8) for Vðθ^ OLS Þ and (9) and (10) in Lemmas 2 and 3, it is easy to see that the A-optimal and D-optimal designs minimize the following loss functions: 1 þ ℓSLS A ðξÞ ¼ trace G2 ℓSLS D ðξÞ ¼
tg 1> G2 2 g 1
1 tg 1> G2 1 g 1
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
;
;
ð11Þ
where ξ is the distribution of x, and t ¼ k1 =s20 ¼ μ23 =s20 ðμ4 s40 Þ. From detðUÞ ¼ s20 ðμ4 s40 Þ μ23 4 0 in Section 2, we have SLS
s20 ðμ4 s40 Þ 4μ23 , which implies 0 r t o1. The optimal designs are denoted by ξSLS A and ξD . For nonlinear models, since g 1 and G2 often involve the unknown parameter vector θ0 , we define locally A- and SLS D-optimal designs to minimize the loss functions ℓSLS A ðξÞ and ℓD ðξÞ, respectively, given the true parameter θ0 . For linear models, g 1 and G2 do not depend on θ0 , but the optimal designs may depend on parameter t and can be locally optimal as well. For simplicity, we just call them A- and D-optimal designs for both linear and nonlinear models in the rest of the paper. Notice that the A-optimal and D-optimal designs based on the OLSE minimize traceðVðθ^ OLS ÞÞ and detðVðθ^ OLS ÞÞ respectively, where Vðθ^ OLS Þ is given in (8), so the corresponding loss functions are 1 ℓOLS ; A ðξÞ ¼ trace G 2 ℓOLS D ðξÞ ¼
1 : detðG2 Þ
ð12Þ OLS
The resulting optimal designs are denoted by ξOLS and ξD . A OLS SLS OLS When the error distribution is symmetric, i.e., μ3 ¼ 0, we have t ¼0, ℓSLS A ðξÞ ¼ ℓA ðξÞ and ℓD ðξÞ ¼ ℓD ðξÞ, and the optimal designs based on the SLSE and OLSE are the same. When μ3 a 0, it is possible that the optimal designs are still the same. The following theorem gives one such case.
144
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
Theorem 1. Let vðxÞ ¼ ∂gðx; θ0 Þ=∂θ. If the first element of vðxÞ is 1, i.e., vðxÞ½1 ¼ 1, then detðVðθ^ SLS ÞÞ ¼ s0 2ðq 1Þ ðs20 k1 Þðq 1Þ detðVðθ^ OLS ÞÞ and s2 k 1 ¼ 0 2 trace V θ^ OLS þk1 : trace V θ^ SLS s0 Thus the D-optimal and A-optimal designs based on the SLSE are the same as those based on the OLSE. The proof of Theorem 1 is in the Appendix. Here is one application of the results in Theorem 1. Consider the linear regression model in (1). The SLSE can be applied to estimate θ with gðx; θÞ ¼ z > ðxÞθ so vðxÞ ¼ ∂gðx; θ0 Þ=∂θ ¼ zðxÞ. If zðxÞ½1 ¼ 1, then by Theorem 1, the optimal designs based on the SLSE are the same as those based on the OLSE. This is a very useful result, since linear regression models often contain a constant term (also called an intercept term). The same optimal designs will yield the most efficient OLSE and SLSE; therefore one can determine which estimator to use after examining the error distribution from residual analysis. However, if zðxÞ½1 a 1, then the optimal designs based on the SLSE may be different from those based on the OLSE. This is illustrated in Example 1. Example 1. The simple linear regression model going through the origin is given by y ¼ θ1 x þ ε;
x A S:
Let ηj ¼ Eðx Þ be the jth moment of x. For this linear model, from (7), g 1 ¼ EðxÞ ¼ η1 , and G2 ¼ Eðx2 Þ ¼ η2 . So detðG2 Þ ¼ η2 and g 1> G2 1 g 1 ¼ η21 =η2 . Two design spaces S1 and S2 are considered here: (1) S1 ¼ ½ 1; 1, a symmetric interval about 0 and (2) S2 ¼ ½0; 1. j
OLS
For the OLSE, the ξD
minimizes the loss function in (12), which is
1 1 ¼ : ℓOLS D ðξÞ ¼ detðG2 Þ η2 OLS
On S1 or S2, 0 r x2 r 1, so η2 ¼ Eðx2 Þ r1 for all the distributions. Thus ℓOLS D ðξÞ Z 1, and the equality is obtained by the ξD with OLS η2 ¼ 1. There are many D-optimal designs on S1, but there is only one on S2. For S1, the ξD has two support points 71 with OLS probabilities Pðx ¼ 1Þ ¼ p0 and Pðx ¼ 1Þ ¼ 1 p0 for any 0 r p0 r 1. For S2, the ξD has one support point þ1 with probability Pðx ¼ 1Þ ¼ 1. SLS For the SLSE, the ξD minimizes the loss function in (11), which is ℓSLS D ðξ Þ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
¼
1 ; η2 tη21 SLS
with 0 r t o1. For S1, ℓSLS D ðξÞ is minimized at η1 ¼ 0 and η2 ¼ 1, and ξD has distribution Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ 1=2, which is one of the D-optimal designs based on the OLSE. For S2, since η1 ¼ EðxÞ ZEðx2 Þ ¼ η2 , and VarðxÞ ¼ η2 η21 Z 0, we need to solve the following optimization problem: minimize
1 η2 tη21
subject to
η21 r η2 r η1 r 1
ξ
The constraint about the two moments η1 and η2 is in Dette and Studden (1997, p. 4). It is clear that the loss function is SLS minimized at η2 ¼ η1 ¼ a1 ¼ minf1; 1=2tg, and the ξD has the following distribution: Pðx ¼ 0Þ ¼ 1 a1 , Pðx ¼ 1Þ ¼ a1 . If SLS OLS SLS OLS SLS 0 rt r 1=2, then a1 ¼ 1 and the ξD and ξD are the same. Otherwise, the ξD and ξD are different, and the ξD has more OLS support points than the ξD . □ OLS SLS SLS In Example 1, since g 1 and G2 are scalars, we have ℓOLS A ðξÞ ¼ ℓD ðξÞ and ℓA ðξÞ ¼ ℓD ðξÞ, and A-optimal and D-optimal designs are equivalent. For the SLSE, some optimal designs depend on parameter t, and some do not. Parameter t is related to the skewness measure δs ¼ μ3 =s30 and the kurtosis measure δk ¼ μ4 =s40 3 as follows: t ¼ δ2s =ðδk þ 2Þ. More examples of optimal designs are given in Section 6. SLS In the next two sections we study the transformation invariance and symmetry properties of the D-optimal designs ξD . SLS
4. Transformation invariance of ξD
For regression model (2), suppose that the design space is S Rp . Consider a transformation T and a distribution ξ of x on S. Denote the transformed design space as ST ¼ fTu; u A Sg Rp and the distribution of Tx as ξT . If a design ξn is a D-optimal design on S and its transformed distribution ξnT is a D-optimal design on ST, then the D-optimal design is said to be invariant under the transformation T.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
145
Theorem 2. If function gðx; θÞ and transformation T satisfy the following condition: ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ SLS
where matrix Q T is nonsingular and does not depend on x, then the D-optimal design ξD is invariant under the transformation T. The proof of Theorem 2 is in the Appendix. Although T can be any transformation, we are usually interested in linear SLS transformations of design spaces. In particular, Theorem 2 provides a sufficient condition to see if ξD is invariant under scale and/or shift transformations for a given model, and the condition is easy to verify. The next two examples are for polynomial regression models without intercept and trigonometric regression models on a SLS partial circle. We examine the invariance of ξD under scale and/or shift transformations. Optimal designs based on the OLSE for these models have been studied by many authors, for example, Chang and Heiligers (1996), Papp (2012), Dette and Studden (1997), Dette et al. (2002), and Chang et al. (2013). Example 2. Consider the qth-order polynomial regression model without intercept y ¼ xθ1 þx2 θ2 þ ⋯ þ xq θq þ ε;
ð13Þ
where x A S ¼ ½a; b. The scale transformation can be defined as Tx ¼ b0 x for b0 4 0, Then ST ¼ ½ab0 ; bb0 , and it is easy to see that 0 1 0 1 b0 x x B b2 x2 C B x2 C C ∂gðTx; θ0 Þ B ∂gðx; θ0 Þ 0 B C C ¼B ; C ¼ QT B ⋮ C ¼ QTB @ ⋮ A ∂θ ∂θ @ A q q q b0 x x 2
q
where Q T is a diagonal matrix with diagonal elements: b0 ; b0 ; …; b0 . Since Q T is nonsingular and does not depend on x, the SLS SLS D-optimal design ξD is invariant under the scale transformation for this model. From Example 1, notice that the ξD is not invariant under the shift transformation Tx ¼ x þ a0 for a0 a 0. □ Example 3. Consider the qth-order trigonometric regression model y ¼ sin ðxÞθ1 þ cos ðxÞθ2 þ ⋯ þ sin ðqxÞθ2q 1 þ cos ðqxÞθ2q þ ε;
ð14Þ
where x A S ¼ ½a; b. In Chang et al. (2013), D-optimal designs are constructed for the first-order model (q ¼1) without SLS OLS intercept. Here we also assume that the model does not have an intercept term, otherwise the ξD is the same as the ξD by Theorem 1. The shift transformation is Tx ¼ x þ a0 , for a0 a0. Then ST ¼ ½a þ a0 ; bþ a0 , and ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ where Q T is a block diagonal matrix with block matrices Q 1 ; …; Q q , and ! sin ðja0 Þ cos ðja0 Þ Qj ¼ ; for j ¼ 1; …; q: sin ðja0 Þ cos ðja0 Þ It is clear that detðQ j Þ ¼ 1, so detðQ T Þ ¼ ∏qj¼ 1 detðQ j Þ ¼ 1. Since Q T is nonsingular and does not depend on x, the D-optimal SLS OLS design ξD is invariant under the shift transformation. The same result for the ξD can be found in Dette et al. (2002). From OLS Chang et al. (2013), the ξD is not invariant under the scale transformation for this model with q ¼1, which implies that the SLS ξD is not scale invariant either. □ Example 4. Consider the exponential regression model in Dette et al. (2006) y ¼ θ1 e θ2 x þ θ3 e θ4 x þ ε; where x A S ¼ ½a; b. This is a nonlinear regression model. Assume that the true parameter vector value is θ0 ¼ ðθ10 ; θ20 ; θ30 ; θ40 Þ > . The shift transformation is Tx ¼ x þ a0 , for a0 a 0. Then ST ¼ ½a þa0 ; b þ a0 , and 0 1 e θ20 ðx þ a0 Þ B C θ10 ðx þa0 Þe θ20 ðx þ a0 Þ C ∂gðTx; θ0 Þ B C ¼ Q T ∂gðx; θ0 Þ ; ¼B θ 40 ðx þ a0 Þ B C e ∂θ ∂θ @ A θ40 ðx þ a0 Þ θ30 ðx þa0 Þe where
0
e θ20 a0 B B θ10 a0 e θ20 a0 QT ¼ B B 0 @ 0
0 e
θ 20 a0
0 0
e
0
0
0
0
θ40 a0
θ30 a0 e θ40 a0
0 e θ40 a0
1 C C C: C A
146
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151 SLS
It is obvious that matrix Q T does not depend on x and is nonsingular. Thus the D-optimal design ξD is invariant under the shift transformation for this model, which implies that we can assume the design space S ¼ ½0; b a to construct D-optimal designs. It is easy to see that the result holds for the general exponential regression model, y ¼ ∑qj¼ 1 θ2j 1 e θ2j x þ ε. □ SLS
5. Symmetry of ξD
SLS
We examine the symmetry of ξD if the design space S is symmetric about 0. For simplicity, we only present the result for the univariate case. However the methodology is quite general, and the result can be extended to multivariate design spaces easily. For the univariate case, the design space is assumed to be S ¼ ½ a; a for a 4 0. Consider transformation Tx ¼ x. Since S is symmetric about 0, ST ¼ S. For any distribution ξ0 ðxÞ of x on S, ξ1 ðxÞ is defined to be the distribution of x, and let distribution ξλ ðxÞ ¼ ð1 λÞξ0 ðxÞ þ λξ1 ðxÞ, for any λ A ½0; 1. Notice that ξ0:5 ðxÞ ¼ 0:5ξ0 ðxÞ þ 0:5ξ1 ðxÞ is a symmetric distribution about 0. To specify the distribution in the expectation in g 1 and G2 , we use the following notation. Define, for any λ A ½0; 1, ∂gðx; θ0 Þ ; g 1;ξλ ¼ Eξλ ∂θ ∂gðx; θ0 Þ ∂gðx; θ0 Þ G2;ξλ ¼ Eξλ ; > ∂θ ∂θ dðλÞ ¼ detðG2;ξλ Þ;
ð15Þ
1 > hðλÞ ¼ g 1;ξ G2;ξ g : λ λ 1;ξλ
The symmetry property of
ð16Þ SLS ξD
is given in the following theorem.
Theorem 3. Consider a regression model and a design space S that is symmetric about 0. There exists a symmetric D-optimal SLS design ξD , if dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, where functions d and h are defined in (15) and (16) respectively. The proof of Theorem 3 is in the Appendix. This theorem also provides a sufficient condition to check for the existence of SLS a symmetric ξD . SLS The next two examples show that there exist symmetric D-optimal designs ξD for the models in Examples 2 and 3 if the design space S is symmetric. SLS
Example 5. Consider the symmetry of ξD for model (13) in Example 2 with a symmetric design space S. From Example 2, SLS we know that ξD is scale invariant, so without loss of generality, the symmetric design space is assumed to be S ¼ ½ 1; 1. SLS To show the existence of a symmetric ξD for this model, we verify the sufficient condition, dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, as follows. From Example 2, we have 20 13 20 13 x x 6 B C7 2 6B x2 C7 6B ð xÞ C7 ∂gðx; θ0 Þ 6B C7 B C7 ¼ Eξ1 6B C7 ¼ Eξ0 6 g 1;ξ1 ¼ Eξ1 6B ⋮ C7 ¼ Pg 1;ξ0 ; 4@ ⋮ A5 ∂θ 4@ A5 ð xÞq xq where P is a diagonal matrix with diagonal elements as ð 1Þ; ð 1Þ2 ; …; ð 1Þq . Similarly, we can show that G2;ξ1 ¼ PG2;ξ0 P > : From the above results and the definitions of dðλÞ and hðλÞ in (15) and (16) respectively, we have dð0Þ ¼ detðG2;ξ0 Þ ¼ detðG2;ξ1 Þ ¼ dð1Þ;
and
1 1 > > G2;ξ g ¼ g 1;ξ G2;ξ g ¼ hð1Þ: hð0Þ ¼ g 1;ξ 0 0 1;ξ0 1 1 1;ξ1 SLS
Thus there exists a symmetric ξD for this model. SLS
□ SLS
Example 6. Consider the symmetry of ξD for model (14) in Example 3, where x A S ¼ ½ a; a for a 40. Since ξD is shift invariant for this model, we can always assume, without loss of generality, the design space S is symmetric about 0. It is easy to verify that ∂gðx; θ0 Þ ¼ Pg 1;ξ0 ; g 1;ξ1 ¼ Eξ1 ∂θ
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
147
where P is a diagonal matrix with diagonal elements as 1; 1; …; 1; 1, and G2;ξ1 ¼ PG2;ξ0 P > : SLS
Then it is obvious that dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ. Thus there exists a symmetric ξD for this model.
□
SLS ξD
for a given model with From Examples 5 and 6, we have the following result for the existence of a symmetric symmetric design space. The proof of Corollary 1 is omitted, since it is similar to the derivation in Examples 5 and 6. Corollary 1. Consider a model with a symmetric design space. If function gðx; θÞ satisfies the following condition: ∂gð x; θ0 Þ ∂gðx; θ0 Þ ¼P ; ∂θ ∂θ SLS
where matrix P does not depend on x and has detðPÞ ¼ 71, then there exists a symmetric D-optimal design ξD for the model. OLS
All the results in Theorems 2 and 3 and Corollary 1 can also be applied to ξD and symmetry properties.
to check for the transformation invariance
6. Examples SLS
SLS
OLS
SLS
Here we will construct ξD for a couple of models, and compare the ξD with ξD . The ξD in Example 7 depends on the SLS SLS value of parameter t, while in Example 8 the ξD does not. The minimum number of support points of ξD is usually bigger OLS than that of ξD . SLS
Example 7. Construct ξD for the second-order polynomial regression model without intercept: y ¼ xθ1 þx2 θ2 þ ε;
x A S ¼ ½ 1; 1:
Let ηj ¼ Eðx Þ be the jth moment of x. Then g 1 ¼ j
Example 5, we will construct a symmetric becomes ℓSLS D ðξÞ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
¼
SLS ξD
η1 η2
, and G2 ¼
η2 η3 η3 η4
SLS
. Since there exists a symmetric ξD on S ¼ ½ 1; 1 from
here. For a symmetric design, we have η1 ¼ η3 ¼ 0 and the loss function in (11)
1 ; η2 η4 tη32
where parameter t A ½0; 1Þ. On S ¼ ½ 1; 1, the even moments of any distribution satisfy 0 rη22 r η4 r η2 r 1 from Dette and SLS Studden (1997). Then we solve the following optimization problem to find a ξD : minimize
1 η2 η4 tη32
subject to
0 rη22 rη4 rη2 r1:
ξ
SLS
It is easy to show that the loss function is minimized at η2 ¼ η4 ¼ a2 ¼ minf1; 2=3tg and a symmetric ξD has the following distribution: Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ a2 =2; Pðx ¼ 0Þ ¼ 1 a2 . When x¼0, the model becomes y ¼ ε. The observations at x¼ 0 provide information on s2 which is estimated together with θ1 and θ2 in the SLSE. OLS OLS To compare with a symmetric ξD , notice that ξD minimizes ℓOLS D ðξÞ ¼
1 1 ¼ : detðG2 Þ η2 η4 η23 OLS
It is clear that ℓOLS has distribution Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ 1=2. If t r2=3, D ðξÞ is minimized at η2 ¼ η4 ¼ 1 and η3 ¼ 0. The ξD OLS SLS SLS OLS then the ξD and ξD are the same. Otherwise, the ξD has one more support point than the ξD . □ SLS
Example 8. Construct ξD for the first-order trigonometric regression model without intercept: y ¼ sin ðxÞθ1 þ cos ðxÞθ2 þ ε;
x A S ¼ ½ π; π:
Define the following expectations to work on the optimal designs: v1 ¼ E½ sin ðxÞ; v2 ¼ E½ cos ðxÞ; v3 ¼ E½ sin 2 ðxÞ; v4 ¼ E½ cos 2 ðxÞ, v5 ¼ E½ sin ðxÞ cos ðxÞ. Then g 1 ¼ vv , and G2 ¼ vv vv . From Chang et al. (2013), there are infinitely many D-optimal 0
OLS OLS designs ξD based on the OLSE, and all the ξD have G2 ¼ 0:5 . Here are three optimal designs: 0 0:5
π π (1) ξOLS D1 :p x ¼ 4 ¼ p x ¼ 4 ¼ 1=2,
π π (2) ξOLS D2 : p x ¼ 4 ¼ p x ¼ 4 ¼ p x ¼
3π π (3) ξOLS D3 : p x ¼ 8 ¼ p x ¼ 8 ¼ 1=2.
3π 4
1
3
5
2
5
4
¼ p x ¼ 3π 4 ¼ 1=4,
SLS
The ξD minimizes the loss function in (11), and for this model the ℓSLS D ðξÞ is minimized when detðG2 Þ is maximized and g 1> G2 1 g 1
OLS
is minimized. From ξD , detðG2 Þ is maximized at v3 ¼ v4 ¼ 1=2 and v5 ¼ 0, and it is easy to show that g 1> G2 1 g 1 is
148
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151 SLS
minimized at v1 ¼ v2 ¼ 0. Among the above three designs, only ξOLS D2 has v1 ¼ v2 ¼ v5 ¼ 0 and v3 ¼ v4 ¼ 1=2, so it is a ξD . In SLS fact, there are also infinitely many ξD for this model including the following designs with four support points: ξSLS Dα : pðx ¼ π=4 þ αÞ ¼ pðx ¼ π=4 þ αÞ ¼ pðx ¼ 3π=4 þαÞ ¼ pðx ¼ 3π=4 þ αÞ ¼ 1=4, for any α A ½ π=4; π=4. Notice that all these optimal designs do not depend on t. Since the model in this example is a specific case from Example 6 when q ¼1 and a ¼ π, there exists a symmetric design about 0. Here it is clear that, with α ¼ 0, ξSLS Dα is a symmetric design about 0. SLS OLS Since all the D-optimal designs ξD have v3 ¼ v4 ¼ 1=2 and v5 ¼ 0 and maximize detðG2 Þ, all of them are ξD . However, OLS SLS OLS SLS only some of the D-optimal designs ξD are ξD . In particular the ξD with two support points are not ξD . □ 7. Discussion New optimal design criteria are proposed based on the SLSE, since the SLSE is more efficient than the OLSE when the error distribution has nonzero third moment. The new design criteria provide good alternatives to the ones based on the OLSE, and many new optimal designs can be constructed for linear and nonlinear regression models. Sufficient conditions are derived to check for transformation invariance and symmetry of D-optimal designs for a given SLS OLS model. These results can be applied to both ξD and ξD , and they are easy to verify and are very useful in practice. As usual, it is hard to obtain similar results for A-optimal designs, which can be a future research topic. The optimal designs based on the SLSE and OLSE are the same for some models. When they are different, the optimal designs based on the SLSE tend to have more support points than those based on the OLSE. Several examples are given for SLS polynomial, trigonometric and exponential regression models. Optimal designs ξD may depend on parameter t. In the case SLS SLS SLS that ξD does not depend on t, it is easy to use it in practice. If ξD does depend on t, sensitivity analysis of ξD can be SLS performed. If ξD is not sensitive to small changes of parameter t, then we do not need very accurate information of t to implement the design. With the new optimal design criteria, constructing optimal designs for some regression models and design spaces produces challenging research problems. For some models, theoretical results may be obtained by using moment theory and convex optimization techniques. For complicated models, the design problems can be simplified and solved by only finding the optimal weights of a set of given support points. In addition, we can define and study other design criteria based on the SLSE.
Acknowledgements This research work is partially supported by Discovery Grants from the Natural Science and Engineering Research Council of Canada. The authors are grateful to the Editor and referees for their helpful comments and suggestions. Appendix A. Proofs Proof of Lemma 1. Since vðxÞ½1 ¼ 1, from (7) we have g 1 ¼ E½vðxÞ ¼ E ð1; …Þ > ¼ ð1; …Þ > ; ∂gðx; θ0 Þ ∂gðx; θ0 Þ G2 ¼ E ¼ E vðxÞv > ðxÞ ¼ g 1 ; … : > ∂θ ∂θ Thus, from G2 1 G2 ¼ I we have G2 1 ½g 1 ; ⋯ ¼ I, which implies G2 1 g 1 ¼ ð1; 0; …; 0Þ > and g 1> G2 1 ¼ ð1; 0; …; 0Þ. Therefore, g 1> G2 1 g 1 ¼ ð1; 0; …; 0Þð1; …Þ > ¼ 1. □ Proof of Lemma 2. From (5), ¼ ðs20 k1 Þq det G2 1 det I q þ det V θ^ SLS ¼ ðs20 k1 Þq det
G2 1 1 þ
!
k1
1=2
s20 k1 g 1> G2 1 g 1 k1
G2
1=2
g 1 g 1> G2
!
g 1> G2 1 g 1
s20 k1 g 1> G2 1 g 1 2 s0 ¼ s0 2q ðs20 k1 Þq det V θ^ OLS 1 2 > s0 k1 g 1 G2 g 1 2ðq 1Þ 2 s ðs0 k1 Þq ¼ 0 det V θ^ OLS : 1 2 > s0 k1 g 1 G2 g 1 From (4) and (17), we have V s^ 2SLS detðC Þ ¼ det V θ^ SLS
k1 Vðs^ 2SLS Þ2 g 1> G2 1 Vðθ^ SLS Þ 1 G2 1 g 1 μ4 s40
ð17Þ
!
¼ det V θ^ SLS
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
V s^ 2SLS
k1 1 Vðs^ 2SLS Þ2 g 1> G2 1 2 s0 k1 μ4 s40
G2
149
! ! k1 1 > g g g G ¼ det V θ^ SLS 1 1 1 2 2 s0 !!
k ðμ s4 Þðs2 k Þ k1 1 4 1 0 0 V s^ 2SLS g 1> G2 1 g 1 1 2 g 1> G2 1 g 1 s0 ðs20 k1 g 1> G2 1 g 1 Þ2 ! k1 > 1 2 ^ ¼ det V θ SLS V s^ SLS 1 2 g 1 G2 g 1 s0 2q 2 q 2 det V θ^ OLS ¼s ðs k1 Þ V ðs^ 0
0
¼
SLS
s0 2q ðs20 k1 Þq þ 1 ðμ4 s40 Þ s20 k1 g 1> G2 1 g 1
det V θ^ OLS :
□
Proof of Lemma 3. From Wang and Leblanc (2008, the proof of Theorem 4), we get
k1 V θ^ SLS ¼ s20 k1 G2 1 þ V s^ 2SLS G2 1 g 1 g 1> G2 1 ; 4 μ4 s0 which leads to
trace V θ^ SLS ¼ s20 k1 trace G2 1 þ
k1 V s^ 2SLS trace G2 1 g 1 g 1> G2 1 4 μ4 s0 s20 k1 ðs20 k1 Þk1 trace V θ^ OLS þ trace G2 1 g 1 g 1> G2 1 ¼ 2 1 s0 s20 k1 g 1> G2 g 1 2 s k1 ðs2 k1 Þk1 g 1> G2 2 g 1 trace V θ^ OLS þ 0 : ¼ 0 2 s0 s20 k1 g 1> G2 1 g 1
ð18Þ
From (4) and (18),
traceðC Þ ¼ trace V θ^ SLS þ V s^ 2SLS 2 > 2 s20 k1 ^ OLS þ ðs0 k1 Þk1 g 1 G2 g 1 þ V s^ 2 trace V θ SLS s20 s20 k1 g 1> G2 1 g 1 2 s k1 ðs2 k1 Þk1 g 1> G2 2 g 1 ðμ4 s40 Þðs20 k1 Þ trace V θ^ OLS þ 0 þ ¼ 0 2 s0 s20 k1 g 1> G2 1 g 1 s20 k1 g 1> G2 1 g 1 ðs2 k Þðk g > G 2 g þμ s4 Þ s2 k1 1 1 1 4 1 2 0 ¼ 0 2 trace V θ^ OLS þ 0 : □ s0 s20 k1 g 1> G2 1 g 1 ¼
Proof of Theorem 1. From Lemma 1, we have g 1> G2 1 g 1 ¼ 1 and g 1> G2 2 g 1 ¼ 1. Putting these results in Lemmas 2 and 3 gives the results for detðVðθ^ SLS ÞÞ and traceðVðθ^ SLS ÞÞ. □ SLS
Proof of Theorem 2. On S, from (11), ξD minimizes ℓ0 ðξÞ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
:
Since ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ we have g 1;T ≔E
∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QTE ¼ Q T g1; ∂θ ∂θ
G2;T ≔E
∂gðTx; θ0 Þ ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ∂gðx; θ0 Þ ¼ QTE Q T> ¼ Q T G2 Q T> : > > ∂θ ∂θ ∂θ ∂θ SLS
1 > G2;T g 1;T ¼ g 1> G2 1 g 1 . On ST, ξD minimizes the following function: Then it is easy to verify that g 1;T
ℓ1 ðξT Þ ¼
1 > G 1g detðG2;T Þð1 tg 1;T 2;T 1;T Þ
150
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
1
¼
ðdetðQ T ÞÞ2 detðG2 Þð1 tg 1> G2 1 g 1 Þ ℓ0 ðξÞ : □ ¼ ðdetðQ T ÞÞ2 SLS
It is clear that if ξn minimizes ℓ0 ðξÞ, then ξnT minimizes ℓ1 ðξT Þ. Thus ξD is invariant under transformation T. Proof of Theorem 3. If dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, we can show that (i) dðλÞ Zdð0Þ and (ii) hðλÞ r hð0Þ, for any λ A ½0; 1. (i) Since ξλ ðxÞ ¼ ð1 λÞξ0 ðxÞ þλξ1 ðxÞ, we have g 1;ξλ ¼ ð1 λÞg 1;ξ0 þλg 1;ξ1 ;
ð19Þ
G2;ξλ ¼ ð1 λÞG2;ξ0 þ λG2;ξ1 :
ð20Þ
From (20) and Minkowski's inequality in Horn and Johnson (1985, p. 482), we get ðdðλÞÞ1=q ¼ ðdetðG2;ξλ ÞÞ1=q ¼ ðdetðð1 λÞG2;ξ0 þ λG2;ξ1 ÞÞ1=q Zðdetðð1 λÞG2;ξ0 ÞÞ1=q þ ðdetðλG2;ξ1 ÞÞ1=q ¼ ð1 λÞdð0Þ1=q þ λdð1Þ1=q ¼ dð0Þ1=q ;
since dð0Þ ¼ dð1Þ:
Thus, for any λ A ½0; 1, we have dðλÞ Z dð0Þ:
ð21Þ
(ii) It is clear that both g 1;ξλ and G2;ξλ are linear functions of λ from (19) and (20). Function hðλÞ is defined as
1 > hðλÞ ¼ g 1;ξ G2;ξ g , and it is a convex function of λ by the result in Wiens (1993, Lemma 2). Therefore, using the convex λ 1;ξλ λ
property and hð0Þ ¼ hð1Þ gives
hðλÞ rð1 λÞhð0Þ þλhð1Þ ¼ hð0Þ:
ð22Þ
SLS
Now, for the SLSE, ξD minimizes the loss function in (11). For any design ξ0, using the results in (21) and (22) with λ ¼ 0:5, we have ℓSLS D ðξ0:5 Þ ¼
1 1 > G2;ξ g Þ detðG2;ξ0:5 Þð1 tg 1;ξ 0:5 1;ξ0:5 0:5
1 dð0:5Þð1 thð0:5ÞÞ 1 r dð0Þð1 thð0ÞÞ ¼
¼ ℓSLS D ðξ0 Þ; which implies that the loss function at the symmetric design ξ0:5 is smaller than or equal to that at design ξ0. Thus the minimum value of ℓSLS D ðξÞ can be always achieved by a symmetric design. □ References Abarin, T., Wang, L., 2009. Second-order least squares estimation of censored regression models. J. Stat. Plan. Inference 139, 125–135. Chang, F.C., Heiligers, B., 1996. E-optimal designs for polynomial regression without intercept. J. Stat. Plan. Inference 55, 371–387. Chang, F.C., Imhof, L., Sun, Y.Y., 2013. Exact D-optimal designs for first-order trigonometric regression models on a partial circle. Metrika 76, 857–872. Chen, R.B., Wong, W.K., Li, K.Y., 2008. Optimal minimax designs over a prespecified interval in a heteroscedastic polynomial model. Stat. Probab. Lett. 78, 1914–1921. Chen, X., Tsao, M., Zhou, J., 2012. Robust second-order least-squares estimator for regression models. Stat. Pap. 53, 371–386. Dette, H., Melas, V.B., Pepelyshev, A., 2002. D-optimal designs for trigonometric regression models on a partial circle. Ann. Inst. Stat. Math. 54, 945–959. Dette, H., Melas, V.B., Wong, W.K., 2006. Locally D-optimal designs for exponential regression models. Stat. Sin. 16, 789–803. Dette, H., Studden, W.J., 1997. The Theory of Canonical Moments with Applications in Statistics, Probability and Analysis. Wiley, New York. Fang, Z., Wiens, D.P., 2000. Integer-valued, minimax robust designs for estimation and extrapolation in heteroscedastic, approximately linear models. J. Am. Stat. Assoc. 95, 807–818. Fedorov, V.V., 1972. Theory of Optimal Experiments. Academic Press, New York. Horn, R.A., Johnson, C.R., 1985. Matrix Analysis. Cambridge University Press, Cambridge. Huber, P.J., 1981. Robust Statistics. Wiley, New York. Li, K.C., Notz, W., 1982. Robust designs for nearly linear regression. J. Stat. Plan. Inference 6, 135–151. Notz, W.I., 1989. Optimal designs for regression models with possible bias. J. Stat. Plan. Inference 22, 43–54. Papp, D., 2012. Optimal designs for rational function regression. J. Am. Stat. Assoc. 107, 400–411. Pukelsheim, F., 1993. Optimal Design of Experiments. Wiley, New York. Wang, L., 2003. Estimation of nonlinear Berkson-type measurement error models. Stat. Sin. 13, 1201–1210.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
151
Wang, L., 2004. Estimation of nonlinear models with Berkson-type measurement errors. Ann. Stat. 32, 2559–2579. Wang, L., Leblanc, A., 2008. Second-order nonlinear least squares estimation. Ann. Inst. Stat. Math. 60, 883–900. Wiens, D.P., 1992. Minimax designs for approximately linear regression. J. Stat. Plan. Inference 31, 353–371. Wiens, D.P., 1993. Designs for approximately linear regression: maximizing the minimum coverage probability of confidence ellipsoids. Can. J. Stat. 21, 59–70. Wiens, D.P., Zhou, J., 1999. Minimax designs for approximately linear models with AR(1) errors. Can. J. Stat. 27, 781–794. Zhou, J., 2001. Integer-valued, minimax robust designs for approximately linear models with correlated errors. Commun. Stat.: Theory Methods 30, 21–39.
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
New optimal design criteria for regression models with asymmetric errors Lucy L. Gao, Julie Zhou n Department of Mathematics and Statistics, University of Victoria, Victoria, BC, Canada V8W 3R4
a r t i c l e in f o
abstract
Article history: Received 3 September 2013 Received in revised form 6 January 2014 Accepted 19 January 2014 Available online 23 January 2014
Optimal regression designs are usually constructed by minimizing some scalar functions of the covariance matrix of the ordinary least squares estimator. However, when the error distribution is not symmetric, the second-order least squares estimator is more efficient than the ordinary least squares estimator. Thus we propose new design criteria to construct optimal regression designs based on the second-order least squares estimator. Transformation invariance and symmetry properties of the new criteria are investigated, and sufficient conditions are derived to check for these properties of D-optimal designs. The results can be applied to both linear and nonlinear regression models. Several examples are given for polynomial, trigonometric and exponential regression models, and new designs are obtained. & 2014 Elsevier B.V. All rights reserved.
Keywords: A-optimal design D-optimal design Non-normal errors Polynomial regression Robust design Second-order least squares estimator Scale invariance Shift invariance Symmetric design Trigonometric regression
1. Introduction Consider a linear regression model yi ¼ z > ðxi Þθ þεi ;
i ¼ 1; ⋯; n;
ð1Þ
where yi is the ith response observed at xi ¼ ðx1i ; …; xpi Þ > of p independent variables x1 ; …; xp , zðxÞ A Rq is a vector of known functions of x; θ A Rq is a vector of unknown regression parameters, and the errors εi's are independent with mean 0 and variance s2. Optimal regression designs aim to choose optimal design points x1 ; …; xn such that we can get the most information about the unknown regression parameters or the regression response z > ðxÞθ. Optimal regression designs have been studied extensively in the literature and many optimal design criteria have been proposed and studied; see, for example, Fedorov (1972) and Pukelsheim (1993). The ordinary least squares estimator (OLSE) is usually used to estimate θ since it is the best linear unbiased estimator. The OLSE is given by θ^ OLS ¼ ðZ > ZÞ 1 Z > y; n
Corresponding author. Tel.: þ1 250 721 7470. E-mail address: [email protected] (J. Zhou).
0378-3758/$ - see front matter & 2014 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.jspi.2014.01.005
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
141
where y ¼ ðy1 ; …; yn Þ > , and the ith row of matrix Z is z > ðxi Þ. The covariance matrix of θ^ OLS is Covðθ^ OLS Þ ¼ s2 ðZ > ZÞ 1 . The commonly used A-optimal, D-optimal and E-optimal designs minimize the trace, the determinant and the largest eigenvalue of Covðθ^ OLS Þ respectively. Optimal regression designs have been derived for various regression models and design spaces, and they perform well under the ideal model assumptions for (1): (i) the response function is correct, (ii) the errors εi's are i.i.d. Nð0; s2 Þ. However, these assumptions are never true in practice. When there are possible violations of the model assumptions, robust regression designs against small departures have been explored. Many robust designs have been constructed to deal with small departures in the response function, in the equal variance of the errors, in the correlation structure of the errors, and combinations of those departures. For example, Huber (1981), Li and Notz (1982), Notz (1989) and Wiens (1992) investigated robust designs against small departures in the response function. Fang and Wiens (2000) studied robust designs against small departures in the response function and in the equal variance of the errors. Chen et al. (2008) worked on minimax designs for polynomial models with various types of heteroscedastic errors, and Wiens and Zhou (1999) and Zhou (2001) worked on minimax designs robust against small departures in the response function and in the correlation structure of the errors. However very little work has been done to study optimal/robust designs when there is a small departure from the assumption of normal distribution. Thus, in this paper we will explore optimal regression designs when the error distribution is asymmetric. If the errors εi's have non-normal distributions, there are more efficient estimators than the OLSE. When the exact distribution of the error term is known, the maximum likelihood estimator can be applied to estimate θ. However the exact distribution of the error term is often unknown in practice. In this situation, the second-order least squares estimator (SLSE) proposed in Wang and Leblanc (2008) is more efficient than the OLSE if the third moment of the error is nonzero, i.e., the error distribution is asymmetric. The SLSE does not depend on the exact distribution of the error term, so we propose to use the SLSE to construct optimal regression designs for asymmetric errors in this paper. In particular, for a given model we construct the optimal distribution of x, and the design points x1 ; …; xn are selected randomly from the optimal distribution. A-optimal and D-optimal designs are defined based on the SLSE, various properties including scale and shift invariance and symmetry of D-optimal designs are explored, and sufficient conditions are derived to check for these properties. In addition, optimal designs based on the SLSE and OLSE are compared. All results can be applied to both linear and nonlinear regression models. The paper is organized as follows. Section 2 reviews the SLSE and its asymptotic distribution. In Section 3, optimal designs based on the SLSE are defined and compared with optimal designs based on the OLSE. In Section 4, transformation invariance of D-optimal designs is studied, and a sufficient condition is obtained to check for transformation invariance. Section 5 investigates the symmetry of D-optimal designs. Section 6 presents examples of D-optimal designs. Concluding remarks are in Section 7. All proofs are given in the Appendix.
2. Second-order least squares estimator The SLSE was first introduced for nonlinear models with measurement errors in Wang (2003, 2004). Since then it has been studied and extended for regression models without measurement errors, for example, Wang and Leblanc (2008) for nonlinear models, Abarin and Wang (2009) for censored regression models, and Chen et al. (2012) for robust SLSE for linear models. The SLSE is asymptotically normally distributed and is more efficient than the OLSE if the third moment of the errors is nonzero. Here we review the SLSE and its related results in Wang and Leblanc (2008) for the following model: yi ¼ gðxi ; θÞ þ εi ;
i ¼ 1; …; n;
ð2Þ
where gðxi ; θÞ can be a linear or nonlinear function of θ A Rq , and the errors εi's are independent and have mean Eðε∣xÞ ¼ 0 and Eðε2 ∣xÞ ¼ s2 . Denote the parameter vector as γ ¼ ðθ > ; s2 Þ > . We assume that y and ϵ have finite fourth moments, where y ¼ ðy1 ; …; yn Þ > and ϵ ¼ ðε1 ; …; εn Þ > . The SLSE γ^ SLS for γ is defined as the measurable function that minimizes n
Q n ðγÞ ¼ ∑ ρi> ðγÞW i ρi ðγÞ; i¼1
where ρi ðγÞ ¼ ðyi gðxi ; θÞ; y2i g 2 ðxi ; θÞ s2 Þ > , and W i ¼ Wðxi Þ is a 2 2 positive semidefinite matrix which may depend on xi . Notice that parameters θ and s2 are estimated together in γ^ SLS , and W i can be any positive semidefinite matrix. Suppose the true parameter value of model (2) is γ 0 ¼ ðθ0> ; s20 Þ > , and design points x1 ; …; xn are randomly selected from a distribution ξ of x. Under some regularity conditions, the asymptotic covariance matrix of the SLSE is shown to be pffiffiffi Vðγ^ SLS Þ ¼ lim Covð nγ^ SLS Þ ¼ A 1 BA 1 ; n- þ 1
142
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
where > ∂ρ ðγ 0 Þ ∂ρðγ 0 Þ W ðx Þ ; > ∂γ ∂γ > ∂ρ ðγ 0 Þ ∂ρðγ0 Þ W ðxÞUW ðxÞ ; B¼E > ∂γ ∂γ A¼E
U ¼ UðxÞ ¼ E½ρðγ 0 Þρ > ðγ0 Þ∣x; and the expectation in A and B is taken with respect to the distribution of x. The regularity conditions include one assumption that A is nonsingular, and see Wang and Leblanc (2008, Assumption 6) for a detailed discussion. It is clear that the covariance matrix Vðγ^ SLS Þ depends on γ0 . Since the asymptotic covariance matrix also depends on the weighting matrix W, we can choose W to obtain the most efficient estimator. Similar to the weighted least squares estimation, it can be shown that > 1 ∂ρ ðγ 0 Þ 1 ∂ρ > ðγ 0 Þ U A 1 BA 1 ≽ E ; ∂γ ∂γ > where ≽ indicates that the difference of the left- and right-hand sides is positive semidefinite, and the lower bound is obtained when WðxÞ ¼ U 1 ðxÞ. It is shown in Wang and Leblanc (2008) that detðUÞ ¼ s20 ðμ4 s40 Þ μ23 , where μ3 ¼ Eðε3 ∣xÞ and μ4 ¼ Eðε4 ∣xÞ, and the optimal weighting matrix is given by W ðx Þ ¼ U 1 ðx Þ ¼
1 detðUÞ
μ4 þ 4μ3 gðx; θ0 Þ þ4s20 g 2 ðx; θ0 Þ s40
μ3 2s20 gðx; θ0 Þ
μ3 2s20 gðx; θ0 Þ
s20
! ;
ð3Þ
assuming detðUÞ a0. From the definition of U above, U is always positive semidefinite. When detðUÞ a0, i.e., detðUÞ ¼ s20 ðμ4 s40 Þ μ23 a0, U is positive definite, which implies that WðxÞ ¼ U 1 ðxÞ is also positive definite. Then the asymptotic covariance matrix of the most efficient SLSE is given by 0 1 μ3 Vðθ^ SLS Þ V s^ 2SLS G2 1 g 1 4 B C μ4 s0 B C C ¼ B μ3 ð4Þ C; 1 2 2 > @ A g V s ^ G Vð s ^ Þ 2 SLS SLS 1 4 μ4 s0 where
k1 V θ^ SLS ¼ s20 k1 G2 2 g 1 g 1> s0
!1 ;
ð5Þ
ðμ s4 Þðs2 k Þ 1 0 0 ; V s^ 2SLS ¼ 4 s20 k1 g 1> G2 1 g 1
ð6Þ
and g1 ¼ E
∂gðx; θ0 Þ ; ∂θ
G2 ¼ E
∂gðx; θ0 Þ ∂gðx; θ0 Þ ; ∂θ ∂θ >
k1 ¼
μ23 : μ4 s40
ð7Þ
The expectation in g 1 and G2 is taken with respect to the distribution of x. Under similar regularity conditions, the > asymptotic covariance matrix for the OLSE γ^ OLS ¼ ðθ^ OLS ; s^ 2OLS Þ > is as follows: 0 1 0 1 s20 G2 1 μ3 G2 1 g 1 Vðθ^ OLS Þ μ3 G2 1 g 1 A¼@ A: ð8Þ D¼@ μ3 g 1> G2 1 Vðs^ 2OLS Þ μ3 g 1> G2 1 μ4 s40 It is easily seen that the SLSE and the OLSE have the same covariance matrix when μ3 ¼ 0. When μ3 a 0, the SLSE for θ and s2 is more efficient than the OLSE, since (1) Vðs^ 2OLS Þ ZVðs^ 2SLS Þ, with equality holding if and only if g 1> G2 1 g 1 ¼ 1. (2) Vðθ^ OLS Þ Vðθ^ SLS Þ is nonnegative definite if g 1> G2 1 g 1 ¼ 1, and is positive definite if g 1> G2 1 g 1 o 1. Notice that 0 r g 1> G2 1 g 1 r1 for any model, from Wang and Leblanc (2008, The Proof of Theorem 4). The SLSE can be computed using the following two-stage procedure as suggested by Wang and Leblanc (2008): minimize Q n ðγÞ using W i ¼ I 2 for i ¼ 1; …; n to obtain the first-stage estimator γ~ SLS , then estimate the elements of U i 1 ¼ U 1 ðxi Þ in (3) 1 1 using γ~ SLS and μ^ 3 and μ^4 from the residuals to obtain U^ i . Finally, minimize Q n ðγÞ again using W i ¼ U^ i to obtain the SLSE γ^ SLS . However, this procedure can be improved by doing several iterations for the second stage.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
143
3. Optimal designs based on the SLSE In this section, we will define and explore D-optimal and A-optimal designs based on the SLSE. First we show the properties of the asymptotic covariance matrix of the SLSE in the next three lemmas. We have 0 rg 1> G2 1 g 1 r1 for all regression models. Lemma 1 gives one condition that g 1> G2 1 g 1 ¼ 1. Lemma 1. Let vðxÞ ¼ ∂gðx; θ0 Þ=∂θ. If the first element of vðxÞ is 1, i.e., vðxÞ½1 ¼ 1, then G2 1 g 1 ¼ ð1; 0; …; 0Þ > ; g 1> G2 1 ¼ ð1; 0; …; 0Þ, and g 1> G2 1 g 1 ¼ 1. The proof of Lemma 1 is in the Appendix. For linear models with an intercept term, it is obvious that vðxÞ½1 ¼ 1 and g 1> G2 1 g 1 ¼ 1. For other models, it may be true that g 1> G2 1 g 1 ¼ 1 for some distributions of x. The next two Lemmas present the results for the determinant and the trace of the covariance matrix of the SLSE, and their relationships with those of the OLSE. Lemma 2. For the optimal SLSE, we have s 2ðq 1Þ ðs2 k Þq 1 0 ¼ 0 det V θ^ OLS det V θ^ SLS 1 s20 k1 g 1> G2 g 1
ð9Þ
and detðC Þ ¼
s0 2q ðs20 k1 Þq þ 1 ðμ4 s40 Þ s20 k1 g 1> G2 1 g 1
det V θ^ OLS ;
where q is the dimension of vector θ. Lemma 3. For the optimal SLSE, we have s2 k ðs2 k Þk g > G 2 g 1 1 1 1 1 2 ¼ 0 2 trace V θ^ OLS þ 0 trace V θ^ SLS s0 s20 k1 g 1> G2 1 g 1
ð10Þ
and traceðC Þ ¼
2 > 2 4 s20 k1 ^ OLS þ ðs0 k1 Þðk1 g 1 G2 g 1 þ μ4 s0 Þ : trace V θ 1 2 > s20 s0 k1 g 1 G2 g 1
The proofs of Lemmas 2 and 3 are in the Appendix. With these results, we can define optimal designs based on the SLSE and investigate their properties. A-optimal and D-optimal designs based on the SLSE are defined to minimize traceðVðθ^ SLS ÞÞ and detðVðθ^ SLS ÞÞ respectively. From (8) for Vðθ^ OLS Þ and (9) and (10) in Lemmas 2 and 3, it is easy to see that the A-optimal and D-optimal designs minimize the following loss functions: 1 þ ℓSLS A ðξÞ ¼ trace G2 ℓSLS D ðξÞ ¼
tg 1> G2 2 g 1
1 tg 1> G2 1 g 1
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
;
;
ð11Þ
where ξ is the distribution of x, and t ¼ k1 =s20 ¼ μ23 =s20 ðμ4 s40 Þ. From detðUÞ ¼ s20 ðμ4 s40 Þ μ23 4 0 in Section 2, we have SLS
s20 ðμ4 s40 Þ 4μ23 , which implies 0 r t o1. The optimal designs are denoted by ξSLS A and ξD . For nonlinear models, since g 1 and G2 often involve the unknown parameter vector θ0 , we define locally A- and SLS D-optimal designs to minimize the loss functions ℓSLS A ðξÞ and ℓD ðξÞ, respectively, given the true parameter θ0 . For linear models, g 1 and G2 do not depend on θ0 , but the optimal designs may depend on parameter t and can be locally optimal as well. For simplicity, we just call them A- and D-optimal designs for both linear and nonlinear models in the rest of the paper. Notice that the A-optimal and D-optimal designs based on the OLSE minimize traceðVðθ^ OLS ÞÞ and detðVðθ^ OLS ÞÞ respectively, where Vðθ^ OLS Þ is given in (8), so the corresponding loss functions are 1 ℓOLS ; A ðξÞ ¼ trace G 2 ℓOLS D ðξÞ ¼
1 : detðG2 Þ
ð12Þ OLS
The resulting optimal designs are denoted by ξOLS and ξD . A OLS SLS OLS When the error distribution is symmetric, i.e., μ3 ¼ 0, we have t ¼0, ℓSLS A ðξÞ ¼ ℓA ðξÞ and ℓD ðξÞ ¼ ℓD ðξÞ, and the optimal designs based on the SLSE and OLSE are the same. When μ3 a 0, it is possible that the optimal designs are still the same. The following theorem gives one such case.
144
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
Theorem 1. Let vðxÞ ¼ ∂gðx; θ0 Þ=∂θ. If the first element of vðxÞ is 1, i.e., vðxÞ½1 ¼ 1, then detðVðθ^ SLS ÞÞ ¼ s0 2ðq 1Þ ðs20 k1 Þðq 1Þ detðVðθ^ OLS ÞÞ and s2 k 1 ¼ 0 2 trace V θ^ OLS þk1 : trace V θ^ SLS s0 Thus the D-optimal and A-optimal designs based on the SLSE are the same as those based on the OLSE. The proof of Theorem 1 is in the Appendix. Here is one application of the results in Theorem 1. Consider the linear regression model in (1). The SLSE can be applied to estimate θ with gðx; θÞ ¼ z > ðxÞθ so vðxÞ ¼ ∂gðx; θ0 Þ=∂θ ¼ zðxÞ. If zðxÞ½1 ¼ 1, then by Theorem 1, the optimal designs based on the SLSE are the same as those based on the OLSE. This is a very useful result, since linear regression models often contain a constant term (also called an intercept term). The same optimal designs will yield the most efficient OLSE and SLSE; therefore one can determine which estimator to use after examining the error distribution from residual analysis. However, if zðxÞ½1 a 1, then the optimal designs based on the SLSE may be different from those based on the OLSE. This is illustrated in Example 1. Example 1. The simple linear regression model going through the origin is given by y ¼ θ1 x þ ε;
x A S:
Let ηj ¼ Eðx Þ be the jth moment of x. For this linear model, from (7), g 1 ¼ EðxÞ ¼ η1 , and G2 ¼ Eðx2 Þ ¼ η2 . So detðG2 Þ ¼ η2 and g 1> G2 1 g 1 ¼ η21 =η2 . Two design spaces S1 and S2 are considered here: (1) S1 ¼ ½ 1; 1, a symmetric interval about 0 and (2) S2 ¼ ½0; 1. j
OLS
For the OLSE, the ξD
minimizes the loss function in (12), which is
1 1 ¼ : ℓOLS D ðξÞ ¼ detðG2 Þ η2 OLS
On S1 or S2, 0 r x2 r 1, so η2 ¼ Eðx2 Þ r1 for all the distributions. Thus ℓOLS D ðξÞ Z 1, and the equality is obtained by the ξD with OLS η2 ¼ 1. There are many D-optimal designs on S1, but there is only one on S2. For S1, the ξD has two support points 71 with OLS probabilities Pðx ¼ 1Þ ¼ p0 and Pðx ¼ 1Þ ¼ 1 p0 for any 0 r p0 r 1. For S2, the ξD has one support point þ1 with probability Pðx ¼ 1Þ ¼ 1. SLS For the SLSE, the ξD minimizes the loss function in (11), which is ℓSLS D ðξ Þ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
¼
1 ; η2 tη21 SLS
with 0 r t o1. For S1, ℓSLS D ðξÞ is minimized at η1 ¼ 0 and η2 ¼ 1, and ξD has distribution Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ 1=2, which is one of the D-optimal designs based on the OLSE. For S2, since η1 ¼ EðxÞ ZEðx2 Þ ¼ η2 , and VarðxÞ ¼ η2 η21 Z 0, we need to solve the following optimization problem: minimize
1 η2 tη21
subject to
η21 r η2 r η1 r 1
ξ
The constraint about the two moments η1 and η2 is in Dette and Studden (1997, p. 4). It is clear that the loss function is SLS minimized at η2 ¼ η1 ¼ a1 ¼ minf1; 1=2tg, and the ξD has the following distribution: Pðx ¼ 0Þ ¼ 1 a1 , Pðx ¼ 1Þ ¼ a1 . If SLS OLS SLS OLS SLS 0 rt r 1=2, then a1 ¼ 1 and the ξD and ξD are the same. Otherwise, the ξD and ξD are different, and the ξD has more OLS support points than the ξD . □ OLS SLS SLS In Example 1, since g 1 and G2 are scalars, we have ℓOLS A ðξÞ ¼ ℓD ðξÞ and ℓA ðξÞ ¼ ℓD ðξÞ, and A-optimal and D-optimal designs are equivalent. For the SLSE, some optimal designs depend on parameter t, and some do not. Parameter t is related to the skewness measure δs ¼ μ3 =s30 and the kurtosis measure δk ¼ μ4 =s40 3 as follows: t ¼ δ2s =ðδk þ 2Þ. More examples of optimal designs are given in Section 6. SLS In the next two sections we study the transformation invariance and symmetry properties of the D-optimal designs ξD . SLS
4. Transformation invariance of ξD
For regression model (2), suppose that the design space is S Rp . Consider a transformation T and a distribution ξ of x on S. Denote the transformed design space as ST ¼ fTu; u A Sg Rp and the distribution of Tx as ξT . If a design ξn is a D-optimal design on S and its transformed distribution ξnT is a D-optimal design on ST, then the D-optimal design is said to be invariant under the transformation T.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
145
Theorem 2. If function gðx; θÞ and transformation T satisfy the following condition: ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ SLS
where matrix Q T is nonsingular and does not depend on x, then the D-optimal design ξD is invariant under the transformation T. The proof of Theorem 2 is in the Appendix. Although T can be any transformation, we are usually interested in linear SLS transformations of design spaces. In particular, Theorem 2 provides a sufficient condition to see if ξD is invariant under scale and/or shift transformations for a given model, and the condition is easy to verify. The next two examples are for polynomial regression models without intercept and trigonometric regression models on a SLS partial circle. We examine the invariance of ξD under scale and/or shift transformations. Optimal designs based on the OLSE for these models have been studied by many authors, for example, Chang and Heiligers (1996), Papp (2012), Dette and Studden (1997), Dette et al. (2002), and Chang et al. (2013). Example 2. Consider the qth-order polynomial regression model without intercept y ¼ xθ1 þx2 θ2 þ ⋯ þ xq θq þ ε;
ð13Þ
where x A S ¼ ½a; b. The scale transformation can be defined as Tx ¼ b0 x for b0 4 0, Then ST ¼ ½ab0 ; bb0 , and it is easy to see that 0 1 0 1 b0 x x B b2 x2 C B x2 C C ∂gðTx; θ0 Þ B ∂gðx; θ0 Þ 0 B C C ¼B ; C ¼ QT B ⋮ C ¼ QTB @ ⋮ A ∂θ ∂θ @ A q q q b0 x x 2
q
where Q T is a diagonal matrix with diagonal elements: b0 ; b0 ; …; b0 . Since Q T is nonsingular and does not depend on x, the SLS SLS D-optimal design ξD is invariant under the scale transformation for this model. From Example 1, notice that the ξD is not invariant under the shift transformation Tx ¼ x þ a0 for a0 a 0. □ Example 3. Consider the qth-order trigonometric regression model y ¼ sin ðxÞθ1 þ cos ðxÞθ2 þ ⋯ þ sin ðqxÞθ2q 1 þ cos ðqxÞθ2q þ ε;
ð14Þ
where x A S ¼ ½a; b. In Chang et al. (2013), D-optimal designs are constructed for the first-order model (q ¼1) without SLS OLS intercept. Here we also assume that the model does not have an intercept term, otherwise the ξD is the same as the ξD by Theorem 1. The shift transformation is Tx ¼ x þ a0 , for a0 a0. Then ST ¼ ½a þ a0 ; bþ a0 , and ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ where Q T is a block diagonal matrix with block matrices Q 1 ; …; Q q , and ! sin ðja0 Þ cos ðja0 Þ Qj ¼ ; for j ¼ 1; …; q: sin ðja0 Þ cos ðja0 Þ It is clear that detðQ j Þ ¼ 1, so detðQ T Þ ¼ ∏qj¼ 1 detðQ j Þ ¼ 1. Since Q T is nonsingular and does not depend on x, the D-optimal SLS OLS design ξD is invariant under the shift transformation. The same result for the ξD can be found in Dette et al. (2002). From OLS Chang et al. (2013), the ξD is not invariant under the scale transformation for this model with q ¼1, which implies that the SLS ξD is not scale invariant either. □ Example 4. Consider the exponential regression model in Dette et al. (2006) y ¼ θ1 e θ2 x þ θ3 e θ4 x þ ε; where x A S ¼ ½a; b. This is a nonlinear regression model. Assume that the true parameter vector value is θ0 ¼ ðθ10 ; θ20 ; θ30 ; θ40 Þ > . The shift transformation is Tx ¼ x þ a0 , for a0 a 0. Then ST ¼ ½a þa0 ; b þ a0 , and 0 1 e θ20 ðx þ a0 Þ B C θ10 ðx þa0 Þe θ20 ðx þ a0 Þ C ∂gðTx; θ0 Þ B C ¼ Q T ∂gðx; θ0 Þ ; ¼B θ 40 ðx þ a0 Þ B C e ∂θ ∂θ @ A θ40 ðx þ a0 Þ θ30 ðx þa0 Þe where
0
e θ20 a0 B B θ10 a0 e θ20 a0 QT ¼ B B 0 @ 0
0 e
θ 20 a0
0 0
e
0
0
0
0
θ40 a0
θ30 a0 e θ40 a0
0 e θ40 a0
1 C C C: C A
146
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151 SLS
It is obvious that matrix Q T does not depend on x and is nonsingular. Thus the D-optimal design ξD is invariant under the shift transformation for this model, which implies that we can assume the design space S ¼ ½0; b a to construct D-optimal designs. It is easy to see that the result holds for the general exponential regression model, y ¼ ∑qj¼ 1 θ2j 1 e θ2j x þ ε. □ SLS
5. Symmetry of ξD
SLS
We examine the symmetry of ξD if the design space S is symmetric about 0. For simplicity, we only present the result for the univariate case. However the methodology is quite general, and the result can be extended to multivariate design spaces easily. For the univariate case, the design space is assumed to be S ¼ ½ a; a for a 4 0. Consider transformation Tx ¼ x. Since S is symmetric about 0, ST ¼ S. For any distribution ξ0 ðxÞ of x on S, ξ1 ðxÞ is defined to be the distribution of x, and let distribution ξλ ðxÞ ¼ ð1 λÞξ0 ðxÞ þ λξ1 ðxÞ, for any λ A ½0; 1. Notice that ξ0:5 ðxÞ ¼ 0:5ξ0 ðxÞ þ 0:5ξ1 ðxÞ is a symmetric distribution about 0. To specify the distribution in the expectation in g 1 and G2 , we use the following notation. Define, for any λ A ½0; 1, ∂gðx; θ0 Þ ; g 1;ξλ ¼ Eξλ ∂θ ∂gðx; θ0 Þ ∂gðx; θ0 Þ G2;ξλ ¼ Eξλ ; > ∂θ ∂θ dðλÞ ¼ detðG2;ξλ Þ;
ð15Þ
1 > hðλÞ ¼ g 1;ξ G2;ξ g : λ λ 1;ξλ
The symmetry property of
ð16Þ SLS ξD
is given in the following theorem.
Theorem 3. Consider a regression model and a design space S that is symmetric about 0. There exists a symmetric D-optimal SLS design ξD , if dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, where functions d and h are defined in (15) and (16) respectively. The proof of Theorem 3 is in the Appendix. This theorem also provides a sufficient condition to check for the existence of SLS a symmetric ξD . SLS The next two examples show that there exist symmetric D-optimal designs ξD for the models in Examples 2 and 3 if the design space S is symmetric. SLS
Example 5. Consider the symmetry of ξD for model (13) in Example 2 with a symmetric design space S. From Example 2, SLS we know that ξD is scale invariant, so without loss of generality, the symmetric design space is assumed to be S ¼ ½ 1; 1. SLS To show the existence of a symmetric ξD for this model, we verify the sufficient condition, dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, as follows. From Example 2, we have 20 13 20 13 x x 6 B C7 2 6B x2 C7 6B ð xÞ C7 ∂gðx; θ0 Þ 6B C7 B C7 ¼ Eξ1 6B C7 ¼ Eξ0 6 g 1;ξ1 ¼ Eξ1 6B ⋮ C7 ¼ Pg 1;ξ0 ; 4@ ⋮ A5 ∂θ 4@ A5 ð xÞq xq where P is a diagonal matrix with diagonal elements as ð 1Þ; ð 1Þ2 ; …; ð 1Þq . Similarly, we can show that G2;ξ1 ¼ PG2;ξ0 P > : From the above results and the definitions of dðλÞ and hðλÞ in (15) and (16) respectively, we have dð0Þ ¼ detðG2;ξ0 Þ ¼ detðG2;ξ1 Þ ¼ dð1Þ;
and
1 1 > > G2;ξ g ¼ g 1;ξ G2;ξ g ¼ hð1Þ: hð0Þ ¼ g 1;ξ 0 0 1;ξ0 1 1 1;ξ1 SLS
Thus there exists a symmetric ξD for this model. SLS
□ SLS
Example 6. Consider the symmetry of ξD for model (14) in Example 3, where x A S ¼ ½ a; a for a 40. Since ξD is shift invariant for this model, we can always assume, without loss of generality, the design space S is symmetric about 0. It is easy to verify that ∂gðx; θ0 Þ ¼ Pg 1;ξ0 ; g 1;ξ1 ¼ Eξ1 ∂θ
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
147
where P is a diagonal matrix with diagonal elements as 1; 1; …; 1; 1, and G2;ξ1 ¼ PG2;ξ0 P > : SLS
Then it is obvious that dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ. Thus there exists a symmetric ξD for this model.
□
SLS ξD
for a given model with From Examples 5 and 6, we have the following result for the existence of a symmetric symmetric design space. The proof of Corollary 1 is omitted, since it is similar to the derivation in Examples 5 and 6. Corollary 1. Consider a model with a symmetric design space. If function gðx; θÞ satisfies the following condition: ∂gð x; θ0 Þ ∂gðx; θ0 Þ ¼P ; ∂θ ∂θ SLS
where matrix P does not depend on x and has detðPÞ ¼ 71, then there exists a symmetric D-optimal design ξD for the model. OLS
All the results in Theorems 2 and 3 and Corollary 1 can also be applied to ξD and symmetry properties.
to check for the transformation invariance
6. Examples SLS
SLS
OLS
SLS
Here we will construct ξD for a couple of models, and compare the ξD with ξD . The ξD in Example 7 depends on the SLS SLS value of parameter t, while in Example 8 the ξD does not. The minimum number of support points of ξD is usually bigger OLS than that of ξD . SLS
Example 7. Construct ξD for the second-order polynomial regression model without intercept: y ¼ xθ1 þx2 θ2 þ ε;
x A S ¼ ½ 1; 1:
Let ηj ¼ Eðx Þ be the jth moment of x. Then g 1 ¼ j
Example 5, we will construct a symmetric becomes ℓSLS D ðξÞ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
¼
SLS ξD
η1 η2
, and G2 ¼
η2 η3 η3 η4
SLS
. Since there exists a symmetric ξD on S ¼ ½ 1; 1 from
here. For a symmetric design, we have η1 ¼ η3 ¼ 0 and the loss function in (11)
1 ; η2 η4 tη32
where parameter t A ½0; 1Þ. On S ¼ ½ 1; 1, the even moments of any distribution satisfy 0 rη22 r η4 r η2 r 1 from Dette and SLS Studden (1997). Then we solve the following optimization problem to find a ξD : minimize
1 η2 η4 tη32
subject to
0 rη22 rη4 rη2 r1:
ξ
SLS
It is easy to show that the loss function is minimized at η2 ¼ η4 ¼ a2 ¼ minf1; 2=3tg and a symmetric ξD has the following distribution: Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ a2 =2; Pðx ¼ 0Þ ¼ 1 a2 . When x¼0, the model becomes y ¼ ε. The observations at x¼ 0 provide information on s2 which is estimated together with θ1 and θ2 in the SLSE. OLS OLS To compare with a symmetric ξD , notice that ξD minimizes ℓOLS D ðξÞ ¼
1 1 ¼ : detðG2 Þ η2 η4 η23 OLS
It is clear that ℓOLS has distribution Pðx ¼ 1Þ ¼ Pðx ¼ 1Þ ¼ 1=2. If t r2=3, D ðξÞ is minimized at η2 ¼ η4 ¼ 1 and η3 ¼ 0. The ξD OLS SLS SLS OLS then the ξD and ξD are the same. Otherwise, the ξD has one more support point than the ξD . □ SLS
Example 8. Construct ξD for the first-order trigonometric regression model without intercept: y ¼ sin ðxÞθ1 þ cos ðxÞθ2 þ ε;
x A S ¼ ½ π; π:
Define the following expectations to work on the optimal designs: v1 ¼ E½ sin ðxÞ; v2 ¼ E½ cos ðxÞ; v3 ¼ E½ sin 2 ðxÞ; v4 ¼ E½ cos 2 ðxÞ, v5 ¼ E½ sin ðxÞ cos ðxÞ. Then g 1 ¼ vv , and G2 ¼ vv vv . From Chang et al. (2013), there are infinitely many D-optimal 0
OLS OLS designs ξD based on the OLSE, and all the ξD have G2 ¼ 0:5 . Here are three optimal designs: 0 0:5
π π (1) ξOLS D1 :p x ¼ 4 ¼ p x ¼ 4 ¼ 1=2,
π π (2) ξOLS D2 : p x ¼ 4 ¼ p x ¼ 4 ¼ p x ¼
3π π (3) ξOLS D3 : p x ¼ 8 ¼ p x ¼ 8 ¼ 1=2.
3π 4
1
3
5
2
5
4
¼ p x ¼ 3π 4 ¼ 1=4,
SLS
The ξD minimizes the loss function in (11), and for this model the ℓSLS D ðξÞ is minimized when detðG2 Þ is maximized and g 1> G2 1 g 1
OLS
is minimized. From ξD , detðG2 Þ is maximized at v3 ¼ v4 ¼ 1=2 and v5 ¼ 0, and it is easy to show that g 1> G2 1 g 1 is
148
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151 SLS
minimized at v1 ¼ v2 ¼ 0. Among the above three designs, only ξOLS D2 has v1 ¼ v2 ¼ v5 ¼ 0 and v3 ¼ v4 ¼ 1=2, so it is a ξD . In SLS fact, there are also infinitely many ξD for this model including the following designs with four support points: ξSLS Dα : pðx ¼ π=4 þ αÞ ¼ pðx ¼ π=4 þ αÞ ¼ pðx ¼ 3π=4 þαÞ ¼ pðx ¼ 3π=4 þ αÞ ¼ 1=4, for any α A ½ π=4; π=4. Notice that all these optimal designs do not depend on t. Since the model in this example is a specific case from Example 6 when q ¼1 and a ¼ π, there exists a symmetric design about 0. Here it is clear that, with α ¼ 0, ξSLS Dα is a symmetric design about 0. SLS OLS Since all the D-optimal designs ξD have v3 ¼ v4 ¼ 1=2 and v5 ¼ 0 and maximize detðG2 Þ, all of them are ξD . However, OLS SLS OLS SLS only some of the D-optimal designs ξD are ξD . In particular the ξD with two support points are not ξD . □ 7. Discussion New optimal design criteria are proposed based on the SLSE, since the SLSE is more efficient than the OLSE when the error distribution has nonzero third moment. The new design criteria provide good alternatives to the ones based on the OLSE, and many new optimal designs can be constructed for linear and nonlinear regression models. Sufficient conditions are derived to check for transformation invariance and symmetry of D-optimal designs for a given SLS OLS model. These results can be applied to both ξD and ξD , and they are easy to verify and are very useful in practice. As usual, it is hard to obtain similar results for A-optimal designs, which can be a future research topic. The optimal designs based on the SLSE and OLSE are the same for some models. When they are different, the optimal designs based on the SLSE tend to have more support points than those based on the OLSE. Several examples are given for SLS polynomial, trigonometric and exponential regression models. Optimal designs ξD may depend on parameter t. In the case SLS SLS SLS that ξD does not depend on t, it is easy to use it in practice. If ξD does depend on t, sensitivity analysis of ξD can be SLS performed. If ξD is not sensitive to small changes of parameter t, then we do not need very accurate information of t to implement the design. With the new optimal design criteria, constructing optimal designs for some regression models and design spaces produces challenging research problems. For some models, theoretical results may be obtained by using moment theory and convex optimization techniques. For complicated models, the design problems can be simplified and solved by only finding the optimal weights of a set of given support points. In addition, we can define and study other design criteria based on the SLSE.
Acknowledgements This research work is partially supported by Discovery Grants from the Natural Science and Engineering Research Council of Canada. The authors are grateful to the Editor and referees for their helpful comments and suggestions. Appendix A. Proofs Proof of Lemma 1. Since vðxÞ½1 ¼ 1, from (7) we have g 1 ¼ E½vðxÞ ¼ E ð1; …Þ > ¼ ð1; …Þ > ; ∂gðx; θ0 Þ ∂gðx; θ0 Þ G2 ¼ E ¼ E vðxÞv > ðxÞ ¼ g 1 ; … : > ∂θ ∂θ Thus, from G2 1 G2 ¼ I we have G2 1 ½g 1 ; ⋯ ¼ I, which implies G2 1 g 1 ¼ ð1; 0; …; 0Þ > and g 1> G2 1 ¼ ð1; 0; …; 0Þ. Therefore, g 1> G2 1 g 1 ¼ ð1; 0; …; 0Þð1; …Þ > ¼ 1. □ Proof of Lemma 2. From (5), ¼ ðs20 k1 Þq det G2 1 det I q þ det V θ^ SLS ¼ ðs20 k1 Þq det
G2 1 1 þ
!
k1
1=2
s20 k1 g 1> G2 1 g 1 k1
G2
1=2
g 1 g 1> G2
!
g 1> G2 1 g 1
s20 k1 g 1> G2 1 g 1 2 s0 ¼ s0 2q ðs20 k1 Þq det V θ^ OLS 1 2 > s0 k1 g 1 G2 g 1 2ðq 1Þ 2 s ðs0 k1 Þq ¼ 0 det V θ^ OLS : 1 2 > s0 k1 g 1 G2 g 1 From (4) and (17), we have V s^ 2SLS detðC Þ ¼ det V θ^ SLS
k1 Vðs^ 2SLS Þ2 g 1> G2 1 Vðθ^ SLS Þ 1 G2 1 g 1 μ4 s40
ð17Þ
!
¼ det V θ^ SLS
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
V s^ 2SLS
k1 1 Vðs^ 2SLS Þ2 g 1> G2 1 2 s0 k1 μ4 s40
G2
149
! ! k1 1 > g g g G ¼ det V θ^ SLS 1 1 1 2 2 s0 !!
k ðμ s4 Þðs2 k Þ k1 1 4 1 0 0 V s^ 2SLS g 1> G2 1 g 1 1 2 g 1> G2 1 g 1 s0 ðs20 k1 g 1> G2 1 g 1 Þ2 ! k1 > 1 2 ^ ¼ det V θ SLS V s^ SLS 1 2 g 1 G2 g 1 s0 2q 2 q 2 det V θ^ OLS ¼s ðs k1 Þ V ðs^ 0
0
¼
SLS
s0 2q ðs20 k1 Þq þ 1 ðμ4 s40 Þ s20 k1 g 1> G2 1 g 1
det V θ^ OLS :
□
Proof of Lemma 3. From Wang and Leblanc (2008, the proof of Theorem 4), we get
k1 V θ^ SLS ¼ s20 k1 G2 1 þ V s^ 2SLS G2 1 g 1 g 1> G2 1 ; 4 μ4 s0 which leads to
trace V θ^ SLS ¼ s20 k1 trace G2 1 þ
k1 V s^ 2SLS trace G2 1 g 1 g 1> G2 1 4 μ4 s0 s20 k1 ðs20 k1 Þk1 trace V θ^ OLS þ trace G2 1 g 1 g 1> G2 1 ¼ 2 1 s0 s20 k1 g 1> G2 g 1 2 s k1 ðs2 k1 Þk1 g 1> G2 2 g 1 trace V θ^ OLS þ 0 : ¼ 0 2 s0 s20 k1 g 1> G2 1 g 1
ð18Þ
From (4) and (18),
traceðC Þ ¼ trace V θ^ SLS þ V s^ 2SLS 2 > 2 s20 k1 ^ OLS þ ðs0 k1 Þk1 g 1 G2 g 1 þ V s^ 2 trace V θ SLS s20 s20 k1 g 1> G2 1 g 1 2 s k1 ðs2 k1 Þk1 g 1> G2 2 g 1 ðμ4 s40 Þðs20 k1 Þ trace V θ^ OLS þ 0 þ ¼ 0 2 s0 s20 k1 g 1> G2 1 g 1 s20 k1 g 1> G2 1 g 1 ðs2 k Þðk g > G 2 g þμ s4 Þ s2 k1 1 1 1 4 1 2 0 ¼ 0 2 trace V θ^ OLS þ 0 : □ s0 s20 k1 g 1> G2 1 g 1 ¼
Proof of Theorem 1. From Lemma 1, we have g 1> G2 1 g 1 ¼ 1 and g 1> G2 2 g 1 ¼ 1. Putting these results in Lemmas 2 and 3 gives the results for detðVðθ^ SLS ÞÞ and traceðVðθ^ SLS ÞÞ. □ SLS
Proof of Theorem 2. On S, from (11), ξD minimizes ℓ0 ðξÞ ¼
1 detðG2 Þð1 tg 1> G2 1 g 1 Þ
:
Since ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QT ; ∂θ ∂θ we have g 1;T ≔E
∂gðTx; θ0 Þ ∂gðx; θ0 Þ ¼ QTE ¼ Q T g1; ∂θ ∂θ
G2;T ≔E
∂gðTx; θ0 Þ ∂gðTx; θ0 Þ ∂gðx; θ0 Þ ∂gðx; θ0 Þ ¼ QTE Q T> ¼ Q T G2 Q T> : > > ∂θ ∂θ ∂θ ∂θ SLS
1 > G2;T g 1;T ¼ g 1> G2 1 g 1 . On ST, ξD minimizes the following function: Then it is easy to verify that g 1;T
ℓ1 ðξT Þ ¼
1 > G 1g detðG2;T Þð1 tg 1;T 2;T 1;T Þ
150
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
1
¼
ðdetðQ T ÞÞ2 detðG2 Þð1 tg 1> G2 1 g 1 Þ ℓ0 ðξÞ : □ ¼ ðdetðQ T ÞÞ2 SLS
It is clear that if ξn minimizes ℓ0 ðξÞ, then ξnT minimizes ℓ1 ðξT Þ. Thus ξD is invariant under transformation T. Proof of Theorem 3. If dð0Þ ¼ dð1Þ and hð0Þ ¼ hð1Þ, we can show that (i) dðλÞ Zdð0Þ and (ii) hðλÞ r hð0Þ, for any λ A ½0; 1. (i) Since ξλ ðxÞ ¼ ð1 λÞξ0 ðxÞ þλξ1 ðxÞ, we have g 1;ξλ ¼ ð1 λÞg 1;ξ0 þλg 1;ξ1 ;
ð19Þ
G2;ξλ ¼ ð1 λÞG2;ξ0 þ λG2;ξ1 :
ð20Þ
From (20) and Minkowski's inequality in Horn and Johnson (1985, p. 482), we get ðdðλÞÞ1=q ¼ ðdetðG2;ξλ ÞÞ1=q ¼ ðdetðð1 λÞG2;ξ0 þ λG2;ξ1 ÞÞ1=q Zðdetðð1 λÞG2;ξ0 ÞÞ1=q þ ðdetðλG2;ξ1 ÞÞ1=q ¼ ð1 λÞdð0Þ1=q þ λdð1Þ1=q ¼ dð0Þ1=q ;
since dð0Þ ¼ dð1Þ:
Thus, for any λ A ½0; 1, we have dðλÞ Z dð0Þ:
ð21Þ
(ii) It is clear that both g 1;ξλ and G2;ξλ are linear functions of λ from (19) and (20). Function hðλÞ is defined as
1 > hðλÞ ¼ g 1;ξ G2;ξ g , and it is a convex function of λ by the result in Wiens (1993, Lemma 2). Therefore, using the convex λ 1;ξλ λ
property and hð0Þ ¼ hð1Þ gives
hðλÞ rð1 λÞhð0Þ þλhð1Þ ¼ hð0Þ:
ð22Þ
SLS
Now, for the SLSE, ξD minimizes the loss function in (11). For any design ξ0, using the results in (21) and (22) with λ ¼ 0:5, we have ℓSLS D ðξ0:5 Þ ¼
1 1 > G2;ξ g Þ detðG2;ξ0:5 Þð1 tg 1;ξ 0:5 1;ξ0:5 0:5
1 dð0:5Þð1 thð0:5ÞÞ 1 r dð0Þð1 thð0ÞÞ ¼
¼ ℓSLS D ðξ0 Þ; which implies that the loss function at the symmetric design ξ0:5 is smaller than or equal to that at design ξ0. Thus the minimum value of ℓSLS D ðξÞ can be always achieved by a symmetric design. □ References Abarin, T., Wang, L., 2009. Second-order least squares estimation of censored regression models. J. Stat. Plan. Inference 139, 125–135. Chang, F.C., Heiligers, B., 1996. E-optimal designs for polynomial regression without intercept. J. Stat. Plan. Inference 55, 371–387. Chang, F.C., Imhof, L., Sun, Y.Y., 2013. Exact D-optimal designs for first-order trigonometric regression models on a partial circle. Metrika 76, 857–872. Chen, R.B., Wong, W.K., Li, K.Y., 2008. Optimal minimax designs over a prespecified interval in a heteroscedastic polynomial model. Stat. Probab. Lett. 78, 1914–1921. Chen, X., Tsao, M., Zhou, J., 2012. Robust second-order least-squares estimator for regression models. Stat. Pap. 53, 371–386. Dette, H., Melas, V.B., Pepelyshev, A., 2002. D-optimal designs for trigonometric regression models on a partial circle. Ann. Inst. Stat. Math. 54, 945–959. Dette, H., Melas, V.B., Wong, W.K., 2006. Locally D-optimal designs for exponential regression models. Stat. Sin. 16, 789–803. Dette, H., Studden, W.J., 1997. The Theory of Canonical Moments with Applications in Statistics, Probability and Analysis. Wiley, New York. Fang, Z., Wiens, D.P., 2000. Integer-valued, minimax robust designs for estimation and extrapolation in heteroscedastic, approximately linear models. J. Am. Stat. Assoc. 95, 807–818. Fedorov, V.V., 1972. Theory of Optimal Experiments. Academic Press, New York. Horn, R.A., Johnson, C.R., 1985. Matrix Analysis. Cambridge University Press, Cambridge. Huber, P.J., 1981. Robust Statistics. Wiley, New York. Li, K.C., Notz, W., 1982. Robust designs for nearly linear regression. J. Stat. Plan. Inference 6, 135–151. Notz, W.I., 1989. Optimal designs for regression models with possible bias. J. Stat. Plan. Inference 22, 43–54. Papp, D., 2012. Optimal designs for rational function regression. J. Am. Stat. Assoc. 107, 400–411. Pukelsheim, F., 1993. Optimal Design of Experiments. Wiley, New York. Wang, L., 2003. Estimation of nonlinear Berkson-type measurement error models. Stat. Sin. 13, 1201–1210.
L.L. Gao, J. Zhou / Journal of Statistical Planning and Inference 149 (2014) 140–151
151
Wang, L., 2004. Estimation of nonlinear models with Berkson-type measurement errors. Ann. Stat. 32, 2559–2579. Wang, L., Leblanc, A., 2008. Second-order nonlinear least squares estimation. Ann. Inst. Stat. Math. 60, 883–900. Wiens, D.P., 1992. Minimax designs for approximately linear regression. J. Stat. Plan. Inference 31, 353–371. Wiens, D.P., 1993. Designs for approximately linear regression: maximizing the minimum coverage probability of confidence ellipsoids. Can. J. Stat. 21, 59–70. Wiens, D.P., Zhou, J., 1999. Minimax designs for approximately linear models with AR(1) errors. Can. J. Stat. 27, 781–794. Zhou, J., 2001. Integer-valued, minimax robust designs for approximately linear models with correlated errors. Commun. Stat.: Theory Methods 30, 21–39.