- Email: [email protected]
On a problem with singularity in comparison of linear experiments
Journal of Statistical Planning and Inference 141 (2011) 2489–2493
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
Short communication
On a problem with singularity in comparison of linear experiments Czes"aw St˛epniak ´w, Al. Rejtana 16 A, 35-959 Rzeszo ´w, Poland Institute of Mathematics, University of Rzeszo
a r t i c l e i n f o
abstract
Article history: Received 23 July 2010 Received in revised form 22 January 2011 Accepted 26 January 2011 Available online 3 February 2011
For consistency, the parameter space in the Gauss–Markov model with singular covariance matrix is usually restricted by observation vector. This restriction arises some difficulties in comparison of linear experiments. To avoid it we reduce the problem of comparison from singular to nonsingular case. & 2011 Elsevier B.V. All rights reserved.
Keywords: Gauss–Markov model Trivial/nontrivial deterministic part Induced model Comparison of experiments
1. Introduction and notation The term statistical experiment is usually identified with a family of probability measures. Comparison of experiments refers to a partial ordering of such families involving a common parameter. In practice the ordering is induced by a class of statistical problems concerning the parameter. In consequence, a wider class of problems leads to a stronger ordering. For a comprehensive information in this subject we refer to the well known book by Torgersen (1999) and to some review articles by Goel and Ginebra (2003) and Heyer (2006). One of the most attractive ways of the ordering of experiments based on convolution, originated by Lehmann (1959), was successfully applied to reliability problems by Shaked and Sua´rez-Llorens (2003). In this note we restrict ourselves to linear experiments, where the whole information reduces to the first two moments of the observation vector. Traditionally, matrix algebra remains the main tool in this area. Throughout this paper the usual vector–matrix notation is used. The space of all n 1 vectors is denoted by Rn and, if S is a set of in Rn then sp(S) will denote its span, i.e. the minimal linear space including S. If M is a matrix then Mu, R(M), N(M) and r(M) denote, respectively, its transposition, range (column space), kernel (null space) and rank. By M and M + is denoted by a generalized inverse and the Moore–Penrose generalized inverse of M. The symbol PM stands for the orthogonal projector onto R(M), i.e. the square matrix P satisfying the conditions Px ¼ x for x 2 RðMÞ and zero for x 2 NðMuÞ. Moreover, if M is a square matrix then tr M denotes its trace and MZ 0 means that M is symmetric and nonnegative definite (nnd, for short). If M is nnd then the symbol M1=2 stands for the matrix satisfying the condition M1=2 M1=2 ¼ M. Let X be n 1 random vector with the expectation EX ¼ Ab and the covariance matrix sV, where A and V are known matrices of n p and n n, respectively, while b and s are unknown parameters. This kind information about X is commonly called the Gauss–Markov model and is denoted by LðAb, sVÞ. The fact that X is subject to the model LðAb, sVÞ will be shortly expressed in the form X LðAb, sVÞ. In the situation when V is singular, many authors, among others Rao (1973, Section 4.9.2) and Seely and Zyskind (1971) suggest to restrict the space of possible parameters b by the observation vector X. The suggested space for b has the form O ¼ fb 2 Rp : CuAb ¼ CuXg, where C is any matrix with n rows such that RðCÞ ¼ RðVÞ. As noted in Seely and Zyskind (1971)
E-mail address: [email protected] 0378-3758/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.jspi.2011.01.022
2490
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
this approach is not acceptable from formal point of view. Moreover, it is inconvenient for comparison of linear experiments since they need have the same parameters (see, e.g. Torgersen, 1999; St˛epniak and Torgersen, 1981; St˛epniak, 1983; St˛epniak et al., 1984). To be more precise, the above-mentioned inconvenience does not refer to all Gauss–Markov models with singular covariance matrix, but only to the models with so-called nontrivial deterministic part. This term, introduced by Torgersen (1984), was investigated in detail in St˛epniak (1999). The idea of this note consists in reducing consideration from singular to nonsingular Gauss–Markov models. Not only this approach leads to convenient tools in comparison of linear experiments but it also opens a new effective way to known results. To demonstrate it, we need several auxiliary results. 2. Auxiliary results Usually we are interested in estimation of a parametric function jðbÞ ¼ cub, where c 2 Rp , by a linear functional duX, of the observation vector X. An estimator duX is said to be unbiased for j if EðduXÞ ¼ jðbÞ for all b. If such estimator exists then the parametric function j is said to be estimable. In the context of the model LðAb, sVÞ we have EðduXÞ ¼ duAb. Thus j ¼ cub is estimable, if and only if c 2 RðAuÞ. In consequence, any unbiased estimator of cub may be presented in the form duX, where d is any solution of the linear equation Aux ¼ c. It is well known that for any estimable cub in the model LðAb, sVÞ there exists a linear unbiased estimator with minimal variance, called the best linear unbiased estimator (BLUE). This estimator may be characterized by a well known Lehmann– Scheffe´ theorem. Since this very important theorem is rather rarely used in statistical literature we shall present it in a simplified form with a complete proof. ´, 1950, Theorem 5.3). Let D ¼ ff ðXÞg be a class of potential estimators. If D constitutes a linear Theorem 1 (Lehmann and Scheffe space then a member f1(X) of this class is a minimum variance unbiased estimator of its expectation in D, if and only if, Covðf1 ðXÞ,f0 ðXÞÞ ¼ 0 for any f0 2 D such that Eðf0 ðXÞÞ ¼ 0 for all b. Proof. Denote by j the expectation of f1 ðXÞ and let U, where U D D, be the class of all unbiased estimators of j. Then f ¼ f ðXÞ belongs to U, if and only if, it can be represented in the form f ¼ f1 þ af0 , where Eðf0 ðXÞÞ ¼ 0, and a 2 R. In consequence, Varðf ðXÞÞ ¼ Var½f1 ðXÞ þ af0 ðXÞ: Thus, Varðf1 ðXÞÞ is minimal, if and only if, Varðf1 ðXÞÞ r a2 Varðf0 ðXÞÞ þ2a Covðf1 ðXÞ,f0 ðXÞÞ þ Varðf1 ðXÞÞ This implies the desired result.
for all a 2 R:
&
From Theorem 1 we get immediately the following corollary. Corollary 2 (Zyskind, 1967, Theorem 3). In the context of the Gauss–Markov model LðAb, sVÞ a linear functional duX is a BLUE of its expectation, if and only if, Vd 2 RðAÞ. We shall say that the Gauss–Markov model LðAb, sVÞ with observation vector X has trivial deterministic part if for any d 2 Rn , the condition VarðduXÞ ¼ 0 implies EðduXÞ ¼ 0. It is worth to note that any model with nontrivial deterministic part has singular covariance matrix, but not vice versa, for instance if 2 3 2 3 1 0 1 1 0 6 7 6 7 A ¼ 4 1 0 5 and V ¼ 4 1 1 0 5: 0 2 0 0 1 In the further consideration we need the following results. Lemma 3 (St˛epniak, 1999, Lemma 1). Let X be subject to the Gauss–Markov model LðAb, sVÞ. Then (a) A linear functional duX has zero variance, if and only if d 2 NðVÞ. (b) A parametric function cub is estimable with zero variance, if and only if c 2 AuNðVÞ. Lemma 4 (St˛epniak, 1999, Lemma 3). In the context of the model LðAb, sVÞ the following are equivalent: (a) The model LðAb, sVÞ has trivial deterministic part. (b) AuNðVÞ ¼ f0g. (c) RðAÞ DRðVÞ. Lemma 5 (Rao, 1973, p. 300). In the context of the Gauss–Markov model LðAb, sVÞ with trivial deterministic part for any estimable j ¼ cub the variance of its BLUE may be presented in the form ^ Þ ¼ cuðAuV AÞ c: Varðj
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
2491
Lemma 6 (St˛epniak, 1985, Theorem 1). For any symmetric nnd matrices M1 and M2 of the same order the following are equivalent: (a) M1 M2 is nnd, (b) RðM1 Þ + RðM2 Þ and xuðM 2 M1 Þx Z 0 for all x 2 RðM2 Þ.
3. A model with trivial deterministic part induced by singular Gauss–Markov model Let LðAb, sVÞ be the Gauss–Markov model with nontrivial deterministic part and let Q be the orthogonal projector from Rp onto the space AuNðVÞ ¼ R½AuðIn PV Þ. We note that AuNðVÞ represents the space of coefficients of the parametric functions in LðAb, sVÞ being estimable with zero variance. Let us introduce an auxiliary model LðA1 b, sVÞ induced by LðAb, sVÞ, where A1 ¼ AðIp Q Þ:
ð1Þ
This model may be considered as a submodel of LðAb, sVÞ with b 2 RðIp Q Þ. It has some interesting properties. Lemma 7 (St˛epniak, 1999, Lemma 7). Let LðAb, sVÞ be arbitrary Gauss–Markov model and let A1 be defined by (1). Then (a) The model LðA1 b, sVÞ has trivial deterministic part. (b) rðVÞrðA1 Þ ¼ rðV þ AAuÞrðAÞ. Theorem 8. Let j ¼ cub be an estimable parametric function in the initial model LðAb, sVÞ with observation vector X. Then j may be presented in the form cu1 b þ cu2 b, where (a) cu1 b is estimable in the induced model LðA1 b, sVÞ and its BLUE in this model coincides with one in the initial model LðAb, sVÞ. (b) cu2 b is estimable in the initial model LðAb, sVÞ with zero variance. (c) cu1 c2 ¼ 0. Proof. If cub is estimable in LðAb, sVÞ then c ¼ Aud for some d 2 Rn . Defining c1 ¼ ðIp Q ÞAud and c2 ¼ QAud we get c ¼ c1 þ c2 and cu1 c2 ¼ 0. Moreover c1 2 RðAu1 Þ and c2 2 AuNðVÞ. Thus cu1 b is estimable in LðA1 b, sVÞ and, if duX is its BLUE in LðA1 b, sVÞ then it is also a BLUE in LðAb, sVÞ since Vd 2 R½AðIp Q Þ implies Vd 2 RðAÞ. Moreover, by Lemma 3, cu2 b is estimable in the initial model LðAb, sVÞ with zero variance. & By Theorem 8 the variance of the BLUE for any estimable parametric function cub depends on c only through its component c1. In consequence, from Lemma 5 we get the following corollary. Corollary 9. In the context of arbitrary model LðAb, sVÞ for any estimable parametric function j ¼ cub the variance of its BLUE may be presented in the form ^ Þ ¼ cu½ðIp Q ÞAuV AðIp Q Þ c, Varðj where Q is the orthogonal projector from Rp onto AuNðVÞ. 4. Application to comparison of linear experiments Consider two linear experiments with observation vectors X 2 Rn and Y 2 Rm such that X LðAb, sVÞ and Y LðBb, sWÞ. We shall say that the experiment designed by LðAb, sVÞ is at least as good (for linear estimation) as one designed by LðBb, sWÞ [notation: LðAb, sVÞgLðBb, sWÞ], if for any parametric function c ¼ cub and for any b 2 Rm there exists a 2 Rn such that Eb, s ðauXcÞ2 r Eb, s ðbuYcÞ2
for all b and s:
Lemma 10 (St˛epniak, 1983, Lemma 1). LðAb, sVÞgLðBb, sWÞ, if and only if, for any b 2 Rm there exists a 2 Rn such that EðauXÞ ¼ EðbuYÞ
and
VarðauXÞ rVarðbuYÞ for all b and s:
It is well known (see, e.g. Ste˛ pniak et al., 1984) that the relation LðAb, sVÞgLðBb, sWÞ with possible singular V and/or W may be expressed in the algebraic form AuðV þAAuÞ ABuðWþ BBuÞ B Z 0:
ð2Þ
2492
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
It appears that the approach used in Section 3 may be used to derive a more convenient characterization of the relation g than (2). To this aim we shall use the following lemmas. Lemma 11. For arbitrary models LðAb, sVÞ and LðBb, sWÞ with trivial deterministic parts the relation LðAb, sVÞ gLðBb, sWÞ holds if and only if AuV ABuW B Z 0. Proof. By the conditions RðAÞ D RðVÞ and RðBÞ DRðWÞ the matrices M1 ¼ AuV A and M2 ¼ BuW B do not depend on the choice of the generalized inverses V and W . Moreover, by setting V ¼ V þ and W ¼ W þ we can present them in the form M1 ¼ ½AuðV þ Þ1=2 ½AuðV þ Þ1=2 u and M2 ¼ ½BuðW þ Þ1=2 ½BuðW þ Þ1=2 u. Since R½AuðV þ Þ1=2 ¼ RðAuVÞ ¼ RðAuÞ and R½BuðW þ Þ1=2 ¼ RðBuÞ we get RðM1 Þ ¼ RðAuÞ and RðM2 Þ ¼ RðBuÞ. Now by Lemmas 5 and 10 we reach the desired result. & Lemma 12 (See St˛epniak et al., 1984, Lemma 3). For arbitrary models LðAb, sVÞ and LðBb, sWÞ with possibly nontrivial deterministic parts the relation LðAb, sVÞgLðBb, sWÞ is equivalent to LðAðIp Q Þb, sVÞgLðBðIp Q Þb, sWÞ, where Q is the orthogonal projector onto AuNðVÞ. From these lemmas we will get the following theorem. Theorem 13. Let LðAb, sVÞ and LðBb, sWÞ be models with possible nontrivial deterministic parts. Then LðAb, sVÞgLðBb, sWÞ, if and only if, (a) BuNðWÞ DAuNðVÞ and (b) ðIp Q ÞðAuV ABuW BÞðIp Q Þ Z0, where Q is as in Lemma 12. Remark 14. Providing (a) the condition (b) does not depend on choice of generalized inverses V and W . In particular in this case one can set V ¼ V þ and W ¼ W þ . Theorem 13 was proved in St˛epniak et al. (1984), Torgersen (1984) and Heyer (2006) by a longer way. Now we shall give a short proof of this theorem. Proof. The necessity of (a) is evident. We note that under this condition the both models LðAðIp Q Þb, sVÞ and LðBðIp Q Þb, sWÞ have trivial deterministic parts. In order to get the desired result we only need to use Lemmas 11 and 12. & In the next section we shall apply Theorem 13 to a study of the relations between a model and its composition with additional runs and/or constraints. Our consideration involves a greater depth inside into the examples A and B in St˛epniak et al. (1984) and leads to a nice explicit result. It appears that in this case the conditions (a) and (b) in Theorem 13 are more useful than the (2). 5. Model with additional runs and constraints Let us consider an arbitrary model L1 ¼ LðA1 b, sV1 Þ and the same model with additional runs and/or constraints. The second one may be presented in the form L ¼ LðAb, rVÞ, where " # " # A1 V1 0 A¼ and V ¼ : ð3Þ A2 0u V2 We shall prove the following theorem. Theorem 15. Under the above notation always LgL1 , while L1 gL if and only if RðAu2 Þ DAu1 NðV1 Þ. Remark 16. It is interesting that the relation L1 gL does not depend on V2. Proof. Since AuNðVÞ ¼ spfAu1 NðV1 Þ,Au2 NðV2 Þg the condition AuNðVÞ D Au1 NðV1 Þ holds if and only if Au2 NðV2 Þ D Au1 NðV1 Þ. Now by the fact that for arbitrary square matrices M1 and M2 " #þ " þ # M1 0 M1 0 ¼ 0u M2 0u M2þ we get ðIp Q ÞðAuV þ AAu1 V1þ A1 ÞðIp Q Þ ¼ ðIp Q ÞAu2 V2þ A2 ðIp Q Þ Z 0: Thus always LgL1 while L1 gL if and only if Au2 NðV2 Þ D Au1 NðV1 Þ and ðIp Q ÞAu2 V2þ A2 ðIp Q Þ ¼ 0. We shall present the last condition in a desirable form. First note that ðIp Q ÞAu2 V2þ A2 ðIp Q Þ ¼ 0 if and only if ½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ ¼ 0 and the second one has a clear interpretation in terms of the model LðA2 ðIp Q Þb, rV2 Þ. By condition Au2 NðV2 Þ DAu1 NðV1 Þ this model has trivial deterministic part. Thus, by Lemma 5 (see also Corollary 9), cu½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ c is the variance of the BLUE estimator of the parametric function c ¼ cub within it. On the other hand, since this model has trivial deterministic part,
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
2493
cu½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ c ¼ 0 for all c 2 Rp if and only if A2 ðIp Q Þ ¼ 0. By definition of Q the last condition is equivalent to RðAu2 Þ DAu1 NðV1 Þ, which implies the earlier one Au2 NðV2 Þ D Au1 NðV1 Þ. In consequence the last condition is redundant. In this way Theorem 15 is proved. & Example 17. Additional runs for the 2 3 2 1 0 1 6 7 6 A1 ¼ 4 1 0 5 and V1 ¼ 4 1 0 2 0
model LðA1 b, sV1 Þ where 3 1 0 7 1 0 5: 0 1
In this case Au1 NðV1 Þ ¼ fcð1,1Þu : c 2 Rg. Thus LðA1 b, sV1 Þ is at least as good as an extended model L ¼ LðAb, rVÞ defined by (3) if and only if RðAu2 Þ D fcð1,1Þu : c 2 Rg. References Goel, P.K., Ginebra, J., 2003. When is one experiment ‘always better than’ another? J Roy. Statist. Soc. D 52, 515–537. Heyer, H., 2006. Order relations for linear models: a survey on recent developments. Statist. Papers 47, 331–372. Lehmann, E.L., 1959. Testing Statistical Hypotheses, first ed. Wiley, New York. Lehmann, E.L., Scheffe´, H., 1950. Completeness, similar regions, and unbiased estimation—Part 1. Sankhya¯ A 10, 305–340. Rao, C.R., 1973. Linear Statistical Inference. Wiley, New York. Seely, J., Zyskind, G., 1971. Linear spaces and minimum variance unbiased estimation. Ann. Math. Statist. 42, 691–701. Shaked, M., Sua´rez-Llorens, A., 2003. On the comparison of reliability experiments based on convolution order. J. Amer. Statist. Assoc. 98, 693–702. Ste˛ pniak, C., 1983. Optimal allocation of units in experimental designs with hierarchical and cross classification. Ann. Inst. Statist. Math. A 35, 461–473. Ste˛ pniak, C., 1985. Ordering of nonnegative definite matrices with application to comparison of linear models. Linear Algebra Appl. 70, 67–71. Ste˛ pniak, C., 1999. Geometry of linear and quadratic estimation in a normal linear model with singular variance–covariance matrix. Demonstratio Math. 32, 639–645. Ste˛ pniak, C., Torgersen, E., 1981. Comparison of linear models with partially known covariances with respect to unbiased estimation. Scand. J. Statist. 8, 183–184. Ste˛ pniak, C., Wang, S.G., Wu, C.F.J., 1984. Comparison of linear experiments with known covariances. Ann. Statist. 12, 358–365. Torgersen, E., 1984. Ordering of linear models. J. Statist. Plann. Inference 9, 1–17. Torgersen, E., 1999. Comparison of Statistical Experiments. Cambridge University Press, Cambridge. Zyskind, G., 1967. On canonical forms, non-negative covariance matrices and best and simple least squares estimators. Ann. Math. Statist. 38, 1092–1109.
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
Short communication
On a problem with singularity in comparison of linear experiments Czes"aw St˛epniak ´w, Al. Rejtana 16 A, 35-959 Rzeszo ´w, Poland Institute of Mathematics, University of Rzeszo
a r t i c l e i n f o
abstract
Article history: Received 23 July 2010 Received in revised form 22 January 2011 Accepted 26 January 2011 Available online 3 February 2011
For consistency, the parameter space in the Gauss–Markov model with singular covariance matrix is usually restricted by observation vector. This restriction arises some difficulties in comparison of linear experiments. To avoid it we reduce the problem of comparison from singular to nonsingular case. & 2011 Elsevier B.V. All rights reserved.
Keywords: Gauss–Markov model Trivial/nontrivial deterministic part Induced model Comparison of experiments
1. Introduction and notation The term statistical experiment is usually identified with a family of probability measures. Comparison of experiments refers to a partial ordering of such families involving a common parameter. In practice the ordering is induced by a class of statistical problems concerning the parameter. In consequence, a wider class of problems leads to a stronger ordering. For a comprehensive information in this subject we refer to the well known book by Torgersen (1999) and to some review articles by Goel and Ginebra (2003) and Heyer (2006). One of the most attractive ways of the ordering of experiments based on convolution, originated by Lehmann (1959), was successfully applied to reliability problems by Shaked and Sua´rez-Llorens (2003). In this note we restrict ourselves to linear experiments, where the whole information reduces to the first two moments of the observation vector. Traditionally, matrix algebra remains the main tool in this area. Throughout this paper the usual vector–matrix notation is used. The space of all n 1 vectors is denoted by Rn and, if S is a set of in Rn then sp(S) will denote its span, i.e. the minimal linear space including S. If M is a matrix then Mu, R(M), N(M) and r(M) denote, respectively, its transposition, range (column space), kernel (null space) and rank. By M and M + is denoted by a generalized inverse and the Moore–Penrose generalized inverse of M. The symbol PM stands for the orthogonal projector onto R(M), i.e. the square matrix P satisfying the conditions Px ¼ x for x 2 RðMÞ and zero for x 2 NðMuÞ. Moreover, if M is a square matrix then tr M denotes its trace and MZ 0 means that M is symmetric and nonnegative definite (nnd, for short). If M is nnd then the symbol M1=2 stands for the matrix satisfying the condition M1=2 M1=2 ¼ M. Let X be n 1 random vector with the expectation EX ¼ Ab and the covariance matrix sV, where A and V are known matrices of n p and n n, respectively, while b and s are unknown parameters. This kind information about X is commonly called the Gauss–Markov model and is denoted by LðAb, sVÞ. The fact that X is subject to the model LðAb, sVÞ will be shortly expressed in the form X LðAb, sVÞ. In the situation when V is singular, many authors, among others Rao (1973, Section 4.9.2) and Seely and Zyskind (1971) suggest to restrict the space of possible parameters b by the observation vector X. The suggested space for b has the form O ¼ fb 2 Rp : CuAb ¼ CuXg, where C is any matrix with n rows such that RðCÞ ¼ RðVÞ. As noted in Seely and Zyskind (1971)
E-mail address: [email protected] 0378-3758/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.jspi.2011.01.022
2490
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
this approach is not acceptable from formal point of view. Moreover, it is inconvenient for comparison of linear experiments since they need have the same parameters (see, e.g. Torgersen, 1999; St˛epniak and Torgersen, 1981; St˛epniak, 1983; St˛epniak et al., 1984). To be more precise, the above-mentioned inconvenience does not refer to all Gauss–Markov models with singular covariance matrix, but only to the models with so-called nontrivial deterministic part. This term, introduced by Torgersen (1984), was investigated in detail in St˛epniak (1999). The idea of this note consists in reducing consideration from singular to nonsingular Gauss–Markov models. Not only this approach leads to convenient tools in comparison of linear experiments but it also opens a new effective way to known results. To demonstrate it, we need several auxiliary results. 2. Auxiliary results Usually we are interested in estimation of a parametric function jðbÞ ¼ cub, where c 2 Rp , by a linear functional duX, of the observation vector X. An estimator duX is said to be unbiased for j if EðduXÞ ¼ jðbÞ for all b. If such estimator exists then the parametric function j is said to be estimable. In the context of the model LðAb, sVÞ we have EðduXÞ ¼ duAb. Thus j ¼ cub is estimable, if and only if c 2 RðAuÞ. In consequence, any unbiased estimator of cub may be presented in the form duX, where d is any solution of the linear equation Aux ¼ c. It is well known that for any estimable cub in the model LðAb, sVÞ there exists a linear unbiased estimator with minimal variance, called the best linear unbiased estimator (BLUE). This estimator may be characterized by a well known Lehmann– Scheffe´ theorem. Since this very important theorem is rather rarely used in statistical literature we shall present it in a simplified form with a complete proof. ´, 1950, Theorem 5.3). Let D ¼ ff ðXÞg be a class of potential estimators. If D constitutes a linear Theorem 1 (Lehmann and Scheffe space then a member f1(X) of this class is a minimum variance unbiased estimator of its expectation in D, if and only if, Covðf1 ðXÞ,f0 ðXÞÞ ¼ 0 for any f0 2 D such that Eðf0 ðXÞÞ ¼ 0 for all b. Proof. Denote by j the expectation of f1 ðXÞ and let U, where U D D, be the class of all unbiased estimators of j. Then f ¼ f ðXÞ belongs to U, if and only if, it can be represented in the form f ¼ f1 þ af0 , where Eðf0 ðXÞÞ ¼ 0, and a 2 R. In consequence, Varðf ðXÞÞ ¼ Var½f1 ðXÞ þ af0 ðXÞ: Thus, Varðf1 ðXÞÞ is minimal, if and only if, Varðf1 ðXÞÞ r a2 Varðf0 ðXÞÞ þ2a Covðf1 ðXÞ,f0 ðXÞÞ þ Varðf1 ðXÞÞ This implies the desired result.
for all a 2 R:
&
From Theorem 1 we get immediately the following corollary. Corollary 2 (Zyskind, 1967, Theorem 3). In the context of the Gauss–Markov model LðAb, sVÞ a linear functional duX is a BLUE of its expectation, if and only if, Vd 2 RðAÞ. We shall say that the Gauss–Markov model LðAb, sVÞ with observation vector X has trivial deterministic part if for any d 2 Rn , the condition VarðduXÞ ¼ 0 implies EðduXÞ ¼ 0. It is worth to note that any model with nontrivial deterministic part has singular covariance matrix, but not vice versa, for instance if 2 3 2 3 1 0 1 1 0 6 7 6 7 A ¼ 4 1 0 5 and V ¼ 4 1 1 0 5: 0 2 0 0 1 In the further consideration we need the following results. Lemma 3 (St˛epniak, 1999, Lemma 1). Let X be subject to the Gauss–Markov model LðAb, sVÞ. Then (a) A linear functional duX has zero variance, if and only if d 2 NðVÞ. (b) A parametric function cub is estimable with zero variance, if and only if c 2 AuNðVÞ. Lemma 4 (St˛epniak, 1999, Lemma 3). In the context of the model LðAb, sVÞ the following are equivalent: (a) The model LðAb, sVÞ has trivial deterministic part. (b) AuNðVÞ ¼ f0g. (c) RðAÞ DRðVÞ. Lemma 5 (Rao, 1973, p. 300). In the context of the Gauss–Markov model LðAb, sVÞ with trivial deterministic part for any estimable j ¼ cub the variance of its BLUE may be presented in the form ^ Þ ¼ cuðAuV AÞ c: Varðj
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
2491
Lemma 6 (St˛epniak, 1985, Theorem 1). For any symmetric nnd matrices M1 and M2 of the same order the following are equivalent: (a) M1 M2 is nnd, (b) RðM1 Þ + RðM2 Þ and xuðM 2 M1 Þx Z 0 for all x 2 RðM2 Þ.
3. A model with trivial deterministic part induced by singular Gauss–Markov model Let LðAb, sVÞ be the Gauss–Markov model with nontrivial deterministic part and let Q be the orthogonal projector from Rp onto the space AuNðVÞ ¼ R½AuðIn PV Þ. We note that AuNðVÞ represents the space of coefficients of the parametric functions in LðAb, sVÞ being estimable with zero variance. Let us introduce an auxiliary model LðA1 b, sVÞ induced by LðAb, sVÞ, where A1 ¼ AðIp Q Þ:
ð1Þ
This model may be considered as a submodel of LðAb, sVÞ with b 2 RðIp Q Þ. It has some interesting properties. Lemma 7 (St˛epniak, 1999, Lemma 7). Let LðAb, sVÞ be arbitrary Gauss–Markov model and let A1 be defined by (1). Then (a) The model LðA1 b, sVÞ has trivial deterministic part. (b) rðVÞrðA1 Þ ¼ rðV þ AAuÞrðAÞ. Theorem 8. Let j ¼ cub be an estimable parametric function in the initial model LðAb, sVÞ with observation vector X. Then j may be presented in the form cu1 b þ cu2 b, where (a) cu1 b is estimable in the induced model LðA1 b, sVÞ and its BLUE in this model coincides with one in the initial model LðAb, sVÞ. (b) cu2 b is estimable in the initial model LðAb, sVÞ with zero variance. (c) cu1 c2 ¼ 0. Proof. If cub is estimable in LðAb, sVÞ then c ¼ Aud for some d 2 Rn . Defining c1 ¼ ðIp Q ÞAud and c2 ¼ QAud we get c ¼ c1 þ c2 and cu1 c2 ¼ 0. Moreover c1 2 RðAu1 Þ and c2 2 AuNðVÞ. Thus cu1 b is estimable in LðA1 b, sVÞ and, if duX is its BLUE in LðA1 b, sVÞ then it is also a BLUE in LðAb, sVÞ since Vd 2 R½AðIp Q Þ implies Vd 2 RðAÞ. Moreover, by Lemma 3, cu2 b is estimable in the initial model LðAb, sVÞ with zero variance. & By Theorem 8 the variance of the BLUE for any estimable parametric function cub depends on c only through its component c1. In consequence, from Lemma 5 we get the following corollary. Corollary 9. In the context of arbitrary model LðAb, sVÞ for any estimable parametric function j ¼ cub the variance of its BLUE may be presented in the form ^ Þ ¼ cu½ðIp Q ÞAuV AðIp Q Þ c, Varðj where Q is the orthogonal projector from Rp onto AuNðVÞ. 4. Application to comparison of linear experiments Consider two linear experiments with observation vectors X 2 Rn and Y 2 Rm such that X LðAb, sVÞ and Y LðBb, sWÞ. We shall say that the experiment designed by LðAb, sVÞ is at least as good (for linear estimation) as one designed by LðBb, sWÞ [notation: LðAb, sVÞgLðBb, sWÞ], if for any parametric function c ¼ cub and for any b 2 Rm there exists a 2 Rn such that Eb, s ðauXcÞ2 r Eb, s ðbuYcÞ2
for all b and s:
Lemma 10 (St˛epniak, 1983, Lemma 1). LðAb, sVÞgLðBb, sWÞ, if and only if, for any b 2 Rm there exists a 2 Rn such that EðauXÞ ¼ EðbuYÞ
and
VarðauXÞ rVarðbuYÞ for all b and s:
It is well known (see, e.g. Ste˛ pniak et al., 1984) that the relation LðAb, sVÞgLðBb, sWÞ with possible singular V and/or W may be expressed in the algebraic form AuðV þAAuÞ ABuðWþ BBuÞ B Z 0:
ð2Þ
2492
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
It appears that the approach used in Section 3 may be used to derive a more convenient characterization of the relation g than (2). To this aim we shall use the following lemmas. Lemma 11. For arbitrary models LðAb, sVÞ and LðBb, sWÞ with trivial deterministic parts the relation LðAb, sVÞ gLðBb, sWÞ holds if and only if AuV ABuW B Z 0. Proof. By the conditions RðAÞ D RðVÞ and RðBÞ DRðWÞ the matrices M1 ¼ AuV A and M2 ¼ BuW B do not depend on the choice of the generalized inverses V and W . Moreover, by setting V ¼ V þ and W ¼ W þ we can present them in the form M1 ¼ ½AuðV þ Þ1=2 ½AuðV þ Þ1=2 u and M2 ¼ ½BuðW þ Þ1=2 ½BuðW þ Þ1=2 u. Since R½AuðV þ Þ1=2 ¼ RðAuVÞ ¼ RðAuÞ and R½BuðW þ Þ1=2 ¼ RðBuÞ we get RðM1 Þ ¼ RðAuÞ and RðM2 Þ ¼ RðBuÞ. Now by Lemmas 5 and 10 we reach the desired result. & Lemma 12 (See St˛epniak et al., 1984, Lemma 3). For arbitrary models LðAb, sVÞ and LðBb, sWÞ with possibly nontrivial deterministic parts the relation LðAb, sVÞgLðBb, sWÞ is equivalent to LðAðIp Q Þb, sVÞgLðBðIp Q Þb, sWÞ, where Q is the orthogonal projector onto AuNðVÞ. From these lemmas we will get the following theorem. Theorem 13. Let LðAb, sVÞ and LðBb, sWÞ be models with possible nontrivial deterministic parts. Then LðAb, sVÞgLðBb, sWÞ, if and only if, (a) BuNðWÞ DAuNðVÞ and (b) ðIp Q ÞðAuV ABuW BÞðIp Q Þ Z0, where Q is as in Lemma 12. Remark 14. Providing (a) the condition (b) does not depend on choice of generalized inverses V and W . In particular in this case one can set V ¼ V þ and W ¼ W þ . Theorem 13 was proved in St˛epniak et al. (1984), Torgersen (1984) and Heyer (2006) by a longer way. Now we shall give a short proof of this theorem. Proof. The necessity of (a) is evident. We note that under this condition the both models LðAðIp Q Þb, sVÞ and LðBðIp Q Þb, sWÞ have trivial deterministic parts. In order to get the desired result we only need to use Lemmas 11 and 12. & In the next section we shall apply Theorem 13 to a study of the relations between a model and its composition with additional runs and/or constraints. Our consideration involves a greater depth inside into the examples A and B in St˛epniak et al. (1984) and leads to a nice explicit result. It appears that in this case the conditions (a) and (b) in Theorem 13 are more useful than the (2). 5. Model with additional runs and constraints Let us consider an arbitrary model L1 ¼ LðA1 b, sV1 Þ and the same model with additional runs and/or constraints. The second one may be presented in the form L ¼ LðAb, rVÞ, where " # " # A1 V1 0 A¼ and V ¼ : ð3Þ A2 0u V2 We shall prove the following theorem. Theorem 15. Under the above notation always LgL1 , while L1 gL if and only if RðAu2 Þ DAu1 NðV1 Þ. Remark 16. It is interesting that the relation L1 gL does not depend on V2. Proof. Since AuNðVÞ ¼ spfAu1 NðV1 Þ,Au2 NðV2 Þg the condition AuNðVÞ D Au1 NðV1 Þ holds if and only if Au2 NðV2 Þ D Au1 NðV1 Þ. Now by the fact that for arbitrary square matrices M1 and M2 " #þ " þ # M1 0 M1 0 ¼ 0u M2 0u M2þ we get ðIp Q ÞðAuV þ AAu1 V1þ A1 ÞðIp Q Þ ¼ ðIp Q ÞAu2 V2þ A2 ðIp Q Þ Z 0: Thus always LgL1 while L1 gL if and only if Au2 NðV2 Þ D Au1 NðV1 Þ and ðIp Q ÞAu2 V2þ A2 ðIp Q Þ ¼ 0. We shall present the last condition in a desirable form. First note that ðIp Q ÞAu2 V2þ A2 ðIp Q Þ ¼ 0 if and only if ½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ ¼ 0 and the second one has a clear interpretation in terms of the model LðA2 ðIp Q Þb, rV2 Þ. By condition Au2 NðV2 Þ DAu1 NðV1 Þ this model has trivial deterministic part. Thus, by Lemma 5 (see also Corollary 9), cu½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ c is the variance of the BLUE estimator of the parametric function c ¼ cub within it. On the other hand, since this model has trivial deterministic part,
C. St˛epniak / Journal of Statistical Planning and Inference 141 (2011) 2489–2493
2493
cu½ðIp Q ÞAu2 V2þ A2 ðIp Q Þ þ c ¼ 0 for all c 2 Rp if and only if A2 ðIp Q Þ ¼ 0. By definition of Q the last condition is equivalent to RðAu2 Þ DAu1 NðV1 Þ, which implies the earlier one Au2 NðV2 Þ D Au1 NðV1 Þ. In consequence the last condition is redundant. In this way Theorem 15 is proved. & Example 17. Additional runs for the 2 3 2 1 0 1 6 7 6 A1 ¼ 4 1 0 5 and V1 ¼ 4 1 0 2 0
model LðA1 b, sV1 Þ where 3 1 0 7 1 0 5: 0 1
In this case Au1 NðV1 Þ ¼ fcð1,1Þu : c 2 Rg. Thus LðA1 b, sV1 Þ is at least as good as an extended model L ¼ LðAb, rVÞ defined by (3) if and only if RðAu2 Þ D fcð1,1Þu : c 2 Rg. References Goel, P.K., Ginebra, J., 2003. When is one experiment ‘always better than’ another? J Roy. Statist. Soc. D 52, 515–537. Heyer, H., 2006. Order relations for linear models: a survey on recent developments. Statist. Papers 47, 331–372. Lehmann, E.L., 1959. Testing Statistical Hypotheses, first ed. Wiley, New York. Lehmann, E.L., Scheffe´, H., 1950. Completeness, similar regions, and unbiased estimation—Part 1. Sankhya¯ A 10, 305–340. Rao, C.R., 1973. Linear Statistical Inference. Wiley, New York. Seely, J., Zyskind, G., 1971. Linear spaces and minimum variance unbiased estimation. Ann. Math. Statist. 42, 691–701. Shaked, M., Sua´rez-Llorens, A., 2003. On the comparison of reliability experiments based on convolution order. J. Amer. Statist. Assoc. 98, 693–702. Ste˛ pniak, C., 1983. Optimal allocation of units in experimental designs with hierarchical and cross classification. Ann. Inst. Statist. Math. A 35, 461–473. Ste˛ pniak, C., 1985. Ordering of nonnegative definite matrices with application to comparison of linear models. Linear Algebra Appl. 70, 67–71. Ste˛ pniak, C., 1999. Geometry of linear and quadratic estimation in a normal linear model with singular variance–covariance matrix. Demonstratio Math. 32, 639–645. Ste˛ pniak, C., Torgersen, E., 1981. Comparison of linear models with partially known covariances with respect to unbiased estimation. Scand. J. Statist. 8, 183–184. Ste˛ pniak, C., Wang, S.G., Wu, C.F.J., 1984. Comparison of linear experiments with known covariances. Ann. Statist. 12, 358–365. Torgersen, E., 1984. Ordering of linear models. J. Statist. Plann. Inference 9, 1–17. Torgersen, E., 1999. Comparison of Statistical Experiments. Cambridge University Press, Cambridge. Zyskind, G., 1967. On canonical forms, non-negative covariance matrices and best and simple least squares estimators. Ann. Math. Statist. 38, 1092–1109.