- Email: [email protected]
On unbiased optimal L -statistics quantile estimators
Statistics and Probability Letters 82 (2012) 1891–1897
Contents lists available at SciVerse ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
On unbiased optimal L-statistics quantile estimators Ling-Wei Li a,∗ , Loo-Hay Lee b , Chun-Hung Chen c , Bo Guo a a
Department of Systems Engineering, College of Information Systems and Management, National University of Defense Technology, Changsha 410073, China
b
Department of Industrial and Systems Engineering, National University of Singapore, 10 Kent Ridge Crescent, 119260, Singapore
c
Department of Systems Engineering and Operations Research, George Mason University, 4400 University Drive, MS 4A6, Fairfax, VA 22030, USA
article
info
Article history: Received 1 March 2012 Received in revised form 31 May 2012 Accepted 31 May 2012 Available online 19 June 2012 Keywords: Unbiased quantile estimator L-statistics Location-scale distributions Best linear unbiased estimator
abstract Recently, Li et al. (2012a,b) have presented two biased Optimal L-statistics Quantile Estimators (OLQEs). In this work, we present two unbiased versions of the two biased OLQEs. Similar to the biased OLQEs, the proposed unbiased OLQEs are able to accommodate a set of scaled populations and a set of location-scale populations, respectively. Furthermore, we compare the proposed unbiased OLQEs with two state-of-the-art efficient unbiased estimators, called Best Linear Unbiased Estimators (BLUEs). Although OLQEs and BLUEs have different aims and models, we point out that the two proposed unbiased OLQEs are closely related to the two BLUEs, respectively. The differences between the unbiased OLQEs and the BLUEs are also provided. We conduct an experimental study to demonstrate that, for a set of location-scale populations and extreme quantiles, if the main concern is large biases, then a proposed unbiased location equivariance OLQE is more appealing. © 2012 Elsevier B.V. All rights reserved.
1. Introduction Recently, Li et al. (2012a,b) have presented two biased Optimal L-statistics Quantile Estimators (OLQEs). Let F (x) be a known distribution function (DF). A symbol ‘‘’’ means that a parameter has a known value. In this paper, the first work (Li et al., 2012a) is called the original OLQE, denoted by OLQE O . OLQE O is able to accommodate a set of scaled populations ˜ is a known location parameter and Xσ ,∆˜ is a random SPS∆˜ = {Xσ ,∆˜ |σ > 0}, where σ is an unknown scale parameter, ∆
˜ ). The second work (Li et al., 2012b) is called the location equivariance OLQE, variable which has a DF Fσ ,∆˜ (x) = F ((x/σ ) − ∆ denoted by OLQE LE . OLQE LE is able to accommodate a set of location-scale populations LSPS = {Xξ ,σ |ξ ∈ R, σ > 0}, where σ and ξ are unknown scale and location parameters, Xξ ,σ is a random variable which has a DF Fξ ,σ (x) = F ((x − ξ )/σ ) and R is the real line. OLQE O and OLQE LE are biased estimators and are derived by minimizing the Mean Square Errors (MSEs). In our earlier works (Li et al., 2012a,b), the experimental studies shown that OLQE O and OLQE LE are superior to a state-of-the-art efficient estimator, called Best Linear Unbiased Estimator (BLUE), in terms of MSE. However, bias is a concern in both theoretical studies and applications of OLQE. Theoretically, methodologies of constructing unbiased OLQEs still remain open questions. Moreover, properties of unbiased OLQEs and a comparison of unbiased OLQEs versus the BLUEs are interesting. In the presence of bias, an estimation can be misleading, say, when we calculate this final estimation by an arithmetic mean of multiple estimation replications. Thus, in this paper, we attempt to design two unbiased versions of OLQE O and OLQE LE which are able to accommodate SPS∆˜ and LSPS, respectively. Moreover, we will show the differences between the two proposed unbiased OLQEs and the BLUEs. Finally, we conduct an experimental study to demonstrate that, for extreme quantiles, OLQE LE has large biases. Thus, for extreme quantiles, if the main concern is the large biases, then the proposed unbiased version of OLQE LE is more appealing. ∗
Corresponding author. E-mail addresses: [email protected] (L.-W. Li), [email protected] (L.-H. Lee), [email protected] (C.-H. Chen), [email protected] (B. Guo).
0167-7152/$ – see front matter © 2012 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2012.05.027
1892
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
2. Unbiased OLQE for a set of scaled populations
(X1:n,σ ,∆˜ , X2:n,σ ,∆˜ , . . . , Xn:n,σ ,∆˜ )T be the n-dimensional random Order Statistics (OS) column vector = (τij,n,σ )n×n (David and Nagaraja, 2003, p. 35) be the covariance matrix of Xσ ,∆˜ and µσ ,∆˜ = (µ1:n,σ ,∆˜ , µ2:n,σ ,∆˜ , . . . , µn:n,σ ,∆˜ )T (David and Nagaraja, 2003, p. 34) be the mean vector of Xσ ,∆˜ . An L-statistics quantile estimator (LQE) is given by L(w, Xσ ,∆˜ ) = w × Xσ ,∆˜ , where w = (w1 , w2 , . . . , wn ) is an n-dimensional weight vector. We Let Xσ ,∆˜ = of Xσ ,∆˜ . Let Σ σ
design an unbiased version of OLQE O , denoted by OLQE U , by minimizing the variance of OLQE U . The variance of the LQE is given by var(L(w, Xσ ,∆˜ )) = w × Σ σ × wT . Let Qσ ,∆˜ (p) = F −1˜ (p), where p is a quantile level. The OLQE U weight vector for σ ,∆
estimating Qσ ,∆˜ (p), denoted by w∗ ˜ (p), is an optimal solution to the following constrained optimization problem (COP) U ,∆ min var L w, Xσ ,∆˜
w∈Rn
s.t. w × µσ ,∆˜ = Qσ ,∆˜ (p) ,
(1)
where Rn is the n-dimensional Euclidean space. Remark 2.1. Compared to the optimization problem of OLQE O , the COP (1) has a different objective function, ˆ σ ,∆˜ )), and a new unbiased constraint, E (L(w, Xσ ,∆˜ )) = w × µσ ,∆˜ = Qσ ,∆˜ (p). Under the unbiased constraint, var(L(w, X we note that var(L(w, Xσ ,∆˜ )) is equal to the MSE of L(w, Xσ ,∆˜ ). Thus, actually, we incorporate the unbiased constraint into the optimization problem of OLQE O and obtain the COP (1). Solving the COP (1), the expression of w∗ ˜ (p) is given in Theorem 2.2. U ,∆ Theorem 2.2. If Xσ ,∆˜ is random, then the COP (1) has a unique optimal solution
1 1 w∗U ,∆˜ (p) = Qσ ,∆˜ (p) µTσ ,∆˜ Σ − µTσ ,∆˜ Σ − ˜ σ σ µσ ,∆
−1
,
(2)
which is the OLQE U weight vector for estimating Qσ ,∆˜ (p). Proof. See Appendix A.
Li et al. (2012a) pointed out that OLQE O is able to accommodate SPS∆˜ , because the OLQE O weight vector is σ -free. We employ the method of Li et al. (2012a) to prove that OLQE U is also able to accommodate SPS∆˜ , as shown in Proposition 2.3. Proposition 2.3. w∗ ˜ (p) is free from the unknown scale parameter σ . U ,∆ 1 1 −2 Proof. Note that µσ ,∆˜ = µ1,∆˜ × σ , Qσ ,∆˜ (p) = Q1,∆˜ (p) × σ and Σ − × Σ− σ =σ 1 . We have
1 1 w∗U ,∆˜ (p) = Q1,∆˜ (p) µT1,∆˜ Σ − µT1,∆˜ Σ − ˜ 1 1 µ1,∆
−1
.
Therefore, w∗ ˜ (p) is free from the unknown scale parameter σ . U ,∆
Remark 2.4. To the best of our knowledge, the existing literature concerning quantile estimations does not study unbiased estimators which are designed to accommodate SPS∆˜ . Thus, OLQE U bridges the gap between current research and practice. 3. Unbiased OLQELE for a set of location-scale populations Similar to the case of OLQE U , we impose an unbiased constraint on the weight vector of OLQE LE and obtain an unbiased version of OLQE LE , denoted by OLQE LEU . Let Xξ ,σ = (X1:n,ξ ,σ , X2:n,ξ ,σ , . . . , Xn:n,ξ ,σ )T be the n-dimensional random OS column vector of Xξ ,σ . µξ ,σ and Σ σ are the mean vector and the covariance matrix of Xξ ,σ , respectively. Let Qξ ,σ (p) = Fξ−,σ1 (p). The OLQE LEU weight vector for estimating Qξ ,σ (p), denoted by w∗LEU (p), is an optimal solution to the following COP min var L w, Xξ ,σ
w∈Rn
s.t. w × µξ ,σ = Qξ ,σ (p) ,
w × 1 = 1,
(3)
where 1 is an n-dimensional all-ones vector and var(L(w, Xξ ,σ )) = w × Σ σ × wT . Remark 3.1. Obviously, the COP (3) is similar to that of OLQE LE except that the COP (3) has the unbiased constraint. Solving the COP (3), the expression of w∗LEU (p) is given in Theorem 3.2. Theorem 3.2. If Xξ ,σ is random, then the COP (3) has a unique optimal solution 1 w∗LEU (p) = cTξ ,σ (p) −TTξ ,σ Σ − σ Tξ ,σ
−1
1 TTξ ,σ Σ − σ ,
where cξ ,σ (p) = (−Qξ ,σ (p), −1)T and Tξ ,σ = (µξ ,σ , 1). w∗LEU (p) is the OLQE LEU weight vector for estimating Qξ ,σ (p).
(4)
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
Proof. See Appendix B.
1893
Li et al. (2012b) pointed out that OLQE LE is able to accommodate LSPS, because the OLQE LE weight vector is free from the unknown parameters, σ and ξ . We prove that OLQE LEU is also able to accommodate LSPS, as shown in Theorem 3.3. Theorem 3.3. w∗LEU (p) is free from the unknown location parameter ξ and the unknown scale parameter σ . Proof. See Appendix C.
1 −1 Remark 3.4. Obviously, the classical weight vector of the BLUE of quantile (Zielinski, 2009), i.e., cT0,1 (p)(−TT0,1 Σ − × 1 T0,1 ) 1 TT0,1 Σ − 1 , is a special case of the expression (4) where ξ = 0 and σ = 1.
4. The differences between OLQEs and BLUEs 4.1. The difference between OLQE U and the BLUE of scale Let Fσ ,∆˜ (x) be a generating function (GF). We can design the weight vector of the BLUE of scale by the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µσ ,∆˜ = 1.
(5)
Multiply both sides of the constraint, w × µσ ,∆˜ = 1, by Qσ ,∆˜ (p), we have (w × Qσ ,∆˜ (p)) × µσ ,∆˜ = Qσ ,∆˜ (p). Let w′ = w × Qσ ,∆˜ (p). Then, the COP (5) can be written as minw′ ∈Rn var(L(w′ , Xσ ,∆˜ ))/Q 2 ˜ (p) s.t. w′ × µσ ,∆˜ = Qσ ,∆˜ (p). σ ,∆ An optimal solution to this modified COP is similar to that to the COP (1). Therefore, an optimal solution to the COP (5) is S ∗ T − 1 − 1 − 1 T ¯ BLUE = w ˜ (p)/Qσ ,∆˜ (p) = (µ ˜ Σ σ µσ ,∆˜ ) µ ˜ Σ σ , which is the weight vector of the BLUE of scale (Balakrishnan and w U ,∆
σ ,∆
Saleh, 2011).
σ ,∆
Remark 4.1. From the above discussion, we know that the COP (1) and the COP (5) are actually similar except for a constant factor, Qσ ,∆˜ (p). For the convenience of comparison, we do not employ the linear model formulation (Khuri, 2010, p. 137) of the BLUE of scale. 4.2. The difference between OLQE LEU and BLUE of location and scale Note that the BLUE of location and scale can be derived by using either a COP model or a linear model (LM). In this subsection, first, from a COP point of view, we show the difference between OLQE LEU and BLUE; second, from a LM point of view, we demonstrate that OLQE LEU (so-called the BLUE of quantile) is actually a best linear unbiased prediction (BLUP) of quantile. 4.2.1. Comparison of OLQE LEU with BLUE from the COP point of view T
We note that the expression (4) can be written as w∗LEU (p) = −cTξ ,σ (p)((wSBLUE )T , (wLBLUE ) )T , where wLBLUE and wSBLUE (Balakrishnan and Rao, 1998, p. 162) are the weight vectors of the BLUE of location and scale, respectively. Thus, an issue of whether the COP (3) is equivalent to that of BLUE becomes a major concern. Let Fξ ,σ (x) be a GF. It is easy to show that wLBLUE is an optimal solution to the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µξ ,σ = 0,
w × 1 = 1.
(6)
Moreover, as pointed out by Balakrishnan and Rao (1998, p. 162), wSBLUE is an optimal solution to the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µξ ,σ = 1,
w × 1 = 0.
(7)
Note that w∗LEU (p) = Qξ ,σ (p) × wSBLUE + wLBLUE . Now, we show that we cannot directly use the COP (6) and the COP (7) to prove that w∗LEU (p) is the optimal solution to the COP (3). Obviously, we have w∗LEU (p)×µξ ,σ = Qξ ,σ (p) and w∗LEU (p)× 1 = 1,
i.e., w∗LEU (p) is a feasible solution of the COP (3). Let Qˆ ξ ,σ (p) = L(w∗LEU (p), Xξ ,σ ). Note that
var Qˆ ξ ,σ (p) = Qξ2,σ (p) wSBLUE Σ σ wSBLUE
T
T T + wLBLUE Σ σ wLBLUE + 2Qξ ,σ (p) wSBLUE Σ σ wLBLUE .
(8)
Thus, if wSBLUE Σ σ (wLBLUE )T = 0, then w∗LEU (p) must be the optimal solution to the COP (3). Since the value of wSBLUE Σ σ (wLBLUE )T depends on the GF Fξ ,σ (x) and is not always zero, the optimality of w∗LEU (p) is not very clear when we employ the COP formulation of BLUE. Remark 4.2. Let σˆ BLUE = wSBLUE Xξ ,σ and ξˆBLUE = wLBLUE Xξ ,σ . Note that wSBLUE Σ σ (wLBLUE )T = cov(σˆ BLUE , ξˆBLUE ). If the GF Fξ ,σ (x) is symmetric and centers at zero, then wSBLUE Σ σ (wLBLUE )T = 0 (David and Nagaraja, 2003, pp. 187–188), and w∗LEU (p) is optimal.
1894
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
4.2.2. Comparison of OLQE LEU with BLUE from the LM point of view From the LM point of view, the procedure for estimating quantiles consists of the following steps: (i) by the LM formulation, we derive the minimizer, (σˆ BLUE , ξˆBLUE ), of the Euclidean norm f (σˆ , ξˆ ) = ∥(σˆ × µξ ,σ + ξˆ × 1) − Xξ ,σ ∥2
by the Least Squares (LS) approach (Khuri, 2010, p. 129); (ii) we employ a linear quantile model (LQM), Qˆ ξ ,σ (p) = σˆ BLUE × Qξ ,σ (p) + ξˆBLUE , to predict quantiles. According to Theorem 4.5 in Rao and Toutenburg (1999, p. 106), Qˆ ξ ,σ (p) is a BLUP (Rao and Toutenburg, 1999, p. 184). Therefore, strictly speaking, LQM is to predict quantiles rather than to estimate quantiles. It should be noted that most authors ignored the aforementioned subtle difference between the optimality of Qˆ ξ ,σ (p) and the optimality of σˆ BLUE and ξˆBLUE . Given σˆ BLUE and ξˆBLUE , they typically treated Qˆ ξ ,σ (p) as the BLUE of quantile
and did not provide a mathematical evidence of the optimality of Qˆ ξ ,σ (p).
Remark 4.3. Obviously, from the LM point of view, BLUE and OLQE LEU have very different aims and methodologies. OLQE LEU aims to minimize the variance of OLQE LEU (i.e., w × Σ σ × wT ); BLUE aims to minimize the Euclidean norm (i.e., the square of the Euclidean distance between σˆ × µξ ,σ + ξˆ × 1 and Xξ ,σ ). The decision variable of OLQE LEU is the n-dimensional weight vector w; the decision variable of BLUE is a 2-dimensional parameter vector (σˆ , ξˆ ). OLQE LEU explicitly imposes two constraints on w; BLUE does not impose any constraint on (σˆ , ξˆ ). 5. An experimental study In this section, we conduct an experimental study to investigate the bias of OLQE LE . The experimental study aims to show that, for extreme quantiles, if the main concern is large biases, then OLQE LEU is better than OLQE LE because the bias of OLQE LE is large. In the rest of this section, first, we briefly describe the experiment design, which is similar to that of Li et al. (2012b); second, we conduct the experimental study and summarize the numerical results. 5.1. Experiment design In this experiment study, we consider three main factors which influence the performances of quantile estimators. The three factors are testing distribution F (x), quantile level p and sample size n. Except for the quantile level, the experiment design of this paper is similar to that of Li et al. (2012b). Since the testing distributions of Li et al. (2012b) are symmetric or right skewed, we only consider high level quantiles. In the literature, the range of an extreme quantile is often 1 − 10−1 ∼1 − 10−4 (Pandey, 2001; Deng et al., 2009; Ferrari and Paterlini, 2009). The level sets of the three factors are given below. (1) F (x) has 4 choices: Normal (0, 1), Laplace (0, 1), Exponential (1) and Pareto (1); (2) p has 7 levels: 1 − 10−1−0.5i , i = 0, 1, . . . , 6; (3) n has 2 levels: 16 and 32. 5.2. Numerical results In this subsection, we summarize the numerical results of the bias of OLQE LE . For each testing distribution, the numerical results are illustrated in a figure. Thus, we obtain 4 figures. From Figs. 1 to 4, it is discovered that the OLQE LE of extreme quantiles has negative biases. Moreover, for each testing distribution: (i) given p, the absolute value of bias becomes small as n increases; (ii) given n, the absolute value of bias becomes large as p increases. Given p and n, the biases of the OLQE LE of extreme quantiles are determined by the shapes of the tails of the testing distributions: (i) in the case of short tails (i.e., Normal (0, 1)), the absolute values of the biases are small; (ii) in the case of exponential tails (i.e., Exponential (1) and Laplace (0, 1)), the absolute values of the biases are moderate; and (iii) in the case of heavy tails (i.e., Pareto (1)), the absolute values of the biases are very large. From the numerical results, we know that, for extreme quantiles, OLQE LE has large biases, especially when the testing distributions do not have light tails. Therefore, if the main concern is large biases, then the biased estimator, i.e., OLQE LE , does not work well for extreme quantiles and OLQE LEU is more appealing. Acknowledgments The authors thank the editor and a referee for their helpful comments, which improved this paper considerably. Li’s research was partially supported by National Natural Science Foundation of China (Grant No. 11101428). Chen’s research has been supported in part by Department of Energy under Award DE-SC0002223, NIH under Grant 1R21DK088368-01, and National Science Council of Taiwan under Award NSC-100-2218-E-002-027-MY3.
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
1895
Fig. 1. Bias for Normal (0, 1).
Fig. 2. Bias for Laplace (0, 1).
Fig. 3. Bias for Exponential (1).
Appendix A. Proof of Theorem 2.2 We employ the Lagrange multiplier method (Sundaram, 2008, pp. 112–144), to solve the COP (1). Let λ be a scalar. The Lagrangian function for the COP (1) is given by Lag1 (w, λ) = var(L(w, Xσ ,∆˜ )) − λ(w × µσ ,∆˜ − Qσ ,∆˜ (p)). Let w∗ ˜ (p) = (w ∗ ˜ (p), w ∗ ˜ (p), . . . , w ∗ ˜ (p)) be an optimal solution to the COP (1). Let λ∗ be a value of the scalar λ U ,∆ U ,∆,1 U ,∆,2 U ,∆ ,n
1896
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
Fig. 4. Bias for Pareto (1).
associated with w∗ ˜ (p). w∗ ˜ (p) and λ∗ must satisfy the following necessary conditions U ,∆ U ,∆
n ∂ Lag1 w∗U ,∆˜ (p) , λ∗ = 2 wU∗ ,∆˜ ,j (p) τij:n,σ − µi:n,σ ,∆˜ λ∗ = 0, i = 1, 2, . . . , n ∂wi j = 1 ∗ ∗ n ∂ Lag w p , λ ( ) 1 ˜ U ,∆ = wU∗ ,∆˜ ,j (p) µj:n,σ ,∆˜ − Qσ ,∆˜ (p) = 0. ∂λ j =1 2Σ σ −µσ ,∆˜ Let Bσ ,∆˜ = −µT . Then, we can rewrite the above system of linear equations as matrix operations, i.e., Bσ ,∆˜ × 0 ˜ σ ,∆
(wU ,∆˜ (p)
λ∗ )T = (0T −Qσ ,∆˜ (p))T , where 0 is an n-dimensional zero vector. Note that if Xσ ,∆˜ is random, then Σ σ is positive definite and µT ˜ Σ σ µσ ,∆˜ > 0. Thus, according to the inverse of a partitioned matrix (Lutkepohl, 1996, p. 30), we σ ,∆ ∗
1
have B−1˜ = σ ,∆
2
1 T −1 −1 T Σ −1 ) × (Σ σ−1 + Σ − ˜ ) µσ ,∆ ˜ (−µσ ,∆ σ µσ ,∆ ˜ Σ σ µσ ,∆ ˜ σ T − 1 − 1 T − (−µ ˜ Σ σ µσ ,∆˜ ) µ ˜ Σ σ 1
σ ,∆
σ ,∆
1 T −1 −1 Σ− ˜ (−µσ ,∆ ˜) σ µσ ,∆ ˜ Σ σ µσ ,∆ T − 1 − 1 2(−µ ˜ Σ σ µσ ,∆ ˜)
. Applying the expression of B−1˜ , σ ,∆
σ ,∆
a solution to the matrix equation is
−1 T 1 T −1 Qσ ,∆˜ (p) Σ − µ µ Σ µ ˜ ˜ σ , ∆ σ , ∆ σ σ ˜ 0 σ ,∆ wU ,∆˜ (p) 1 . −1 = B− ˜ −Q ˜ (p) = σ ,∆ 1 σ ,∆ λ∗ 2Qσ ,∆˜ (p) µT ˜ Σ − µ ˜ σ ,∆ σ
∗
σ ,∆
Note that Li et al. (2012a) pointed out that if Xσ ,∆˜ is random, then MSE (L(w, Xσ ,∆˜ )) = wΣ σ wT + (wµσ ,∆˜ − Qσ ,∆˜ (p))2 has a minimum. Thus, according to Proposition 5.6 in Sundaram (2008, p. 122), we know that w∗ ˜ (p) is the unique optimal solution to the COP (1).
U ,∆
Appendix B. Proof of Theorem 3.2 Similar to the proof of Theorem 2.2, we employ the Lagrange multiplier method to solve the COP (3). The Lagrangian function for the COP (3) is given by Lag2 (w, λ1 , λ2 ) = var(L(w, Xξ ,σ )) − λ1 (w × µξ ,σ − Qξ ,σ (p)) − λ2 (w × 1 − 1), where ∗ ∗ ∗ ∗ λ1 and λ2 are scalars. Let w∗LEU (p) = (wLEU ,1 (p), wLEU ,2 (p), . . . , wLEU ,n (p)) be an optimal solution to the COP (3). Let λ1 and λ∗2 be values of the scalars λ1 and λ2 associated with w∗LEU (p), respectively. w∗LEU (p), λ∗1 and λ∗2 must satisfy the following necessary conditions
n ∂ Lag2 w∗LEU (p) , λ∗1 , λ∗2 ∗ ∗ ∗ = 2 wLEU ,j (p) τij:n,σ − µi:n,ξ ,σ λ1 − λ2 = 0, ∂wi j =1 n ∂ Lag2 w∗ (p) , λ∗ , λ∗ LEU 1 2 ∗ = wLEU ,j (p) µj:n,ξ ,σ − Qξ ,σ (p) = 0 ∂λ 1 j =1 n ∂ Lag2 w∗ (p) , λ∗ , λ∗ LEU 1 2 ∗ = wLEU ,j (p) − 1 = 0. ∂λ2 j =1
i = 1, 2, . . . , n
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897 2Σ σ
Let Wξ ,σ = (−TT
ξ ,σ
−Tξ ,σ 02×2
) and λ∗ = (λ∗1
1897
λ∗2 ), where 02×2 is a 2 × 2 zero matrix. Then, we can rewrite the above
∗ system of linear equations as matrix operations, i.e., Wξ ,σ × (wLEU (p)
λ∗ )T = (0T
cTξ ,σ (p))T . Note that if Xξ ,σ is random,
then Σ σ and TTξ ,σ Σ σ Tξ ,σ are positive definite (Lutkepohl, 1996, 152). Therefore, according to the inverse of a partitioned matrix (Lutkepohl, 1996, p. 30), we have 1 W− ξ ,σ
−1 T 1 1 1 −1 −1 Tξ ,σ Σ − Tξ ,σ Σ + Σ Tξ ,σ −TTξ ,σ Σ − σ σ σ σ = 2 −1 T T 1 1 Tξ ,σ Σ − −Tξ ,σ Σ − σ σ Tξ ,σ
−1 1 −Tξ ,σ Σ − σ Tξ ,σ . −1 1 2 −TTξ ,σ Σ − T ξ ,σ σ
1 Σ− σ Tξ ,σ
T
1 Thus, applying the expression of W− ξ ,σ , a solution to the matrix equation is
w∗LEU (p)
T 1 λ∗ = W− ξ ,σ
0
cξ ,σ (p)
=
1 T −1 Σ− σ Tξ ,σ −Tξ ,σ Σ σ Tξ ,σ
1 2 −TTξ ,σ Σ − σ Tξ ,σ
−1
−1
cξ ,σ (p)
cξ ,σ (p)
.
Similar to the proof of Theorem 2.2, we know that w∗LEU (p) is the unique optimal solution to the COP (3). Appendix C. Proof of Theorem 3.3 The COP (3) has two constraints, w × µξ ,σ = Qξ ,σ (p) and w × 1 = 1. Note that Qξ ,σ (p) = σ × Q0,1 (p) + ξ and µξ ,σ = ξ ×1+σ ×µ0,1 . The first constraint can be written as w×µ0,1 = Q0,1 (p). Note that var(L(w, Xξ ,σ )) = w×Σ σ ×wT = σ 2 var(L(w, X0,1 )). The COP (3) can be written as min var L w, X0,1 s.t. w × µ0,1 = Q0,1 (p) , w × 1 = 1. n w∈R
1 1 −1 T Applying the expression (4), an optimal solution to the above modified COP is w∗LEU (p) = cT0,1 (p)(−TT0,1 Σ − T0,1 Σ − 1 T0,1 ) 1 . ∗ Comparing the aforementioned expression with the expression (4), we know that wLEU (p) is free from the unknown parameters ξ and σ .
References Balakrishnan, N., Rao, C.R., 1998. Handbook of Statistics. In: Order Statistics and Their Applications, vol. 17. Elsevier, New York. Balakrishnan, N., Saleh, H.M., 2011. Relations for moments of progressively type-II censored order statistics from half-logistic distribution with applications to inference. Computational Statistics and Data Analysis 55, 2775–2792. David, H.A., Nagaraja, H.N., 2003. Order Statistics, third ed. Wiley, New York. Deng, J., Pandey, M.D., Gu, D., 2009. Extreme quantile estimation from censored sample using partial cross-entropy and fractional partial probability weighted moments. Structural Safety 31, 43–54. Ferrari, D., Paterlini, S., 2009. The maximum Lq-likelihood method: an application to extreme quantile estimation in finance. Methodology and Computing in Applied Probability 11, 3–19. Khuri, A.I., 2010. Linear Model Methodology. Chapman & Hall/CRC, New York. Li, L.W., Lee, L.H., Chen, C.H., Guo, B., 2012a, Optimal L-statistics quantile estimator for a set of scaled populations, Manuscript. Li, L.W., Lee, L.H., Chen, C.H., Guo, B., Liu, Y.J., 2012b. Optimal L-statistics quantile estimator for a set of location-scale populations. Statistics and Probability Letters. URL: http://dx.doi.org/10.1016/j.spl.2012.05.015. Lutkepohl, H., 1996. Handbook of Matrices. John Wiley & Sons, England, West Sussex. Pandey, M.D., 2001. Extreme quantile estimation using order statistics with minimum cross-entropy principle. Probabilistic Engineering Mechanics 16, 31–42. Rao, C.R., Toutenburg, H., 1999. Linear Models: Least Squares and Alternatives, second ed. Springer, New York. Sundaram, R.K., 2008. A First Course in Optimization Theory. Posts & Telecom Press, Beijing. Zielinski, R., 2009. Optimal nonparametric quantile estimators—towards a general theory: a survey. Communications in Statistics—Theory and Methods 38, 980–992.
Contents lists available at SciVerse ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
On unbiased optimal L-statistics quantile estimators Ling-Wei Li a,∗ , Loo-Hay Lee b , Chun-Hung Chen c , Bo Guo a a
Department of Systems Engineering, College of Information Systems and Management, National University of Defense Technology, Changsha 410073, China
b
Department of Industrial and Systems Engineering, National University of Singapore, 10 Kent Ridge Crescent, 119260, Singapore
c
Department of Systems Engineering and Operations Research, George Mason University, 4400 University Drive, MS 4A6, Fairfax, VA 22030, USA
article
info
Article history: Received 1 March 2012 Received in revised form 31 May 2012 Accepted 31 May 2012 Available online 19 June 2012 Keywords: Unbiased quantile estimator L-statistics Location-scale distributions Best linear unbiased estimator
abstract Recently, Li et al. (2012a,b) have presented two biased Optimal L-statistics Quantile Estimators (OLQEs). In this work, we present two unbiased versions of the two biased OLQEs. Similar to the biased OLQEs, the proposed unbiased OLQEs are able to accommodate a set of scaled populations and a set of location-scale populations, respectively. Furthermore, we compare the proposed unbiased OLQEs with two state-of-the-art efficient unbiased estimators, called Best Linear Unbiased Estimators (BLUEs). Although OLQEs and BLUEs have different aims and models, we point out that the two proposed unbiased OLQEs are closely related to the two BLUEs, respectively. The differences between the unbiased OLQEs and the BLUEs are also provided. We conduct an experimental study to demonstrate that, for a set of location-scale populations and extreme quantiles, if the main concern is large biases, then a proposed unbiased location equivariance OLQE is more appealing. © 2012 Elsevier B.V. All rights reserved.
1. Introduction Recently, Li et al. (2012a,b) have presented two biased Optimal L-statistics Quantile Estimators (OLQEs). Let F (x) be a known distribution function (DF). A symbol ‘‘’’ means that a parameter has a known value. In this paper, the first work (Li et al., 2012a) is called the original OLQE, denoted by OLQE O . OLQE O is able to accommodate a set of scaled populations ˜ is a known location parameter and Xσ ,∆˜ is a random SPS∆˜ = {Xσ ,∆˜ |σ > 0}, where σ is an unknown scale parameter, ∆
˜ ). The second work (Li et al., 2012b) is called the location equivariance OLQE, variable which has a DF Fσ ,∆˜ (x) = F ((x/σ ) − ∆ denoted by OLQE LE . OLQE LE is able to accommodate a set of location-scale populations LSPS = {Xξ ,σ |ξ ∈ R, σ > 0}, where σ and ξ are unknown scale and location parameters, Xξ ,σ is a random variable which has a DF Fξ ,σ (x) = F ((x − ξ )/σ ) and R is the real line. OLQE O and OLQE LE are biased estimators and are derived by minimizing the Mean Square Errors (MSEs). In our earlier works (Li et al., 2012a,b), the experimental studies shown that OLQE O and OLQE LE are superior to a state-of-the-art efficient estimator, called Best Linear Unbiased Estimator (BLUE), in terms of MSE. However, bias is a concern in both theoretical studies and applications of OLQE. Theoretically, methodologies of constructing unbiased OLQEs still remain open questions. Moreover, properties of unbiased OLQEs and a comparison of unbiased OLQEs versus the BLUEs are interesting. In the presence of bias, an estimation can be misleading, say, when we calculate this final estimation by an arithmetic mean of multiple estimation replications. Thus, in this paper, we attempt to design two unbiased versions of OLQE O and OLQE LE which are able to accommodate SPS∆˜ and LSPS, respectively. Moreover, we will show the differences between the two proposed unbiased OLQEs and the BLUEs. Finally, we conduct an experimental study to demonstrate that, for extreme quantiles, OLQE LE has large biases. Thus, for extreme quantiles, if the main concern is the large biases, then the proposed unbiased version of OLQE LE is more appealing. ∗
Corresponding author. E-mail addresses: [email protected] (L.-W. Li), [email protected] (L.-H. Lee), [email protected] (C.-H. Chen), [email protected] (B. Guo).
0167-7152/$ – see front matter © 2012 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2012.05.027
1892
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
2. Unbiased OLQE for a set of scaled populations
(X1:n,σ ,∆˜ , X2:n,σ ,∆˜ , . . . , Xn:n,σ ,∆˜ )T be the n-dimensional random Order Statistics (OS) column vector = (τij,n,σ )n×n (David and Nagaraja, 2003, p. 35) be the covariance matrix of Xσ ,∆˜ and µσ ,∆˜ = (µ1:n,σ ,∆˜ , µ2:n,σ ,∆˜ , . . . , µn:n,σ ,∆˜ )T (David and Nagaraja, 2003, p. 34) be the mean vector of Xσ ,∆˜ . An L-statistics quantile estimator (LQE) is given by L(w, Xσ ,∆˜ ) = w × Xσ ,∆˜ , where w = (w1 , w2 , . . . , wn ) is an n-dimensional weight vector. We Let Xσ ,∆˜ = of Xσ ,∆˜ . Let Σ σ
design an unbiased version of OLQE O , denoted by OLQE U , by minimizing the variance of OLQE U . The variance of the LQE is given by var(L(w, Xσ ,∆˜ )) = w × Σ σ × wT . Let Qσ ,∆˜ (p) = F −1˜ (p), where p is a quantile level. The OLQE U weight vector for σ ,∆
estimating Qσ ,∆˜ (p), denoted by w∗ ˜ (p), is an optimal solution to the following constrained optimization problem (COP) U ,∆ min var L w, Xσ ,∆˜
w∈Rn
s.t. w × µσ ,∆˜ = Qσ ,∆˜ (p) ,
(1)
where Rn is the n-dimensional Euclidean space. Remark 2.1. Compared to the optimization problem of OLQE O , the COP (1) has a different objective function, ˆ σ ,∆˜ )), and a new unbiased constraint, E (L(w, Xσ ,∆˜ )) = w × µσ ,∆˜ = Qσ ,∆˜ (p). Under the unbiased constraint, var(L(w, X we note that var(L(w, Xσ ,∆˜ )) is equal to the MSE of L(w, Xσ ,∆˜ ). Thus, actually, we incorporate the unbiased constraint into the optimization problem of OLQE O and obtain the COP (1). Solving the COP (1), the expression of w∗ ˜ (p) is given in Theorem 2.2. U ,∆ Theorem 2.2. If Xσ ,∆˜ is random, then the COP (1) has a unique optimal solution
1 1 w∗U ,∆˜ (p) = Qσ ,∆˜ (p) µTσ ,∆˜ Σ − µTσ ,∆˜ Σ − ˜ σ σ µσ ,∆
−1
,
(2)
which is the OLQE U weight vector for estimating Qσ ,∆˜ (p). Proof. See Appendix A.
Li et al. (2012a) pointed out that OLQE O is able to accommodate SPS∆˜ , because the OLQE O weight vector is σ -free. We employ the method of Li et al. (2012a) to prove that OLQE U is also able to accommodate SPS∆˜ , as shown in Proposition 2.3. Proposition 2.3. w∗ ˜ (p) is free from the unknown scale parameter σ . U ,∆ 1 1 −2 Proof. Note that µσ ,∆˜ = µ1,∆˜ × σ , Qσ ,∆˜ (p) = Q1,∆˜ (p) × σ and Σ − × Σ− σ =σ 1 . We have
1 1 w∗U ,∆˜ (p) = Q1,∆˜ (p) µT1,∆˜ Σ − µT1,∆˜ Σ − ˜ 1 1 µ1,∆
−1
.
Therefore, w∗ ˜ (p) is free from the unknown scale parameter σ . U ,∆
Remark 2.4. To the best of our knowledge, the existing literature concerning quantile estimations does not study unbiased estimators which are designed to accommodate SPS∆˜ . Thus, OLQE U bridges the gap between current research and practice. 3. Unbiased OLQELE for a set of location-scale populations Similar to the case of OLQE U , we impose an unbiased constraint on the weight vector of OLQE LE and obtain an unbiased version of OLQE LE , denoted by OLQE LEU . Let Xξ ,σ = (X1:n,ξ ,σ , X2:n,ξ ,σ , . . . , Xn:n,ξ ,σ )T be the n-dimensional random OS column vector of Xξ ,σ . µξ ,σ and Σ σ are the mean vector and the covariance matrix of Xξ ,σ , respectively. Let Qξ ,σ (p) = Fξ−,σ1 (p). The OLQE LEU weight vector for estimating Qξ ,σ (p), denoted by w∗LEU (p), is an optimal solution to the following COP min var L w, Xξ ,σ
w∈Rn
s.t. w × µξ ,σ = Qξ ,σ (p) ,
w × 1 = 1,
(3)
where 1 is an n-dimensional all-ones vector and var(L(w, Xξ ,σ )) = w × Σ σ × wT . Remark 3.1. Obviously, the COP (3) is similar to that of OLQE LE except that the COP (3) has the unbiased constraint. Solving the COP (3), the expression of w∗LEU (p) is given in Theorem 3.2. Theorem 3.2. If Xξ ,σ is random, then the COP (3) has a unique optimal solution 1 w∗LEU (p) = cTξ ,σ (p) −TTξ ,σ Σ − σ Tξ ,σ
−1
1 TTξ ,σ Σ − σ ,
where cξ ,σ (p) = (−Qξ ,σ (p), −1)T and Tξ ,σ = (µξ ,σ , 1). w∗LEU (p) is the OLQE LEU weight vector for estimating Qξ ,σ (p).
(4)
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
Proof. See Appendix B.
1893
Li et al. (2012b) pointed out that OLQE LE is able to accommodate LSPS, because the OLQE LE weight vector is free from the unknown parameters, σ and ξ . We prove that OLQE LEU is also able to accommodate LSPS, as shown in Theorem 3.3. Theorem 3.3. w∗LEU (p) is free from the unknown location parameter ξ and the unknown scale parameter σ . Proof. See Appendix C.
1 −1 Remark 3.4. Obviously, the classical weight vector of the BLUE of quantile (Zielinski, 2009), i.e., cT0,1 (p)(−TT0,1 Σ − × 1 T0,1 ) 1 TT0,1 Σ − 1 , is a special case of the expression (4) where ξ = 0 and σ = 1.
4. The differences between OLQEs and BLUEs 4.1. The difference between OLQE U and the BLUE of scale Let Fσ ,∆˜ (x) be a generating function (GF). We can design the weight vector of the BLUE of scale by the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µσ ,∆˜ = 1.
(5)
Multiply both sides of the constraint, w × µσ ,∆˜ = 1, by Qσ ,∆˜ (p), we have (w × Qσ ,∆˜ (p)) × µσ ,∆˜ = Qσ ,∆˜ (p). Let w′ = w × Qσ ,∆˜ (p). Then, the COP (5) can be written as minw′ ∈Rn var(L(w′ , Xσ ,∆˜ ))/Q 2 ˜ (p) s.t. w′ × µσ ,∆˜ = Qσ ,∆˜ (p). σ ,∆ An optimal solution to this modified COP is similar to that to the COP (1). Therefore, an optimal solution to the COP (5) is S ∗ T − 1 − 1 − 1 T ¯ BLUE = w ˜ (p)/Qσ ,∆˜ (p) = (µ ˜ Σ σ µσ ,∆˜ ) µ ˜ Σ σ , which is the weight vector of the BLUE of scale (Balakrishnan and w U ,∆
σ ,∆
Saleh, 2011).
σ ,∆
Remark 4.1. From the above discussion, we know that the COP (1) and the COP (5) are actually similar except for a constant factor, Qσ ,∆˜ (p). For the convenience of comparison, we do not employ the linear model formulation (Khuri, 2010, p. 137) of the BLUE of scale. 4.2. The difference between OLQE LEU and BLUE of location and scale Note that the BLUE of location and scale can be derived by using either a COP model or a linear model (LM). In this subsection, first, from a COP point of view, we show the difference between OLQE LEU and BLUE; second, from a LM point of view, we demonstrate that OLQE LEU (so-called the BLUE of quantile) is actually a best linear unbiased prediction (BLUP) of quantile. 4.2.1. Comparison of OLQE LEU with BLUE from the COP point of view T
We note that the expression (4) can be written as w∗LEU (p) = −cTξ ,σ (p)((wSBLUE )T , (wLBLUE ) )T , where wLBLUE and wSBLUE (Balakrishnan and Rao, 1998, p. 162) are the weight vectors of the BLUE of location and scale, respectively. Thus, an issue of whether the COP (3) is equivalent to that of BLUE becomes a major concern. Let Fξ ,σ (x) be a GF. It is easy to show that wLBLUE is an optimal solution to the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µξ ,σ = 0,
w × 1 = 1.
(6)
Moreover, as pointed out by Balakrishnan and Rao (1998, p. 162), wSBLUE is an optimal solution to the following COP min w × Σ σ × wT
w∈Rn
s.t. w × µξ ,σ = 1,
w × 1 = 0.
(7)
Note that w∗LEU (p) = Qξ ,σ (p) × wSBLUE + wLBLUE . Now, we show that we cannot directly use the COP (6) and the COP (7) to prove that w∗LEU (p) is the optimal solution to the COP (3). Obviously, we have w∗LEU (p)×µξ ,σ = Qξ ,σ (p) and w∗LEU (p)× 1 = 1,
i.e., w∗LEU (p) is a feasible solution of the COP (3). Let Qˆ ξ ,σ (p) = L(w∗LEU (p), Xξ ,σ ). Note that
var Qˆ ξ ,σ (p) = Qξ2,σ (p) wSBLUE Σ σ wSBLUE
T
T T + wLBLUE Σ σ wLBLUE + 2Qξ ,σ (p) wSBLUE Σ σ wLBLUE .
(8)
Thus, if wSBLUE Σ σ (wLBLUE )T = 0, then w∗LEU (p) must be the optimal solution to the COP (3). Since the value of wSBLUE Σ σ (wLBLUE )T depends on the GF Fξ ,σ (x) and is not always zero, the optimality of w∗LEU (p) is not very clear when we employ the COP formulation of BLUE. Remark 4.2. Let σˆ BLUE = wSBLUE Xξ ,σ and ξˆBLUE = wLBLUE Xξ ,σ . Note that wSBLUE Σ σ (wLBLUE )T = cov(σˆ BLUE , ξˆBLUE ). If the GF Fξ ,σ (x) is symmetric and centers at zero, then wSBLUE Σ σ (wLBLUE )T = 0 (David and Nagaraja, 2003, pp. 187–188), and w∗LEU (p) is optimal.
1894
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
4.2.2. Comparison of OLQE LEU with BLUE from the LM point of view From the LM point of view, the procedure for estimating quantiles consists of the following steps: (i) by the LM formulation, we derive the minimizer, (σˆ BLUE , ξˆBLUE ), of the Euclidean norm f (σˆ , ξˆ ) = ∥(σˆ × µξ ,σ + ξˆ × 1) − Xξ ,σ ∥2
by the Least Squares (LS) approach (Khuri, 2010, p. 129); (ii) we employ a linear quantile model (LQM), Qˆ ξ ,σ (p) = σˆ BLUE × Qξ ,σ (p) + ξˆBLUE , to predict quantiles. According to Theorem 4.5 in Rao and Toutenburg (1999, p. 106), Qˆ ξ ,σ (p) is a BLUP (Rao and Toutenburg, 1999, p. 184). Therefore, strictly speaking, LQM is to predict quantiles rather than to estimate quantiles. It should be noted that most authors ignored the aforementioned subtle difference between the optimality of Qˆ ξ ,σ (p) and the optimality of σˆ BLUE and ξˆBLUE . Given σˆ BLUE and ξˆBLUE , they typically treated Qˆ ξ ,σ (p) as the BLUE of quantile
and did not provide a mathematical evidence of the optimality of Qˆ ξ ,σ (p).
Remark 4.3. Obviously, from the LM point of view, BLUE and OLQE LEU have very different aims and methodologies. OLQE LEU aims to minimize the variance of OLQE LEU (i.e., w × Σ σ × wT ); BLUE aims to minimize the Euclidean norm (i.e., the square of the Euclidean distance between σˆ × µξ ,σ + ξˆ × 1 and Xξ ,σ ). The decision variable of OLQE LEU is the n-dimensional weight vector w; the decision variable of BLUE is a 2-dimensional parameter vector (σˆ , ξˆ ). OLQE LEU explicitly imposes two constraints on w; BLUE does not impose any constraint on (σˆ , ξˆ ). 5. An experimental study In this section, we conduct an experimental study to investigate the bias of OLQE LE . The experimental study aims to show that, for extreme quantiles, if the main concern is large biases, then OLQE LEU is better than OLQE LE because the bias of OLQE LE is large. In the rest of this section, first, we briefly describe the experiment design, which is similar to that of Li et al. (2012b); second, we conduct the experimental study and summarize the numerical results. 5.1. Experiment design In this experiment study, we consider three main factors which influence the performances of quantile estimators. The three factors are testing distribution F (x), quantile level p and sample size n. Except for the quantile level, the experiment design of this paper is similar to that of Li et al. (2012b). Since the testing distributions of Li et al. (2012b) are symmetric or right skewed, we only consider high level quantiles. In the literature, the range of an extreme quantile is often 1 − 10−1 ∼1 − 10−4 (Pandey, 2001; Deng et al., 2009; Ferrari and Paterlini, 2009). The level sets of the three factors are given below. (1) F (x) has 4 choices: Normal (0, 1), Laplace (0, 1), Exponential (1) and Pareto (1); (2) p has 7 levels: 1 − 10−1−0.5i , i = 0, 1, . . . , 6; (3) n has 2 levels: 16 and 32. 5.2. Numerical results In this subsection, we summarize the numerical results of the bias of OLQE LE . For each testing distribution, the numerical results are illustrated in a figure. Thus, we obtain 4 figures. From Figs. 1 to 4, it is discovered that the OLQE LE of extreme quantiles has negative biases. Moreover, for each testing distribution: (i) given p, the absolute value of bias becomes small as n increases; (ii) given n, the absolute value of bias becomes large as p increases. Given p and n, the biases of the OLQE LE of extreme quantiles are determined by the shapes of the tails of the testing distributions: (i) in the case of short tails (i.e., Normal (0, 1)), the absolute values of the biases are small; (ii) in the case of exponential tails (i.e., Exponential (1) and Laplace (0, 1)), the absolute values of the biases are moderate; and (iii) in the case of heavy tails (i.e., Pareto (1)), the absolute values of the biases are very large. From the numerical results, we know that, for extreme quantiles, OLQE LE has large biases, especially when the testing distributions do not have light tails. Therefore, if the main concern is large biases, then the biased estimator, i.e., OLQE LE , does not work well for extreme quantiles and OLQE LEU is more appealing. Acknowledgments The authors thank the editor and a referee for their helpful comments, which improved this paper considerably. Li’s research was partially supported by National Natural Science Foundation of China (Grant No. 11101428). Chen’s research has been supported in part by Department of Energy under Award DE-SC0002223, NIH under Grant 1R21DK088368-01, and National Science Council of Taiwan under Award NSC-100-2218-E-002-027-MY3.
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
1895
Fig. 1. Bias for Normal (0, 1).
Fig. 2. Bias for Laplace (0, 1).
Fig. 3. Bias for Exponential (1).
Appendix A. Proof of Theorem 2.2 We employ the Lagrange multiplier method (Sundaram, 2008, pp. 112–144), to solve the COP (1). Let λ be a scalar. The Lagrangian function for the COP (1) is given by Lag1 (w, λ) = var(L(w, Xσ ,∆˜ )) − λ(w × µσ ,∆˜ − Qσ ,∆˜ (p)). Let w∗ ˜ (p) = (w ∗ ˜ (p), w ∗ ˜ (p), . . . , w ∗ ˜ (p)) be an optimal solution to the COP (1). Let λ∗ be a value of the scalar λ U ,∆ U ,∆,1 U ,∆,2 U ,∆ ,n
1896
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897
Fig. 4. Bias for Pareto (1).
associated with w∗ ˜ (p). w∗ ˜ (p) and λ∗ must satisfy the following necessary conditions U ,∆ U ,∆
n ∂ Lag1 w∗U ,∆˜ (p) , λ∗ = 2 wU∗ ,∆˜ ,j (p) τij:n,σ − µi:n,σ ,∆˜ λ∗ = 0, i = 1, 2, . . . , n ∂wi j = 1 ∗ ∗ n ∂ Lag w p , λ ( ) 1 ˜ U ,∆ = wU∗ ,∆˜ ,j (p) µj:n,σ ,∆˜ − Qσ ,∆˜ (p) = 0. ∂λ j =1 2Σ σ −µσ ,∆˜ Let Bσ ,∆˜ = −µT . Then, we can rewrite the above system of linear equations as matrix operations, i.e., Bσ ,∆˜ × 0 ˜ σ ,∆
(wU ,∆˜ (p)
λ∗ )T = (0T −Qσ ,∆˜ (p))T , where 0 is an n-dimensional zero vector. Note that if Xσ ,∆˜ is random, then Σ σ is positive definite and µT ˜ Σ σ µσ ,∆˜ > 0. Thus, according to the inverse of a partitioned matrix (Lutkepohl, 1996, p. 30), we σ ,∆ ∗
1
have B−1˜ = σ ,∆
2
1 T −1 −1 T Σ −1 ) × (Σ σ−1 + Σ − ˜ ) µσ ,∆ ˜ (−µσ ,∆ σ µσ ,∆ ˜ Σ σ µσ ,∆ ˜ σ T − 1 − 1 T − (−µ ˜ Σ σ µσ ,∆˜ ) µ ˜ Σ σ 1
σ ,∆
σ ,∆
1 T −1 −1 Σ− ˜ (−µσ ,∆ ˜) σ µσ ,∆ ˜ Σ σ µσ ,∆ T − 1 − 1 2(−µ ˜ Σ σ µσ ,∆ ˜)
. Applying the expression of B−1˜ , σ ,∆
σ ,∆
a solution to the matrix equation is
−1 T 1 T −1 Qσ ,∆˜ (p) Σ − µ µ Σ µ ˜ ˜ σ , ∆ σ , ∆ σ σ ˜ 0 σ ,∆ wU ,∆˜ (p) 1 . −1 = B− ˜ −Q ˜ (p) = σ ,∆ 1 σ ,∆ λ∗ 2Qσ ,∆˜ (p) µT ˜ Σ − µ ˜ σ ,∆ σ
∗
σ ,∆
Note that Li et al. (2012a) pointed out that if Xσ ,∆˜ is random, then MSE (L(w, Xσ ,∆˜ )) = wΣ σ wT + (wµσ ,∆˜ − Qσ ,∆˜ (p))2 has a minimum. Thus, according to Proposition 5.6 in Sundaram (2008, p. 122), we know that w∗ ˜ (p) is the unique optimal solution to the COP (1).
U ,∆
Appendix B. Proof of Theorem 3.2 Similar to the proof of Theorem 2.2, we employ the Lagrange multiplier method to solve the COP (3). The Lagrangian function for the COP (3) is given by Lag2 (w, λ1 , λ2 ) = var(L(w, Xξ ,σ )) − λ1 (w × µξ ,σ − Qξ ,σ (p)) − λ2 (w × 1 − 1), where ∗ ∗ ∗ ∗ λ1 and λ2 are scalars. Let w∗LEU (p) = (wLEU ,1 (p), wLEU ,2 (p), . . . , wLEU ,n (p)) be an optimal solution to the COP (3). Let λ1 and λ∗2 be values of the scalars λ1 and λ2 associated with w∗LEU (p), respectively. w∗LEU (p), λ∗1 and λ∗2 must satisfy the following necessary conditions
n ∂ Lag2 w∗LEU (p) , λ∗1 , λ∗2 ∗ ∗ ∗ = 2 wLEU ,j (p) τij:n,σ − µi:n,ξ ,σ λ1 − λ2 = 0, ∂wi j =1 n ∂ Lag2 w∗ (p) , λ∗ , λ∗ LEU 1 2 ∗ = wLEU ,j (p) µj:n,ξ ,σ − Qξ ,σ (p) = 0 ∂λ 1 j =1 n ∂ Lag2 w∗ (p) , λ∗ , λ∗ LEU 1 2 ∗ = wLEU ,j (p) − 1 = 0. ∂λ2 j =1
i = 1, 2, . . . , n
L.-W. Li et al. / Statistics and Probability Letters 82 (2012) 1891–1897 2Σ σ
Let Wξ ,σ = (−TT
ξ ,σ
−Tξ ,σ 02×2
) and λ∗ = (λ∗1
1897
λ∗2 ), where 02×2 is a 2 × 2 zero matrix. Then, we can rewrite the above
∗ system of linear equations as matrix operations, i.e., Wξ ,σ × (wLEU (p)
λ∗ )T = (0T
cTξ ,σ (p))T . Note that if Xξ ,σ is random,
then Σ σ and TTξ ,σ Σ σ Tξ ,σ are positive definite (Lutkepohl, 1996, 152). Therefore, according to the inverse of a partitioned matrix (Lutkepohl, 1996, p. 30), we have 1 W− ξ ,σ
−1 T 1 1 1 −1 −1 Tξ ,σ Σ − Tξ ,σ Σ + Σ Tξ ,σ −TTξ ,σ Σ − σ σ σ σ = 2 −1 T T 1 1 Tξ ,σ Σ − −Tξ ,σ Σ − σ σ Tξ ,σ
−1 1 −Tξ ,σ Σ − σ Tξ ,σ . −1 1 2 −TTξ ,σ Σ − T ξ ,σ σ
1 Σ− σ Tξ ,σ
T
1 Thus, applying the expression of W− ξ ,σ , a solution to the matrix equation is
w∗LEU (p)
T 1 λ∗ = W− ξ ,σ
0
cξ ,σ (p)
=
1 T −1 Σ− σ Tξ ,σ −Tξ ,σ Σ σ Tξ ,σ
1 2 −TTξ ,σ Σ − σ Tξ ,σ
−1
−1
cξ ,σ (p)
cξ ,σ (p)
.
Similar to the proof of Theorem 2.2, we know that w∗LEU (p) is the unique optimal solution to the COP (3). Appendix C. Proof of Theorem 3.3 The COP (3) has two constraints, w × µξ ,σ = Qξ ,σ (p) and w × 1 = 1. Note that Qξ ,σ (p) = σ × Q0,1 (p) + ξ and µξ ,σ = ξ ×1+σ ×µ0,1 . The first constraint can be written as w×µ0,1 = Q0,1 (p). Note that var(L(w, Xξ ,σ )) = w×Σ σ ×wT = σ 2 var(L(w, X0,1 )). The COP (3) can be written as min var L w, X0,1 s.t. w × µ0,1 = Q0,1 (p) , w × 1 = 1. n w∈R
1 1 −1 T Applying the expression (4), an optimal solution to the above modified COP is w∗LEU (p) = cT0,1 (p)(−TT0,1 Σ − T0,1 Σ − 1 T0,1 ) 1 . ∗ Comparing the aforementioned expression with the expression (4), we know that wLEU (p) is free from the unknown parameters ξ and σ .
References Balakrishnan, N., Rao, C.R., 1998. Handbook of Statistics. In: Order Statistics and Their Applications, vol. 17. Elsevier, New York. Balakrishnan, N., Saleh, H.M., 2011. Relations for moments of progressively type-II censored order statistics from half-logistic distribution with applications to inference. Computational Statistics and Data Analysis 55, 2775–2792. David, H.A., Nagaraja, H.N., 2003. Order Statistics, third ed. Wiley, New York. Deng, J., Pandey, M.D., Gu, D., 2009. Extreme quantile estimation from censored sample using partial cross-entropy and fractional partial probability weighted moments. Structural Safety 31, 43–54. Ferrari, D., Paterlini, S., 2009. The maximum Lq-likelihood method: an application to extreme quantile estimation in finance. Methodology and Computing in Applied Probability 11, 3–19. Khuri, A.I., 2010. Linear Model Methodology. Chapman & Hall/CRC, New York. Li, L.W., Lee, L.H., Chen, C.H., Guo, B., 2012a, Optimal L-statistics quantile estimator for a set of scaled populations, Manuscript. Li, L.W., Lee, L.H., Chen, C.H., Guo, B., Liu, Y.J., 2012b. Optimal L-statistics quantile estimator for a set of location-scale populations. Statistics and Probability Letters. URL: http://dx.doi.org/10.1016/j.spl.2012.05.015. Lutkepohl, H., 1996. Handbook of Matrices. John Wiley & Sons, England, West Sussex. Pandey, M.D., 2001. Extreme quantile estimation using order statistics with minimum cross-entropy principle. Probabilistic Engineering Mechanics 16, 31–42. Rao, C.R., Toutenburg, H., 1999. Linear Models: Least Squares and Alternatives, second ed. Springer, New York. Sundaram, R.K., 2008. A First Course in Optimization Theory. Posts & Telecom Press, Beijing. Zielinski, R., 2009. Optimal nonparametric quantile estimators—towards a general theory: a survey. Communications in Statistics—Theory and Methods 38, 980–992.