- Email: [email protected]
Rényi entropy properties of records
Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
Re´nyi entropy properties of records M. Abbasnejad , N.R. Arghami Department of Statistics, School of Mathematical Sciences, Ferdowsi University of Mashhad, P.O. Box 1159, Mashhad 91775, Iran
a r t i c l e in f o
abstract
Article history: Received 17 September 2007 Received in revised form 13 October 2009 Accepted 14 January 2011 Available online 25 January 2011
We provide bounds for Re´nyi entropy of records. We also show that the Re´nyi entropy ordering of random variables determines the Re´nyi entropy ordering of their respective records. We characterize exponential distribution by maximization of Re´nyi entropy under some conditions. We show that Re´nyi distance between distribution of records and parent distribution is distribution free. & 2011 Elsevier B.V. All rights reserved.
Keywords: Exponential distribution Hazard function Maximum entropy principal Ordering Records Re´nyi entropy bounds Re´nyi distance
1. Introduction Let Xi ,iZ 1, be a sequence of iid continuous random variables with the cdf F(x) and the pdf f(x). An observation Xj will be called an upper record value if its value is greater than that of all previous observations. Thus Xj is an upper record value if Xj 4Xi for all i o j. By convention X1 is the first upper record value. An analogous definition can be given for lower record values. The times at which upper record values appear are given by the random variables Tj which are called record times and are defined by T1 =1 with probability 1 and, for j Z 2, Tj ¼ Minfi : Xi 4 XTj1 g: The waiting time between the ith upper record value and the (i+ 1)th upper record value is called the inter-record time (IRT), and is denoted by Di ¼ Ti þ 1 Ti ,i ¼ 1,2, . . . . Record times and inter-record times for lower record values are defined analogously. Let U1 ,U2 , . . . ,Un be the first n upper record values from a distribution with the cdf F(x) and the pdf f(x). Then the joint pdf of the first n upper record values and the marginal density of Un (the nth upper record value, n Z1) are given, respectively, by qðuÞ ¼
n 1 Y
rðui Þf ðun Þ,
u1 o oun ,
i¼1
fUn ðun Þ ¼
Rn1 ðun Þ f ðun Þ, ðn1Þ!
1 oun o þ 1,
where RðtÞ ¼ logð1FðtÞÞ, and rðtÞ ¼ RuðtÞ ¼ f ðtÞ=1FðtÞ is the hazard function. Corresponding author. Tel./fax: +98 511 8828605.
E-mail address: [email protected] (M. Abbasnejad). 0378-3758/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.jspi.2011.01.017
ð1Þ
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2313
Let L1 ,L2 , . . . ,Ln be the first n lower record values from a distribution with the cdf F(x) and the pdf f(x). Then the joint pdf of the first n lower record values and the marginal density of Ln (the nth lower record value, n Z 1) are given, respectively, by pðlÞ ¼
n 1 Y
r~ ðli Þf ðln Þ,
l1 4 4 ln ,
i¼1
fLn ðln Þ ¼
n1 R~ ðln Þ f ðln Þ, ðn1Þ!
1o ln o þ1,
~ ¼ logðFðtÞÞ, and r~ ðtÞ ¼ RuðtÞ ~ where RðtÞ ¼ f ðtÞ=FðtÞ is the reversed hazard function. See Arnold et al. (1998) for more details. Records can be used in a wide range of problems, including seismology, sporting and athletic events, meteorological analysis, industrial stress testing and hydrology. Record values have been studied extensively in the literature. For example, Chandler (1952) studied the behavior of random record values arising from the ‘‘classical record model’’, that is, the record model, where underlying sample from which records are observed, is considered to consist of iid observations from continuous probability distribution. Some authors compared records with the same number of iid observations based on the amount of their information. Ahmadi and Arghami (2001b, 2003), Hofmann and Nagaraja (2003), Hofmann and Balakrishnan (2004), Stepanov et al. (2003) compared them based on Fisher information. Habibi et al. (2006) compared Kullback–Leibler information of records with the same number of iid observations. Abbasnejad and Arghami (2006) did the same thing in terms of Re´nyi information. Recently, Baratpur et al. (2007) studied some information properties of records based on Shannon entropy and mutual information. They provide some bounds for the entropy of records. We generalize some of their results to the case of Re´nyi entropy. The rest of the paper is organized as follows. In Section 2, we express Re´nyi entropy of nth upper record based on Re´nyi entropy of nth record of standard exponential distribution and study some properties of it. Section 3 provides bounds for Re´nyi entropy of records. In Section 4, we characterize exponential distribution by maximizing Re´nyi entropy of record values. In Section 5, it is shown that the Re´nyi information between record value and data distribution is distribution free. 2. Re´nyi entropy of records Ever since Shannon (1948), has proposed a measure of uncertainty in a discrete distribution based on the Bolltzmann entropy, there has been a great deal of interest in the measurement of uncertainty associated with a probability distribution. There is now a huge literature devoted to the applications, generalizations and properties of Shannon’s measure of uncertainty. Let X be a non-negative random variable with an absolutely continuous distribution with probability density function f(x). Then the Shannon entropy of the density f is defined by Z þ1 HðXÞ ¼ f ðxÞlogf ðxÞ dx: 0
With his work, a new branch of mathematics with applications in different areas such as physics, probability and statistics, communication theory and economics was opened up. Numerous entropy and information indices, among them Re´nyi entropy, have been developed and used in various disciplines and contexts. If X is a random variable having an absolutely continuous cdf F(x) and pdf f(x), then Re´nyi entropy of order a of the random variable X is defined by Z þ1 1 log Ha ðXÞ ¼ f a ðxÞ dx 8a 40ðaa1Þ, ð2Þ a1 1 where lim Ha ðXÞ ¼ HðXÞ ¼
a-1
Z
f ðxÞlogf ðxÞ dx
is the Shannon entropy of X (Re´nyi, 1961). In reliability theory, record values are used for statistical modeling, shock models, and they are closely connected with the occurrence times of a corresponding nonhomogeneous Poisson process, with some minimal repair scheme, and with the relation transform (see, Kamps, 1994, 1995). In this section, we explore properties of Re´nyi entropy for record values. In the following Lemma, we express Re´nyi entropy of nth upper record from an arbitrary distribution in terms of Re´nyi entropy of nth upper record of standard exponential distribution. Lemma 2.1. Let fXn ,n Z 1g be a sequence of iid continuous random variables from the distribution F(x) with density function f(x) and the quantile function F 1 ð:Þ. Let Un denote the nth upper record. Then the Re´nyi entropy of Un can be expressed as Ha ðUn Þ ¼ Ha ðUn Þ
aðn1Þ þ1 1 loga logEgn ½f a1 ðF 1 ð1eVn ÞÞ, a1 a1
ð3Þ
where Un Gðn,1Þ denotes nth upper record of standard exponential distribution and Vn is a random variable, whose pdf is denoted by gn and is distributed as Gðaðn1Þ þ 1,1Þ.
2314
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Proof. By formulas (1) and (2), and using transformation U= R(X) we have Z þ 1 aðn1Þ u 1 u e a 1 log logGðnÞ logGðaðn1Þ þ1Þ f a1 ðF 1 ð1eu ÞÞ du ¼ Ha ðUn Þ ¼ a1 ½ðn1Þ!a a1 a1 0 Z þ1 a ðn1Þ v 1 v e log f a1 ðF 1 ð1ev ÞÞ dv: a1 Gðaðn1Þ þ 1Þ 0 The result follows from the fact that the expression for Re´nyi entropy of Un is Ha ðUn Þ ¼
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ loga: a1 a1 a1
&
Lemma 2.2. Under the assumptions of Lemma 2.1, if Ln denotes the nth lower record, then the Re´nyi entropy of Ln can be expressed as Ha ðLn Þ ¼
a 1 1 logGðnÞ logGðaðn1Þ þ 1Þ logEgn ½f a1 ðF 1 ðeVn ÞÞ, a1 a1 a1
ð4Þ
where Vn is as in Lemma 2.1. Proof. The proof is similar to that of Lemma 2.1. Several ordering and reliability properties of 1995), Ahmadi and Arghami (2001a), and Ahmadi entropy of records in terms of ordering properties and Y denote random variables with distribution F ðxÞ ¼ 1FðxÞ and GðyÞ ¼ 1GðyÞ.
& record values have been studied by Kochar (1990), Kamps (1994, and Balakrishnan (2005). We now provide some results on the Re´nyi of parent distributions. We need the following definitions in which X functions F and G, density functions f and g, and survival functions st
Definition 2.1. The random variable X is said to be stochastically less than or equal to Y, denoted by X r Y, if F ðvÞ rGðvÞ for all v. st
Shaked and Shanthikumar (1994) showed that X r Y implies that for any non-decreasing (non-increasing) function d, E½dðXÞr ð Z ÞE½dðYÞ, provided the expectations exist. lr
Definition 2.2. The random variable X is said to be less than or equal to Y in likelihood ratio ordering, denoted by X r Y, if f ðxÞ=gðxÞ is non-increasing in x. lr
st
Note that X r Y implies X r Y (Bickel and Lehmann, 1976). Re
Definition 2.3. The random variable X is said to be less than or equal to Y in Re´nyi entropy ordering, denoted by X r Y, if Ha ðXÞ r Ha ðYÞ for all a 4 0. Theorem 2.1. Let fXn ,n Z 1g be as in Lemma 2.1. Let Un denote its nth upper record. If f(x) is non-decreasing in x, then Ha ðUn Þ is non-increasing in n. Proof. Using (3), we have Ha ðUn Þ ¼ cn
1
a1
logEgn ½f a1 ðF 1 ð1eVn ÞÞ,
where cn ¼ ða=ða1ÞÞlogGðnÞð1=ða1ÞÞGðaðn1Þ þ1Þ: So Ha ðUn þ 1 ÞHa ðUn Þ ¼ cn þ 1 cn
Eg ½f a1 ðF 1 ð1eVn þ 1 ÞÞ 1 log n þ 1 a1 1 : a1 ðF ð1eVn ÞÞ Egn ½f
Treating n as a continuous variable and taking derivative with respect to n gives dcn a ¼ ½cðnÞcðaðn1Þ þ1Þ, dn a1 where cðzÞ ¼ dlogGðzÞ=dz is the digamma function. Note that since cðzÞ is an increasing function, for all z, cn is a decreasing function of n. lr st It is easy to show that Vn r Vn þ 1 and then Vn r Vn þ 1 . By assumption, f a1 ðF 1 ð1ev ÞÞ is non-decreasing in v for a 4 1, thus Egn þ 1 ½f a1 ðF 1 ð1eVn þ 1 ÞÞ 4Egn ½f a1 ðF 1 ð1eVn ÞÞ: Therefore, Ha ðUn þ 1 ÞHa ðUn Þ r 0 and thus the result follows. The proof for 0 o a o 1 is similar.
&
Theorem 2.2. Let fXn ,n Z 1g be as in Lemma 2.1. Let Ln, denotes its nth lower record. If f(x) is non-increasing in x, then Ha ðLn Þ is non-increasing in n.
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Proof. The proof is similar to that of Theorem 2.1, except that (4) is used to obtain the result.
2315
&
In the next theorem we show that Re´nyi entropy ordering of two random Variables X and Y is transferred to their upper records. Theorem 2.3. Let X and Y be two continuous random variables with distribution functions F(x) and G(y) and density functions f(x) and g(y), respectively. Suppose gðG1 ð1ev ÞÞ r1 , A1 ¼ v 40 1 v f ðF ð1e ÞÞ gðG1 ð1ev ÞÞ 41 A2 ¼ v 40 1 f ðF ð1ev ÞÞ Re
Re
and X r Y. If infA1 ZsupA2 , then UnX r UnY 8n Z 1: Re
Proof. If A1 ¼ | or A2 ¼ | the result is obvious. So, let A1 a| and A2 a|. Since X r Y, for a 41 we have Z þ1 Z þ1 Z þ1 f a ðxÞ dx g a ðyÞ dy ¼ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z0, eða1ÞHa ðXÞ eða1ÞHa ðYÞ ¼ 1
ð5Þ
0
1
where we use transformations v ¼ logð1FðxÞÞ and v ¼ logð1GðxÞÞ to obtain the second equality. And for 0 o a o 1 we have Z þ1 ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dvr 0: ð6Þ 0
On the other hand, by noting (1) we can write Z þ1 Z X Y fUan ðxÞ dx eða1ÞHa ðUn Þ eða1ÞHa ðUn Þ ¼ 1
1 ½ðn1Þ!a
Z
þ1
1
þ1
gUan ðxÞ dx ¼
1 ½ðn1Þ!a
Z
þ1
½ðlogð1GðxÞÞÞn1 gðxÞa dx ¼
1
½ðlogð1FðxÞÞÞn1 f ðxÞa dx
1
1 D: ½ðn1Þ!a
For a 4 1, again using transformations v ¼ logð1FðxÞÞ, v ¼ logð1GðxÞÞ, we have Z þ1 vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1e vÞÞ dv D¼ Z0 Z ¼ vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv þ vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv A1 A2 Z ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv ZðinfA1 Þaðn1Þ A1 Z þ ðsupA2 Þaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z A2 ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv ZðinfA1 Þaðn1Þ A1 Z ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv þ ðinfA1 Þaðn1Þ A2 Z þ1 ¼ ðinfA1 Þaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z 0, 0
where we use condition infA1 Z supA2 to obtain the second inequality. The third inequality is obtained by using (5). In a similar way, but using (6), it can be shown that for 0 o a o 1 Z þ1 ev vaðn1Þ ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv r 0: 0
The result thus follows.
&
Example 2.1. Let X Weiðd, yÞ and Y Weiðd, yuÞ, where ða1Þ=a o yu o y o1. It can be easily shown that 1 1 ayða1Þ ayða1Þ logG loga, þ Ha ðXÞ ¼ logy y a1 y y Re
which is a decreasing function of y. So Ha ðXÞ r Ha ðYÞ, that is, X r Y. 1=y ðy1Þ=y x
Also f ðF 1 ð1ex ÞÞ ¼ yd
x
e
Re
and infA1 ¼ supA2 . Thus by Theorem 2.3 it follows that UnX r UnY
8n Z 1:
2316
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Theorem 2.4. Let X and Y be as in Theorem2.3. Suppose gðG1 ðev ÞÞ A1 ¼ v 4 0 1 v r1 , f ðF ðe ÞÞ gðG1 ðev ÞÞ A2 ¼ v 4 0 1 v 41 f ðF ðe ÞÞ Re
Re
and X r Y. If infA1 Z supA2 , then LXn r LYn 8n Z1: Proof. The proof is similar to the proof of Theorem 2.3 and is thus omitted.
&
3. Bounds for Re´nyi entropy of records Baratpur et al. (2007) obtained some bounds for Shannon entropy of records. In this section, we use the relations (3) and (4) to obtain bounds for the Re´nyi entropy of upper and lower records. Theorem 3.1. For any random variable X with Re´nyi entropy Ha ðXÞ o 1 the Re´nyi entropy of upper record Un ,n ¼ 1,2, . . . , is bounded as follows. (a) For all a 41
and
Ha ðUn Þ ZHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn þ Sa ðXÞ,
(b) for 0 o a o1
where
Ha ðUn Þ rHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn þ Sa ðXÞ,
(i) Un Gðn,1Þ, (ii) Bn ¼ ððaðn1ÞÞaðn1Þ =Gðaðn1Þ þ 1ÞÞeaðn1Þ , and R þ1 (iii) Sa ðXÞ ¼ ð1=ða1ÞÞlog 1 rðxÞf a1 ðxÞ dx, where r(x) is the hazard function of X. Proof. Since the mode of the gamma distribution with pdf gn is mn ¼ aðn1Þ, we have gn ðvÞ rBn ¼ gn ðmn Þ ¼
ðaðn1ÞÞaðn1Þ aðn1Þ e : Gðaðn1Þ þ 1Þ
Now for a 4 1 we can write
Z þ1 1 1 logEgn ½f a1 ðF 1 ð1eVn ÞÞ ¼ log gn ðvÞf a1 ðF 1 ð1ev ÞÞ dv a1 a1 0 Z þ1 1 1 logBn log f a1 ðF 1 ð1ev ÞÞ dv Z a1 a1 Z0þ1 1 1 logBn log rðyÞf a1 ðyÞ dy, ¼ a1 a1 1
where the last equality is reached by transformation y ¼ F 1 ð1ev Þ. Substituting the above in (3) gives the result. For 0 o a o1 the proof is similar. & Example 3.1. Let fXi ,i Z1g be a sequence of iid random variables having the exponential distribution with parameter y. R þ1 a1 Then rðyÞ ¼ y and Sa ðYÞ ¼ ð1=ða1ÞÞlog 0 rðyÞf a1 ðyÞ dy ¼ ð1=ða1ÞÞlog½y =ða1Þ: Therefore, we obtain the following bounds: Ha ðUn Þ ZHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn logða1Þlogy,
8a 41 and Ha ðUn Þ rHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn logða1Þlogy,
80 o a o1. On the other hand, it is easy to show that Ha ðUn Þ ¼
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ logalogy: a1 a1 a1
Another bound for Re´nyi entropy of Un is given in the following.
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2317
Theorem 3.2. Under the assumptions of Theorem3.1, Ha ðUn Þ Z Ha ðUn Þ
aðn1Þ þ 1 logalogM 8a 40, a1
where M ¼ fX ðmÞ o1, and m ¼ supfx : fX ðxÞ r Mg is the mode of the distribution. Proof. Since f ðxÞ r M, we have Z 1 log f a1 ðF 1 ð1ev ÞÞ ZlogM: a1 The result follows from (3).
&
When both lower bounds given in Theorems 3.1 and 3.2 can be computed, one may use the maximum of the two lower bounds. Example 3.2. Let fXi ,i Z1g be a sequence of iid random variables having the Pareto distribution with parameters y an b, that is f ðxÞ ¼
yby xy þ 1
x Z b 40, y 40: 1
Here, we observe M ¼ yb
and rðxÞ ¼ y=x.
By using Theorem 3.1, the following bounds are obtained: " # 1 ya byða1Þ1 ðaðn1Þ þ 1Þloga þ logBn þ log Ha ðUn Þ Z Ha ðUn Þ a1 y þ1 8a 4 1 and Ha ðUn Þ r Ha ðUn Þ
" # 1 ya byða1Þ1 ðaðn1Þ þ 1Þloga þ logBn þ log a1 y þ1
80 o a o 1: By using Theorem 3.2 we have Ha ðUn Þ Z Ha ðUn Þ
aðn1Þ þ 1 y logalog a1 b
8a 4 0:
Also, it is easy to see that
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ logð2y þ 1Þ a1 a1 a1 aðn1Þ þ 2 1 logy þ logb: þ a1 a1 The difference between this lower bound and Ha ðUn Þ is ððaðn1Þ þ 1Þ=ða1ÞÞlogð2y þ1Þ þððan þ 1Þ=ða1ÞÞlogðyÞða=ða1ÞÞlogb which is increasing in n for y 4 12, a 4 1 and y o 12,0 o a o 1 and it is decreasing in n for y o 12, a 41 and y 4 12,0 o a o1. Ha ðUn Þ ¼
Remark 3.1. Similar bounds for upper records, given in Theorems 3.1 and 3.2, are obtained for lower records. 4. Characterization of exponential distribution by Re´nyi entropy of records ‘‘To produce a model for the data-generating distribution the well-known maximum entropy (ME) paradigm can be employed. In the ME procedure, we begin with the fact that the distribution F is unknown. However, we may have access to some information about this distribution and based on this distribution we wish to derive a model that best approximates the distribution F. We proceed by formulating the partial knowledge about F in terms of a set of information constraints which are usually moment constraints’’, See Ebrahimi (2000). Then, the inference is based on the model f that maximizes the entropy of X, subject to the information constraints. The exponential model plays an important role in survival analysis and in the life testing of a system. In this section, we derive exponential distribution as the distribution that maximizes the Re´nyi entropy of record values under some information constraints. Let C be a class of distributions F(x) of non-negative random variables X with F(0) =0 such that ðiÞ ðiiÞ
rðx, yÞ ¼ aðyÞbðxÞ, bðxÞ Z M,
M 40,
where bðxÞ ¼ BuðxÞ is a non-negative function of x and aðyÞ is a non-negative function of y. In the following theorem, we characterize C by the Re´nyi entropy of nth upper record value.
2318
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Theorem 4.1. The nth upper record of the distribution F(x) has maximum Re´nyi entropy in C, if and only if Fðx; yÞ ¼ 1eMaðyÞx : Proof. Let Un be the nth upper record of Fðx; yÞ, where Fðx; yÞ is in class C. Then by (3) we have a1 x 1 Z xaðn1Þ ex aðyÞb B1 eaðyÞBðB ðx=aðyÞÞÞ aðn1Þ þ 1 1 aðyÞ loga log dx Ha ðUn Þ ¼ Ha ðUn Þ a1 a1 Gðaðn1Þ þ 1Þ
Z ba1 B1 x eax xaðn1Þ aðn1Þ þ1 1 aðyÞ ¼ Ha ðUn Þ logalogaðyÞ log dx: a1 a1 Gðaðn1Þ þ1Þ Noting that bðxÞ Z M we have Ha ðUn Þ rHa ðUn Þ
aðn1Þ þ 1 1 logalogMlogaðyÞ log a1 a1
Z
xaðn1Þ eax
Gðaðn1Þ þ 1Þ
dx
¼ Ha ðUn ÞlogMlogaðyÞ, which is, by (3), the Re´nyi entropy of nth upper record of Fðx; yÞ ¼ 1eMaðyÞx . Therefore, the nth upper record of exponential distribution has maximum Re´nyi entropy in class C. On the other hand, suppose the nth upper record of Fðx; yÞ ¼ 1eaðyÞBðxÞ has maximum Re´nyi entropy in class C. Then by (2.2) we have x Z xaðn1Þ eax ba1 B1 aðn1Þ þ 1 1 aðyÞ Ha ðUn Þ ¼ Ha ðUn Þ logalogaðyÞ log dx a1 a1 Gðaðn1Þ þ 1Þ ¼ Ha ðUn ÞlogMlogaðyÞ, so 1
a1
log
Z
3a1 2 x b B1 6 7 xaðn1Þ eax aaðn1Þ þ 1 aðyÞ 6 7 dx ¼ 0: 4 5 M Gðaðn1Þ þ 1Þ
Hence 82 3a1 9 > > x > > 1 > > Z < b B = aðn1Þ eax aaðn1Þ þ 1 6 7 að y Þ 6 7 1 x dx ¼ 0: 4 5 > > M Gðaðn1Þ þ 1Þ > > > > ; : By noting bðxÞ ZM, the function under the integral sign is a non-negative function of x on ð0,1Þ. Therefore, x b B1 aðyÞ ¼ 1, x 4 0, M so d x 1 B1 ¼ , dx aðyÞ MaðyÞ which implies x 1 ¼ x þ gðyÞ: B1 aðyÞ MaðyÞ
ð7Þ
By noting X is a non-negative random variable, we have B1 ð0Þ ¼ 0 and so gðyÞ ¼ 0. Using the transformation y ¼ x=aðyÞ in (7) we conclude B1 ðyÞ ¼ y=M. Hence BðxÞ ¼ Mx, that is, X has exponential distribution. & 5. Re´nyi information of records The Re´nyi information of order a between two random variables X and Y with density functions f(x) and g(y), respectively, is given by Da ðf ,gÞ ¼
1 log a1
Z
þ1 1
f ðxÞ gðxÞ
a1
f ðxÞ dx:
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2319
Note that lim Da ðf ,gÞ ¼ Dðf ,gÞ ¼
a-1
Z
þ1
f ðxÞlog
1
f ðxÞ dx gðxÞ
is the Kullback–Leibler information between f and g. For testing a general hypothesis about parameters of one population, the likelihood ratio test statistic is of general use. The likelihood ratio test statistic is a measure of deviation between the maximum likelihood achieved under the null hypothesis and the maximum achieved over the whole parameter space. Following this philosophy, a different measure of deviation, like a divergence, can be used. Some tests based on divergences have already been proposed in the literature, and it has been shown that in many cases they represent good competitors to classical tests. For example, Salicru et al. (1994) and Morales et al. (1997, 2000, 2004) suggested to test composite hypotheses, using some families of divergence, like fdivergence or Re´nyi distance. Also, some goodness-of-fit test statistics have been presented based on entropy and information measures. Vasicek (1976) used the sample Shannon entropy estimate to test normality. Ebrahimi et al. (1992) proposed a test for exponentiality based on Kullback–Leibler information. Abbasnejad (to appear) obtained a test statistic for exponentiality based on Re´nyi information. Habibi et al. (2007) presented a goodness-of-fit test for exponentiality based on Kullback–Leibler information of records. In the following, we study Re´nyi information of records. Theorem 5.1. (i) The Re´nyi information between the nth upper record and the parent distribution is given by
aðn1Þ þ 1 loga: a1 where fn is pdf of Un and Y Gðn,1=aÞ. Da ðfn ,f Þ ¼ Ha ðYÞ ¼ Ha ðUn Þ þ
(ii) Da ðfn ,f Þ is an increasing function of n. Proof. (i) By transformations U ¼ logð1FðXÞÞ we have Z þ 1 aðn1Þ 1 u log Da ðfn ,f Þ ¼ eu du a1 ½ðn1Þ!a 0 ¼ Ha ðYÞ: Hence, the Re´nyi information between the distribution of the nth upper record and the parent distribution is distribution free. (ii) By part (i) Da ðfn ,f Þ ¼
1
a1
½alogGðnÞ þ logGðaðn1Þ þ1Þ:
By differentiating with respect to n and noting that c is an increasing function, we have dDa ðfn ,f Þ a ¼ ½cðnÞcðaðn1Þ þ1Þ Z 0: dn a1 Thus the result follows.
&
Since, by increasing n, we expect that the distance between the distribution of the nth upper record and the original distribution increases, Theorem 5.1 confirms our intuition. Remark 5.1. Similar results given in Theorem 5.1 hold for lower record values.
Acknowledgments The authors would like to thank the referees for their attention to the topic and useful comments and suggestions. The authors acknowledge the support received from Ordered and Spatial Data Center of Excellence of Ferdowsi University of Mashhad, Iran. References Abbasnejad, M., Arghami, N.R., 2006. Potential statistical evidence in experiments and Re´nyi information. J. Iranian Statist. Soc. (JIRSS) 5, 39–52. Abbasnejad, M., in press. Some goodness of fit tests based on Renyi information. Applied Mathematical Sciences. Ahmadi, J., Arghami, N.R., 2001a. Some univariate stochastic orders on record values. Comm. Statist. Theory Methods 30, 69–74. Ahmadi, J., Arghami, N.R., 2001b. On the Fisher information in record values. Metrika 53, 195–206. Ahmadi, J., Arghami, N.R., 2003. Comparing the Fisher information in record values and iid observations. Statistics 37, 435–441. Ahmadi, J., Balakrishnan, N., 2005. Preservation of some reliability properties by current records and record ranges. Statistics 39, 347–354. Arnold, B.C., Balakrishnan, N., Nagaraja, H.N., 1998. Records. John Wiley and Sons, New York. Baratpur, S., Ahmadi, J., Arghami, N.R., 2007. Entropy properties of record statistics. Statist. Papers 48, 197–213. Bickel, P.J., Lehmann, E.L., 1976. Descriptive statistics for nonparametric models. III. Dispersion. Ann. Statist. 4, 1139–1158. Chandler, K.N., 1952. The distribution and frequency of record values. J. Roy. Statist. Soc. Ser. B 14, 220–228. Ebrahimi, N., Habibullah, M., Soofi, E.S., 1992. Testing exponentiality based on Kullback–Leibler information. J. Roy. Statist. Soc. 54, 739–748.
2320
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Ebrahimi, N., 2000. The maximum entropy method for lifetime distributions. Sankhya 62, 236–243. Habibi, A., Ahmadi, J., Arghmi, N.R., 2006. Statistical evidence in experiments and in record values. Comm. Statist. Theory Methods 35, 1971–1983. Habibi, A., Yousefzadeh, F., Amini, M., Arghmi, N.R., 2007. Testing exponentiality based on record values. J. Iranian Statist. Soc. (JIRSS) 6, 77–87. Hofmann, G., Balakrishnan, N., 2004. Fisher information in k-records. Ann. Inst. Statist. Math. 56, 383–396. Hofmann, G., Nagaraja, H.N., 2003. Fisher information in record data. Metrika 57, 177–193. Kamps, U., 1994. Reliability properties of record values from non-identically distributed random variables. Comm. Statist. Theory Methods 23, 2101–2112. Kamps, U., 1995. A Concept of Generalized Order Statistics. Teubner, Stuttgart, Germany. Kochar, S.C., 1990. Some partial ordering results on record values. Comm. Statist. Theory Methods 19, 299–306. Morales, D., Pardo, L., Vajda, I., 1997. Some new statistics for testing hypotheses in parametric models. Multivariate Anal. 62, 137–168. Morales, D., Pardo, L., Vajda, I., 2000. Renyi statistic in directed families of exponential experiments. Statistics 34, 151–174. Morales, D., Pardo, L., Pardo, M.C., Vajda, I., 2004. Renyi statistics for testing composite hypotheses in general exponential models. Statistics 38, 133–147. Re´nyi, A., 1961. On measures of entropy and information. Proceedings of Fourth Berkeley Symposium on Mathematics, Statistics and Probability 1960, vol. I. University of California Press, Berkeley, pp. 547–561. Salicru, M., Morales, D., Mendenenz, M.L., Pardo, L., 1994. On the applications of divergence type measures in testing statistical hypotheses. Multivariate Anal. 51, 372–391. Shaked, M., Shanthikumar, J.G., 1994. Stochastic Orders and Their Applications. Academic Press, New York. Shannon, C.E., 1948. A mathematical theory of communication. Bell System Technical Journal 27, 379–423. Stepanov, A.V., Balakrishnan, N., Hofmann, G., 2003. Exact distribution and Fisher information of weak record values. Statist. Probab. Lett. 64, 69–81. Vasicek, O., 1976. A test for normality based on sample entropy. J. Amer. Statist. Soc. B 38, 54–59.
Contents lists available at ScienceDirect
Journal of Statistical Planning and Inference journal homepage: www.elsevier.com/locate/jspi
Re´nyi entropy properties of records M. Abbasnejad , N.R. Arghami Department of Statistics, School of Mathematical Sciences, Ferdowsi University of Mashhad, P.O. Box 1159, Mashhad 91775, Iran
a r t i c l e in f o
abstract
Article history: Received 17 September 2007 Received in revised form 13 October 2009 Accepted 14 January 2011 Available online 25 January 2011
We provide bounds for Re´nyi entropy of records. We also show that the Re´nyi entropy ordering of random variables determines the Re´nyi entropy ordering of their respective records. We characterize exponential distribution by maximization of Re´nyi entropy under some conditions. We show that Re´nyi distance between distribution of records and parent distribution is distribution free. & 2011 Elsevier B.V. All rights reserved.
Keywords: Exponential distribution Hazard function Maximum entropy principal Ordering Records Re´nyi entropy bounds Re´nyi distance
1. Introduction Let Xi ,iZ 1, be a sequence of iid continuous random variables with the cdf F(x) and the pdf f(x). An observation Xj will be called an upper record value if its value is greater than that of all previous observations. Thus Xj is an upper record value if Xj 4Xi for all i o j. By convention X1 is the first upper record value. An analogous definition can be given for lower record values. The times at which upper record values appear are given by the random variables Tj which are called record times and are defined by T1 =1 with probability 1 and, for j Z 2, Tj ¼ Minfi : Xi 4 XTj1 g: The waiting time between the ith upper record value and the (i+ 1)th upper record value is called the inter-record time (IRT), and is denoted by Di ¼ Ti þ 1 Ti ,i ¼ 1,2, . . . . Record times and inter-record times for lower record values are defined analogously. Let U1 ,U2 , . . . ,Un be the first n upper record values from a distribution with the cdf F(x) and the pdf f(x). Then the joint pdf of the first n upper record values and the marginal density of Un (the nth upper record value, n Z1) are given, respectively, by qðuÞ ¼
n 1 Y
rðui Þf ðun Þ,
u1 o oun ,
i¼1
fUn ðun Þ ¼
Rn1 ðun Þ f ðun Þ, ðn1Þ!
1 oun o þ 1,
where RðtÞ ¼ logð1FðtÞÞ, and rðtÞ ¼ RuðtÞ ¼ f ðtÞ=1FðtÞ is the hazard function. Corresponding author. Tel./fax: +98 511 8828605.
E-mail address: [email protected] (M. Abbasnejad). 0378-3758/$ - see front matter & 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.jspi.2011.01.017
ð1Þ
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2313
Let L1 ,L2 , . . . ,Ln be the first n lower record values from a distribution with the cdf F(x) and the pdf f(x). Then the joint pdf of the first n lower record values and the marginal density of Ln (the nth lower record value, n Z 1) are given, respectively, by pðlÞ ¼
n 1 Y
r~ ðli Þf ðln Þ,
l1 4 4 ln ,
i¼1
fLn ðln Þ ¼
n1 R~ ðln Þ f ðln Þ, ðn1Þ!
1o ln o þ1,
~ ¼ logðFðtÞÞ, and r~ ðtÞ ¼ RuðtÞ ~ where RðtÞ ¼ f ðtÞ=FðtÞ is the reversed hazard function. See Arnold et al. (1998) for more details. Records can be used in a wide range of problems, including seismology, sporting and athletic events, meteorological analysis, industrial stress testing and hydrology. Record values have been studied extensively in the literature. For example, Chandler (1952) studied the behavior of random record values arising from the ‘‘classical record model’’, that is, the record model, where underlying sample from which records are observed, is considered to consist of iid observations from continuous probability distribution. Some authors compared records with the same number of iid observations based on the amount of their information. Ahmadi and Arghami (2001b, 2003), Hofmann and Nagaraja (2003), Hofmann and Balakrishnan (2004), Stepanov et al. (2003) compared them based on Fisher information. Habibi et al. (2006) compared Kullback–Leibler information of records with the same number of iid observations. Abbasnejad and Arghami (2006) did the same thing in terms of Re´nyi information. Recently, Baratpur et al. (2007) studied some information properties of records based on Shannon entropy and mutual information. They provide some bounds for the entropy of records. We generalize some of their results to the case of Re´nyi entropy. The rest of the paper is organized as follows. In Section 2, we express Re´nyi entropy of nth upper record based on Re´nyi entropy of nth record of standard exponential distribution and study some properties of it. Section 3 provides bounds for Re´nyi entropy of records. In Section 4, we characterize exponential distribution by maximizing Re´nyi entropy of record values. In Section 5, it is shown that the Re´nyi information between record value and data distribution is distribution free. 2. Re´nyi entropy of records Ever since Shannon (1948), has proposed a measure of uncertainty in a discrete distribution based on the Bolltzmann entropy, there has been a great deal of interest in the measurement of uncertainty associated with a probability distribution. There is now a huge literature devoted to the applications, generalizations and properties of Shannon’s measure of uncertainty. Let X be a non-negative random variable with an absolutely continuous distribution with probability density function f(x). Then the Shannon entropy of the density f is defined by Z þ1 HðXÞ ¼ f ðxÞlogf ðxÞ dx: 0
With his work, a new branch of mathematics with applications in different areas such as physics, probability and statistics, communication theory and economics was opened up. Numerous entropy and information indices, among them Re´nyi entropy, have been developed and used in various disciplines and contexts. If X is a random variable having an absolutely continuous cdf F(x) and pdf f(x), then Re´nyi entropy of order a of the random variable X is defined by Z þ1 1 log Ha ðXÞ ¼ f a ðxÞ dx 8a 40ðaa1Þ, ð2Þ a1 1 where lim Ha ðXÞ ¼ HðXÞ ¼
a-1
Z
f ðxÞlogf ðxÞ dx
is the Shannon entropy of X (Re´nyi, 1961). In reliability theory, record values are used for statistical modeling, shock models, and they are closely connected with the occurrence times of a corresponding nonhomogeneous Poisson process, with some minimal repair scheme, and with the relation transform (see, Kamps, 1994, 1995). In this section, we explore properties of Re´nyi entropy for record values. In the following Lemma, we express Re´nyi entropy of nth upper record from an arbitrary distribution in terms of Re´nyi entropy of nth upper record of standard exponential distribution. Lemma 2.1. Let fXn ,n Z 1g be a sequence of iid continuous random variables from the distribution F(x) with density function f(x) and the quantile function F 1 ð:Þ. Let Un denote the nth upper record. Then the Re´nyi entropy of Un can be expressed as Ha ðUn Þ ¼ Ha ðUn Þ
aðn1Þ þ1 1 loga logEgn ½f a1 ðF 1 ð1eVn ÞÞ, a1 a1
ð3Þ
where Un Gðn,1Þ denotes nth upper record of standard exponential distribution and Vn is a random variable, whose pdf is denoted by gn and is distributed as Gðaðn1Þ þ 1,1Þ.
2314
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Proof. By formulas (1) and (2), and using transformation U= R(X) we have Z þ 1 aðn1Þ u 1 u e a 1 log logGðnÞ logGðaðn1Þ þ1Þ f a1 ðF 1 ð1eu ÞÞ du ¼ Ha ðUn Þ ¼ a1 ½ðn1Þ!a a1 a1 0 Z þ1 a ðn1Þ v 1 v e log f a1 ðF 1 ð1ev ÞÞ dv: a1 Gðaðn1Þ þ 1Þ 0 The result follows from the fact that the expression for Re´nyi entropy of Un is Ha ðUn Þ ¼
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ loga: a1 a1 a1
&
Lemma 2.2. Under the assumptions of Lemma 2.1, if Ln denotes the nth lower record, then the Re´nyi entropy of Ln can be expressed as Ha ðLn Þ ¼
a 1 1 logGðnÞ logGðaðn1Þ þ 1Þ logEgn ½f a1 ðF 1 ðeVn ÞÞ, a1 a1 a1
ð4Þ
where Vn is as in Lemma 2.1. Proof. The proof is similar to that of Lemma 2.1. Several ordering and reliability properties of 1995), Ahmadi and Arghami (2001a), and Ahmadi entropy of records in terms of ordering properties and Y denote random variables with distribution F ðxÞ ¼ 1FðxÞ and GðyÞ ¼ 1GðyÞ.
& record values have been studied by Kochar (1990), Kamps (1994, and Balakrishnan (2005). We now provide some results on the Re´nyi of parent distributions. We need the following definitions in which X functions F and G, density functions f and g, and survival functions st
Definition 2.1. The random variable X is said to be stochastically less than or equal to Y, denoted by X r Y, if F ðvÞ rGðvÞ for all v. st
Shaked and Shanthikumar (1994) showed that X r Y implies that for any non-decreasing (non-increasing) function d, E½dðXÞr ð Z ÞE½dðYÞ, provided the expectations exist. lr
Definition 2.2. The random variable X is said to be less than or equal to Y in likelihood ratio ordering, denoted by X r Y, if f ðxÞ=gðxÞ is non-increasing in x. lr
st
Note that X r Y implies X r Y (Bickel and Lehmann, 1976). Re
Definition 2.3. The random variable X is said to be less than or equal to Y in Re´nyi entropy ordering, denoted by X r Y, if Ha ðXÞ r Ha ðYÞ for all a 4 0. Theorem 2.1. Let fXn ,n Z 1g be as in Lemma 2.1. Let Un denote its nth upper record. If f(x) is non-decreasing in x, then Ha ðUn Þ is non-increasing in n. Proof. Using (3), we have Ha ðUn Þ ¼ cn
1
a1
logEgn ½f a1 ðF 1 ð1eVn ÞÞ,
where cn ¼ ða=ða1ÞÞlogGðnÞð1=ða1ÞÞGðaðn1Þ þ1Þ: So Ha ðUn þ 1 ÞHa ðUn Þ ¼ cn þ 1 cn
Eg ½f a1 ðF 1 ð1eVn þ 1 ÞÞ 1 log n þ 1 a1 1 : a1 ðF ð1eVn ÞÞ Egn ½f
Treating n as a continuous variable and taking derivative with respect to n gives dcn a ¼ ½cðnÞcðaðn1Þ þ1Þ, dn a1 where cðzÞ ¼ dlogGðzÞ=dz is the digamma function. Note that since cðzÞ is an increasing function, for all z, cn is a decreasing function of n. lr st It is easy to show that Vn r Vn þ 1 and then Vn r Vn þ 1 . By assumption, f a1 ðF 1 ð1ev ÞÞ is non-decreasing in v for a 4 1, thus Egn þ 1 ½f a1 ðF 1 ð1eVn þ 1 ÞÞ 4Egn ½f a1 ðF 1 ð1eVn ÞÞ: Therefore, Ha ðUn þ 1 ÞHa ðUn Þ r 0 and thus the result follows. The proof for 0 o a o 1 is similar.
&
Theorem 2.2. Let fXn ,n Z 1g be as in Lemma 2.1. Let Ln, denotes its nth lower record. If f(x) is non-increasing in x, then Ha ðLn Þ is non-increasing in n.
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Proof. The proof is similar to that of Theorem 2.1, except that (4) is used to obtain the result.
2315
&
In the next theorem we show that Re´nyi entropy ordering of two random Variables X and Y is transferred to their upper records. Theorem 2.3. Let X and Y be two continuous random variables with distribution functions F(x) and G(y) and density functions f(x) and g(y), respectively. Suppose gðG1 ð1ev ÞÞ r1 , A1 ¼ v 40 1 v f ðF ð1e ÞÞ gðG1 ð1ev ÞÞ 41 A2 ¼ v 40 1 f ðF ð1ev ÞÞ Re
Re
and X r Y. If infA1 ZsupA2 , then UnX r UnY 8n Z 1: Re
Proof. If A1 ¼ | or A2 ¼ | the result is obvious. So, let A1 a| and A2 a|. Since X r Y, for a 41 we have Z þ1 Z þ1 Z þ1 f a ðxÞ dx g a ðyÞ dy ¼ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z0, eða1ÞHa ðXÞ eða1ÞHa ðYÞ ¼ 1
ð5Þ
0
1
where we use transformations v ¼ logð1FðxÞÞ and v ¼ logð1GðxÞÞ to obtain the second equality. And for 0 o a o 1 we have Z þ1 ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dvr 0: ð6Þ 0
On the other hand, by noting (1) we can write Z þ1 Z X Y fUan ðxÞ dx eða1ÞHa ðUn Þ eða1ÞHa ðUn Þ ¼ 1
1 ½ðn1Þ!a
Z
þ1
1
þ1
gUan ðxÞ dx ¼
1 ½ðn1Þ!a
Z
þ1
½ðlogð1GðxÞÞÞn1 gðxÞa dx ¼
1
½ðlogð1FðxÞÞÞn1 f ðxÞa dx
1
1 D: ½ðn1Þ!a
For a 4 1, again using transformations v ¼ logð1FðxÞÞ, v ¼ logð1GðxÞÞ, we have Z þ1 vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1e vÞÞ dv D¼ Z0 Z ¼ vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv þ vaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv A1 A2 Z ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv ZðinfA1 Þaðn1Þ A1 Z þ ðsupA2 Þaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z A2 ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv ZðinfA1 Þaðn1Þ A1 Z ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv þ ðinfA1 Þaðn1Þ A2 Z þ1 ¼ ðinfA1 Þaðn1Þ ev ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv Z 0, 0
where we use condition infA1 Z supA2 to obtain the second inequality. The third inequality is obtained by using (5). In a similar way, but using (6), it can be shown that for 0 o a o 1 Z þ1 ev vaðn1Þ ½f a1 ðF 1 ð1ev ÞÞg a1 ðG1 ð1ev ÞÞ dv r 0: 0
The result thus follows.
&
Example 2.1. Let X Weiðd, yÞ and Y Weiðd, yuÞ, where ða1Þ=a o yu o y o1. It can be easily shown that 1 1 ayða1Þ ayða1Þ logG loga, þ Ha ðXÞ ¼ logy y a1 y y Re
which is a decreasing function of y. So Ha ðXÞ r Ha ðYÞ, that is, X r Y. 1=y ðy1Þ=y x
Also f ðF 1 ð1ex ÞÞ ¼ yd
x
e
Re
and infA1 ¼ supA2 . Thus by Theorem 2.3 it follows that UnX r UnY
8n Z 1:
2316
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Theorem 2.4. Let X and Y be as in Theorem2.3. Suppose gðG1 ðev ÞÞ A1 ¼ v 4 0 1 v r1 , f ðF ðe ÞÞ gðG1 ðev ÞÞ A2 ¼ v 4 0 1 v 41 f ðF ðe ÞÞ Re
Re
and X r Y. If infA1 Z supA2 , then LXn r LYn 8n Z1: Proof. The proof is similar to the proof of Theorem 2.3 and is thus omitted.
&
3. Bounds for Re´nyi entropy of records Baratpur et al. (2007) obtained some bounds for Shannon entropy of records. In this section, we use the relations (3) and (4) to obtain bounds for the Re´nyi entropy of upper and lower records. Theorem 3.1. For any random variable X with Re´nyi entropy Ha ðXÞ o 1 the Re´nyi entropy of upper record Un ,n ¼ 1,2, . . . , is bounded as follows. (a) For all a 41
and
Ha ðUn Þ ZHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn þ Sa ðXÞ,
(b) for 0 o a o1
where
Ha ðUn Þ rHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn þ Sa ðXÞ,
(i) Un Gðn,1Þ, (ii) Bn ¼ ððaðn1ÞÞaðn1Þ =Gðaðn1Þ þ 1ÞÞeaðn1Þ , and R þ1 (iii) Sa ðXÞ ¼ ð1=ða1ÞÞlog 1 rðxÞf a1 ðxÞ dx, where r(x) is the hazard function of X. Proof. Since the mode of the gamma distribution with pdf gn is mn ¼ aðn1Þ, we have gn ðvÞ rBn ¼ gn ðmn Þ ¼
ðaðn1ÞÞaðn1Þ aðn1Þ e : Gðaðn1Þ þ 1Þ
Now for a 4 1 we can write
Z þ1 1 1 logEgn ½f a1 ðF 1 ð1eVn ÞÞ ¼ log gn ðvÞf a1 ðF 1 ð1ev ÞÞ dv a1 a1 0 Z þ1 1 1 logBn log f a1 ðF 1 ð1ev ÞÞ dv Z a1 a1 Z0þ1 1 1 logBn log rðyÞf a1 ðyÞ dy, ¼ a1 a1 1
where the last equality is reached by transformation y ¼ F 1 ð1ev Þ. Substituting the above in (3) gives the result. For 0 o a o1 the proof is similar. & Example 3.1. Let fXi ,i Z1g be a sequence of iid random variables having the exponential distribution with parameter y. R þ1 a1 Then rðyÞ ¼ y and Sa ðYÞ ¼ ð1=ða1ÞÞlog 0 rðyÞf a1 ðyÞ dy ¼ ð1=ða1ÞÞlog½y =ða1Þ: Therefore, we obtain the following bounds: Ha ðUn Þ ZHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn logða1Þlogy,
8a 41 and Ha ðUn Þ rHa ðUn Þ
1
a1
½ðaðn1Þ þ 1Þloga þ logBn logða1Þlogy,
80 o a o1. On the other hand, it is easy to show that Ha ðUn Þ ¼
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ logalogy: a1 a1 a1
Another bound for Re´nyi entropy of Un is given in the following.
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2317
Theorem 3.2. Under the assumptions of Theorem3.1, Ha ðUn Þ Z Ha ðUn Þ
aðn1Þ þ 1 logalogM 8a 40, a1
where M ¼ fX ðmÞ o1, and m ¼ supfx : fX ðxÞ r Mg is the mode of the distribution. Proof. Since f ðxÞ r M, we have Z 1 log f a1 ðF 1 ð1ev ÞÞ ZlogM: a1 The result follows from (3).
&
When both lower bounds given in Theorems 3.1 and 3.2 can be computed, one may use the maximum of the two lower bounds. Example 3.2. Let fXi ,i Z1g be a sequence of iid random variables having the Pareto distribution with parameters y an b, that is f ðxÞ ¼
yby xy þ 1
x Z b 40, y 40: 1
Here, we observe M ¼ yb
and rðxÞ ¼ y=x.
By using Theorem 3.1, the following bounds are obtained: " # 1 ya byða1Þ1 ðaðn1Þ þ 1Þloga þ logBn þ log Ha ðUn Þ Z Ha ðUn Þ a1 y þ1 8a 4 1 and Ha ðUn Þ r Ha ðUn Þ
" # 1 ya byða1Þ1 ðaðn1Þ þ 1Þloga þ logBn þ log a1 y þ1
80 o a o 1: By using Theorem 3.2 we have Ha ðUn Þ Z Ha ðUn Þ
aðn1Þ þ 1 y logalog a1 b
8a 4 0:
Also, it is easy to see that
a 1 aðn1Þ þ 1 logGðnÞ logGðaðn1Þ þ 1Þ þ logð2y þ 1Þ a1 a1 a1 aðn1Þ þ 2 1 logy þ logb: þ a1 a1 The difference between this lower bound and Ha ðUn Þ is ððaðn1Þ þ 1Þ=ða1ÞÞlogð2y þ1Þ þððan þ 1Þ=ða1ÞÞlogðyÞða=ða1ÞÞlogb which is increasing in n for y 4 12, a 4 1 and y o 12,0 o a o 1 and it is decreasing in n for y o 12, a 41 and y 4 12,0 o a o1. Ha ðUn Þ ¼
Remark 3.1. Similar bounds for upper records, given in Theorems 3.1 and 3.2, are obtained for lower records. 4. Characterization of exponential distribution by Re´nyi entropy of records ‘‘To produce a model for the data-generating distribution the well-known maximum entropy (ME) paradigm can be employed. In the ME procedure, we begin with the fact that the distribution F is unknown. However, we may have access to some information about this distribution and based on this distribution we wish to derive a model that best approximates the distribution F. We proceed by formulating the partial knowledge about F in terms of a set of information constraints which are usually moment constraints’’, See Ebrahimi (2000). Then, the inference is based on the model f that maximizes the entropy of X, subject to the information constraints. The exponential model plays an important role in survival analysis and in the life testing of a system. In this section, we derive exponential distribution as the distribution that maximizes the Re´nyi entropy of record values under some information constraints. Let C be a class of distributions F(x) of non-negative random variables X with F(0) =0 such that ðiÞ ðiiÞ
rðx, yÞ ¼ aðyÞbðxÞ, bðxÞ Z M,
M 40,
where bðxÞ ¼ BuðxÞ is a non-negative function of x and aðyÞ is a non-negative function of y. In the following theorem, we characterize C by the Re´nyi entropy of nth upper record value.
2318
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Theorem 4.1. The nth upper record of the distribution F(x) has maximum Re´nyi entropy in C, if and only if Fðx; yÞ ¼ 1eMaðyÞx : Proof. Let Un be the nth upper record of Fðx; yÞ, where Fðx; yÞ is in class C. Then by (3) we have a1 x 1 Z xaðn1Þ ex aðyÞb B1 eaðyÞBðB ðx=aðyÞÞÞ aðn1Þ þ 1 1 aðyÞ loga log dx Ha ðUn Þ ¼ Ha ðUn Þ a1 a1 Gðaðn1Þ þ 1Þ
Z ba1 B1 x eax xaðn1Þ aðn1Þ þ1 1 aðyÞ ¼ Ha ðUn Þ logalogaðyÞ log dx: a1 a1 Gðaðn1Þ þ1Þ Noting that bðxÞ Z M we have Ha ðUn Þ rHa ðUn Þ
aðn1Þ þ 1 1 logalogMlogaðyÞ log a1 a1
Z
xaðn1Þ eax
Gðaðn1Þ þ 1Þ
dx
¼ Ha ðUn ÞlogMlogaðyÞ, which is, by (3), the Re´nyi entropy of nth upper record of Fðx; yÞ ¼ 1eMaðyÞx . Therefore, the nth upper record of exponential distribution has maximum Re´nyi entropy in class C. On the other hand, suppose the nth upper record of Fðx; yÞ ¼ 1eaðyÞBðxÞ has maximum Re´nyi entropy in class C. Then by (2.2) we have x Z xaðn1Þ eax ba1 B1 aðn1Þ þ 1 1 aðyÞ Ha ðUn Þ ¼ Ha ðUn Þ logalogaðyÞ log dx a1 a1 Gðaðn1Þ þ 1Þ ¼ Ha ðUn ÞlogMlogaðyÞ, so 1
a1
log
Z
3a1 2 x b B1 6 7 xaðn1Þ eax aaðn1Þ þ 1 aðyÞ 6 7 dx ¼ 0: 4 5 M Gðaðn1Þ þ 1Þ
Hence 82 3a1 9 > > x > > 1 > > Z < b B = aðn1Þ eax aaðn1Þ þ 1 6 7 að y Þ 6 7 1 x dx ¼ 0: 4 5 > > M Gðaðn1Þ þ 1Þ > > > > ; : By noting bðxÞ ZM, the function under the integral sign is a non-negative function of x on ð0,1Þ. Therefore, x b B1 aðyÞ ¼ 1, x 4 0, M so d x 1 B1 ¼ , dx aðyÞ MaðyÞ which implies x 1 ¼ x þ gðyÞ: B1 aðyÞ MaðyÞ
ð7Þ
By noting X is a non-negative random variable, we have B1 ð0Þ ¼ 0 and so gðyÞ ¼ 0. Using the transformation y ¼ x=aðyÞ in (7) we conclude B1 ðyÞ ¼ y=M. Hence BðxÞ ¼ Mx, that is, X has exponential distribution. & 5. Re´nyi information of records The Re´nyi information of order a between two random variables X and Y with density functions f(x) and g(y), respectively, is given by Da ðf ,gÞ ¼
1 log a1
Z
þ1 1
f ðxÞ gðxÞ
a1
f ðxÞ dx:
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
2319
Note that lim Da ðf ,gÞ ¼ Dðf ,gÞ ¼
a-1
Z
þ1
f ðxÞlog
1
f ðxÞ dx gðxÞ
is the Kullback–Leibler information between f and g. For testing a general hypothesis about parameters of one population, the likelihood ratio test statistic is of general use. The likelihood ratio test statistic is a measure of deviation between the maximum likelihood achieved under the null hypothesis and the maximum achieved over the whole parameter space. Following this philosophy, a different measure of deviation, like a divergence, can be used. Some tests based on divergences have already been proposed in the literature, and it has been shown that in many cases they represent good competitors to classical tests. For example, Salicru et al. (1994) and Morales et al. (1997, 2000, 2004) suggested to test composite hypotheses, using some families of divergence, like fdivergence or Re´nyi distance. Also, some goodness-of-fit test statistics have been presented based on entropy and information measures. Vasicek (1976) used the sample Shannon entropy estimate to test normality. Ebrahimi et al. (1992) proposed a test for exponentiality based on Kullback–Leibler information. Abbasnejad (to appear) obtained a test statistic for exponentiality based on Re´nyi information. Habibi et al. (2007) presented a goodness-of-fit test for exponentiality based on Kullback–Leibler information of records. In the following, we study Re´nyi information of records. Theorem 5.1. (i) The Re´nyi information between the nth upper record and the parent distribution is given by
aðn1Þ þ 1 loga: a1 where fn is pdf of Un and Y Gðn,1=aÞ. Da ðfn ,f Þ ¼ Ha ðYÞ ¼ Ha ðUn Þ þ
(ii) Da ðfn ,f Þ is an increasing function of n. Proof. (i) By transformations U ¼ logð1FðXÞÞ we have Z þ 1 aðn1Þ 1 u log Da ðfn ,f Þ ¼ eu du a1 ½ðn1Þ!a 0 ¼ Ha ðYÞ: Hence, the Re´nyi information between the distribution of the nth upper record and the parent distribution is distribution free. (ii) By part (i) Da ðfn ,f Þ ¼
1
a1
½alogGðnÞ þ logGðaðn1Þ þ1Þ:
By differentiating with respect to n and noting that c is an increasing function, we have dDa ðfn ,f Þ a ¼ ½cðnÞcðaðn1Þ þ1Þ Z 0: dn a1 Thus the result follows.
&
Since, by increasing n, we expect that the distance between the distribution of the nth upper record and the original distribution increases, Theorem 5.1 confirms our intuition. Remark 5.1. Similar results given in Theorem 5.1 hold for lower record values.
Acknowledgments The authors would like to thank the referees for their attention to the topic and useful comments and suggestions. The authors acknowledge the support received from Ordered and Spatial Data Center of Excellence of Ferdowsi University of Mashhad, Iran. References Abbasnejad, M., Arghami, N.R., 2006. Potential statistical evidence in experiments and Re´nyi information. J. Iranian Statist. Soc. (JIRSS) 5, 39–52. Abbasnejad, M., in press. Some goodness of fit tests based on Renyi information. Applied Mathematical Sciences. Ahmadi, J., Arghami, N.R., 2001a. Some univariate stochastic orders on record values. Comm. Statist. Theory Methods 30, 69–74. Ahmadi, J., Arghami, N.R., 2001b. On the Fisher information in record values. Metrika 53, 195–206. Ahmadi, J., Arghami, N.R., 2003. Comparing the Fisher information in record values and iid observations. Statistics 37, 435–441. Ahmadi, J., Balakrishnan, N., 2005. Preservation of some reliability properties by current records and record ranges. Statistics 39, 347–354. Arnold, B.C., Balakrishnan, N., Nagaraja, H.N., 1998. Records. John Wiley and Sons, New York. Baratpur, S., Ahmadi, J., Arghami, N.R., 2007. Entropy properties of record statistics. Statist. Papers 48, 197–213. Bickel, P.J., Lehmann, E.L., 1976. Descriptive statistics for nonparametric models. III. Dispersion. Ann. Statist. 4, 1139–1158. Chandler, K.N., 1952. The distribution and frequency of record values. J. Roy. Statist. Soc. Ser. B 14, 220–228. Ebrahimi, N., Habibullah, M., Soofi, E.S., 1992. Testing exponentiality based on Kullback–Leibler information. J. Roy. Statist. Soc. 54, 739–748.
2320
M. Abbasnejad, N.R. Arghami / Journal of Statistical Planning and Inference 141 (2011) 2312–2320
Ebrahimi, N., 2000. The maximum entropy method for lifetime distributions. Sankhya 62, 236–243. Habibi, A., Ahmadi, J., Arghmi, N.R., 2006. Statistical evidence in experiments and in record values. Comm. Statist. Theory Methods 35, 1971–1983. Habibi, A., Yousefzadeh, F., Amini, M., Arghmi, N.R., 2007. Testing exponentiality based on record values. J. Iranian Statist. Soc. (JIRSS) 6, 77–87. Hofmann, G., Balakrishnan, N., 2004. Fisher information in k-records. Ann. Inst. Statist. Math. 56, 383–396. Hofmann, G., Nagaraja, H.N., 2003. Fisher information in record data. Metrika 57, 177–193. Kamps, U., 1994. Reliability properties of record values from non-identically distributed random variables. Comm. Statist. Theory Methods 23, 2101–2112. Kamps, U., 1995. A Concept of Generalized Order Statistics. Teubner, Stuttgart, Germany. Kochar, S.C., 1990. Some partial ordering results on record values. Comm. Statist. Theory Methods 19, 299–306. Morales, D., Pardo, L., Vajda, I., 1997. Some new statistics for testing hypotheses in parametric models. Multivariate Anal. 62, 137–168. Morales, D., Pardo, L., Vajda, I., 2000. Renyi statistic in directed families of exponential experiments. Statistics 34, 151–174. Morales, D., Pardo, L., Pardo, M.C., Vajda, I., 2004. Renyi statistics for testing composite hypotheses in general exponential models. Statistics 38, 133–147. Re´nyi, A., 1961. On measures of entropy and information. Proceedings of Fourth Berkeley Symposium on Mathematics, Statistics and Probability 1960, vol. I. University of California Press, Berkeley, pp. 547–561. Salicru, M., Morales, D., Mendenenz, M.L., Pardo, L., 1994. On the applications of divergence type measures in testing statistical hypotheses. Multivariate Anal. 51, 372–391. Shaked, M., Shanthikumar, J.G., 1994. Stochastic Orders and Their Applications. Academic Press, New York. Shannon, C.E., 1948. A mathematical theory of communication. Bell System Technical Journal 27, 379–423. Stepanov, A.V., Balakrishnan, N., Hofmann, G., 2003. Exact distribution and Fisher information of weak record values. Statist. Probab. Lett. 64, 69–81. Vasicek, O., 1976. A test for normality based on sample entropy. J. Amer. Statist. Soc. B 38, 54–59.