- Email: [email protected]
Results on residual Rényi entropy of order statistics and record values
Information Sciences 180 (2010) 4195–4206
Contents lists available at ScienceDirect
Information Sciences journal homepage: www.elsevier.com/locate/ins
Results on residual Rényi entropy of order statistics and record values S. Zarezadeh *, M. Asadi Department of Statistics, University of Isfahan, Isfahan 81744, Iran
a r t i c l e
i n f o
a b s t r a c t
Article history: Received 22 July 2009 Received in revised form 9 June 2010 Accepted 13 June 2010
Keywords: Order statistics Record values Rényi entropy Residual lifetime Shannon entropy Reliability (n k + 1)-out-of-n systems
This paper explores properties of the residual Rényi entropy of some ordered random variables. The residual Rényi entropy of the kth order statistic from a continuous distribution function is represented in terms of the residual Rényi entropy of the kth order statistic from uniform distribution. The monotone behavior of the residual Rényi entropy of order statistic under various conditions is discussed. Analogues results for the residual Rényi entropy of record values are also given. Ó 2010 Elsevier Inc. All rights reserved.
1. Introduction Let X be a non-negative continuous random variable with density function f, distribution function F and the survival function F ¼ 1 F. Shannon entropy of F is defined by
HðXÞ ¼ HðFÞ ¼
Z
1
f ðxÞ log f ðxÞ dx:
0
In continuous case, H(X) is also referred to as the differential entropy. Shannon entropy plays a central role in the field of information theory and has a wide range of applications in many fields. It is known that H(X) measures the uniformity of f. When H(F1) > H(F2), it is more difficult to predict outcomes of F1 as compared with predicting outcomes of F2. Rényi [25] introduced a one parameter extension of Shannon entropy which is more flexible than H(X) and also has been used in various fields. Rényi entropy of X is defined by
Ha ðXÞ ¼
1 log 1a
Z
1
f a ðxÞ dx;
0
where a > 0, a – 1. It can be easily shown that lima?1Ha(X) = H(X). Properties of Rényi entropy have been studied by many authors, including Rényi [25], Morales et al. [18], Song [27], Nadarjah and Zografos [19,20], Bercher [7], De Gregorio and Iacus [9] and Golshani et al. [13]. In reliability theory and survival analysis, X usually denotes a duration such as the lifetime. The residual lifetime of the system when it is still operating at time t, is Xt = X tjX > t which has probability density function * Corresponding author. E-mail address: [email protected] (S. Zarezadeh). 0020-0255/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.ins.2010.06.019
4196
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
f ðx; tÞ ¼
f ðxÞ ; FðtÞ
x P t > 0;
where FðtÞ > 0. Ebrahimi [10] proposed using the entropy of residual lifetime distribution
HðX; tÞ ¼
Z
1
f ðx; tÞ log f ðx; tÞ dx;
t > 0:
t
The residual entropy is time-dependent and measures the uncertainty of the residual lifetime of the system when it is still operating at time t. Several authors have studied properties of H(X; t); see, for example, Ebrahimi and Kirmani [11], Asadi and Ebrahimi [2] and Belzunce et al. [6]. The residual Rényi entropy (RRE) is defined similarly:
Ha ðX; tÞ ¼
1 log 1a
Z t
1
f a ðxÞ F a ðtÞ
dx;
ð1Þ
where a > 0, a – 1. Properties of RRE have been investigated by several authors including Gupta and Nanda [15], Asadi et al. [3], Nanda and Paul [22], Nanda and Maiti [21] and Mahmoudi and Asadi [17]. The aim of the present paper is to study properties of RRE of order statistics and record values. Let X1, . . . , Xn denote a random sample of size n from F. Order statistics refer to the arrangement of X1, . . . , Xn from the smallest to the largest, denoted as X1:n 6 X2:n 6 6 Xn:n. Order statistics are used in many branches of probability and statistics including characterization of probability distributions, goodness-of-fit tests, quality control, reliability analysis and many other problems. Order statistics are of particular interest in reliability theory in the study of the lifetime properties of the coherent systems and in life testing, when data are collected based on different censoring mechanisms. For a comprehensive review on the theory and applications of order statistics one can refer to Daivid and Nagaraja [8]. A record value of a sequence of independent identically distributed (i.i.d.) random variables {Xi; i P 1} is an observation Xj whose value exceeds or is less than the values of all previous observations. For example, Xj is an upper record if Xi < Xj for every i < j. Record values arise naturally in problems such as industrial stress testing, meteorological analysis, hydrology, sporting and athletic events, and economics. In reliability, records model are used to study, for example, technical systems which are subject to shocks, e.g. peaks of voltages. Successive large shocks may be viewed as realizations of records from a sequence of identically independent voltages. For more details about records and their applications, one may refer to Arnold et al. [1] and Nevzorov [23]. Several authors have studied the information properties of ordered data. Wong and Chen [28] showed that the difference between the average entropy of order statistics and the entropy of the parent distribution is a constant. They also showed that when the distribution of the data is symmetric, the entropy of order statistics is symmetric about the median. Park [24] obtained some recurrence relations for the entropy of order statistics. Ebrahimi et al. [12] explored some properties of the Shannon entropy of the order statistics and showed that the Kullback–Leibler information functions involving order statistics are distribution free. Baratpour et al. [4,5] obtained some results for the Shannon entropy and Rényi entropy of the order statistics and record values. We continue this line of research by exploring properties of RRE of order statistics and record values. In Section 2, we represent the RRE of order statistics Xk:n of a sample from any continuous distribution function F in terms of RRE of order statistics of a sample from uniform distribution. Since for many statistical models the functional form of the RRE of order statistics cannot be obtained in a closed form, we obtain upper and lower bounds for RRE of order statistics. Several illustrative examples are given. We also show that, under some mild conditions, the RRE’s of the minimum and maximum of a random sample are monotone functions of the number of observations of sample. We give a counter example to show that the RRE of other order statistics Xk:n is not necessary monotone function of n. We also study the monotone behavior of RRE of order statistics Xk:n in terms of k. It is shown that the RRE of Xk:n is not a monotone function of k on the entire support of F. In Section 3, we investigate properties of RRE of record values. We give bounds for RRE of record values. We also show that, under some mild conditions, the RRE of record values is monotone function of number of records in the sequence. Proofs of the results are gathered in an Appendix. Before proceeding to give the main results of the paper, we overview some preliminary concepts on partial orderings between random variables. (For more details of these concepts one can see [26].) Let X and Y be two random variables with survival functions F and G and density functions f and g, respectively. Definition 1.1 (a) The random variable Y is said to be smaller than X in the usual stochastic order (denoted by Y 6stX) if GðxÞ 6 FðxÞ for all x. f ðxÞ (b) The random variable Y is said to be smaller than X in likelihood ratio order (denoted by Y 6lrX) if gðxÞ is an increasing function of x. It can be shown that if Y 6 lrX, then Y 6 stX (see [26], for more details). Throughout the paper increasing (decreasing) means non-decreasing (non-increasing).
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4197
2. The residual Rényi entropy of order statistics In this section, we focus on the RRE of order statistics. First note that the density function and survival function of Xk:n denoted by fk:n(x) and F k:n ðxÞ, k = 1, . . . , n, respectively, are
fk:n ðxÞ ¼
1 ½FðxÞk1 ½1 FðxÞnk f ðxÞ Bðk; n k þ 1Þ
ð2Þ
and
F k:n ðxÞ ¼
k1 X n
i
i¼0
F i ðxÞF ni ðxÞ;
ð3Þ
where
Bða; bÞ ¼
Z
1
xa1 ð1 xÞb1 dx;
a > 0; b > 0:
0
See, for example, David and Nagaraja [8, pp. 9–10]. The survival function F k:n ðxÞ can also be represented as
F k:n ðxÞ ¼
BFðxÞ ðk; n k þ 1Þ ; Bðk; n k þ 1Þ
ð4Þ
where
Bx ða; bÞ ¼
Z
1
ua1 ð1 uÞb1 du;
0 < x < 1:
x
B(a, b) and Bx ða; bÞ are known as the beta and the incomplete beta functions, respectively (see, for example [8]). Notation: Throughout this section we use the notation Y Bt ða; bÞ to show that Y has a truncated beta distribution with density function
fY ðyÞ ¼
1 Bt ða; bÞ
ya1 ð1 yÞb1 ;
t 6 y 6 1:
ð5Þ
Remark 2.1. In reliability engineering (n k + 1)-out-of-n systems are very important kind of structures. A (n k + 1)-outof-n system functions if and only if at least (n k + 1) components out of n components function. If X1, X2, . . . , Xn denote the independent lifetimes of the components of such system, then the lifetime of the system is equal to the order statistics Xk:n. The special cases of k = 1 and k = n are corresponding to series and parallel systems, respectively. Assuming that a (n k + 1)out-of-n system is working at time t, then the RRE of Xk:n measures the entropy of the residual lifetime of the system. Hence the RRE, as a dynamic measure of entropy, can be important for system designers to get information about the entropy of used (n k + 1)-out-of-n systems at any time t. The following lemma shows that the RRE of order statistics of uniform distribution can be written in terms of incomplete beta function which is important in computational point of view. Its proof, which follows from the definition of RRE, is an easy exercise and hence is omitted. Lemma 2.2. Let Uk:n be kth order statistic based on a random sample of size n from uniform distribution on (0, 1). Then
Ha ðU k:n ; tÞ ¼
1 a log Bt ðaðk 1Þ þ 1; aðn kÞ þ 1Þ log Bt ðk; n k þ 1Þ: 1a 1a
If F is a continuous distribution function then it is well known, from the probability integral transformation, that d
U k:n ¼ FðX k:n Þ;
k ¼ 1; . . . ; n;
where ‘d’ stands for equality in distribution and Xk:n is the kth order statistic based on a random sample of size n from F (see, for example [8]). Using this, in the following theorem, we will show that the RRE of order statistics Xk:n can be represented in terms of RRE of order statistics of uniform distribution. The proof of the theorem is straightforward and hence is omitted. Theorem 2.3. Let F be an absolutely continuous distribution function with density f. Then the RRE of the kth order statistic can be represented in terms of the RRE of kth order statistic from uniform distribution, over the unit interval, as follows:
Ha ðX k:n ; tÞ ¼ Ha ðU k:n ; FðtÞÞ þ
1 log E½f a1 ðF 1 ðY k ÞÞ; 1a
where Y k BFðtÞ ðaðk 1Þ þ 1; aðn kÞ þ 1Þ:
ð6Þ
4198
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
It can be seen, after some calculations, that when in (6) a ? 1, the Shannon entropy of kth order statistic from a sample of F can be written as follows:
HðX k:n ; tÞ ¼ HðU k:n ; FðtÞÞ E½log f ðF 1 ðY k ÞÞ;
ð7Þ
where Y k BFðtÞ ðk; n k þ 1Þ: The specialized version of this result for t = 0, was already obtained by Ebrahimi et al. [12]. Remark 2.4. The quantity f(F1(x)) is known, in the literature, as the density-quantile function and is used to approximate the moments of order statistics (see [8]). In the following we give some examples. Example 2.5. Suppose that F is exponential with mean 1h . Then f(F1(y)) = h(1 y) and we have
E½f a1 ðF 1 ðY 1 ÞÞ ¼
ha1 ehtð1aÞ : na½aðn 1Þ þ 1
For k = 1, Theorem 2.3 gives
Ha ðX 1:n ; tÞ ¼
log a logðnhÞ: a1
On the other hand, we have
Ha ðX; tÞ ¼
log a log h: a1
This gives
Ha ðX 1:n ; tÞ Ha ðX; tÞ ¼ log n: That is, in the exponential case the difference between of RRE of the lifetime of a series system and RRE of the lifetime of each components is free of both of the time and a depends only on the number of components of the system. Example 2.6. Let X have Pareto distribution with distribution function
FðxÞ ¼ 1
h b ; x
x P b > 0;
h>0
and density function
f ðxÞ ¼
hbh xhþ1
x P b > 0;
h > 0:
Then
h 1 f F 1 ðyÞ ¼ ð1 yÞ1þh : b Therefore for the first order statistic of a random sample of size n we have
E½f a1 ðF 1 ðY 1 ÞÞ ¼
ða1Þð1þhÞ
aðn 1Þ þ 1 a 1a b h b t aðnh þ 1Þ 1
:
Hence Theorem 2.3 gives
Ha ðX 1:n ; tÞ ¼ log t þ
a 1a
log nh
1 log ½aðnh þ 1Þ 1; 1a
a>
1 : nh þ 1
The difference between RRE of X1:n and RRE of X is
Ha ðX 1:n ; tÞ Ha ðX; tÞ ¼
a 1a
log n
1 aðnh þ 1Þ 1 log ; 1a aðh þ 1Þ 1
a>
1 ; hþ1
which is free of t and depends only on a and n. We obtained the closed form of RRE of the first order statistics in exponential and Pareto distributions. However, we do not have a closed form for RRE of other order statistics for these distributions. This is true, in general for other distributions, that there is no closed form for the RRE of order statistics. This gives a motivation for obtaining some bounds for RRE of order statistics. Hence we prove the following theorem. Theorem 2.7. Let X be a non-negative continuous random variable with density function f and distribution function F. Let also Ha (X; t) and Ha(Xk:n; t) denote the RREs of X and Xk:n, respectively.
4199
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206 k1 (a) Let Ha(X; t) be finite. If mk ¼ maxfFðtÞ; n1 g, then for a > 1 (0 < a < 1)
Ha ðX k:n ; tÞ P ð6Þbk ðtÞ þ Ha ðX; tÞ þ
a 1a
log FðtÞ;
where
bk ðtÞ ¼
a 1a
ðk 1Þ log mk þ ðn kÞ logð1 mk Þ log BFðtÞ ðk; n k þ 1Þ :
(b) Let M = f (m) < 1, where m = sup{x:f (x) 6 M} is mode of the distribution. Then for a > 0
Ha ðX k:n ; tÞ P Ha ðU k:n ; FðtÞÞ log M: Part (a) of the theorem gives an lower bound, in the case of a > 1 (upper bound, in the case of 0 < a < 1), for RRE of Xk:n in terms of incomplete beta function and the RRE of the parent distribution. Part (b) of the theorem shows a lower bound for RRE of Xk:n in terms of RRE of order statistics of uniform distribution and the mode of the underlying distribution. The bound in part (b) is particulary interesting since it shows that the difference between the RRE of Xk:n at t and RRE of Uk:n at F(t) is at least log M, where M = f(m) and m is the mode of X. In Table 1, we list the bounds of the RRE of the order statistics based on Theorem 2.7 for some well known distributions. In the following we explore monotone behavior of RRE of order statistics. First we prove the following lemma which plays a crucial role in the subsequent results. Lemma 2.8. Let u (x) and vk(x),k > 0, be non-negative functions where u (x) is increasing. Assume that 0 6 t < c 6 1 and Wk has a density function fk where
umk ðwÞv k ðwÞ ; fk ðwÞ ¼ R c mk u ðxÞv k ðxÞ dx t
w 2 ðt; cÞ:
ð8Þ
Let m be real valued and define function ha as follows.
R c ma u ðxÞv a ðxÞ dx 1 ha ðmÞ ¼ log Rtc a ; a > 0; 1a um ðxÞv 1 ðxÞ dx
a – 1:
ð9Þ
t
(i) If for a > 1(0 < a < 1),Wa 6st(Pst) W1 then ha(m) is an increasing function of m. (ii) If for a > 1(0 < a < 1),Wa Pst(6st) W1 then ha(m) is a decreasing function of m. Remark 2.9. Under the assumptions of Lemma 2.8, it can also be proved that when u(x) is decreasing then (a) For a > 1(0 < a < 1), Wa 6st(Pst)W1 implies that ha(m) is a decreasing function of m. (b) For a > 1(0 < a < 1), Wa Pst(6st) W1 implies that ha(m) is an increasing function of m. Table 1 Bounds for Ha(Xk:n; t) based on Parts (a) and (b) of Theorem 2.7, respectively. Density function
Bounds
Triangular distribution ( 2x 06x6d d ; f ðxÞ ¼ 2ð1xÞ d>0 1d ; d 6 x 6 1;
( P ð6Þ
bk ðtÞ þ 11 a ½a log 2d logð1 þ aÞ þ logðda t1þa Þ; 1t 1t bk ðtÞ þ 11 a ½a log 2 þ log 1þ a þ a log 1d;
06t6d d6t61
PHa(Uk:n;F(t)) log2 Standard half-Cauchy distribution 2 f ðxÞ ¼ pð1þx 2Þ ;
xP0
Standard half-normal distribution 2 ffi exp ðxl2Þ , f ðxÞ ¼ rp2ffiffiffiffi 2r 2p
P ð6Þ bk ðtÞ log 2 þ 11 a a log p þ log B
t2 1þt2
1
2;
a 12
P Ha ðU k:n ; FðtÞÞ log p2 h pffiffiffi i a lÞ P ð6Þbk ðtÞ þ 12 logð2pr2 Þ þ 11 a log U aðt þ log p2 affiffiffi r P Ha ðU k:n ; FðtÞÞ þ 12 log pr 2
2
x>lP0 Generalized exponential distribution
b1 1 exp lx f ðxÞ ¼ bh exp lx , h h x > l > 0,b,h > 0. Generalized gamma distribution a
c
f ðxÞ ¼ CcrðaÞ xca1 erx ,
h i P ð6Þbk ðtÞ þ log h þ 11 a a log b þ log B1exp ðltÞ ðaðb 1Þ þ 1; aÞ h P Ha ðU k:n ; FðtÞÞ þ log h þ ð1 bÞ log 1 1b ; b > 1 h i P ð6Þbk ðtÞ 1c logðarÞ log c þ 11 a log C aðca1Þþ1 ; artc a logðaa CðaÞÞ c
a1 logðca 1Þ þ ðca1Þ ; ca > 1 P Ha ðU k:n ; FðtÞÞ þ log CðaÞc 1 ca1 c c ðrcÞ c
x > 0,a,c,r > 0
4200
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Using Lemma 2.8, we can now prove the following corollary for (n k + 1)-out-of-n systems with components having uniform distributions. Corollary 2.10 (a) Consider a parallel (series) system consists of n components where the components have uniform distribution over unit interval. Then the RRE of the system lifetime is a decreasing function of the number of components. 2 1 (b) Let Uk:n denote the kth order statistic of uniform distribution over unit interval. If k1 6 k2 6 n are integers then for t P kn1 ,
Ha ðU k1 :n ; tÞ 6 Ha ðU k2 :n ; tÞ: The class of distribution functions with decreasing density functions is a wide class of distributions. Examples are exponential, Pareto, mixture of exponential and Pareto distributions, etc. There are also distribution functions with increasing density; for example the power distribution with density f(x) = bxb1, 0 < x < 1, b > 1. We use the result of part (a) of Corollary 2.10 to prove the following theorem in the class of distribution functions with monotone density. Theorem 2.11. Let X1, . . . , Xn be a set of i.i.d random variables representing the lifetime of the components of a parallel (series) system having a common distribution function F. Assume that F has a density function f which is increasing (decreasing) in its support. Then the RRE of system lifetime is decreasing in n. The following example shows that the result of Theorem 2.11 is not in general valid for any (n k + 1)-out-of-n system. Example 2.12. Assume that the structure of the system is (n 1)-out-of-n. Then the lifetime of the system is X2:n. Let the components of the system have uniform distribution on (0,1). Fig. 1 depicts the graph of RRE of X2:n for a = 2 and n = 2, . . . , 30 at time t = 0.2. This is evident from the graph that the RRE of the system is not a decreasing function of n. In fact the graph shows that RRE of X2:2 is less than that of X2:3. Remark 2.13. In the context of reliability theory a situation in which the density function is decreasing and hence the RRE of a series system is decreasing in the number of the components of the system arises as follows. If for a lifetime model one can f ðtÞ ) is a decreasing function of time then the density funcmake sure that the failure rate of the data distribution (i.e. rðtÞ FðtÞ tion of the data must be a decreasing function. Examples of well known lifetime distributions in reliability with decreasing failure rate are Weibull distribution with shape parameter less than one and Gamma distribution with shape parameter less than one. Hence, the RRE of a series system with components having these distributions is a decreasing function of the number of components. Now, we use part (b) of Corollary 2.10 to study the monotone behavior of RRE of order statistics Xk:n in terms of k. Theorem 2.14. Let X be a non-negative continuous random variable with distribution function F. Let F have a density function f
which is decreasing over its support. If k1 and k2 are integers such that k1 6 k2 6 n, then Ha X k1 :n ; t 6 Ha X k2 :n ; t for 1 k2 1 tPF n1 .
Fig. 1. The plot of RRE of (n 1)-out-of-n system, n = 2, . . . , 30 at time t = 0.2, for a = 2, when the parent distribution is uniform.
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4201
Fig. 2. The plot of RRE of Xk:15, k = 13, 14, for a = 2, based on the survival function (10).
The following example shows that the condition t P F 1
k2 1 n1
cannot be dropped from the conditions of theorem.
Example 2.15. Let the survival function of X be given by 4x
FðxÞ ¼
ð1 xÞeð3þ3xÞ ð1 þ xÞ1=3
;
0 < x < 1:
ð10Þ
Fig. 2 displays the plots of RRE of order statistics Xk:15, for k = 13, 14 and a = 2 based on survival function (10). It is seen from the plots that the RRE of the order statistics are not ordered in terms of k for all values of t, t 2 (0, 1). In fact the plot shows that for the values of t < F 1 13 the RRE is not necessarily monotone in terms of k. 14 Theorem 2.14 gives the following interesting corollary. Corollary 2.16. Let X be a non-negative continuous random variable with distribution function F and a decreasing density function f. If k 6 nþ1 2 then Ha(Xk:n; t) is increasing in k for values of t greater than the median of distribution. 3. The Residual Rényi entropy of record values In this section we obtain some results on the RRE of record values. Let U1, U2, . . . be a sequence of upper record values based on a sequence of non-negative continuous random variables Xi’s with distribution function F and density function f. Then the density function and survival function of Un, which are denoted by fUn and F Un , respectively, are given by
fUn ðxÞ ¼
½ log FðxÞn1 f ðxÞ; ðn 1Þ!
F Un ðxÞ ¼
x > 0;
n P 1;
n1 X ½ log FðxÞj Cðn; log FðxÞÞ FðxÞ ¼ ; j! CðnÞ j¼0
ð11Þ
where C(a; x) is known as the incomplete gamma function and is defined as
Cða; xÞ ¼
Z
1
ua1 ex dx;
a; x > 0
x
(see, for example [1]). Remark 3.1. The survival function (11) arises naturally in reliability theory. Assume that a system is put in operation at time t = 0. When the system fails, it may be restored to a condition identical to that immediately before failure. That is, the failure rate after repair remains the same as that immediately prior to failure. This kind of repair is called the minimal repair. It is
4202
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
shown that the epoch times of repairs follow a nonhomogeneous Poisson process with cumulative intensity function RðtÞ ¼ log FðtÞ. Hence if T1 6 T2 6 denote the epoch times of the repairs then the survival function of Tn, n = 1, 2, . . . is equal to the survival function of record values in (11) (see [14]). Using the definition of RRE, the result of the following lemma is easy to verify. Lemma 3.2. Let U n denote the nth upper record value from a sequence of observations from U (0, 1). Then
Ha U n ; t ¼
1 Cðaðn 1Þ þ 1; logð1 tÞÞ log : 1a Ca ðn; logð1 tÞÞ
The next theorem represents the RRE of upper record Un in terms of upper record U n of uniform distribution. First, we need the following notation. Notation: We use the notation X Ct(a, k) to show that X has a truncated Gamma distribution with density function
f ðxÞ ¼
ka xa1 ekx ; Cða; tÞ
x > t > 0;
where a > 0 and k > 0. Theorem 3.3. The RRE of Un can be written in terms of the RRE of U n as follows:
Ha ðU n ; tÞ ¼ Ha U n ; FðtÞ þ
h
i 1 log E f a1 F 1 1 eV n ; 1a
ð12Þ
where V n C log FðtÞ ðaðn 1Þ þ 1; 1Þ: From Eq. (12), it is easily seen that the residual Shannon entropy of nth upper record value of an absolutely continuous distribution function F can be written in terms of the residual Shannon entropy of nth upper record value of U(0, 1) as follows:
h
i HðU n ; tÞ ¼ H U n ; FðtÞ E log f F 1 1 eV n ;
ð13Þ
where V n C log FðtÞ ðn; 1Þ: Example 3.4. Let X have Weibull distribution with density b
f ðxÞ ¼ bkb ðx lÞb1 e½kðxlÞ Here, F
1
ðxÞ ¼
1 ð logð1 k
; x P l:
b
xÞÞ þ l: Then we have for b P 1,
h
i E f a1 F 1 1 eV n ¼
0
C
ðkbÞa1
Cðaðn 1Þ þ 1; ðkðt lÞÞb Þ
@
1 ð1 b
aÞ þ na; aðkðt lÞÞb
1
1
abð1aÞþna
A:
Therefore
C 1 Ha ðU n ; tÞ ¼ log 1a
1 ð1 b
aÞ þ na; aðkðt lÞÞb a
b
C ðn; ðkðt lÞÞ Þ
logðkbÞ
1 na log a: log a b 1a
The following theorem gives a lower bound for RRE of nth record in terms of RRE of uniform distribution. The proof of the theorem is similar to proof of Theorem 2.7 and hence is omitted. Theorem 3.5. Suppose that M = f (m) < 1 where m is the mode of X and assume that the assumptions of Theorem 3.3 are met. Then for a > 0,
Ha ðU n ; tÞ P Ha U n ; FðtÞ log M: It is seen that the bound in Theorem 3.5 is very similar to the bound in part (b) of Theorem 2.7. This similarity results from assumptions of theorems and representations (6) and (12) where ‘‘ in both of them” the corresponding RRE of uniform distribution and density-quantile function appear. Example 3.6. The density function of the mixture of two Pareto distributions with parameters h1 and h2 is
f ðxÞ ¼ ch1 xh1 1 þ ð1 cÞh2 xh2 1 ;
x P 1; 0 < c < 1; h1 > h2 > 0:
Since, the mode of this distribution is m = 1, we have
4203
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Ha ðU n ; tÞ P
1 Cðaðn 1Þ þ 1; log FðtÞÞ logðch1 þ ð1 cÞh2 Þ; log 1a Ca ðn; log FðtÞÞ
where log FðtÞ ¼ ðh1 þ h2 Þ log t log cth2 þ ð1 cÞth1 : The following theorem investigates the monotone behavior of RRE of upper records in terms of n. Theorem 3.7. Let {Xi, i P 1} be a sequence of i.i.d random variables from distribution function F having an increasing density function f. If {Un, n P 1} represents the sequence of upper record values corresponding to F, then Ha(Un;t) is decreasing in n. An example of the distributions for which this theorem can be applied is the power distribution with distribution function F(x) = xb, 0 < x < 1, b > 1. Remark 3.8. The reader may have observed that some of the results of the paper on order statistics and record values have analogies in the proofs and conclusions. The main reason (rationale) is the analogy between the definition of maximum of a set of order statistics and record values. In fact, record values can be viewed as the maximum of a set of order statistics from a sample whose size is determined by the values of occurrence of the observations (for more details on similarity between distributional properties of order statistics and record values see [16]).
4. Conclusion In this paper, we studied the residual Rényi entropy (RRE) of order statistics and record values. We obtained the relation between the RRE of order statistics (record values) of a continuous distribution in terms of RRE of order statistics (record values) of uniform distribution. As the RRE usually cannot be obtained in an closed form, some bounds are provided for RRE of order statistics and record values. We investigated the monotone behavior of RRE of order statistics Xk:n, in term of k and n and the monotone behavior of record values in terms of number of observations. Several illustrative examples were also provided. Acknowledgments The authors thank the editor and two referees for helpful and constructive comments which have considerably improved the presentation of the paper. The authors would also like to thank Professor E.S. Soofi for his suggestions in preparing the paper. This work is supported in part by the ‘‘Ordered and Spacial Data Center of Excellence of Ferdowsi University of Mashhad”.
Appendix A. Proofs A.1. Proof of Theorem 2.7 h i
k1 (a) According to Theorem 2.3, it is enough to obtain a bound for 11 a log E f a1 ðF 1 ðY k ÞÞ . Note that mk ¼ max FðtÞ; n1 is the mode of the distribution of Yk. Let M k ¼ fY k ðmk Þ, then for a > 1 (0 < a < 1) we have
1 1 log E½f a1 ðF 1 ðY k ÞÞ ¼ log 1a 1a
Z
FðtÞ
1 þ log 1a ¼
yaðk1Þ ð1 yÞaðnkÞ
1
Z
BFðtÞ ðaðk 1Þ þ 1; aðn kÞ þ 1Þ 1
FðtÞ
f a1 ðF 1 ðyÞÞ dy ¼
f a1 ðF 1 ðyÞÞ dy P ð6Þ
1 1 log M k þ log 1a 1a
Z
1
1 log Mk 1a
f a ðuÞ du
t
1 a log Mk þ Ha ðX; tÞ þ log FðtÞ: 1a 1a
(b) We have for a > 1(0 < a < 1),
f a1 ðF 1 ðyÞÞ 6 ðPÞM a1 : Thus for the RRE of kth order statistic, using Theorem 2.3, we can write
Ha ðX k:n ; tÞ P Ha ðU k:n ; FðtÞÞ log M: For the case when a ? 1, using relation (7), we obtain the following result for residual Shannon entropy
HðX k:n ; tÞ P HðU k:n ; FðtÞÞ log M:
4204
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
A.2. Proof of Lemma 2.8 We prove part (i). Part (ii) can be proved similarly. Under the assumption that ha(m) is differentiable in terms of m, we obtain
! @g a ðmÞ @ha ðmÞ 1 @m ; ¼ @m 1 a g a ðmÞ where
Rc
uma ðxÞv a ðxÞ ðdx a um ðxÞv 1 ðxÞ dx t
g a ðmÞ ¼ Rt c and
@g a ðmÞ a ¼ R c aþ1 @m um ðxÞv 1 ðxÞdx
Z
c
log uðxÞuma ðxÞv a ðxÞdx
t
t
Z
c
um ðxÞv 1 ðxÞdx
Z
t
c
log uðxÞum ðxÞv 1 ðxÞdx
t
Z
c
uma ðxÞv a ðxÞdx :
t
ð14Þ Now using the fact that Wa 6st(Pst)W1 and that log is increasing function we get
E½log uðW a Þ 6 ðPÞE½log uðW 1 Þ (see, for example [26]). This shows that (14) is non-positive (non-negative) and hence ha(m) is an increasing function of m. h A.3. Proof of Corollary 2.10 (a) We assume that the system is parallel. For series system similar arguments can be used to prove the result on using Remark 2.9. From Lemma 2.2 it is easily seen that Ha(Un:n;t) can be written as (9) with u(x) = x and va(x) = xa in which we assume, without loss of generality, n P 1 is a continuous variable. Since for a > 1 (0 < a < 1) the ratio
R1
xaðn1Þ dx t R1 xn1 dx t
is increasing (decreasing) in t, for the chosen u(x) and va(x), we have
W a Pst ð6ÞW 1 ; where density function of Wk, k > 0, is given in (8). Hence from Lemma 2.8 we conclude that the RRE of the parallel system is a decreasing function of the number of components. na x (b) The result can be proved using the same arguments as used to prove part (a) on taking uðxÞ ¼ 1x and v a ðxÞ ¼ ð1xÞ . In xa k1 this case it can be easily seen that for t P n1 and a > 1 (0 < a < 1),
W a 6st ðPÞW 1 : Hence it can be concluded that for k1 6 k2 6 n,
Ha ðU k1 :n ; tÞ 6 Ha ðU k2 :n ; tÞ;
tP
k2 1 : n1
ð15Þ
A.4. Proof of Theorem 2.11 Let Y n BFðtÞ ðaðn 1Þ þ 1; 1Þ and assume that fY n shows the density function of Yn. Then we have
fY nþ1 ðwÞ BFðtÞ ðaðn 1Þ þ 1; 1Þ a y ; ¼ fY n ðyÞ BFðtÞ ðan þ 1; 1Þ
FðtÞ < y < 1;
which is an increasing function of y. This implies that Yn 6lrYn+1 and hence Yn 6 stYn+1. On the other hand, for a > 1 (0 < a < 1), fa1(F1(x)) is increasing (decreasing) in x. Hence
E½f a1 ðF 1 ðY n ÞÞ 6 ðPÞE½f a1 ðF 1 ðY nþ1 ÞÞ:
ð16Þ
From (16) we get that
1 E½f a1 ðF 1 ðY nþ1 ÞÞ log 6 0: 1a E½f a1 ðF 1 ðY n ÞÞ
ð17Þ
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4205
From Theorem 2.3 we have
Ha ðX nþ1:nþ1 ; tÞ Ha ðX n:n ; tÞ ¼ Da ðn; tÞ þ
1 E½f a1 ðF 1 ðY nþ1 ÞÞ log ; 1a E½f a1 ðF 1 ðY n ÞÞ
ð18Þ
where Da(n; t) = Ha(Un+1:n+1; F(t)) Ha(Un:n; F(t)). Now using part (a) of Corollary 2.10 and inequality (17), it can be seen that the left hand side of (18) is non-positive. That is, the RRE of the parallel system is a decreasing function of the number of components. Similar arguments can be used to prove the case when the components of the system are connected in series. h A.5. Proof of Theorem 2.14 From Theorem 2.3, we have for k1 6 k2 6 n
Ha ðX k2 :n ; tÞ Ha ðX k1 :n ; tÞ ¼ Da ðk2 ; k1 ; tÞ þ
E½f a1 ðF 1 ðY k2 ÞÞ 1 log ; 1a E½f a1 ðF 1 ðY k1 ÞÞ
ð19Þ
where
Da ðk2 ; k1 ; tÞ ¼ Ha ðU k2 :n ; FðtÞÞ Ha ðU k1 :n ; FðtÞÞ and
Y ki BFðtÞ ðaðki 1Þ þ 1; aðn ki Þ þ 1Þ;
i ¼ 1; 2:
It is easy to verify that Y k1 6lr Y k2 and hence Y k1 6st Y k2 . Now the result follows using part (b) of Corollary 2.10 and the same arguments as used to prove Theorem 2.11. h A.6. Proof of Corollary 2.16 Let k1 6 k2 6 nþ1 : The right hand side of this inequality implies that 2
m P F 1
k2 1 ; n1
where m ¼ F 1 ð12Þ denotes the median of F. Now using Theorem 2.14, we have for t P m
Ha ðX k1 :n ; tÞ 6 Ha ðX k2 :n ; tÞ:
A.7. Proof of Theorem 3.3 The proof follows easily from the definition of RRE of Un, in which one needs to make substitution u ¼ log FðxÞ in the integrant. Then after some simple algebraic manipulations the result follows. h A.8. Proof of Theorem 3.7 Using Theorem 3.3, we have
Ha ðU nþ1 ; tÞ Ha ðU n ; tÞ ¼ Da ðn; tÞ þ
1 E½f a1 ðF 1 ð1 eV nþ1 ÞÞ log ; 1a E½f a1 ðF 1 ð1 eV n ÞÞ
ð20Þ
where
Da ðn; tÞ ¼ Ha ðU nþ1 ; FðtÞÞ Ha U n ; FðtÞ
ð21Þ
and
V n C log FðtÞ ðaðn 1Þ þ 1; 1Þ:
On taking u(x) = x and va(x) = xaex and using Lemma 2.8, we can show that Ha U n ; FðtÞ is decreasing in n and hence Da(n;t) 6 0. One can show that Vn 6 lrVn+1 and hence Vn 6 st Vn+1. This implies that, for 0 < a < 1(a > 1),
E½f a1 ðF 1 ð1 eV n ÞÞ P ð6ÞE½f a1 ðF 1 ð1 eV nþ1 ÞÞ: Hence, we have for a > 0,a – 1
1 E½f a1 ðF 1 ð1 eV nþ1 ÞÞ log 6 0: 1a E½f a1 ðF 1 ð1 eV n ÞÞ
4206
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Thus, from the last inequality and inequalities (20) and (21), we conclude that Ha(Un+1; t) Ha(Un; t) 6 0. This completes the theorem. h References [1] B.C. Arnold, N. Balakrishnan, H.N. Nagaraja, Records, Wiley, New York, 1998. [2] M. Asadi, N. Ebrahimi, Residual entropy and its characterizations in terms of hazard function and mean residual life function, Statistics and Probability Letters 49 (2000) 263–269. [3] M. Asadi, N. Ebrahimi, E.S. Soofi, Dynamic generalized information measures, Statistics and Probability Letters 71 (2005) 85–98. [4] S. Baratpour, J. Ahmadi, N.R. Arghami, Entropy properties of record statistics, Statistical Papers 48 (2007) 197–213. [5] S. Baratpur, J. Ahmadi, N.R. Arghami, Characterizations based on Rényi entropy of order statistics and record values, Journal of Statistical Planning and Inference 138 (2008) 2544–2551. [6] F. Belzunce, J. Navarro, J.M. Ruiz, Some results on residual entropy functions, Metrika 59 (2004) 147–161. [7] J.F. Bercher, On some entropy functionals derived from Rényi information divergence, Information Sciences 178 (2008) 2489–2506. [8] H.A. David, H.N. Nagaraja, Order Statistics, third ed., Wiley, New York, 2003. [9] A. De Gregorio, S.M. Iacus, On Rényi information for ergodic diffusion processes, Information Sciences (2009) 279–291. [10] N. Ebrahimi, How to measure uncertainty in the residual lifetime distributions, Sankhya A 58 (1996) 48–57. [11] N. Ebrahimi, S.N.U.A. Kirmani, Some results on ordering of survival functions through uncertainty, Statistics and Probability Letters 11 (1996) 167–176. [12] N. Ebrahimi, E.S. Soofi, H. Zahedi, Information properties of order statistics and spacings, IEEE Transactions on Information Theory IT50 (2004) 177– 183. [13] L. Golshani, E. Pasha, G. Yari, Some properties of Rényi entropy and Rényi entropy rate, Information Sciences 179 (2009) 2426–2433. [14] R.C. Gupta, S.N.U.A. Kirmani, Closure and monotonicity properties of nonhomogeneous poisson processes and record values, Probability in the Engineering and Informational Sciences 2 (1988) 475–484. [15] R.D. Gupta, A.K. Nanda, a- and b-Entropies and relative entropies of distributions, Journal of Statistical Theory and Applications 1 (3) (2002) 177–190. [16] U. Kamps, A Concept of Generalized Order Statistics. Teubner, Stuttgart, 1995. [17] M. Mahmoudi, M. Asadi, On the monotone behavior of time dependent entropy of order a, Journal of Iranian Statistical Society 9 (2010). [18] D. Morales, L. Pardo, I. Vajda, Some new statistics for testing hypotheses in parametric models, Journal of Multivariate Analysis 62 (1997) 137–168. [19] S. Nadarjah, K. Zografos, Formulas for Rényi information and related measures for univariate distributions, Information Sciences 155 (2003) 119–138. [20] S. Nadarjah, K. Zografos, Expressions for Rényi and Shannon entropies for bivariate distributions, Information Sciences 170 (2005) 173–189. [21] A.K. Nanda, S.S. Maiti, Rényi information measure for a used item, Information Sciences 177 (2007) 4161–4175. [22] A.K. Nanda, P. Paul, Some results on generalized residual entropy, Information Sciences 176 (2006) 27–47. [23] V. Nevzorov, Records: Mathematical Theory, Translation of Mathematical Monographs, vol. 194, American Mathematical Society Providence, RI, USA, 2001. [24] S. Park, The entropy of consecutive order statistics, IEEE Transaction on Information Theory IT41 (1995) 2003–2007. [25] A. Rényi, On Measures of entropy and information, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. I, University of California Press, 1961. [26] M. Shaked, J.G. Shanthikumar, Stochastic Orders, Springer, New York, 2007. [27] K. Song, Rényi information, loglikelihood and an intrinsic distribution measure, Journal of Statistical Planning and Inference 93 (2001) 51–69. [28] K.M. Wong, S. Chen, The entropy of ordered sequences and order statistics, IEEE Transaction on Information Theory IT36 (1990) 276–284.
Contents lists available at ScienceDirect
Information Sciences journal homepage: www.elsevier.com/locate/ins
Results on residual Rényi entropy of order statistics and record values S. Zarezadeh *, M. Asadi Department of Statistics, University of Isfahan, Isfahan 81744, Iran
a r t i c l e
i n f o
a b s t r a c t
Article history: Received 22 July 2009 Received in revised form 9 June 2010 Accepted 13 June 2010
Keywords: Order statistics Record values Rényi entropy Residual lifetime Shannon entropy Reliability (n k + 1)-out-of-n systems
This paper explores properties of the residual Rényi entropy of some ordered random variables. The residual Rényi entropy of the kth order statistic from a continuous distribution function is represented in terms of the residual Rényi entropy of the kth order statistic from uniform distribution. The monotone behavior of the residual Rényi entropy of order statistic under various conditions is discussed. Analogues results for the residual Rényi entropy of record values are also given. Ó 2010 Elsevier Inc. All rights reserved.
1. Introduction Let X be a non-negative continuous random variable with density function f, distribution function F and the survival function F ¼ 1 F. Shannon entropy of F is defined by
HðXÞ ¼ HðFÞ ¼
Z
1
f ðxÞ log f ðxÞ dx:
0
In continuous case, H(X) is also referred to as the differential entropy. Shannon entropy plays a central role in the field of information theory and has a wide range of applications in many fields. It is known that H(X) measures the uniformity of f. When H(F1) > H(F2), it is more difficult to predict outcomes of F1 as compared with predicting outcomes of F2. Rényi [25] introduced a one parameter extension of Shannon entropy which is more flexible than H(X) and also has been used in various fields. Rényi entropy of X is defined by
Ha ðXÞ ¼
1 log 1a
Z
1
f a ðxÞ dx;
0
where a > 0, a – 1. It can be easily shown that lima?1Ha(X) = H(X). Properties of Rényi entropy have been studied by many authors, including Rényi [25], Morales et al. [18], Song [27], Nadarjah and Zografos [19,20], Bercher [7], De Gregorio and Iacus [9] and Golshani et al. [13]. In reliability theory and survival analysis, X usually denotes a duration such as the lifetime. The residual lifetime of the system when it is still operating at time t, is Xt = X tjX > t which has probability density function * Corresponding author. E-mail address: [email protected] (S. Zarezadeh). 0020-0255/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.ins.2010.06.019
4196
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
f ðx; tÞ ¼
f ðxÞ ; FðtÞ
x P t > 0;
where FðtÞ > 0. Ebrahimi [10] proposed using the entropy of residual lifetime distribution
HðX; tÞ ¼
Z
1
f ðx; tÞ log f ðx; tÞ dx;
t > 0:
t
The residual entropy is time-dependent and measures the uncertainty of the residual lifetime of the system when it is still operating at time t. Several authors have studied properties of H(X; t); see, for example, Ebrahimi and Kirmani [11], Asadi and Ebrahimi [2] and Belzunce et al. [6]. The residual Rényi entropy (RRE) is defined similarly:
Ha ðX; tÞ ¼
1 log 1a
Z t
1
f a ðxÞ F a ðtÞ
dx;
ð1Þ
where a > 0, a – 1. Properties of RRE have been investigated by several authors including Gupta and Nanda [15], Asadi et al. [3], Nanda and Paul [22], Nanda and Maiti [21] and Mahmoudi and Asadi [17]. The aim of the present paper is to study properties of RRE of order statistics and record values. Let X1, . . . , Xn denote a random sample of size n from F. Order statistics refer to the arrangement of X1, . . . , Xn from the smallest to the largest, denoted as X1:n 6 X2:n 6 6 Xn:n. Order statistics are used in many branches of probability and statistics including characterization of probability distributions, goodness-of-fit tests, quality control, reliability analysis and many other problems. Order statistics are of particular interest in reliability theory in the study of the lifetime properties of the coherent systems and in life testing, when data are collected based on different censoring mechanisms. For a comprehensive review on the theory and applications of order statistics one can refer to Daivid and Nagaraja [8]. A record value of a sequence of independent identically distributed (i.i.d.) random variables {Xi; i P 1} is an observation Xj whose value exceeds or is less than the values of all previous observations. For example, Xj is an upper record if Xi < Xj for every i < j. Record values arise naturally in problems such as industrial stress testing, meteorological analysis, hydrology, sporting and athletic events, and economics. In reliability, records model are used to study, for example, technical systems which are subject to shocks, e.g. peaks of voltages. Successive large shocks may be viewed as realizations of records from a sequence of identically independent voltages. For more details about records and their applications, one may refer to Arnold et al. [1] and Nevzorov [23]. Several authors have studied the information properties of ordered data. Wong and Chen [28] showed that the difference between the average entropy of order statistics and the entropy of the parent distribution is a constant. They also showed that when the distribution of the data is symmetric, the entropy of order statistics is symmetric about the median. Park [24] obtained some recurrence relations for the entropy of order statistics. Ebrahimi et al. [12] explored some properties of the Shannon entropy of the order statistics and showed that the Kullback–Leibler information functions involving order statistics are distribution free. Baratpour et al. [4,5] obtained some results for the Shannon entropy and Rényi entropy of the order statistics and record values. We continue this line of research by exploring properties of RRE of order statistics and record values. In Section 2, we represent the RRE of order statistics Xk:n of a sample from any continuous distribution function F in terms of RRE of order statistics of a sample from uniform distribution. Since for many statistical models the functional form of the RRE of order statistics cannot be obtained in a closed form, we obtain upper and lower bounds for RRE of order statistics. Several illustrative examples are given. We also show that, under some mild conditions, the RRE’s of the minimum and maximum of a random sample are monotone functions of the number of observations of sample. We give a counter example to show that the RRE of other order statistics Xk:n is not necessary monotone function of n. We also study the monotone behavior of RRE of order statistics Xk:n in terms of k. It is shown that the RRE of Xk:n is not a monotone function of k on the entire support of F. In Section 3, we investigate properties of RRE of record values. We give bounds for RRE of record values. We also show that, under some mild conditions, the RRE of record values is monotone function of number of records in the sequence. Proofs of the results are gathered in an Appendix. Before proceeding to give the main results of the paper, we overview some preliminary concepts on partial orderings between random variables. (For more details of these concepts one can see [26].) Let X and Y be two random variables with survival functions F and G and density functions f and g, respectively. Definition 1.1 (a) The random variable Y is said to be smaller than X in the usual stochastic order (denoted by Y 6stX) if GðxÞ 6 FðxÞ for all x. f ðxÞ (b) The random variable Y is said to be smaller than X in likelihood ratio order (denoted by Y 6lrX) if gðxÞ is an increasing function of x. It can be shown that if Y 6 lrX, then Y 6 stX (see [26], for more details). Throughout the paper increasing (decreasing) means non-decreasing (non-increasing).
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4197
2. The residual Rényi entropy of order statistics In this section, we focus on the RRE of order statistics. First note that the density function and survival function of Xk:n denoted by fk:n(x) and F k:n ðxÞ, k = 1, . . . , n, respectively, are
fk:n ðxÞ ¼
1 ½FðxÞk1 ½1 FðxÞnk f ðxÞ Bðk; n k þ 1Þ
ð2Þ
and
F k:n ðxÞ ¼
k1 X n
i
i¼0
F i ðxÞF ni ðxÞ;
ð3Þ
where
Bða; bÞ ¼
Z
1
xa1 ð1 xÞb1 dx;
a > 0; b > 0:
0
See, for example, David and Nagaraja [8, pp. 9–10]. The survival function F k:n ðxÞ can also be represented as
F k:n ðxÞ ¼
BFðxÞ ðk; n k þ 1Þ ; Bðk; n k þ 1Þ
ð4Þ
where
Bx ða; bÞ ¼
Z
1
ua1 ð1 uÞb1 du;
0 < x < 1:
x
B(a, b) and Bx ða; bÞ are known as the beta and the incomplete beta functions, respectively (see, for example [8]). Notation: Throughout this section we use the notation Y Bt ða; bÞ to show that Y has a truncated beta distribution with density function
fY ðyÞ ¼
1 Bt ða; bÞ
ya1 ð1 yÞb1 ;
t 6 y 6 1:
ð5Þ
Remark 2.1. In reliability engineering (n k + 1)-out-of-n systems are very important kind of structures. A (n k + 1)-outof-n system functions if and only if at least (n k + 1) components out of n components function. If X1, X2, . . . , Xn denote the independent lifetimes of the components of such system, then the lifetime of the system is equal to the order statistics Xk:n. The special cases of k = 1 and k = n are corresponding to series and parallel systems, respectively. Assuming that a (n k + 1)out-of-n system is working at time t, then the RRE of Xk:n measures the entropy of the residual lifetime of the system. Hence the RRE, as a dynamic measure of entropy, can be important for system designers to get information about the entropy of used (n k + 1)-out-of-n systems at any time t. The following lemma shows that the RRE of order statistics of uniform distribution can be written in terms of incomplete beta function which is important in computational point of view. Its proof, which follows from the definition of RRE, is an easy exercise and hence is omitted. Lemma 2.2. Let Uk:n be kth order statistic based on a random sample of size n from uniform distribution on (0, 1). Then
Ha ðU k:n ; tÞ ¼
1 a log Bt ðaðk 1Þ þ 1; aðn kÞ þ 1Þ log Bt ðk; n k þ 1Þ: 1a 1a
If F is a continuous distribution function then it is well known, from the probability integral transformation, that d
U k:n ¼ FðX k:n Þ;
k ¼ 1; . . . ; n;
where ‘d’ stands for equality in distribution and Xk:n is the kth order statistic based on a random sample of size n from F (see, for example [8]). Using this, in the following theorem, we will show that the RRE of order statistics Xk:n can be represented in terms of RRE of order statistics of uniform distribution. The proof of the theorem is straightforward and hence is omitted. Theorem 2.3. Let F be an absolutely continuous distribution function with density f. Then the RRE of the kth order statistic can be represented in terms of the RRE of kth order statistic from uniform distribution, over the unit interval, as follows:
Ha ðX k:n ; tÞ ¼ Ha ðU k:n ; FðtÞÞ þ
1 log E½f a1 ðF 1 ðY k ÞÞ; 1a
where Y k BFðtÞ ðaðk 1Þ þ 1; aðn kÞ þ 1Þ:
ð6Þ
4198
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
It can be seen, after some calculations, that when in (6) a ? 1, the Shannon entropy of kth order statistic from a sample of F can be written as follows:
HðX k:n ; tÞ ¼ HðU k:n ; FðtÞÞ E½log f ðF 1 ðY k ÞÞ;
ð7Þ
where Y k BFðtÞ ðk; n k þ 1Þ: The specialized version of this result for t = 0, was already obtained by Ebrahimi et al. [12]. Remark 2.4. The quantity f(F1(x)) is known, in the literature, as the density-quantile function and is used to approximate the moments of order statistics (see [8]). In the following we give some examples. Example 2.5. Suppose that F is exponential with mean 1h . Then f(F1(y)) = h(1 y) and we have
E½f a1 ðF 1 ðY 1 ÞÞ ¼
ha1 ehtð1aÞ : na½aðn 1Þ þ 1
For k = 1, Theorem 2.3 gives
Ha ðX 1:n ; tÞ ¼
log a logðnhÞ: a1
On the other hand, we have
Ha ðX; tÞ ¼
log a log h: a1
This gives
Ha ðX 1:n ; tÞ Ha ðX; tÞ ¼ log n: That is, in the exponential case the difference between of RRE of the lifetime of a series system and RRE of the lifetime of each components is free of both of the time and a depends only on the number of components of the system. Example 2.6. Let X have Pareto distribution with distribution function
FðxÞ ¼ 1
h b ; x
x P b > 0;
h>0
and density function
f ðxÞ ¼
hbh xhþ1
x P b > 0;
h > 0:
Then
h 1 f F 1 ðyÞ ¼ ð1 yÞ1þh : b Therefore for the first order statistic of a random sample of size n we have
E½f a1 ðF 1 ðY 1 ÞÞ ¼
ða1Þð1þhÞ
aðn 1Þ þ 1 a 1a b h b t aðnh þ 1Þ 1
:
Hence Theorem 2.3 gives
Ha ðX 1:n ; tÞ ¼ log t þ
a 1a
log nh
1 log ½aðnh þ 1Þ 1; 1a
a>
1 : nh þ 1
The difference between RRE of X1:n and RRE of X is
Ha ðX 1:n ; tÞ Ha ðX; tÞ ¼
a 1a
log n
1 aðnh þ 1Þ 1 log ; 1a aðh þ 1Þ 1
a>
1 ; hþ1
which is free of t and depends only on a and n. We obtained the closed form of RRE of the first order statistics in exponential and Pareto distributions. However, we do not have a closed form for RRE of other order statistics for these distributions. This is true, in general for other distributions, that there is no closed form for the RRE of order statistics. This gives a motivation for obtaining some bounds for RRE of order statistics. Hence we prove the following theorem. Theorem 2.7. Let X be a non-negative continuous random variable with density function f and distribution function F. Let also Ha (X; t) and Ha(Xk:n; t) denote the RREs of X and Xk:n, respectively.
4199
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206 k1 (a) Let Ha(X; t) be finite. If mk ¼ maxfFðtÞ; n1 g, then for a > 1 (0 < a < 1)
Ha ðX k:n ; tÞ P ð6Þbk ðtÞ þ Ha ðX; tÞ þ
a 1a
log FðtÞ;
where
bk ðtÞ ¼
a 1a
ðk 1Þ log mk þ ðn kÞ logð1 mk Þ log BFðtÞ ðk; n k þ 1Þ :
(b) Let M = f (m) < 1, where m = sup{x:f (x) 6 M} is mode of the distribution. Then for a > 0
Ha ðX k:n ; tÞ P Ha ðU k:n ; FðtÞÞ log M: Part (a) of the theorem gives an lower bound, in the case of a > 1 (upper bound, in the case of 0 < a < 1), for RRE of Xk:n in terms of incomplete beta function and the RRE of the parent distribution. Part (b) of the theorem shows a lower bound for RRE of Xk:n in terms of RRE of order statistics of uniform distribution and the mode of the underlying distribution. The bound in part (b) is particulary interesting since it shows that the difference between the RRE of Xk:n at t and RRE of Uk:n at F(t) is at least log M, where M = f(m) and m is the mode of X. In Table 1, we list the bounds of the RRE of the order statistics based on Theorem 2.7 for some well known distributions. In the following we explore monotone behavior of RRE of order statistics. First we prove the following lemma which plays a crucial role in the subsequent results. Lemma 2.8. Let u (x) and vk(x),k > 0, be non-negative functions where u (x) is increasing. Assume that 0 6 t < c 6 1 and Wk has a density function fk where
umk ðwÞv k ðwÞ ; fk ðwÞ ¼ R c mk u ðxÞv k ðxÞ dx t
w 2 ðt; cÞ:
ð8Þ
Let m be real valued and define function ha as follows.
R c ma u ðxÞv a ðxÞ dx 1 ha ðmÞ ¼ log Rtc a ; a > 0; 1a um ðxÞv 1 ðxÞ dx
a – 1:
ð9Þ
t
(i) If for a > 1(0 < a < 1),Wa 6st(Pst) W1 then ha(m) is an increasing function of m. (ii) If for a > 1(0 < a < 1),Wa Pst(6st) W1 then ha(m) is a decreasing function of m. Remark 2.9. Under the assumptions of Lemma 2.8, it can also be proved that when u(x) is decreasing then (a) For a > 1(0 < a < 1), Wa 6st(Pst)W1 implies that ha(m) is a decreasing function of m. (b) For a > 1(0 < a < 1), Wa Pst(6st) W1 implies that ha(m) is an increasing function of m. Table 1 Bounds for Ha(Xk:n; t) based on Parts (a) and (b) of Theorem 2.7, respectively. Density function
Bounds
Triangular distribution ( 2x 06x6d d ; f ðxÞ ¼ 2ð1xÞ d>0 1d ; d 6 x 6 1;
( P ð6Þ
bk ðtÞ þ 11 a ½a log 2d logð1 þ aÞ þ logðda t1þa Þ; 1t 1t bk ðtÞ þ 11 a ½a log 2 þ log 1þ a þ a log 1d;
06t6d d6t61
PHa(Uk:n;F(t)) log2 Standard half-Cauchy distribution 2 f ðxÞ ¼ pð1þx 2Þ ;
xP0
Standard half-normal distribution 2 ffi exp ðxl2Þ , f ðxÞ ¼ rp2ffiffiffiffi 2r 2p
P ð6Þ bk ðtÞ log 2 þ 11 a a log p þ log B
t2 1þt2
1
2;
a 12
P Ha ðU k:n ; FðtÞÞ log p2 h pffiffiffi i a lÞ P ð6Þbk ðtÞ þ 12 logð2pr2 Þ þ 11 a log U aðt þ log p2 affiffiffi r P Ha ðU k:n ; FðtÞÞ þ 12 log pr 2
2
x>lP0 Generalized exponential distribution
b1 1 exp lx f ðxÞ ¼ bh exp lx , h h x > l > 0,b,h > 0. Generalized gamma distribution a
c
f ðxÞ ¼ CcrðaÞ xca1 erx ,
h i P ð6Þbk ðtÞ þ log h þ 11 a a log b þ log B1exp ðltÞ ðaðb 1Þ þ 1; aÞ h P Ha ðU k:n ; FðtÞÞ þ log h þ ð1 bÞ log 1 1b ; b > 1 h i P ð6Þbk ðtÞ 1c logðarÞ log c þ 11 a log C aðca1Þþ1 ; artc a logðaa CðaÞÞ c
a1 logðca 1Þ þ ðca1Þ ; ca > 1 P Ha ðU k:n ; FðtÞÞ þ log CðaÞc 1 ca1 c c ðrcÞ c
x > 0,a,c,r > 0
4200
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Using Lemma 2.8, we can now prove the following corollary for (n k + 1)-out-of-n systems with components having uniform distributions. Corollary 2.10 (a) Consider a parallel (series) system consists of n components where the components have uniform distribution over unit interval. Then the RRE of the system lifetime is a decreasing function of the number of components. 2 1 (b) Let Uk:n denote the kth order statistic of uniform distribution over unit interval. If k1 6 k2 6 n are integers then for t P kn1 ,
Ha ðU k1 :n ; tÞ 6 Ha ðU k2 :n ; tÞ: The class of distribution functions with decreasing density functions is a wide class of distributions. Examples are exponential, Pareto, mixture of exponential and Pareto distributions, etc. There are also distribution functions with increasing density; for example the power distribution with density f(x) = bxb1, 0 < x < 1, b > 1. We use the result of part (a) of Corollary 2.10 to prove the following theorem in the class of distribution functions with monotone density. Theorem 2.11. Let X1, . . . , Xn be a set of i.i.d random variables representing the lifetime of the components of a parallel (series) system having a common distribution function F. Assume that F has a density function f which is increasing (decreasing) in its support. Then the RRE of system lifetime is decreasing in n. The following example shows that the result of Theorem 2.11 is not in general valid for any (n k + 1)-out-of-n system. Example 2.12. Assume that the structure of the system is (n 1)-out-of-n. Then the lifetime of the system is X2:n. Let the components of the system have uniform distribution on (0,1). Fig. 1 depicts the graph of RRE of X2:n for a = 2 and n = 2, . . . , 30 at time t = 0.2. This is evident from the graph that the RRE of the system is not a decreasing function of n. In fact the graph shows that RRE of X2:2 is less than that of X2:3. Remark 2.13. In the context of reliability theory a situation in which the density function is decreasing and hence the RRE of a series system is decreasing in the number of the components of the system arises as follows. If for a lifetime model one can f ðtÞ ) is a decreasing function of time then the density funcmake sure that the failure rate of the data distribution (i.e. rðtÞ FðtÞ tion of the data must be a decreasing function. Examples of well known lifetime distributions in reliability with decreasing failure rate are Weibull distribution with shape parameter less than one and Gamma distribution with shape parameter less than one. Hence, the RRE of a series system with components having these distributions is a decreasing function of the number of components. Now, we use part (b) of Corollary 2.10 to study the monotone behavior of RRE of order statistics Xk:n in terms of k. Theorem 2.14. Let X be a non-negative continuous random variable with distribution function F. Let F have a density function f
which is decreasing over its support. If k1 and k2 are integers such that k1 6 k2 6 n, then Ha X k1 :n ; t 6 Ha X k2 :n ; t for 1 k2 1 tPF n1 .
Fig. 1. The plot of RRE of (n 1)-out-of-n system, n = 2, . . . , 30 at time t = 0.2, for a = 2, when the parent distribution is uniform.
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4201
Fig. 2. The plot of RRE of Xk:15, k = 13, 14, for a = 2, based on the survival function (10).
The following example shows that the condition t P F 1
k2 1 n1
cannot be dropped from the conditions of theorem.
Example 2.15. Let the survival function of X be given by 4x
FðxÞ ¼
ð1 xÞeð3þ3xÞ ð1 þ xÞ1=3
;
0 < x < 1:
ð10Þ
Fig. 2 displays the plots of RRE of order statistics Xk:15, for k = 13, 14 and a = 2 based on survival function (10). It is seen from the plots that the RRE of the order statistics are not ordered in terms of k for all values of t, t 2 (0, 1). In fact the plot shows that for the values of t < F 1 13 the RRE is not necessarily monotone in terms of k. 14 Theorem 2.14 gives the following interesting corollary. Corollary 2.16. Let X be a non-negative continuous random variable with distribution function F and a decreasing density function f. If k 6 nþ1 2 then Ha(Xk:n; t) is increasing in k for values of t greater than the median of distribution. 3. The Residual Rényi entropy of record values In this section we obtain some results on the RRE of record values. Let U1, U2, . . . be a sequence of upper record values based on a sequence of non-negative continuous random variables Xi’s with distribution function F and density function f. Then the density function and survival function of Un, which are denoted by fUn and F Un , respectively, are given by
fUn ðxÞ ¼
½ log FðxÞn1 f ðxÞ; ðn 1Þ!
F Un ðxÞ ¼
x > 0;
n P 1;
n1 X ½ log FðxÞj Cðn; log FðxÞÞ FðxÞ ¼ ; j! CðnÞ j¼0
ð11Þ
where C(a; x) is known as the incomplete gamma function and is defined as
Cða; xÞ ¼
Z
1
ua1 ex dx;
a; x > 0
x
(see, for example [1]). Remark 3.1. The survival function (11) arises naturally in reliability theory. Assume that a system is put in operation at time t = 0. When the system fails, it may be restored to a condition identical to that immediately before failure. That is, the failure rate after repair remains the same as that immediately prior to failure. This kind of repair is called the minimal repair. It is
4202
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
shown that the epoch times of repairs follow a nonhomogeneous Poisson process with cumulative intensity function RðtÞ ¼ log FðtÞ. Hence if T1 6 T2 6 denote the epoch times of the repairs then the survival function of Tn, n = 1, 2, . . . is equal to the survival function of record values in (11) (see [14]). Using the definition of RRE, the result of the following lemma is easy to verify. Lemma 3.2. Let U n denote the nth upper record value from a sequence of observations from U (0, 1). Then
Ha U n ; t ¼
1 Cðaðn 1Þ þ 1; logð1 tÞÞ log : 1a Ca ðn; logð1 tÞÞ
The next theorem represents the RRE of upper record Un in terms of upper record U n of uniform distribution. First, we need the following notation. Notation: We use the notation X Ct(a, k) to show that X has a truncated Gamma distribution with density function
f ðxÞ ¼
ka xa1 ekx ; Cða; tÞ
x > t > 0;
where a > 0 and k > 0. Theorem 3.3. The RRE of Un can be written in terms of the RRE of U n as follows:
Ha ðU n ; tÞ ¼ Ha U n ; FðtÞ þ
h
i 1 log E f a1 F 1 1 eV n ; 1a
ð12Þ
where V n C log FðtÞ ðaðn 1Þ þ 1; 1Þ: From Eq. (12), it is easily seen that the residual Shannon entropy of nth upper record value of an absolutely continuous distribution function F can be written in terms of the residual Shannon entropy of nth upper record value of U(0, 1) as follows:
h
i HðU n ; tÞ ¼ H U n ; FðtÞ E log f F 1 1 eV n ;
ð13Þ
where V n C log FðtÞ ðn; 1Þ: Example 3.4. Let X have Weibull distribution with density b
f ðxÞ ¼ bkb ðx lÞb1 e½kðxlÞ Here, F
1
ðxÞ ¼
1 ð logð1 k
; x P l:
b
xÞÞ þ l: Then we have for b P 1,
h
i E f a1 F 1 1 eV n ¼
0
C
ðkbÞa1
Cðaðn 1Þ þ 1; ðkðt lÞÞb Þ
@
1 ð1 b
aÞ þ na; aðkðt lÞÞb
1
1
abð1aÞþna
A:
Therefore
C 1 Ha ðU n ; tÞ ¼ log 1a
1 ð1 b
aÞ þ na; aðkðt lÞÞb a
b
C ðn; ðkðt lÞÞ Þ
logðkbÞ
1 na log a: log a b 1a
The following theorem gives a lower bound for RRE of nth record in terms of RRE of uniform distribution. The proof of the theorem is similar to proof of Theorem 2.7 and hence is omitted. Theorem 3.5. Suppose that M = f (m) < 1 where m is the mode of X and assume that the assumptions of Theorem 3.3 are met. Then for a > 0,
Ha ðU n ; tÞ P Ha U n ; FðtÞ log M: It is seen that the bound in Theorem 3.5 is very similar to the bound in part (b) of Theorem 2.7. This similarity results from assumptions of theorems and representations (6) and (12) where ‘‘ in both of them” the corresponding RRE of uniform distribution and density-quantile function appear. Example 3.6. The density function of the mixture of two Pareto distributions with parameters h1 and h2 is
f ðxÞ ¼ ch1 xh1 1 þ ð1 cÞh2 xh2 1 ;
x P 1; 0 < c < 1; h1 > h2 > 0:
Since, the mode of this distribution is m = 1, we have
4203
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Ha ðU n ; tÞ P
1 Cðaðn 1Þ þ 1; log FðtÞÞ logðch1 þ ð1 cÞh2 Þ; log 1a Ca ðn; log FðtÞÞ
where log FðtÞ ¼ ðh1 þ h2 Þ log t log cth2 þ ð1 cÞth1 : The following theorem investigates the monotone behavior of RRE of upper records in terms of n. Theorem 3.7. Let {Xi, i P 1} be a sequence of i.i.d random variables from distribution function F having an increasing density function f. If {Un, n P 1} represents the sequence of upper record values corresponding to F, then Ha(Un;t) is decreasing in n. An example of the distributions for which this theorem can be applied is the power distribution with distribution function F(x) = xb, 0 < x < 1, b > 1. Remark 3.8. The reader may have observed that some of the results of the paper on order statistics and record values have analogies in the proofs and conclusions. The main reason (rationale) is the analogy between the definition of maximum of a set of order statistics and record values. In fact, record values can be viewed as the maximum of a set of order statistics from a sample whose size is determined by the values of occurrence of the observations (for more details on similarity between distributional properties of order statistics and record values see [16]).
4. Conclusion In this paper, we studied the residual Rényi entropy (RRE) of order statistics and record values. We obtained the relation between the RRE of order statistics (record values) of a continuous distribution in terms of RRE of order statistics (record values) of uniform distribution. As the RRE usually cannot be obtained in an closed form, some bounds are provided for RRE of order statistics and record values. We investigated the monotone behavior of RRE of order statistics Xk:n, in term of k and n and the monotone behavior of record values in terms of number of observations. Several illustrative examples were also provided. Acknowledgments The authors thank the editor and two referees for helpful and constructive comments which have considerably improved the presentation of the paper. The authors would also like to thank Professor E.S. Soofi for his suggestions in preparing the paper. This work is supported in part by the ‘‘Ordered and Spacial Data Center of Excellence of Ferdowsi University of Mashhad”.
Appendix A. Proofs A.1. Proof of Theorem 2.7 h i
k1 (a) According to Theorem 2.3, it is enough to obtain a bound for 11 a log E f a1 ðF 1 ðY k ÞÞ . Note that mk ¼ max FðtÞ; n1 is the mode of the distribution of Yk. Let M k ¼ fY k ðmk Þ, then for a > 1 (0 < a < 1) we have
1 1 log E½f a1 ðF 1 ðY k ÞÞ ¼ log 1a 1a
Z
FðtÞ
1 þ log 1a ¼
yaðk1Þ ð1 yÞaðnkÞ
1
Z
BFðtÞ ðaðk 1Þ þ 1; aðn kÞ þ 1Þ 1
FðtÞ
f a1 ðF 1 ðyÞÞ dy ¼
f a1 ðF 1 ðyÞÞ dy P ð6Þ
1 1 log M k þ log 1a 1a
Z
1
1 log Mk 1a
f a ðuÞ du
t
1 a log Mk þ Ha ðX; tÞ þ log FðtÞ: 1a 1a
(b) We have for a > 1(0 < a < 1),
f a1 ðF 1 ðyÞÞ 6 ðPÞM a1 : Thus for the RRE of kth order statistic, using Theorem 2.3, we can write
Ha ðX k:n ; tÞ P Ha ðU k:n ; FðtÞÞ log M: For the case when a ? 1, using relation (7), we obtain the following result for residual Shannon entropy
HðX k:n ; tÞ P HðU k:n ; FðtÞÞ log M:
4204
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
A.2. Proof of Lemma 2.8 We prove part (i). Part (ii) can be proved similarly. Under the assumption that ha(m) is differentiable in terms of m, we obtain
! @g a ðmÞ @ha ðmÞ 1 @m ; ¼ @m 1 a g a ðmÞ where
Rc
uma ðxÞv a ðxÞ ðdx a um ðxÞv 1 ðxÞ dx t
g a ðmÞ ¼ Rt c and
@g a ðmÞ a ¼ R c aþ1 @m um ðxÞv 1 ðxÞdx
Z
c
log uðxÞuma ðxÞv a ðxÞdx
t
t
Z
c
um ðxÞv 1 ðxÞdx
Z
t
c
log uðxÞum ðxÞv 1 ðxÞdx
t
Z
c
uma ðxÞv a ðxÞdx :
t
ð14Þ Now using the fact that Wa 6st(Pst)W1 and that log is increasing function we get
E½log uðW a Þ 6 ðPÞE½log uðW 1 Þ (see, for example [26]). This shows that (14) is non-positive (non-negative) and hence ha(m) is an increasing function of m. h A.3. Proof of Corollary 2.10 (a) We assume that the system is parallel. For series system similar arguments can be used to prove the result on using Remark 2.9. From Lemma 2.2 it is easily seen that Ha(Un:n;t) can be written as (9) with u(x) = x and va(x) = xa in which we assume, without loss of generality, n P 1 is a continuous variable. Since for a > 1 (0 < a < 1) the ratio
R1
xaðn1Þ dx t R1 xn1 dx t
is increasing (decreasing) in t, for the chosen u(x) and va(x), we have
W a Pst ð6ÞW 1 ; where density function of Wk, k > 0, is given in (8). Hence from Lemma 2.8 we conclude that the RRE of the parallel system is a decreasing function of the number of components. na x (b) The result can be proved using the same arguments as used to prove part (a) on taking uðxÞ ¼ 1x and v a ðxÞ ¼ ð1xÞ . In xa k1 this case it can be easily seen that for t P n1 and a > 1 (0 < a < 1),
W a 6st ðPÞW 1 : Hence it can be concluded that for k1 6 k2 6 n,
Ha ðU k1 :n ; tÞ 6 Ha ðU k2 :n ; tÞ;
tP
k2 1 : n1
ð15Þ
A.4. Proof of Theorem 2.11 Let Y n BFðtÞ ðaðn 1Þ þ 1; 1Þ and assume that fY n shows the density function of Yn. Then we have
fY nþ1 ðwÞ BFðtÞ ðaðn 1Þ þ 1; 1Þ a y ; ¼ fY n ðyÞ BFðtÞ ðan þ 1; 1Þ
FðtÞ < y < 1;
which is an increasing function of y. This implies that Yn 6lrYn+1 and hence Yn 6 stYn+1. On the other hand, for a > 1 (0 < a < 1), fa1(F1(x)) is increasing (decreasing) in x. Hence
E½f a1 ðF 1 ðY n ÞÞ 6 ðPÞE½f a1 ðF 1 ðY nþ1 ÞÞ:
ð16Þ
From (16) we get that
1 E½f a1 ðF 1 ðY nþ1 ÞÞ log 6 0: 1a E½f a1 ðF 1 ðY n ÞÞ
ð17Þ
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
4205
From Theorem 2.3 we have
Ha ðX nþ1:nþ1 ; tÞ Ha ðX n:n ; tÞ ¼ Da ðn; tÞ þ
1 E½f a1 ðF 1 ðY nþ1 ÞÞ log ; 1a E½f a1 ðF 1 ðY n ÞÞ
ð18Þ
where Da(n; t) = Ha(Un+1:n+1; F(t)) Ha(Un:n; F(t)). Now using part (a) of Corollary 2.10 and inequality (17), it can be seen that the left hand side of (18) is non-positive. That is, the RRE of the parallel system is a decreasing function of the number of components. Similar arguments can be used to prove the case when the components of the system are connected in series. h A.5. Proof of Theorem 2.14 From Theorem 2.3, we have for k1 6 k2 6 n
Ha ðX k2 :n ; tÞ Ha ðX k1 :n ; tÞ ¼ Da ðk2 ; k1 ; tÞ þ
E½f a1 ðF 1 ðY k2 ÞÞ 1 log ; 1a E½f a1 ðF 1 ðY k1 ÞÞ
ð19Þ
where
Da ðk2 ; k1 ; tÞ ¼ Ha ðU k2 :n ; FðtÞÞ Ha ðU k1 :n ; FðtÞÞ and
Y ki BFðtÞ ðaðki 1Þ þ 1; aðn ki Þ þ 1Þ;
i ¼ 1; 2:
It is easy to verify that Y k1 6lr Y k2 and hence Y k1 6st Y k2 . Now the result follows using part (b) of Corollary 2.10 and the same arguments as used to prove Theorem 2.11. h A.6. Proof of Corollary 2.16 Let k1 6 k2 6 nþ1 : The right hand side of this inequality implies that 2
m P F 1
k2 1 ; n1
where m ¼ F 1 ð12Þ denotes the median of F. Now using Theorem 2.14, we have for t P m
Ha ðX k1 :n ; tÞ 6 Ha ðX k2 :n ; tÞ:
A.7. Proof of Theorem 3.3 The proof follows easily from the definition of RRE of Un, in which one needs to make substitution u ¼ log FðxÞ in the integrant. Then after some simple algebraic manipulations the result follows. h A.8. Proof of Theorem 3.7 Using Theorem 3.3, we have
Ha ðU nþ1 ; tÞ Ha ðU n ; tÞ ¼ Da ðn; tÞ þ
1 E½f a1 ðF 1 ð1 eV nþ1 ÞÞ log ; 1a E½f a1 ðF 1 ð1 eV n ÞÞ
ð20Þ
where
Da ðn; tÞ ¼ Ha ðU nþ1 ; FðtÞÞ Ha U n ; FðtÞ
ð21Þ
and
V n C log FðtÞ ðaðn 1Þ þ 1; 1Þ:
On taking u(x) = x and va(x) = xaex and using Lemma 2.8, we can show that Ha U n ; FðtÞ is decreasing in n and hence Da(n;t) 6 0. One can show that Vn 6 lrVn+1 and hence Vn 6 st Vn+1. This implies that, for 0 < a < 1(a > 1),
E½f a1 ðF 1 ð1 eV n ÞÞ P ð6ÞE½f a1 ðF 1 ð1 eV nþ1 ÞÞ: Hence, we have for a > 0,a – 1
1 E½f a1 ðF 1 ð1 eV nþ1 ÞÞ log 6 0: 1a E½f a1 ðF 1 ð1 eV n ÞÞ
4206
S. Zarezadeh, M. Asadi / Information Sciences 180 (2010) 4195–4206
Thus, from the last inequality and inequalities (20) and (21), we conclude that Ha(Un+1; t) Ha(Un; t) 6 0. This completes the theorem. h References [1] B.C. Arnold, N. Balakrishnan, H.N. Nagaraja, Records, Wiley, New York, 1998. [2] M. Asadi, N. Ebrahimi, Residual entropy and its characterizations in terms of hazard function and mean residual life function, Statistics and Probability Letters 49 (2000) 263–269. [3] M. Asadi, N. Ebrahimi, E.S. Soofi, Dynamic generalized information measures, Statistics and Probability Letters 71 (2005) 85–98. [4] S. Baratpour, J. Ahmadi, N.R. Arghami, Entropy properties of record statistics, Statistical Papers 48 (2007) 197–213. [5] S. Baratpur, J. Ahmadi, N.R. Arghami, Characterizations based on Rényi entropy of order statistics and record values, Journal of Statistical Planning and Inference 138 (2008) 2544–2551. [6] F. Belzunce, J. Navarro, J.M. Ruiz, Some results on residual entropy functions, Metrika 59 (2004) 147–161. [7] J.F. Bercher, On some entropy functionals derived from Rényi information divergence, Information Sciences 178 (2008) 2489–2506. [8] H.A. David, H.N. Nagaraja, Order Statistics, third ed., Wiley, New York, 2003. [9] A. De Gregorio, S.M. Iacus, On Rényi information for ergodic diffusion processes, Information Sciences (2009) 279–291. [10] N. Ebrahimi, How to measure uncertainty in the residual lifetime distributions, Sankhya A 58 (1996) 48–57. [11] N. Ebrahimi, S.N.U.A. Kirmani, Some results on ordering of survival functions through uncertainty, Statistics and Probability Letters 11 (1996) 167–176. [12] N. Ebrahimi, E.S. Soofi, H. Zahedi, Information properties of order statistics and spacings, IEEE Transactions on Information Theory IT50 (2004) 177– 183. [13] L. Golshani, E. Pasha, G. Yari, Some properties of Rényi entropy and Rényi entropy rate, Information Sciences 179 (2009) 2426–2433. [14] R.C. Gupta, S.N.U.A. Kirmani, Closure and monotonicity properties of nonhomogeneous poisson processes and record values, Probability in the Engineering and Informational Sciences 2 (1988) 475–484. [15] R.D. Gupta, A.K. Nanda, a- and b-Entropies and relative entropies of distributions, Journal of Statistical Theory and Applications 1 (3) (2002) 177–190. [16] U. Kamps, A Concept of Generalized Order Statistics. Teubner, Stuttgart, 1995. [17] M. Mahmoudi, M. Asadi, On the monotone behavior of time dependent entropy of order a, Journal of Iranian Statistical Society 9 (2010). [18] D. Morales, L. Pardo, I. Vajda, Some new statistics for testing hypotheses in parametric models, Journal of Multivariate Analysis 62 (1997) 137–168. [19] S. Nadarjah, K. Zografos, Formulas for Rényi information and related measures for univariate distributions, Information Sciences 155 (2003) 119–138. [20] S. Nadarjah, K. Zografos, Expressions for Rényi and Shannon entropies for bivariate distributions, Information Sciences 170 (2005) 173–189. [21] A.K. Nanda, S.S. Maiti, Rényi information measure for a used item, Information Sciences 177 (2007) 4161–4175. [22] A.K. Nanda, P. Paul, Some results on generalized residual entropy, Information Sciences 176 (2006) 27–47. [23] V. Nevzorov, Records: Mathematical Theory, Translation of Mathematical Monographs, vol. 194, American Mathematical Society Providence, RI, USA, 2001. [24] S. Park, The entropy of consecutive order statistics, IEEE Transaction on Information Theory IT41 (1995) 2003–2007. [25] A. Rényi, On Measures of entropy and information, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. I, University of California Press, 1961. [26] M. Shaked, J.G. Shanthikumar, Stochastic Orders, Springer, New York, 2007. [27] K. Song, Rényi information, loglikelihood and an intrinsic distribution measure, Journal of Statistical Planning and Inference 93 (2001) 51–69. [28] K.M. Wong, S. Chen, The entropy of ordered sequences and order statistics, IEEE Transaction on Information Theory IT36 (1990) 276–284.