Estimation of arithmetic means and their applications in guessing theory

Mathl. Comput. Modelling Vol. 28, No. 10, pp. 31-43, 1998 @ 1998 Elsevier Science Ltd. All rights reserved Printed in Great Britain 089%7177/98 $19.00 + 0.00 PIL: SO8957177(98)00153-S

Estimation of Arithmetic Means and Their Applications in Guessing Theory S. S. DRAGOMIR Applied Mathematics Department, University of Transkei UNITRA Private Bag Xl UMTATA, 5100, South Africa

S. BOZTAS Department of Mathematics, Royal Melbourne Institute of Technology GPO Box 2476V, Melbourne 3601, Australia (Received and accepted March 1998)

Abstract-we make use of come estimation results for arithmetic means to develop some new estimates for the moments of guessing mappings which complement, in a sense, the results from the recent papers 11-4). @ 1998 Elsevier Science Ltd. All rights reserved. Keywords-Inequalities,

Guessing, Entropy, Information theory.

1. INTRODUCTION Recently, the problem of bounding the expected number of guesses required to determine the value of a random variable has attracted some interest in the information theory literature. This problem, which we will refer to as the guessing problem has applications in sequential decision problems in information theory, including source and channel coding. Arikan [l] derived upper and lower bounds to the moments of this quantity and used these bounds to determine the cutoff rate region of sequential decoding for multiple access channels. Arikan’s upper bound was later improved by Boztq in 121. Subsequently, Arikan and Merhav considered guessing subject to distortion in the context of lossy block source coding in their paper [3]. The same authors generalized their work to joint source-channel guessing with distortion in [4]. Dragomir and Boztaq obtained two-sided bounds on the guessing moments in [5]. Here, we investigate the problem of bounding these moments in a general setting and obtain a number of twosided bounds. These results generalize and extend those in [5]. 2.

THE

Consider the arithmetic

RESULTS

ON

ARITHMETIC

MEANS

means

A&, 4 =

=&a,

wherepi>Oand

&~=l, i=l

i=l

The second author would like to thank E. Arikan and who provided him with the preprints of [3,4].

31

S. S. DFLAGOMIR AND S. BOZTAQ

32

n

A&, x) =

c

n

qzxir

whereqizoand

Cqi=l, kl

i=l

where x, (i = l,.. . ,n) is a sequence of real numbers. result which uses some well-known inequalities. THEOREM2.1.

We then have the following estimation

Under the above assumptions,we have

PROOF. It is clear that

IA&,x)-A,(q,z)l=

&m) IicI z

Note that the first and last inequalities from Hiilder’s inequality.

) xi/

&lw~illz,l~ i=l

in (1) are now obvious.

The middle inequality

follows I

If we define

A&)

= ; $xi a=1

to be the unweighted COROLLARY2.2.

mean of the xi, then we have the following corollary.

Under the above assumptions,

IA&,z)

we have

- An(z)I I 4

max pi-b a=l,...,n l

(2.2)

i

I

&lxil

Now, let us define the bounds

mp= +yfn..,n Pir I

mq =

mm

qi,

i=l,...,n

It is clear that

mp - Mq P mpVqIMp,qSMp-mq. The following holds if we utilize the celebrated Griiss inequality.

Estimation of Arithmetic Means

THEOREM 2.3.

33

Under the above assumptions, we have IA&, z) - An(q,4

I ;(M,

- mz)(M,,q

I f(M,

- m,)(M,

- mp,J (2.3) + Mq - mp - mp).

PROOF. First, we shall prove the following discrete version of Griiss’ inequality starting with the integral version:

-I

1

b-a

b

J

a

which holds provided f, g are integrable z E [a,b]. We claim that

mappings with y < f(z)

5 I?, 4 < g(z) 5 @, for all

l~~o&i-i$e~~$bi~S

:(A-o)(B-b),

(2.5)

provided a 5 ai 5 A, b 5 bi 5 B, for all i = 1,. . , n. We choose in Grtiss’ integral inequality

f(z) =

al,

2 E [O,I),

o2, .

2 E [L2),

1: an, T E [n - 1,721, ( h, 2 E [O,l),

1: b2,

m

=

x

E

[La,

x

E

[n-

.

1 bn,

l,n].

Then, y = a, I? = A, 4 = b, 3 = B, and the integrals can be replaced by the corresponding sums proving (2.5). Now, we have

A,&, 4 - Ah, 4 =

&i - q&i. i=l

Applying inequality (2.5) to the sequences a; = (pi - qi), bi = xi, and taking into account that Cz, ai = 0, we deduce the estimate in the theorem. I COROLLARY2.4. With the above assumptions, we have IAn(P, z) - An(x)1 I z(&

- m&M,

- m,J.

(2.6)

We now present some different estimates using the classical identity due to Korkine. THEOREM 2.5.

Under the above assumptions, we have

IA,(P, ~1 - Anta 41

(2.7)

S. S. DRAGOMIR AND S. BOZTAB

34

PROOF. We shall use the following identity due to Korkine (see, for example, [S, p. 2421) which can be easily verified for arbitrary real numbers ai, bi (i = 1,. . . , n):

Then, we have

= i

,k IPi-

Qi -

4j)l(5i- 5j)

(Pj -

x,3=1

=

&$

[(Pi-

qi) -

(Pj

-

flj)]

(xi - zj)

1,pl

and thus, IA&~)

- A&,2)1

I ,~z:,

_I_

lzi - “jle

lpi - 4. i=l

Similarly, we have

I&@, 4 - A&

z)I = &

-5 [(pi - pj) - (qi - qj)l bi - xjj) %,J=l

and the first part of the theorem is proved. On the other hand, we have that

35

Estimation of Arithmetic Means

and similarly,

and the theorem is thus proved. The following corollary also holds. COROLLARY 2.6. With the above assumptions,

we have

I&(P, xl- An(s)I 5

We can also obtain the following result, also based on the Korkine identity. THEOREM 2.7. With the above assumptions,

I-4&,4

- &(q,z)l I n [i_I &

we have

’- q.1’ * y2 [tg+

PROOF. Using the Cauchy-Schwarz-Buniakowski

lAn(~>s) - Mq>z)/ I: & .c lb r,j=1 1 r n

-pj)

inequality - (qi -

(:gg2]“2. for double sums, we have that

a)1 (xi - q)l i

w rn

1 l/2

It is simple to show that

i=l

i,j=l

and that &(zi-zj)2=2

[ngz?-

($zi)‘].

Hence, we deduce the theorem. COROLLARY 2.8. With the above assumptions,

we have

We will use the identity below which was obtained by Dragomir and Mond in [7]. LEMMA 2.9.

for an arbitrary

sequence of real numbers ai, bi (i = 1, . . . , n).

PROOF. Straightforward

by expanding the right-hand

Using this lemma we can prove the following.

side.

36

S. S. DRAGOMIR ANDS. BOZTA~

THEOREM 2.10. With tJ.ie above assumptions,

PROOF. Substitute

we have

ai = pi - qi and bi = xi - (l/n) cT,‘=i Xj, for i = 1,. . . , n in Lemma 2.9 to

deduce A,@, 5) - &(q, 5) = e(~i

- %)

i=l

Now, the first and last inequalities by Holder’s inequality.

in the theorem are obvious and the second inequality

follows I

Note that for p = q = l/2, we obtain Theorem 2.7. COROLLARY 2.11. With the above assumptions,

3. ESTIMATES

FOR

THE

we have

MOMENTS

OF GUESSING

Massey [8] considered the problem of guessing the value of a random variable X by asking questions of the form “is X equal to x?” until the answer is “yes”. Consider a random variable X with finite range X given by X={x1,xz

,‘..I 2*}.

The above problem can be formalized by calling a mapping G(X) of the random variable X a guessing mapping for X whenever G : X --) (1,. . . , n} is one-to-one. If the guessing process for X is being carried out while the value of a related random variable Y is known, then we call a mapping G(X ) Y) a guessing mapping for X given Y, if for any fixed value Y = y, G(X ( y) is a guessing mapping for X. A guessing sequence corresponding to G is optimal in the sense of minimizing the quantity E[G(X)] if J’x(G-l(k)) The following inequalities the recent paper [l].

2 Px (G-‘(k

+ I)),

k 1 1.

(34

on the moments of G(X) and G(X 1 Y) were proved by Arikan in

Estimation of Arithmetic Means

37

THEOREM 3.1. For an arbitrary guessing mapping G(X) or G(X ) Y) and for any p 2 0, we have [G(X)Pl

E

[CiL px(~k)l”+ql+p

>

(1 + Inn)@

E

[G(X

,

1

y)p,

CyEYEL

PX,Y(WC, YY'l+pl (1+

l+p

lnn)p

Note that, for p = 1, we get the following estimates on the average number of guesses:

E[G(x)]

[c;z,, px(xk)1’2]2

2

+Inn)

(1 and

E IG(X I UpI L

I&y [CL PX,Ybk,YP2]

2

(1+1nn)

.

For further results concerning the guessing moments and their applications see [2], as well as [3,4]. 3.1. Bounds

on Moments

of Two Guessing

in information

theory,

Mappings

To simplify the notation further, we assume that the zz are numbered such that xk is always the kth guess by the guessing mapping G(X). Then, consider an arbitrary guessing mapping L(X) and note that we can write (by suppressing the dependence on X and letting Px(zi) = pi)

E(GP) = &pi,

(P

i=l

E (W =

2 iPP,(i), i=l

2 (97

(P 2

017

for some permutation of the indices {1,2, . . . , n}. We now use results from the previous section to obtain some bounds on moments of guessing. THEOREM 3.2. Let G(X) estimate:

and L(X)

IE(G(X))’- E(W))7 I

be two guessing mappings,

I

i=y,y,n IPi - P,(i)

where Sp(n) = Cz,

r > 1,

I S,(n),

P with p, any nonnegative real number.

then we have the following

r-1

+8-l

= 1,

30

S.S.DRAGOMIRAND S.BOZTAS

PROOF. The proof uses Theorem

2.1, by choosing qi = p,(j) and Zi = P.

The details are

omitted.

I

It is instructive to note that (l/n)S,(n) is the pth moment of any guessing mapping when the distribution of X is uniform. If we let p = 1, T = s = 2, we get

I

IWGW))- ~W(-‘3)lI

G(X)

+ ‘)Q2n + ‘)) 1’2 ,

(3.2)

IE(G(X)Y’- -V(X))pl I ; (d’- 1) (Mm,,- n-+) I ; (nP- 1) (PM - I’,,,),

(3.3)

[& Ipi - p,,(i) ,“] 1’2 (n(n

Now, let us define

Mv = i=y”n (Pi- P,(i))

1

9.

m 0,p

=

,

%=Tin (pi- P,(Q), ,...,n

PM =

max pi, i=l,...,?Z

P* =

min pi, i=l,...,n

and note that we can obtain the estimate. THEOREM

3.4. With the assumptions

of Theorem 2.3, we have

for any two guessing mappings and any p 2 0. PROOF. Apply the proof of Theorem

a

2.3.

The following theorem also holds. THEOREM

3.4. With the above assumptions,

IE(GG)Y’- %W))PI

where

and n 2 2.

we have

Estimation of ArithmeticMeans

39

PROOF. We need only to compute

c

tip-jP]=

c

]jP-1Q]++

c

+

]jP--2P(

2
l
lli
c

]jP - 3P] +‘..

-I- ,_Z<.

IjP - (n -

1N

s
C

jp -

lP(n-2)+

C

+ np - (n - l)PIP

f..

=

jG-zP(n-3)

3
2
jp - lP(n - 1) + 2

C

+

jp - 2O(n - 2) - [I@]

j=l

llj
jP - 3P(n - 2) - [ZP + lP]

2 j=l

+

. *. + &

- [(n - 1)P + (n - 2)P + * *. + 2p + lP]

3=1 =

(n

-

l)S,(n)

= (n - l)S,(n)

[2(n - 1)lP + 2(n - 1)2P +. . . + 2(n - l)P]

-

- 22

P(n - i)

ill n-l =

(n

1)$(n)

-

- 2n c

ip + 2nf

=

(n

-

l)S,(n)

-

2nqn

ip+l

is1

i=l

- 1) + 2S,+r(n

- l),

and the theorem is proved. Note that when p = 1, we obtain Kr(n) = ; [(n - l)Sr(n) =

- 2n&(n

- 1) + 2S2(n - l)]

(n - Nn + 1) 6

’

which leads to

while if p = 2, we get Ks(n) = ((n - l)(n + 1)2)/6, which yields (W(X))2

- E(W))2( (n” - l) min

I

{

,$r

IPi -Wi)l~ f

(n - l)(n + 1>z (

3

min

C lb - RI + l
IPi -PA}. - _ { i=n,yn IJW- WAI 3l_jycy
By utilizing Theorem 2.7, we obtain the following theorem.

y

S. S. DRAGOMIR ANDS. BOZTAS

40 THEOREM 3.5.

Under the above assumptions, we have

)‘”(i&(n)-(iSp(n))l)l12.

IE(-W)Y - E(WON 5 n PROOF. Details are omitted. Note that if we consider the case p = 1, we obtain

IE(L(X)) - E(G(X))I

(n- ‘tn +‘)) 1’ [f& 2 -,,,,Y] 1’2.

5 ;

kl

The following theorem also holds. THEOREM 3.6.

IE(G(X)Y

With the above sssumptions, we have

IPi - Pdi) IS

5 { (

- E(L(X))Pl

(3.4) r > 1,

r-i

+ s-i

= 1,

PROOF. Details are omitted.

I

If we put p = 1 in (3.4)) we deduce /

IE(G(X))

(g IPi

- JW(X))I

I t

l/s

Pa(i) I*

>

where it is straightforward (but tedious) to show that

where LzJ is the greatest

integer I 2 and [zl is the least integer > x. If we put p = 2 in (3.4), we deduce

(n - l)?

(E(G(X))2 - E(L(X))2j

5

I ($

F

+ l) g1 IPi -

Ipi -~,(i)fs

(2 r>l,

P,(i)

I1

Ii2 - (n + ‘)y

+ 1)/r)1”

7

T-i+s-1=1,

[(n + 1)(2n + 1) - (no + 1)(2n0 + l)] m,yJp; - p,(i)lj

with 120= L,/(n + 1)(2n + 1)/S] a natural number.

We omit the details.

(3.6)

41

Estimationof ArithmeticMeans

3.2. Bounds on the Distance from the Uniform

In what follows, arithmetic mean in proximation results random variable as THEOREM 3.7.

Case

using the results in Section 2, concerning the approximation of a weighted terms of unweighted arithmetic means, we are going to point out more apfor the moments of a given guessing mapping G(X) associated with a discrete in Section 3.1 above.

For a guessing mapping G(X), we have r

np

2 pi - 1 ,

is1

I I

(j+:~ ) P

E(G(X))p -

+&)

l/r

(s&P

5 (

T > 1, \ S,(n)

9

(3.7) r-r + s-r = 1,

1 ma pi - ; , %=l,...,TZ I I

where S,(n) = ~~=, P. PROOF.

I

Use Corollary 2.2 with zi = ip.

As usual, a special case of interest is p = 1. Moreover, by using Corollary 2.4, we can also prove the following. THEOREM 3.8.

For a guessing mapping G(X), we have

lWG(X)Y - iSp( where PM = maxi,1 ,...,n pi and P,

I

n(nle‘) (pM -

(3.3)

P,,,),

= mmi,l,..., n pi.

If we let p = 1 in the above theorem, we get a result proved by Dragomir and Boztq E(G(X))

-

Fi

5

v

in [5],

(PM - Pm).

The following theorem also holds. THEOREM 3.9.

For a guessing mapping G(X), we have

where KP(n) = (l/n)[(n

- l)Sp(n) - 2nS,(n

- 1) + 2S,+r(n

- l)] as in Theorem 3.7.

PROOF. The proof follows by Corollary 2.6 and using an identical argument to get the bound K,(n).

as in Theorem 3.7 I

Once again, the cases of most interest are p = 1,2. A related result of Dragomir and Boztq from [5] to the case p = 1 is E(G(X))

while Theorem 3.9

would yield

_

?I

5

cn - ‘F

+ ‘)

S. S. DRAGOMIR ANDS. BOZTA~

42

THEJOREM3.10.

With the above assumptions,

we have

PROOF. The proof follows by Corollary 2.8. If we choose p = 1 in the above inequality, we obtain

E(G(X))

_

!!+i 1 5

;

[

tn- ‘tn + ‘1y/2 [g(*~-;)2]1’2

=(n- lh

12

where llpll2 is the Euclidean norm of p = (pr, . and Boztq

1l’*

+ 1)

(nllPII;

_

1)‘/2

,

,pn). This result was first proved by Dragomir

in [5].

THEOREM 3.11.

For a guessing mapping G(X),

we aSo have

PROOF. The proof follows by Corollary 1.12. We omit the details. The case p = 1 includes the inequality

first proved in [5]. 3.3.

Discussion

of Results

If we assume that G(X) is the optimal guessing mapping (in the sense of minimizing E(G(X))p), then all the inequalities in this paper can be used to obtain explicit upper bounds of the form 0 I F(G(X)Y

- F(L(X)Y’

-< M(p)

or equivalently F(L(X))” where M(p)

I @G(X))’

+ M(p),

is the right-hand side of the typical two sided inequalities obtained in this paper,

namely,

Such upper bounds characterize worst case performance in guessing strategies, while the corresponding lower bounds characterize best case performance in guessing strategies.

Estimation of Arithmetic Means

43

4. CONCLUSION We have obtained

a range of bounds

on guessing

moments.

These results

have applications

in

information theory, especially sequential decoding. The interested reader can refer to [l-4] for detailed examples of applications for similar inequalities to information theory problems such as sequential decoding and source coding.

REFERENCES 1. E. A&an, An inequality on guessing and its application to sequential decoding, IEEE ‘Z’knsactions on Infonatzon Theory 42, 99-105, (January 1996). 2. S. Boztag, Comments on an inequality on guessing and its application to sequential decoding, IEEE lensactions on Information Theory 43, (November 1997). ovaInformation Theory, (July 3. E. Arikan and N. Merhav, Guessing subject to distortion, IEEE tinsactions 1996). 4. E. Arikan and N. Merhav, Joint source-channel coding and guessing with application to sequential decoding, IEEE Tmnsactions on Information Theory, (January 1997). 5. S.S. Dragomir and S. Boztaq, Some estimates of the average number of guesses to determine a random variable, In Proceedings of the IEEE International Symposium on Information Theory, June 30-July 4, 1997, Ulm, Germany. 6. D.S. Mitrinovic, J.E. Pecaric and A.M. Fink, Classical and New Inequalities in Analysis, Kluwer Academic, (1993). 7. S.S. Dragomir and B. Mend, Some mappings associated with Chebyshev’s inequality for sequences of real numbers, under review for publication, (1996). 8. J.L. Massey, Guessing and entropy, In Proceedings of the 1994 IEEE International Symposium on Information Theory, Trondheim, Norway, July 1994.