- Email: [email protected]
A lower bound for expectation of a convex functional
Statistics & Probability North-Holland
Letters
15 October
18 (1993) 191-194
1993
A lower bound for expectation of a convex functional Mei-Hui
Guo
National Sun Yat-Sen Uniuersity, Kaohsiung, Taiwan, ROC, and Worcester Polytechnic Institute, Worcester, MA, USA
Ching-Zong
Wei
Academia Sinica, Taipei, Taiwan, ROC Received April 1992 Revised January 1993
Abstract: Let 4 be a symmetric convex function from I%” to Iw. Under certain conditional symmetric conditions on the random variables Xi,. , X,,, the inequality:E[d(X,, . , X,)] > E[max,, iG &O,. . ,O, X,, 0,. ,O)] is derived. Conditions under which the strict inequality holds are also obtained. Application to nonlinear autoregressive models and symmetrization of random variables are given. Keywords: Consistency;
convexity;
symmetry;
stationarity.
1. Introduction Let Xl, x,,..., X,, be a sequence of random variables and C#Jbe a convex function from R” to R. In this paper, we investigate the conditions under which
where f. is a known function which depends on the unknown parameter 6 and E,‘S are i.i.d random variables with mean zero. To estimate 6, one may use the minimum contrast estimator 6n which is defined to be the minimizer of the penalty function fl-P
1
r=l
aE lyyn 4(0,...,0, Xi,O,...,O) . [.. The motivation of this inequality originated from the study of the estimation of a nonlinear autoregressive time series (Z,}:
Correspondence to: Mei-Hui Guo, Department of Applied Mathematics, National Sun Yat-Sen University, Kaohsiung, Taiwan 804, ROC, or Ching-Zong Wei, Institute of Statistical Science, Academia Sinica, Taipei 11529, Taiwan ROC. 0167-7152/93/$06.00
0 1993 - Elsevier
Science
Publishers
where the loss function p is chosen according to one’s interest. For example, P(Z, >...Y Zp+l; fi) = (Zp+,
-f&q
T..., z,,)‘,
if 6,, is the conditional least squares estimator. When (Z,) is ergodic with true parameter 6,, to ensure the almost sure convergence of I?,, to 6, (strong consistency), one common assumption is that
Q(o) =+(Z,,...,Z,+,;
B.V. All rights reserved
a)] 191
Volume
18, Number
3
STATISTICS
has a unique minimizer 1967). In the conditional set
& PROBABILITY
(Guo, 1989; and Huber, least squares case, if we
LETTERS
15 October
if and only if there exist 1 < i 0.
(3)
X, =fil,,(zl,...,z,)-f~(zl,...,z,),
Remark 1. Since 4 is convex and symmetric,
Xz=a,+,
4(i)
and
All expectations well defined.
then this assumption E[4(X,,
X,)]
is equivalent
>E[4(0,
X,)]
to for all fi+fi,.
In Section 2, the main theorem is presented. In Section 3, two applications are given. One deals with the strong consistency of the maximum likelihood type estimator for an exponential autoregressive model. Another provides an upper bound for the expectation of the extreme order statistics through the absolute moment of the symmetrized partial sum.
=i[@)
+4(-i)]
a@).
considered
(4)
in Theorem
In the following, a function f< . ) from R” into R is said to be symmetric (about zero) if f(i) = f< -i_) for all _? E R”. A random vector 2 is said to be symmetric (about zero) if 2 and -.?! have the same distribution. Theorem 2.1. Let 4 be a symmetric convex function from R” into R and X,, . . . , X,, be random variables such that for all 1 < i < n, either X, given t.={X,:
X, and pi. If X,
+$(X1,
%)I%]
F,)lfi]
=-+$(-Xi, =E(+(X,,
I?)
and +(X,3
%)]
=E(:[4(X,,
2;1)
E[~(X,Y..?X,)l
J?)]).
+$(X1,
I?) +4(-x,,
J?)]
=t[~(X,>~J+~(X,, By the convexity 3[4(X,?
-Q]. of 4,
Pi) +4J( -X1, v 6(X,,
E[4(Xi,
I?)]
Q] fi),
the maximum
+(O,
(6) of u and v. In
pi) @(Xi,
=.+(X)]~ Furthermore, E[+(O ,...,
if 4 is strictly convex and 0, X,, 0 ,...,
0)]
E[4(Xw.,X,)]
192
l~lyn 4(0 ,... 20, X,, O,...,O)] 1 .-.
G)] (7)
where X=(X,,..., XJ and $(JZ)=+(O, 91)V +(x1, 6). It is not difficult to see that $ is symmetric and convex. Replacing (4, Xi, I!i) in (7) by ($,, X2, F2,>, we have
for 1 < i < n then
> E
(5)
If pi given X, is symmetric, then identity (5) can also be obtained by similar arguments. Now, since 4 is symmetric,
where u v v denotes view of (5) and (6),
is symmetric of q. given X, is symmetric. Then
Pi)] I q
+4(-X,,
> 4(o,Q
l
2.1 are
Proof of Theorem 2.1. Consider given fi is symmetric then
+4(-X,, 2. The main result
1993
E[$@)] (2)
aE[$(X,,
0, X,,...,X,)
vlcr(O, X,, O,...,O)]
(8)
Volume
18. Number
STATISTICS
3
& PROBABILITY
=E[cb(O, 0, x, >...,X,) V4(XlY6) v$J(O, x,7
o,...,0117
(9)
where the last identity is ensured by (4). In view of (7) and (91, inequality (1) is then a consequence of induction arguments. Now let us consider the case when 4 is strictly convex. If (3) is violated then with probability one, either X= 6 or there is a j such that Xj z 0 and < = 6. In both cases, = imfx_ $(O,. . .,o, xi, . .
#J(X)
0,. . . ,O).
Therefore, (2) is violated. Now assume that (31 is satisfied. Since E[$(O, . . . ,O, Xi, 0,. . . ,011 < ~0 for all i and max
+(O ,...,
lGi9n
=G i i=l
0, Xi,0 ,..., )...)
[4@
0,
xi,
0)
0 )...)
0)
-m]
+ 4(@>
E(+$(X,:
I?) +4(-X,, -4(O,
I?)]
Yi) v 4(X,,
O)> > 0.
(10)
By (5) and E[&J?)] < TV,(10) in turn implies that E[&!?)l > E[+(O, Yi,> V 4(X,, 611. However, from above arguments E[ +(0, >E
Yi) v 4(X,,
1. .
G)]
imzyn $(O,...,O,
Combining
these
Xi, O,...,O)
two inequalities,
1.
we obtain
Remark 2. Let Xi = (Xi, Xi+r, . . . , XJ. derived from the arguments above that E[@)]
>E[
lm:-k . .
+(O,...,O,
(2).
It can be
X,, O,...,O>
v@,...,o,
Remar_k 3. If one is only interested in showing E[+(X)] > E[ 4(0, Y,)] then from the proof above it is not difficult to see that we only have to assume that (9 4 is convex and symmetric; or Yi (ii) either Xi given Yi is symmetric given Xd is symmetric; (iii) E[@, Y,)l < rl”; Y,> + 4(-X,, IQ > (iv) PI2-‘[~(x,,
$No,IQ, > 0. Remark 4. A referee of using the arguments that the sequence =E[+(X,
Xk)]
,...,
pointed out the possibility given in the proof to show
Xi, 0 ,...,
0)]
is nondecreasing for 1 G i G II under further assumption that (Xi,. . . , X,) is permutation symmetric. This is indeed the case even without added assumption. The reason is that as a function of (X,,...,XJ, h(X,,...,
X,)
=4(X,
)...)
xi, 0 )...) 0)
is also convex and symmetric. Replacing 4 by h and exchanging the roles of X, and X, in our proof, one obtains C(i) a C(i - 1) in terms of (7). This completes our observation.
3. Applications Example model
q
1993
if for all 1 G i < k, either Xi given Yi is symmetric or f. given Xi is symmetric. This general form is useful when neither Xj given k;; nor <. given X, is symmetric for some k G j Q II.
C(i)
(2) holds trivially if E[4(X)l = W. Therefore, we are going to assume that E[4(X)I < cc. Also observe that without loss of generality, we can assume that i = 1 in (3). Since 4 is strictly convex, on the event [Xi # 0, Yi # 61, inequality (6) holds strictly. By this, (3) and (6) imply that
15 October
LETTERS
3.1.
Consider
Z, = (u +be-CZ~-l)Z,_l where E,‘S are i.i.d random mon density g(x)
= const:
the
exponential
AR(l)
+E,
variables,
with a com-
ePixtp
with p 2 1. Denote the true parameter by 6, = Z, to be car,, b,, cc,) and assume the process 193
Volume
18, Number
3
STATISTICS
& PROBABILITY
stationary. (See Tweedie (1975) for a condition that ensures the uniqueness of a stationary solution.) Then, the stationary distribution rr of 2, equals F * G, where G is the distribution of E, and F is the distribution of (a, + b,e-“oz$Z,. Hence r has a density F * g(y)
= /g(_v -x)
dF(x)
which is positive for all y. We can also show that E( I Z, 1’) < 03 (Tweedie, 1983 and Guo, 1989). For the consistency property of the maximum likelihood type estimator of fi = (a, 6, c>, the function Q(s) defined by E[IZ,-(a-bepczf)Z,~]
II
=E[ I( a, + b,e-“OZf - a - be-“‘f ) Z, + Ed p
is usually assumed to have a unique minimum point at 6,. Let
42-‘[4(X,> X,) + 4(-X,, =l-P((x*+x,)(x,-X,)>O)>O
and IX,+X*IP.
Note that 4 is convex and is strictly convex when p > 1. Obviously X, given Xi is symmetric. Since g and r have positive density, X,#O)
=P(X,
#O)
= P( aa + b,e -cd? + a + bepcz?) > () for all 19+ 6,. Thus, for p > 1, by Theorem 2.1,
Q(+E(IX,IPV = Q(%J
194
X,)}
and Q(s) > Qfor all 6 # 6,. Example 3.2. Let Z,, . . . , Z, be random variables with EIIZil]
PIEi=l]=PIEi=
-l]=&
Assume that {EJ is independent of {ZJ. Define X, = E,Z, then Xi is a symmetrization of Zi and I X, I = 1Zi 1. It is not difficult to see that conditional on {Zj), Xi given y = {Xj: 1 Q j < IZ, j # i1 is symmetric. From Theorem 2.1, we then have max I Zi I < E ( 1Xi + *** +X, iI{ Zj}) . l
. .
E(imiyn X*)=
P(X,#O,
X*)1 >4(0,
lzil)QE(IX,+...+X~I).
This inequality is of special interest when Zi are independent and symmetric. In this case, we have
x, = &2
4(X,,
1993
for all 6 # 6,. For p = 1, by Remark 3 and the fact that
E(lylyn Xi = (a, + boe-QZf - a - be-czf)Z,,
15 October
LETTERS
IX~12)~~(lX~lP)
. .
IZiI)
... +ZJ).
References Guo, M. (19891, Inference
for nonlinear time series, Doctoral Dissertation, Univ. of Maryland (College Park, MD). Huber, P. (1967), The behavior of maximum likelihood estimates under nonstandard conditions, Proc. of the Fifth Berkeley Symp. on Math. Statist. and Probab., Vol. 1 (Univ. of California Press, Berkeley and Los Angeles, CA) pp. 221-233. Tweedie, R.L. (1975), Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space, Stochastic Process. Appl. 3, 385-403. Tweedie, R.L. (19831, The existence of moments for stationary Markov chains, J. Appl. Probab. 20, 191-196.
Letters
15 October
18 (1993) 191-194
1993
A lower bound for expectation of a convex functional Mei-Hui
Guo
National Sun Yat-Sen Uniuersity, Kaohsiung, Taiwan, ROC, and Worcester Polytechnic Institute, Worcester, MA, USA
Ching-Zong
Wei
Academia Sinica, Taipei, Taiwan, ROC Received April 1992 Revised January 1993
Abstract: Let 4 be a symmetric convex function from I%” to Iw. Under certain conditional symmetric conditions on the random variables Xi,. , X,,, the inequality:E[d(X,, . , X,)] > E[max,, iG &O,. . ,O, X,, 0,. ,O)] is derived. Conditions under which the strict inequality holds are also obtained. Application to nonlinear autoregressive models and symmetrization of random variables are given. Keywords: Consistency;
convexity;
symmetry;
stationarity.
1. Introduction Let Xl, x,,..., X,, be a sequence of random variables and C#Jbe a convex function from R” to R. In this paper, we investigate the conditions under which
where f. is a known function which depends on the unknown parameter 6 and E,‘S are i.i.d random variables with mean zero. To estimate 6, one may use the minimum contrast estimator 6n which is defined to be the minimizer of the penalty function fl-P
1
r=l
aE lyyn 4(0,...,0, Xi,O,...,O) . [.. The motivation of this inequality originated from the study of the estimation of a nonlinear autoregressive time series (Z,}:
Correspondence to: Mei-Hui Guo, Department of Applied Mathematics, National Sun Yat-Sen University, Kaohsiung, Taiwan 804, ROC, or Ching-Zong Wei, Institute of Statistical Science, Academia Sinica, Taipei 11529, Taiwan ROC. 0167-7152/93/$06.00
0 1993 - Elsevier
Science
Publishers
where the loss function p is chosen according to one’s interest. For example, P(Z, >...Y Zp+l; fi) = (Zp+,
-f&q
T..., z,,)‘,
if 6,, is the conditional least squares estimator. When (Z,) is ergodic with true parameter 6,, to ensure the almost sure convergence of I?,, to 6, (strong consistency), one common assumption is that
Q(o) =+(Z,,...,Z,+,;
B.V. All rights reserved
a)] 191
Volume
18, Number
3
STATISTICS
has a unique minimizer 1967). In the conditional set
& PROBABILITY
(Guo, 1989; and Huber, least squares case, if we
LETTERS
15 October
if and only if there exist 1 < i
(3)
X, =fil,,(zl,...,z,)-f~(zl,...,z,),
Remark 1. Since 4 is convex and symmetric,
Xz=a,+,
4(i)
and
All expectations well defined.
then this assumption E[4(X,,
X,)]
is equivalent
>E[4(0,
X,)]
to for all fi+fi,.
In Section 2, the main theorem is presented. In Section 3, two applications are given. One deals with the strong consistency of the maximum likelihood type estimator for an exponential autoregressive model. Another provides an upper bound for the expectation of the extreme order statistics through the absolute moment of the symmetrized partial sum.
=i[@)
+4(-i)]
a@).
considered
(4)
in Theorem
In the following, a function f< . ) from R” into R is said to be symmetric (about zero) if f(i) = f< -i_) for all _? E R”. A random vector 2 is said to be symmetric (about zero) if 2 and -.?! have the same distribution. Theorem 2.1. Let 4 be a symmetric convex function from R” into R and X,, . . . , X,, be random variables such that for all 1 < i < n, either X, given t.={X,:
X, and pi. If X,
+$(X1,
%)I%]
F,)lfi]
=-+$(-Xi, =E(+(X,,
I?)
and +(X,3
%)]
=E(:[4(X,,
2;1)
E[~(X,Y..?X,)l
J?)]).
+$(X1,
I?) +4(-x,,
J?)]
=t[~(X,>~J+~(X,, By the convexity 3[4(X,?
-Q]. of 4,
Pi) +4J( -X1, v 6(X,,
E[4(Xi,
I?)]
Q] fi),
the maximum
+(O,
(6) of u and v. In
pi) @(Xi,
=.+(X)]~ Furthermore, E[+(O ,...,
if 4 is strictly convex and 0, X,, 0 ,...,
0)]
E[4(Xw.,X,)]
192
l~lyn 4(0 ,... 20, X,, O,...,O)] 1 .-.
G)] (7)
where X=(X,,..., XJ and $(JZ)=+(O, 91)V +(x1, 6). It is not difficult to see that $ is symmetric and convex. Replacing (4, Xi, I!i) in (7) by ($,, X2, F2,>, we have
for 1 < i < n then
> E
(5)
If pi given X, is symmetric, then identity (5) can also be obtained by similar arguments. Now, since 4 is symmetric,
where u v v denotes view of (5) and (6),
is symmetric of q. given X, is symmetric. Then
Pi)] I q
+4(-X,,
> 4(o,Q
l
2.1 are
Proof of Theorem 2.1. Consider given fi is symmetric then
+4(-X,, 2. The main result
1993
E[$@)] (2)
aE[$(X,,
0, X,,...,X,)
vlcr(O, X,, O,...,O)]
(8)
Volume
18. Number
STATISTICS
3
& PROBABILITY
=E[cb(O, 0, x, >...,X,) V4(XlY6) v$J(O, x,7
o,...,0117
(9)
where the last identity is ensured by (4). In view of (7) and (91, inequality (1) is then a consequence of induction arguments. Now let us consider the case when 4 is strictly convex. If (3) is violated then with probability one, either X= 6 or there is a j such that Xj z 0 and < = 6. In both cases, = imfx_ $(O,. . .,o, xi, . .
#J(X)
0,. . . ,O).
Therefore, (2) is violated. Now assume that (31 is satisfied. Since E[$(O, . . . ,O, Xi, 0,. . . ,011 < ~0 for all i and max
+(O ,...,
lGi9n
=G i i=l
0, Xi,0 ,..., )...)
[4@
0,
xi,
0)
0 )...)
0)
-m]
+ 4(@>
E(+$(X,:
I?) +4(-X,, -4(O,
I?)]
Yi) v 4(X,,
O)> > 0.
(10)
By (5) and E[&J?)] < TV,(10) in turn implies that E[&!?)l > E[+(O, Yi,> V 4(X,, 611. However, from above arguments E[ +(0, >E
Yi) v 4(X,,
1. .
G)]
imzyn $(O,...,O,
Combining
these
Xi, O,...,O)
two inequalities,
1.
we obtain
Remark 2. Let Xi = (Xi, Xi+r, . . . , XJ. derived from the arguments above that E[@)]
>E[
lm:-k . .
+(O,...,O,
(2).
It can be
X,, O,...,O>
v@,...,o,
Remar_k 3. If one is only interested in showing E[+(X)] > E[ 4(0, Y,)] then from the proof above it is not difficult to see that we only have to assume that (9 4 is convex and symmetric; or Yi (ii) either Xi given Yi is symmetric given Xd is symmetric; (iii) E[@, Y,)l < rl”; Y,> + 4(-X,, IQ > (iv) PI2-‘[~(x,,
$No,IQ, > 0. Remark 4. A referee of using the arguments that the sequence =E[+(X,
Xk)]
,...,
pointed out the possibility given in the proof to show
Xi, 0 ,...,
0)]
is nondecreasing for 1 G i G II under further assumption that (Xi,. . . , X,) is permutation symmetric. This is indeed the case even without added assumption. The reason is that as a function of (X,,...,XJ, h(X,,...,
X,)
=4(X,
)...)
xi, 0 )...) 0)
is also convex and symmetric. Replacing 4 by h and exchanging the roles of X, and X, in our proof, one obtains C(i) a C(i - 1) in terms of (7). This completes our observation.
3. Applications Example model
q
1993
if for all 1 G i < k, either Xi given Yi is symmetric or f. given Xi is symmetric. This general form is useful when neither Xj given k;; nor <. given X, is symmetric for some k G j Q II.
C(i)
(2) holds trivially if E[4(X)l = W. Therefore, we are going to assume that E[4(X)I < cc. Also observe that without loss of generality, we can assume that i = 1 in (3). Since 4 is strictly convex, on the event [Xi # 0, Yi # 61, inequality (6) holds strictly. By this, (3) and (6) imply that
15 October
LETTERS
3.1.
Consider
Z, = (u +be-CZ~-l)Z,_l where E,‘S are i.i.d random mon density g(x)
= const:
the
exponential
AR(l)
+E,
variables,
with a com-
ePixtp
with p 2 1. Denote the true parameter by 6, = Z, to be car,, b,, cc,) and assume the process 193
Volume
18, Number
3
STATISTICS
& PROBABILITY
stationary. (See Tweedie (1975) for a condition that ensures the uniqueness of a stationary solution.) Then, the stationary distribution rr of 2, equals F * G, where G is the distribution of E, and F is the distribution of (a, + b,e-“oz$Z,. Hence r has a density F * g(y)
= /g(_v -x)
dF(x)
which is positive for all y. We can also show that E( I Z, 1’) < 03 (Tweedie, 1983 and Guo, 1989). For the consistency property of the maximum likelihood type estimator of fi = (a, 6, c>, the function Q(s) defined by E[IZ,-(a-bepczf)Z,~]
II
=E[ I( a, + b,e-“OZf - a - be-“‘f ) Z, + Ed p
is usually assumed to have a unique minimum point at 6,. Let
42-‘[4(X,> X,) + 4(-X,, =l-P((x*+x,)(x,-X,)>O)>O
and IX,+X*IP.
Note that 4 is convex and is strictly convex when p > 1. Obviously X, given Xi is symmetric. Since g and r have positive density, X,#O)
=P(X,
#O)
= P( aa + b,e -cd? + a + bepcz?) > () for all 19+ 6,. Thus, for p > 1, by Theorem 2.1,
Q(+E(IX,IPV = Q(%J
194
X,)}
and Q(s) > Q
PIEi=l]=PIEi=
-l]=&
Assume that {EJ is independent of {ZJ. Define X, = E,Z, then Xi is a symmetrization of Zi and I X, I = 1Zi 1. It is not difficult to see that conditional on {Zj), Xi given y = {Xj: 1 Q j < IZ, j # i1 is symmetric. From Theorem 2.1, we then have max I Zi I < E ( 1Xi + *** +X, iI{ Zj}) . l
. .
E(imiyn X*)=
P(X,#O,
X*)1 >4(0,
lzil)QE(IX,+...+X~I).
This inequality is of special interest when Zi are independent and symmetric. In this case, we have
x, = &2
4(X,,
1993
for all 6 # 6,. For p = 1, by Remark 3 and the fact that
E(lylyn Xi = (a, + boe-QZf - a - be-czf)Z,,
15 October
LETTERS
IX~12)~~(lX~lP)
. .
IZiI)
... +ZJ).
References Guo, M. (19891, Inference
for nonlinear time series, Doctoral Dissertation, Univ. of Maryland (College Park, MD). Huber, P. (1967), The behavior of maximum likelihood estimates under nonstandard conditions, Proc. of the Fifth Berkeley Symp. on Math. Statist. and Probab., Vol. 1 (Univ. of California Press, Berkeley and Los Angeles, CA) pp. 221-233. Tweedie, R.L. (1975), Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space, Stochastic Process. Appl. 3, 385-403. Tweedie, R.L. (19831, The existence of moments for stationary Markov chains, J. Appl. Probab. 20, 191-196.