On estimator efficiency in stochastic processes

Stochastic Processes and their Applications North-Holland Publishing Company

SHORT

93

15 (1983) 93-98

COMMUNICATION

ON ESTIMATOR EFFICIENCY IN STOCHASTIC PROCESSES Trevor

SWEETING

Department of Mathematics, University of Surrey, Guildford, Surrey GlJ2 5XH, England

Received

20 June 1981

It is shown, under mild regularity conditions on the random information matrix, that the maximum likelihood estimator is efficient in the sense of having asymptotically maximum probability of concentration about the true parameter value. In the case of a single parameter, the conditions are improvements of those used by Heyde (1978). The proof is based on the idea of maximum probability estimators introduced by Weiss and Wolfowitz (1967).

Class.: Primary 62M99; Maximum likelihood estimation inference from stochastic processes limiting probability of concentration

Secondary

62F20

1. Introduction There

has been much interest

recently

in large sample

inference

from stochastic

processes and in particular in the ‘nonergodic’ cases, where the random information matrix does not behave asymptotically like a constant (for example, branching processes, the pure birth process). With regard to estimating efficiency, Heyde [2] showed that, under certain regularity conditions, the maximum likelihood estimator (m.1.e.) of a single parameter in a stochastic process is efficient in the sense of having asymptotically maximum probability of concentration in symmetric intervals. The purpose of the present paper is two-fold: firstly, to demonstrate that this result holds under a set of conditions rather simpler than those in [2], and secondly, to treat the general multiparameter case, the role of symmetric intervals being taken by convex, symmetric sets. The only conditions which are employed are regularity conditions on the random information matrix; it was shown in [3] that this small set of conditions is sufficient to deduce the uniform asymptotic normality of the m.1.e. (without invoking any martingale central limit theory, for example). The proof is based on the idea of maximum probability estimators introduced by Weiss and Wolfowitz [5]. 0304-4149/83/0000-0000/$03.00

@ 1983 North-Holland

94

T. Sweefing / On estimator eficiency

2. Regularity

conditions

and statement

Let t be a discrete or continuous a family of probability distributions by 8 E 0, an open densityp,(o)

subset

of result

parameter and, for each value of f, let (Pk) be defined on a measurable space (n,, ~4~) indexed

of (Wk.We assume

w.r.t. a u-finite

measure

that, for each t and 8 E 0, Pk has a

A,, and that the second-order

of pr(8) exist and are continuous a.e. for all 19E 0. Let I,(0) = log ~~(0) and 9a,(f3) = -K’(0) be the random

partial derivatives

information

matrix,

where

f:‘(0) is the matrix of second-order derivatives. The symbol &will mean uniform convergence in distribution in compact subsets of 0 (ordinary uniform convergence for non-random quantities). Let r be the k X k matrix (0,, . . . , ok), Bi E 0, i = 1 . . >k, and define 9, (r) to be 4, with row i evaluated at 19,.The following conditions were used in [3]. Condition Cl. There satisfying {At(e)}4 W,(e) where

A,(B),

matrices

continuous

in 0,

~(At(e)}~‘~a,(e)[{A,(~)}-‘l= :Ww(N

C2. For all c > 0 supl{At(e)}~‘A,(e’)-I,1

where the supremum (ii)

square

W(0) > 0 a.s.

Condition (i)

exist nonrandom such that

0

$0

is taken

over the set I{A,(0)}‘(@‘-

fl)l c c,

and

supl~~,~~~~~‘~~a,~~~-~~~~~ll~~~~~~~-’lTl $0

in probability,

where

the supremum

is taken

over

the set I{At(0)}‘(O,

- f~)l s c,

l
It was proved in [3] that if Conditions m.1.e. is uniformly

asymptotically

yt(O) = M~)I=(~~

Cl and C2 hold, then the randomly

normal;

precisely,

-0).

Theorems 1 and 2 in [3] imply that, with probability a local maximum 6, of I,(O) satisfying (WWWWe),

W,(O))>

{ w(e)}P”2z.

tending

to one, there

exists

(1)

(Z, W(O))

where Z is a standard normal random vector approximate large-sample sampling distribution the result Yt(@) :

normed

define

in Rk, independent of & is therefore

of W(0). obtained

The from

(2)

We shall show that as t -*CD the m.1.e. has ‘maximum concentration’ about 0 amongst a class of ‘reasonable’ competing estimators. Let 92 be the class of sets in

95

T. Sweeting / On estimator efficienq

Rk which

are convex

and symmetric

about

the origin;

from

(2) and [3, Lemmas

1 (ii) and 31 it follows that for R E 68 P; (Y,(@)E R) &F,(R) where FO is the distribution

of {lV(19)}-“~2.

Next, define the class of estimators %? -19) converges uniformly in distribution,

by the property that (7’,) E %Yiff {A,(O)}‘(T, with limit LO, say. This is a reasonable class of competing estimators for large-sample properties, since uniform convergence is an essential if large-sample confidence regions based on T, are to be constructed. In the next section we prove the following result.

Theorem

for

to consider requirement

2.1. Let (T,) E %’ and suppose Conditions Cl and C2 hold. Then

all R E 9, 0 E 0.

Suppose E({ W(O)}-‘) < 00, so that the mean vector and covariance matrix of Fe are 0 and E{ W(O)}-’ respectively. Let M(Le) denote the asymptotic mean-square error matrix, l XXT dL,(x), of Le whenever it exists. We can easily deduce the following corollary. Corollary 2.2. If M(Le) exists then

is nonnegative definite. (Note that we make no assumption

about the form of L, here.)

We briefly relate our conditions and results in the case k = 1 to those in [2]. Firstly, Conditions Cl and C2 do not involve expectations (although {AI(O)}’ will often be E{4,(8)} in practice). In particular, the conditional information need not be introduced at this stage (cf. [2, Assumptions l(i), 2(ii)]). Our conditions are similar to [2, Assumptions l(i), 2(i), (iii)] on replacing {E,Jn(8)}“2 in [2] by A,(@), but the uniform nature of our Condition C2 implies (l), and so [2, Assumption l(ii)] can be removed. Conditions Cl and C2 will be simpler to check in practice and are, as already stated, precisely the conditions which will give rise to uniform asymptotic normality. We could have taken our class of competing estimators to be all estimators satisfying (3) in Section 3, but the class %? seems to be a more natural one to consider. Finally, the mean-square error efficiency property is not restricted to asymptotically normal estimators; when E{W(O)}-’ < 00, it is easily seen that Corollary 2.2 implies the final statement of the theorem in [2].

96

T. Sweeting / On estimator eficiency

3. Preliminaries

and proof of Theorem

2.1.

In the case k = 1, the simplest proof consists of checking that conditions A and for [4, Theorem 3.11 to apply, as was done in [23. (We do appear to need the distribution of I+‘(@) continuous in this derivation though.) The multi-

B are satisfied

parameter case for general convex sets (and general normalizing constants A,(8)) is, however, not covered by [4, Theorem 3.21, and instead we base the proof on the idea of maximum probability [5,6]. Rather than adapt the results probability estimator, we shall find Fix tiOE 0 and write A, = A,(@& set of points

satisfying

estimators introduced by Weiss and Wolfowitz in [5] and attempt to show that & is a maximum it easier to proceed directly. 9, = 9a,(00), W, = W,(fl,). Let h > 0 and H, be the

IAT(0 - 0,)l< h. Note that (Tt) E %?implies

P;(A~(~‘,-~)ER)-P&JA~(~‘-~&R)+O

(3)

uniformly in B E H, for all R E 92. (The distribution of {A,(B)}~(T, - ~9)is continuous in 0 for each t since A,(8) is continuous, and hence Lo is continuous in 0 from uniformity of convergence; (3) now follows from Condition C2(i)-c.f. [3, Lemma 31.) We state properties (4)-(6) below implied by Conditions Cl and C2; the details are straightforward and hence omitted. Let e > 0; then for all 0 E H, and t > to it follows from Condition C2 that P:,(sup~A,‘[4,(8’)-~a,]{A;‘}‘~~~/h*)<~ where

the supremum

is taken

(4)

over 8 E H,, and from Conditions

Cl and C2(i) and

(4) that P;(W,>o)>l-&. Finally,

there

(5)

exist M and a sequence

b, + 00 such that for all B satisfying

IAT(t9 -

0,,)l c b, and t > tl P~((A:(&-~‘)~>M)
(6)

Here the uniform stochastic boundedness Condition C2(i). We are now in a position Proof of Theorem be the volume

V,’

2.1. We first prove the result for bounded

41) and

R E 9. Let V, = jH, dt9

of H,. Note that, from (3),

Iim V;’ and similarly

of Y,(H) is used (see [3, Lemma to prove the theorem.

IH,

Pk(Af(T,-8)~R)d8=limPk,,(A~(T,-80)~R)

for (it). We shall show that, for sufficiently Pk(A:(T,

-0)

E R) de ce4’V;’

from which the result will follow, in view of (7).

(7) large h and t > t’,

P’,(A:(&@ER)d0+4F I H<

(8)

T. Sweeting

Let z be the radius

of a sphere

S, c 0, be the set of points

I wA&)l>

centered

97

eficiency

at the origin

which contains

R. Let

satisfying 5 F/h*, the supremum

s~p]A;‘[.9~(f3’)-9~]{A~‘}‘]

(i) (ii) (iii)

/ On estimator

being over O’EX,

0,

lAT(e^, - tY,)l s h -z.

Write B,(d) = H, n {AT(d - 0) E R}; the left-hand pr(O) de dA, + V,’

side of (8) is less than

P;(S,)

d6’ = Ii + 12

say. Consider first the quantity Ii. By Taylor expansion of l,(8) about I$ (which exists whenever x E S,) it is easily seen that, whenever 19E H,, x E S,, we have

where A:($

I&[ G 2~. But the integral

w.r.t. 0 of exp {-g(f3 - $,)T9,(r3 -k)}

over the set

- 0) E R is maximized

restricted that

at d = 8, (Anderson [l]), and hence the same integral to 8 E H, is also a maximum at d = ff,, since IAT(it - 130)ld h -2. It follows

I1 se”V;’

I H,

P;(A;(&@ER)d&

Consider next the quantity I*. Let _&be the set of points 8 satisfying jAT(e - eo)l s (1 -~)“~h; then it follows from (4), (5) and (6) that Pf(.!?,)<3~ whenever 8 EJ,. The argument here is identical to the proof of the theorem in [S] and details are omitted. Finally, VF1 J,,,, dt9 O be such that P~(I{A,(B)}T(7’, -O)l> K)
seen that L@(R) s F,(R)

holds for the bounded set R A {Ix 1G K}, it is easily the theorem holds for all R E 3.

+ E, and hence

Proof of Corollary 2.2. Let X -Lo, Y -Fe Then

and x E [Wk; let L*e,F”e be the distributions

of ]XTX12, ]XTY12 respectively.

Elx=X]* = i:; Y G&y)

2 and the corollary

= loa 11 -G(y)1

m{l-F;(p)}dy I0

dy

=E]xTY12

follows immediately.

Remark. Thee, em 2.1 gives an optimal asymptotic property of the sampling distribution of I!?~.A conditionality principle would, however, dictate the use of the

98

T. Sweeting / On estimator

asymptotic sampling distribution asymptotically like an ancillary

efficiency

of $t conditional on W,(e”,), since the latter behaves statistic for 13 under our conditions. It is probable

that e^, will be efficient also in this restricted space under suitable conditions, but this has not yet been explored; there are difficulties in even deducing that, conditional

on W,(C?~),the asymptotic

distribution

of iC is normal.

References [1] T.W. Anderson, The integral of a symmetric unimodal function, Proc. Amer. Math. Sot. 6 (1955) 170-176. [2] C.C. Heyde, On an optimal asymptotic property of the maximum likelihood estimator of a parameter from a stochastic process, Stochastic Process. Appl. 8 (1978) l-9. [3] T.J. Sweeting, Uniform asymptotic normality of the maximum likelihood estimator, Ann. Statist. 8 (1980) 1375-1381. [4] L. Weiss and J. Wolfowitz, Generalized maximum likelihood estimators, Theory Probab. Appl. 11 (1966) 58-81. [5] L. Weiss and J. Wolfowitz, Maximum probability estimators, Ann. Inst. Statist, Math. 19 (1967) 193-206. [6] L. Weiss and J. Wolfowitz, Maximum Probability Estimators and Related Topics (Springer, Berlin, 1974).