- Email: [email protected]
Higher-order asymptotic theory for discriminant analysis of Gaussian ARMA processes
Statistics
ELSEVIER
Higher-order
& Probability
Letters
20 (1994) 259-268
asymptotic theory for discriminant of Gaussian ARMA processes Guoqiang
Department
ofMathematical
Science.
analysis
Zhang Osaka University,
Received June 1993; Revised September
Toyonaka
560. Japan
1993
Abstract This paper deals with the higher-order asymptotic theory for discriminant analysis of Gaussian ARMA processes. Two discriminant statistics are given, then the higher-order approximations for the misclassification probabilities of them are derived. Using them, the higher-order difference of the misclassification probabilities is elucidated. Some numerical examples are also given.
Keywords:
ARMA processes;
Higher-order
asymptotic
distribution;
Discriminant
statistic; Misclassification
probability
1. Introduction In multivariate analysis, higher-order asymptotic investigations for discriminant analysis have been investigated in detail. For example, Okamoto (1963) gave an asymptotic expansion of the distribution of the linear discriminant function due to Anderson. However, in time-series analysis, there are few literatures in this field. KrzySko (1983) investigated the asymptotic distribution of a discriminant function for multivariate AR processes by Bayes method. Wakaki and Oh (1986) studied the asymptotic distribution of a linear discriminant function for AR(l) processes and gave an approximation for the distribution up to order n- ‘. In this paper, we develop the higher-order asymptotic theory for discriminant analysis in time series. For this, we often use the higher-order asymptotic theory and techniques in time-series analysis (see Taniguchi, 1991). Let{X,,t=O,fl, 22 ,... > be a Gaussian ARMA process with zero mean and spectral densities j(n) O,), f(llO,) under different populations 7c1 and TC~,wheretJiEOciW’.LetX,=[X,,X,,...,X,]’beasample from one of the two populations x1 and 7~~.In order to decide the source population of X,, it is known that the statistic
zcfl,f2) =
&
(1)
which is an approximation of the n-l log-likelihood ratio(n -I LR), can be used as a discriminant criterion (see Shumway, 1982), where Z,(n) = (1/2nn)lC:= 1 X,eiAt12 is the periodogram of X,. Zhang and Taniguchi 0167-7152/94/$7.00 0 1994 Elsevier Science B.V. All rights reserved SSDI 0167-7152(93)E0178-V
260
G. Zhang 1 Statistics & Probability Letters 20 (1994) 259-268
(1994) proved the asymptotic normality of 1(fi&), and gave the asymptotic probability of misclassification. However in the real problem it is often difficult to get a sufficiently large sample. If the sample size is not large, the asymptotic (limiting) distribution does not often work well, which motivates the higher-order asymptotic analysis. First, for a Gaussian ARMA process, we give the asymptotic expansion of the misclassification probability of Z(fi,f2) up to order n- ‘/’ b y the Edgeworth expansion method under the conditions
(2) x2:
.ftAle,)
=/(wo
+
2)
where t30E 0 c Iw‘, and x is a constant. Second, for an AR(l) process, we give the asymptotic expansion of the misclassification probabilities of an estimated version of Z(fi,f2) up to order n-i. Finally, we illustrate some simulation results for the ARMA(1,l) model.
2. Second-order asymptotic
that {X,} 1s a Gaussian
Suppose 5
approximation
fijX,_j
= i
j=O
of misclassification
ARMA(p,q)
process
probabilities for Gaussian ARMA processes
generated
by
with CI,,= p0 = 1, c(~ # 0, /?, # 0,
CljEf-j,
(3)
j=O
where c4=o/?jzj # 0, Cy=occjzj # 0, on 1zI d 1, and E, are i.i.d. N(0,a2). Let us introduce the following class of functions on [ - rr, rt]; f:f(A)
=
f
a(u)em
cc(U)= a(-u),
u=-02
It is known
f
lullcx(u)l < cc .
(4)
Il=-CC
that the spectral
density
of ARMA(p,q)
process
(5) We set down the following
assumptions.
Assumption 2.1. The spectral density f(nl0) is continuously four times differentiable with respect to 8, and the first, second and third derivatives (8/6V3)[f(n(0)], (a2/%P)[f(n10)], (a3/af13)[f(n10)] belong to D, the fourth derivative is bounded on C-x, rc]. Assumption
2.2. For Q1 # Q2, f(nl6,)
Assumption
2.3. There exists d > 0 such that
z(e)
=-&
ww CfUl WI
#f(nle,),
2d2
2 d
on a set of positive
Lebesgue
measure.
(6)
for all 0 d 0.
f(w)
Henceforth, we denote the jth cumulants of [ .] by Cumj[.]. The following calculation of the asymptotic cumulants of LR evaluated by Taniguchi (1991).
lemma
is parallel
to the
G. Zhang J Statistics
Lemma 1. Assume that Assumptions Cum,
& Probability
Letters
2.1-2.3 hold. Then, under hypothesis
Cn~(fi,fi)l = c I(Qd + - x3 [3J(&) + K(d,)] 6&
Cum,[nZ(fl,fi)]
= x21(&)
+ ?.Z(S,) &
Cum3[nZ(fi,f2)]
= - x” &
Cum,[nZ(f,,f2)]
= 0(n(-Ji2)+1)
261
20 (1994) 259-268
x1, we have
+ !??-(!d + O(n_‘), A
+ O(n-‘),
K(6),) + O(n-I), for all J > 4,
where
We use the discriminant rule: (i) if Z(fi,fi) > 0, assign X, to rci; (ii) if Z(fi ,f2) < 0, assign X,, to 7r2. We denote the misclassification probability of misclassifying X, from rci into x2 by Pr(211). Then, we get Theorem 1. We put the proofs of theorems, etc. in the Appendix if they are not straightforward. Theorem 1. Assume that Assumptions
2.1-2.3 hold. Then under condition (2), we have
+wb) 1+O(n-‘), 1
+ fqe,) where Q(z) = J’,4(t)dt,
4(t) = (l/fi)exp(-t2/2),
which is equivalent
lim P1(1j2)=@ “-02
P(112) are different
up to higher
order
,
-!$a
term.
(8)
)
(
to the result by Zhang
and Taniguchi
It is well known that LR is an optimal discriminant misclassification loss. For the Gaussian ARMA process,
.qe,) where PLR( .) is the misclassification
(7)
O(n-‘),
and v = -(lxl/2)m.
Remark. The misclassification probabilities P(211) and However, when n tends to infinite, (7) becomes lim P,(211)= n-02
+
probability
+ f qe,) of LR.
(1994). In the following,
we only consider
P(211).
statistic in the sense of the minimization of the using the results of Taniguchi (1991), we have
1K(80) 6 wo) 1+
+ - -
O(n-‘),
(9)
262
G. Zhang / Statistics & Probability
Theorem 2. Assume that Assumption I’(fiLL)
= r(fIJ2)
Example. Let {X,} be a Gaussian spectral density
where only parameter
I(@,)=
2.1-2.3 hold. Then under condition (2), the misclassification
n J n are equioalent
- I~lW~,)/
ARMA(1,l)
0
J(Oo)
=
-
to PLR(21 1) up to O(n
process satisfying
condition
280
(1 _
2 and Assumption
680
@?)’
K(eO)
= (1 _
pobabilities
of
‘).
and 0 < 1ccl, 101< 1, c( # 8. In this case, we can directly
8 is unknown,
j&p,
Letters 20 (1994) 259-268
2.1-2.3 with
get
(e, - cr)(l - 280c( + a”)
Of)“’
we,)
= (1 -
@$)(I
-
Q,x)(l
-
a”)’
Thus
P1(2( 1) = P,(112) = @(a) -
‘x”(u)
[Gl(eo)
XJqq
+ G2(Q0)] + O(n-‘),
P,+(211) = P,+(112) = PLR(211) = PLR(112) = (a(u) -
1x1@(u)
Gr(8,)
XJm
+ G(n-‘),
where
x280 GI(Qo>x) = 00 + 4(1 _ e;)>
3. Third-order asymptotic
G2(e0)=
approximation
(e, - cC)(l - 2ae, + c?)
(1 _ Q,a)(l _ a’)
of misclassification
’
1x1 u= -2&q’
probabilities for Gaussian AR(l)
processes
In this section, we shall develop the third-order asymptotic theory for discriminant analysis of the process {X,}. To avoid unnecessarily complex formulae, we restrict ourselves to the situation where the process is a Gaussian AR(l) process with spectral densities f(nlfZ,) and f(nl0,) under hypotheses rrl and 7t2, where le,I < 1, lo2 I < 1, and 19~# Oz. As a discriminant statistic, we propose a new estimated version of Z(fi,f2);
~tdfiJ2)
= &
,y S{ n
f(nle2)
logf(Alel)
+.me
^
1 [ .ml~z)
1
f(4QI)
11
(10)
dA,
where
(g=
c:=2 x,x,c:i: x: + c1x: + czx; 1
with cl > 0, c2 > 0 constants. Here e is proposed by Ochi (1983), and includes special cases. We investigate the higher-order asymptotics of Za(fi,f2). Initially, we give the stochastic expansion of Ze(fi ,f2).
(11) various
famous
statisic
as
G. Zhang / Statistics & Probability Letters 20 (1994j 259-268
263
Theorem 3. Under condition (2), we have
for both x1 and n,, where L&, = ,,&(8 - 6,). Using Theorem 2.3.2 of Brillinger (1981), we can prove the following lemma. This lemma enables one to evaluate the asymptotic cumulants of Z,(fi ,f2). Lemma 2. Suppose that the cumulants of a random variable Q are evaluated as follows:
Cum, [Q] = an- i/2 + o(n- ‘),
Cum,[Q]
= 1 + bn-’ + o(n-‘),
Cum,[O]
= ~n-“~ + o(n-‘),
Cum,[fZ]
= dn-’ + o(n-‘),
CumJ[Q]
= O(n-(J’2)+1)
for all J > 5.
Set 9 = A0 + AIQ + (l/$)A,S2’
+ (l/n)A,Q3 the asymptotic cumulants of rj are evaluated as
+ op(tzW1),where A,, . . . , A3 are nonrandom constants. Then
Cum,[e]
= A0 + [aA, + A2]n-1/2 + o(n-‘),
Cum,[fi]
= A: + [4aA1A2 + bA: + 2cA1Az + 6A,A3 + 2A$]n-’
Cum,[e]
= [CA: + 6AfA2]n-112
Cum,[fi]
= [dA’: + 24cA:A2 + 24A:A3 + 48A:A$]n-’
Cum,[fj]
= 0(n-1J’2)+ ‘) for all f 3 5.
+ o(n-I),
+ o(n-‘), + o(n-‘),
It follows from Ochi (1983) that the standardized version of his estimator a = &(6 the following asymptotic cumulants: Cum&U
= -J&(2
Cum3[Ql
= - d>i
Cum,[Q]
= 0(n-(J’21+‘)
+ c)n-“2 f o(n-‘),
n-l/’ + o(n-i),
g(&c) Cum2[SZ] = 1 + =n
Cum,[Q]
=
- 0,)/J=
has
+
(13)
-1
6(--l + 110:) _ n 1 + o(n-l), I - eg
for all J > 5,
Cum1
Under condition (2), we have X2
X&C
tWfiJi)l = 2c1 _ eij + -_a-112 1_ sg
(14) (15)
where g(0,c) = 98’ - 1 - 402(c, - 1)(c2 - 1) f 2c(d2 f 02c - 1) and c = cl + c2 - 1. From Theorem 3, Lemma 2 and (13)-(15), we have the following proposition. Proposition.
O(U-I),
+
x2[1 - 0; - 2tgc] _ n ’ + o(n-I), 2(1 - @)”
264
G. Zhang / Statistics
Cum2Cn~dfi,f2)1 =
=
Cum4CWflJ2)l
=
Letters 20 (1994) 259-268
& - ,12z33, nel” 0 982
Cum3C~Mf~J2)l
& Probability
+
[ (1 -
-
c1 _
+ x2(6 - 14e; - seic) + x2g(eo,c) ei)3
(1 - e:)’
6x3eo
eij2 n-l/’
6x4(3 + 78:) c1 _ eij3
n
CumJ[nZ6(fl,f2)] = O(n-(J’2)+1)
_
+
1
n -l + o(n_I),
3x4(1 + se;) _ c1 _ eg,3
n
’ + oW’),
’ + o(n-‘I,
for all J > 5.
Thus p
n~ii(fiJ2)
-
2
XJrce,,
[
e; 2~1 =
WV)
-
+(‘)
_
1 xe;
2 0;)
n
+
(1
+
31
i
_
3 - 13e; - 4e:c &I,
+
[
41 - a7 + 24)
+
282
~4 n
[-&%$j+
4(1 - e;)
1 ~3
Y
1 [ - J;;;1:e:) 19e; + 2e;c2 - 4e;c + 2g(eo,c) x2e2 v+$(:y:ijI +o(n-‘) 4(1 - e;) - (1 - e& )I v2
+
2(1 - e$)3’2n
+1 3 n( under 7r1. Theorem
4. Under condition (2), we have
(c + l)eo +
P,.(f f ,(2/ 1) = O(v) - !A gqv) B 1>2 X
i[
Jm
x2eo
4(1 - ep
1
‘-“’
x(3 - i9e; + 2eic2 - 4eic + 2g(eo,c) + ~~(1 + e: - 4eic)
8(1 - efp
x5e2 - 64(1 - &7/Z We next give the third-order ratio, which is optimal. From Cum, [LR] = G &
Cum2[LR]
= &
;
-
1-’1+ n
o(n-‘).
(16)
asymptotic expansion for the misclassification Taniguchi (1991), we can get +
0
32(1 - 8;)5’2
probabilities
x2(38; - 1) n- 1 + o(n-‘) 2(1 - e;)2
2x3eo (1 - ey
n- 112
1
x2(3e; - 1)
(1 - e:)”
n-l + o(n-‘),
by the likelihood
G. Zhang / Statistics
& Probability
Letters 20 (1994) 259-268
265
6x3&, -l,z + 6x4(1 + 20;) ,-1 + o(n_‘), = - (I _ eg,Z n (1 - ey
Cnm&RI
6x4(78; + 3) _
Cum&RI = t1_ ezj3 n ’ + o(n-‘), Cum,[LR]
=-I
O(n-(Jiz)+l)
for all J 2 5.
Thus
= Q(v) - cb(v)
1 ei n xe;v4 2c1 _ 0;) "'+ (1 - 8;4)3’2JL +
5 l?i --d&+(1 _@;)Wn "+ -,/;;;A+; ~~~~~~2)+ ;;--@;; ' I[ )I x(1 - 486)
+
J--$&+2;l’“B,;‘!.} +OW’).
(17)
Summarizing the above, we have the foflowing theorem. Theorem 5. Under condition (2), we have the following:
(i) PLR(2[1) = a>(v)- IxI Cp(v)
J&~
+
+
x(5@ X
4(1
“f&W
1
n-“2
1) 982 x3(1 + e;, 8(1 - 6$)3’2 + 32(1 - 8;)5’2 - 64(1 - &7!2
(ii) p1,(f,,f2)(41) - fM211) = - ; +
+
o(n-‘).
$(v)
x(40; + 66;~’ + 88$(c, - 1)(c2 - 1) - 4~) X(1 - 0;)“” [
x3 e;c - 8(1 - 8;)5’2
n-l + II
o(n-‘).
(iii) Zfcl + c2 = 1, then ?J
pI,(~I,fJ21 1) - PLR(211) = 4(v) lxle ;l-~;:&-
cl)1 n-1 + o(n-‘). 0
Here, the term of order n - 1 1s . positive, whence, the statistic LR is better than Ih(fI ,f2). The minimum of the term of order n - ’ is attained at cl = c2 = 4.
266
G. Zhang / Statistics & Probability Letters 20 (1994) 259-268
4. Simulation In this section, x, = 8X,_,
we give a numerical
study for the Gaussian
(1,1) process
+ E, - IX&,_1,
(18)
where E, - i.i.d.N(O, l), to verify the results of Section
I(fl J2) =
ARMA
241:
@_‘) ,;yg
2’
x,x,+,,,[(e;
2. For this model, it can be shown that
- e:)cP + (e, - e2)(cF1’ + cd”“)]
(n 1) f=l
(19)
and
Z’(flJ2) = ~(flLf2) -
(e, - a)(1 - 2aBo + a2) Ix’ ?zJtl(l - a”) (1 - Q3(1 - a&)
(20)
under condition (2). We calculated Z(fi,f2) and Z’(fi,f2) for n = 100, and iterated this procedure 1000 times for 80 = 0.1, . ..) 0.9, CC= 0.55, and x = 0.6, x = - 0.6. Then we get Table 1. From this table, we can see that (1) the simulation results of discriminat statistics Z(fi,f2) and Z’(fi,f2) are both close to the theoretical detection probabilities of LR; (2) the MSE of Z’(f1,f2) are smaller than those of Z(fi,fi), so Z’(fi,f2) is better than Z(fi,f2) in the sense of minimizing the misclassification loss. That is, the simulation results agree with the theoretical ones given in Theorems 1 and 2. Conclusion. For the discriminant problems of the Gaussian ARMA processes with small samples, we can use an approximation of the nP1 LR Z(fi,f2) as a classification criterion. The misclassification probabilities of this criterion under local condition can be evaluated with explicit expression up to O(K”~). We can also get a modified version of Z(fi ,f2) which minimizes the misclassification loss. The theorems established in Section 2 can be expended to order O(n- ‘).
Table 1 Comparison of the simulated detection theoretical detection probabilities
probabilities
of I(f: g) > 0 and l’(fi,f2)
> 0 with the
SDP ofI(.f-i,fz) > 0
SDP of I+(fi&)
x = 0.6
x = -0.6
x = 0.6
x = -0.6
x = 0.6
x = -0.6
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.614 0.701 0.685 0.658 0.682 0.650 0.646 0.647 0.677
0.659 0.631 0.630 0.642 0.684 0.620 0.68 1 0.678 0.634
0.563 0.673 0.681 0.665 0.67 1 0.638 0.638 0.638 0.698
0.660 0.642 0.652 0.606 0.675 0.630 0.701 0.670 0.659
0.6188 0.6195 0.6246 0.6265 0.6381 0.6422 0.6691 0.6797 0.7851
0.6181 0.6210 0.6222 0.6301 0.6329 0.6501 0.6565 0.7032 0.7236
MSE
0.0070
0.0060
0.0040
0.0037
> 0
TDPofLR
>0
Note: (a) SDP = simulated detection probability, TDP = theoretical detection probability; (b) TDP = 1 - PLR(l 12);(c) MSE = variance[(TDP - SDP)/TDP] + {Mean[(TDP - SDP)/TDP]}‘.
267
G. Zhang / Statistics & Probability Letters 20 j1994) 259.-268
Appendix
< 01, wh ere P,, [ ‘1 stands for the probability of [ .] under hypothesis 7Ei.Applying a general Edgeworth expansion formula (Taniguchi, 1991, p. 15) to Lemma 1, we have Proof of Theorem 1. Note that P,(2/ 1) = P,,[l(fi,fi)
x J(@o)
1
KtQo)[v” +2,:nji8o)D6JI;Wo)J%G Setting v = - (Ix j/2)m
I]
I
+ O(n_‘).
in the above equation, we have
p7c,c~(flJ2)< 01= 1”,(211) = i Px,[l(fI,f,)
for x >O
> 0] = 1 - P1(211) for x < 0,
which proves (7) for P,(Z/ 1). On the other hand, PI(l 12) = 1 - P,, [Z(fr ,fi) d 01, Under hypothesis 7r2, using Lemma 1 and the relation I(&) = Z(4)) + -z- [2J(B,) + K(@,)] + O(n - ‘), 4 we
have Cumr[~$!$!$]=
-~Jij&j-$[3J(eeJI!?$(eo)]--j$&+O(n~1),
Cum,[yx]=
1 +~J(Hy:~o~(eo)+O(n-l),
forall J&4.
Cum,[:lli~]=O(n-““~+‘) Similarly, we obtain
= 1 - Q(v) +
4(v) Jm
1
+ Otn-‘)
268
G. Zhang / Statistics
1 - P,(112)
p
n~(.f1?.A)
a similar
Letters 20 (1994) 259-268
for x > 0, for x < 0.
= i P1012) Proof of Theorem 2. Using
& Probability
argument
0
as in the above, under
7c1, we can get
B(b)
+ O(n_1).
xJ@x,j;l~
1
Thus P I(f,,S*) + (211) = @i(u) - E --!!%L Qac) = PLR(211) + O(C’).
;
J(0,)
+ ;
1 K(b) K(B,) + - ~ 6 I(b)
1+O(n-‘)
17
Acknowledgements The author, would like to thank the referee, Professors helpful suggestions and encouragement.
K. Isii, N. Inagaki
and M. Taniguchi
for their
References Brillinger, D.R. (1981), Time Series: Data Analysis and Theory, Expended Edition (San Francisco, CA, Holden day). KrzyBko, M. (1983), Asymptotic distribution of the discriminant function, Statist. Prob. Lett. I, 243-250. Ochi, Y. (1983), Asymptotic expansions for the distribution of an estimator in the first-order autoregressive process, J. Time Ser. Anal. 4, 51-67. Okamoto, M. (1963), An asymptotic expansion for the distribution of the linear discriminant function, Ann. Math. Statist. 34, 1286-1301. Shumway, R.H. (1982), Discriminant analysis for time series, in: P.R. Krishnaiah and L.N Kanal, eds., Handbook of Statistics. Vol. 2 (North-Holland, Amsterdam) pp. l-46. Taniguchi, M. (1991), Higher Order Asymptotic Theory for Time Series Analysis, Lecture Notes in Statistics 68 (Springer, Berlin). Wakaki, H. and Y.S. Oh (1986), Asymptotic expansion for the distribution of the discriminant function in the first order autoregressive processes, Hiroshima Math. J. 16, 625-629. Zhang, G.Q. and M. Taniguchi (1994), Discriminant analysis for stationary vector time series, J. Time Ser. Anal. 15, 117- 126.
ELSEVIER
Higher-order
& Probability
Letters
20 (1994) 259-268
asymptotic theory for discriminant of Gaussian ARMA processes Guoqiang
Department
ofMathematical
Science.
analysis
Zhang Osaka University,
Received June 1993; Revised September
Toyonaka
560. Japan
1993
Abstract This paper deals with the higher-order asymptotic theory for discriminant analysis of Gaussian ARMA processes. Two discriminant statistics are given, then the higher-order approximations for the misclassification probabilities of them are derived. Using them, the higher-order difference of the misclassification probabilities is elucidated. Some numerical examples are also given.
Keywords:
ARMA processes;
Higher-order
asymptotic
distribution;
Discriminant
statistic; Misclassification
probability
1. Introduction In multivariate analysis, higher-order asymptotic investigations for discriminant analysis have been investigated in detail. For example, Okamoto (1963) gave an asymptotic expansion of the distribution of the linear discriminant function due to Anderson. However, in time-series analysis, there are few literatures in this field. KrzySko (1983) investigated the asymptotic distribution of a discriminant function for multivariate AR processes by Bayes method. Wakaki and Oh (1986) studied the asymptotic distribution of a linear discriminant function for AR(l) processes and gave an approximation for the distribution up to order n- ‘. In this paper, we develop the higher-order asymptotic theory for discriminant analysis in time series. For this, we often use the higher-order asymptotic theory and techniques in time-series analysis (see Taniguchi, 1991). Let{X,,t=O,fl, 22 ,... > be a Gaussian ARMA process with zero mean and spectral densities j(n) O,), f(llO,) under different populations 7c1 and TC~,wheretJiEOciW’.LetX,=[X,,X,,...,X,]’beasample from one of the two populations x1 and 7~~.In order to decide the source population of X,, it is known that the statistic
zcfl,f2) =
&
(1)
which is an approximation of the n-l log-likelihood ratio(n -I LR), can be used as a discriminant criterion (see Shumway, 1982), where Z,(n) = (1/2nn)lC:= 1 X,eiAt12 is the periodogram of X,. Zhang and Taniguchi 0167-7152/94/$7.00 0 1994 Elsevier Science B.V. All rights reserved SSDI 0167-7152(93)E0178-V
260
G. Zhang 1 Statistics & Probability Letters 20 (1994) 259-268
(1994) proved the asymptotic normality of 1(fi&), and gave the asymptotic probability of misclassification. However in the real problem it is often difficult to get a sufficiently large sample. If the sample size is not large, the asymptotic (limiting) distribution does not often work well, which motivates the higher-order asymptotic analysis. First, for a Gaussian ARMA process, we give the asymptotic expansion of the misclassification probability of Z(fi,f2) up to order n- ‘/’ b y the Edgeworth expansion method under the conditions
(2) x2:
.ftAle,)
=/(wo
+
2)
where t30E 0 c Iw‘, and x is a constant. Second, for an AR(l) process, we give the asymptotic expansion of the misclassification probabilities of an estimated version of Z(fi,f2) up to order n-i. Finally, we illustrate some simulation results for the ARMA(1,l) model.
2. Second-order asymptotic
that {X,} 1s a Gaussian
Suppose 5
approximation
fijX,_j
= i
j=O
of misclassification
ARMA(p,q)
process
probabilities for Gaussian ARMA processes
generated
by
with CI,,= p0 = 1, c(~ # 0, /?, # 0,
CljEf-j,
(3)
j=O
where c4=o/?jzj # 0, Cy=occjzj # 0, on 1zI d 1, and E, are i.i.d. N(0,a2). Let us introduce the following class of functions on [ - rr, rt]; f:f(A)
=
f
a(u)em
cc(U)= a(-u),
u=-02
It is known
f
lullcx(u)l < cc .
(4)
Il=-CC
that the spectral
density
of ARMA(p,q)
process
(5) We set down the following
assumptions.
Assumption 2.1. The spectral density f(nl0) is continuously four times differentiable with respect to 8, and the first, second and third derivatives (8/6V3)[f(n(0)], (a2/%P)[f(n10)], (a3/af13)[f(n10)] belong to D, the fourth derivative is bounded on C-x, rc]. Assumption
2.2. For Q1 # Q2, f(nl6,)
Assumption
2.3. There exists d > 0 such that
z(e)
=-&
ww CfUl WI
#f(nle,),
2d2
2 d
on a set of positive
Lebesgue
measure.
(6)
for all 0 d 0.
f(w)
Henceforth, we denote the jth cumulants of [ .] by Cumj[.]. The following calculation of the asymptotic cumulants of LR evaluated by Taniguchi (1991).
lemma
is parallel
to the
G. Zhang J Statistics
Lemma 1. Assume that Assumptions Cum,
& Probability
Letters
2.1-2.3 hold. Then, under hypothesis
Cn~(fi,fi)l = c I(Qd + - x3 [3J(&) + K(d,)] 6&
Cum,[nZ(fl,fi)]
= x21(&)
+ ?.Z(S,) &
Cum3[nZ(fi,f2)]
= - x” &
Cum,[nZ(f,,f2)]
= 0(n(-Ji2)+1)
261
20 (1994) 259-268
x1, we have
+ !??-(!d + O(n_‘), A
+ O(n-‘),
K(6),) + O(n-I), for all J > 4,
where
We use the discriminant rule: (i) if Z(fi,fi) > 0, assign X, to rci; (ii) if Z(fi ,f2) < 0, assign X,, to 7r2. We denote the misclassification probability of misclassifying X, from rci into x2 by Pr(211). Then, we get Theorem 1. We put the proofs of theorems, etc. in the Appendix if they are not straightforward. Theorem 1. Assume that Assumptions
2.1-2.3 hold. Then under condition (2), we have
+wb) 1+O(n-‘), 1
+ fqe,) where Q(z) = J’,4(t)dt,
4(t) = (l/fi)exp(-t2/2),
which is equivalent
lim P1(1j2)=@ “-02
P(112) are different
up to higher
order
,
-!$a
term.
(8)
)
(
to the result by Zhang
and Taniguchi
It is well known that LR is an optimal discriminant misclassification loss. For the Gaussian ARMA process,
.qe,) where PLR( .) is the misclassification
(7)
O(n-‘),
and v = -(lxl/2)m.
Remark. The misclassification probabilities P(211) and However, when n tends to infinite, (7) becomes lim P,(211)= n-02
+
probability
+ f qe,) of LR.
(1994). In the following,
we only consider
P(211).
statistic in the sense of the minimization of the using the results of Taniguchi (1991), we have
1K(80) 6 wo) 1+
+ - -
O(n-‘),
(9)
262
G. Zhang / Statistics & Probability
Theorem 2. Assume that Assumption I’(fiLL)
= r(fIJ2)
Example. Let {X,} be a Gaussian spectral density
where only parameter
I(@,)=
2.1-2.3 hold. Then under condition (2), the misclassification
n J n are equioalent
- I~lW~,)/
ARMA(1,l)
0
J(Oo)
=
-
to PLR(21 1) up to O(n
process satisfying
condition
280
(1 _
2 and Assumption
680
@?)’
K(eO)
= (1 _
pobabilities
of
‘).
and 0 < 1ccl, 101< 1, c( # 8. In this case, we can directly
8 is unknown,
j&p,
Letters 20 (1994) 259-268
2.1-2.3 with
get
(e, - cr)(l - 280c( + a”)
Of)“’
we,)
= (1 -
@$)(I
-
Q,x)(l
-
a”)’
Thus
P1(2( 1) = P,(112) = @(a) -
‘x”(u)
[Gl(eo)
XJqq
+ G2(Q0)] + O(n-‘),
P,+(211) = P,+(112) = PLR(211) = PLR(112) = (a(u) -
1x1@(u)
Gr(8,)
XJm
+ G(n-‘),
where
x280 GI(Qo>x) = 00 + 4(1 _ e;)>
3. Third-order asymptotic
G2(e0)=
approximation
(e, - cC)(l - 2ae, + c?)
(1 _ Q,a)(l _ a’)
of misclassification
’
1x1 u= -2&q’
probabilities for Gaussian AR(l)
processes
In this section, we shall develop the third-order asymptotic theory for discriminant analysis of the process {X,}. To avoid unnecessarily complex formulae, we restrict ourselves to the situation where the process is a Gaussian AR(l) process with spectral densities f(nlfZ,) and f(nl0,) under hypotheses rrl and 7t2, where le,I < 1, lo2 I < 1, and 19~# Oz. As a discriminant statistic, we propose a new estimated version of Z(fi,f2);
~tdfiJ2)
= &
,y S{ n
f(nle2)
logf(Alel)
+.me
^
1 [ .ml~z)
1
f(4QI)
11
(10)
dA,
where
(g=
c:=2 x,x,c:i: x: + c1x: + czx; 1
with cl > 0, c2 > 0 constants. Here e is proposed by Ochi (1983), and includes special cases. We investigate the higher-order asymptotics of Za(fi,f2). Initially, we give the stochastic expansion of Ze(fi ,f2).
(11) various
famous
statisic
as
G. Zhang / Statistics & Probability Letters 20 (1994j 259-268
263
Theorem 3. Under condition (2), we have
for both x1 and n,, where L&, = ,,&(8 - 6,). Using Theorem 2.3.2 of Brillinger (1981), we can prove the following lemma. This lemma enables one to evaluate the asymptotic cumulants of Z,(fi ,f2). Lemma 2. Suppose that the cumulants of a random variable Q are evaluated as follows:
Cum, [Q] = an- i/2 + o(n- ‘),
Cum,[Q]
= 1 + bn-’ + o(n-‘),
Cum,[O]
= ~n-“~ + o(n-‘),
Cum,[fZ]
= dn-’ + o(n-‘),
CumJ[Q]
= O(n-(J’2)+1)
for all J > 5.
Set 9 = A0 + AIQ + (l/$)A,S2’
+ (l/n)A,Q3 the asymptotic cumulants of rj are evaluated as
+ op(tzW1),where A,, . . . , A3 are nonrandom constants. Then
Cum,[e]
= A0 + [aA, + A2]n-1/2 + o(n-‘),
Cum,[fi]
= A: + [4aA1A2 + bA: + 2cA1Az + 6A,A3 + 2A$]n-’
Cum,[e]
= [CA: + 6AfA2]n-112
Cum,[fi]
= [dA’: + 24cA:A2 + 24A:A3 + 48A:A$]n-’
Cum,[fj]
= 0(n-1J’2)+ ‘) for all f 3 5.
+ o(n-I),
+ o(n-‘), + o(n-‘),
It follows from Ochi (1983) that the standardized version of his estimator a = &(6 the following asymptotic cumulants: Cum&U
= -J&(2
Cum3[Ql
= - d>i
Cum,[Q]
= 0(n-(J’21+‘)
+ c)n-“2 f o(n-‘),
n-l/’ + o(n-i),
g(&c) Cum2[SZ] = 1 + =n
Cum,[Q]
=
- 0,)/J=
has
+
(13)
-1
6(--l + 110:) _ n 1 + o(n-l), I - eg
for all J > 5,
Cum1
Under condition (2), we have X2
X&C
tWfiJi)l = 2c1 _ eij + -_a-112 1_ sg
(14) (15)
where g(0,c) = 98’ - 1 - 402(c, - 1)(c2 - 1) f 2c(d2 f 02c - 1) and c = cl + c2 - 1. From Theorem 3, Lemma 2 and (13)-(15), we have the following proposition. Proposition.
O(U-I),
+
x2[1 - 0; - 2tgc] _ n ’ + o(n-I), 2(1 - @)”
264
G. Zhang / Statistics
Cum2Cn~dfi,f2)1 =
=
Cum4CWflJ2)l
=
Letters 20 (1994) 259-268
& - ,12z33, nel” 0 982
Cum3C~Mf~J2)l
& Probability
+
[ (1 -
-
c1 _
+ x2(6 - 14e; - seic) + x2g(eo,c) ei)3
(1 - e:)’
6x3eo
eij2 n-l/’
6x4(3 + 78:) c1 _ eij3
n
CumJ[nZ6(fl,f2)] = O(n-(J’2)+1)
_
+
1
n -l + o(n_I),
3x4(1 + se;) _ c1 _ eg,3
n
’ + oW’),
’ + o(n-‘I,
for all J > 5.
Thus p
n~ii(fiJ2)
-
2
XJrce,,
[
e; 2~1 =
WV)
-
+(‘)
_
1 xe;
2 0;)
n
+
(1
+
31
i
_
3 - 13e; - 4e:c &I,
+
[
41 - a7 + 24)
+
282
~4 n
[-&%$j+
4(1 - e;)
1 ~3
Y
1 [ - J;;;1:e:) 19e; + 2e;c2 - 4e;c + 2g(eo,c) x2e2 v+$(:y:ijI +o(n-‘) 4(1 - e;) - (1 - e& )I v2
+
2(1 - e$)3’2n
+1 3 n( under 7r1. Theorem
4. Under condition (2), we have
(c + l)eo +
P,.(f f ,(2/ 1) = O(v) - !A gqv) B 1>2 X
i[
Jm
x2eo
4(1 - ep
1
‘-“’
x(3 - i9e; + 2eic2 - 4eic + 2g(eo,c) + ~~(1 + e: - 4eic)
8(1 - efp
x5e2 - 64(1 - &7/Z We next give the third-order ratio, which is optimal. From Cum, [LR] = G &
Cum2[LR]
= &
;
-
1-’1+ n
o(n-‘).
(16)
asymptotic expansion for the misclassification Taniguchi (1991), we can get +
0
32(1 - 8;)5’2
probabilities
x2(38; - 1) n- 1 + o(n-‘) 2(1 - e;)2
2x3eo (1 - ey
n- 112
1
x2(3e; - 1)
(1 - e:)”
n-l + o(n-‘),
by the likelihood
G. Zhang / Statistics
& Probability
Letters 20 (1994) 259-268
265
6x3&, -l,z + 6x4(1 + 20;) ,-1 + o(n_‘), = - (I _ eg,Z n (1 - ey
Cnm&RI
6x4(78; + 3) _
Cum&RI = t1_ ezj3 n ’ + o(n-‘), Cum,[LR]
=-I
O(n-(Jiz)+l)
for all J 2 5.
Thus
= Q(v) - cb(v)
1 ei n xe;v4 2c1 _ 0;) "'+ (1 - 8;4)3’2JL +
5 l?i --d&+(1 _@;)Wn "+ -,/;;;A+; ~~~~~~2)+ ;;--@;; ' I[ )I x(1 - 486)
+
J--$&+2;l’“B,;‘!.} +OW’).
(17)
Summarizing the above, we have the foflowing theorem. Theorem 5. Under condition (2), we have the following:
(i) PLR(2[1) = a>(v)- IxI Cp(v)
J&~
+
+
x(5@ X
4(1
“f&W
1
n-“2
1) 982 x3(1 + e;, 8(1 - 6$)3’2 + 32(1 - 8;)5’2 - 64(1 - &7!2
(ii) p1,(f,,f2)(41) - fM211) = - ; +
+
o(n-‘).
$(v)
x(40; + 66;~’ + 88$(c, - 1)(c2 - 1) - 4~) X(1 - 0;)“” [
x3 e;c - 8(1 - 8;)5’2
n-l + II
o(n-‘).
(iii) Zfcl + c2 = 1, then ?J
pI,(~I,fJ21 1) - PLR(211) = 4(v) lxle ;l-~;:&-
cl)1 n-1 + o(n-‘). 0
Here, the term of order n - 1 1s . positive, whence, the statistic LR is better than Ih(fI ,f2). The minimum of the term of order n - ’ is attained at cl = c2 = 4.
266
G. Zhang / Statistics & Probability Letters 20 (1994) 259-268
4. Simulation In this section, x, = 8X,_,
we give a numerical
study for the Gaussian
(1,1) process
+ E, - IX&,_1,
(18)
where E, - i.i.d.N(O, l), to verify the results of Section
I(fl J2) =
ARMA
241:
@_‘) ,;yg
2’
x,x,+,,,[(e;
2. For this model, it can be shown that
- e:)cP + (e, - e2)(cF1’ + cd”“)]
(n 1) f=l
(19)
and
Z’(flJ2) = ~(flLf2) -
(e, - a)(1 - 2aBo + a2) Ix’ ?zJtl(l - a”) (1 - Q3(1 - a&)
(20)
under condition (2). We calculated Z(fi,f2) and Z’(fi,f2) for n = 100, and iterated this procedure 1000 times for 80 = 0.1, . ..) 0.9, CC= 0.55, and x = 0.6, x = - 0.6. Then we get Table 1. From this table, we can see that (1) the simulation results of discriminat statistics Z(fi,f2) and Z’(fi,f2) are both close to the theoretical detection probabilities of LR; (2) the MSE of Z’(f1,f2) are smaller than those of Z(fi,fi), so Z’(fi,f2) is better than Z(fi,f2) in the sense of minimizing the misclassification loss. That is, the simulation results agree with the theoretical ones given in Theorems 1 and 2. Conclusion. For the discriminant problems of the Gaussian ARMA processes with small samples, we can use an approximation of the nP1 LR Z(fi,f2) as a classification criterion. The misclassification probabilities of this criterion under local condition can be evaluated with explicit expression up to O(K”~). We can also get a modified version of Z(fi ,f2) which minimizes the misclassification loss. The theorems established in Section 2 can be expended to order O(n- ‘).
Table 1 Comparison of the simulated detection theoretical detection probabilities
probabilities
of I(f: g) > 0 and l’(fi,f2)
> 0 with the
SDP ofI(.f-i,fz) > 0
SDP of I+(fi&)
x = 0.6
x = -0.6
x = 0.6
x = -0.6
x = 0.6
x = -0.6
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.614 0.701 0.685 0.658 0.682 0.650 0.646 0.647 0.677
0.659 0.631 0.630 0.642 0.684 0.620 0.68 1 0.678 0.634
0.563 0.673 0.681 0.665 0.67 1 0.638 0.638 0.638 0.698
0.660 0.642 0.652 0.606 0.675 0.630 0.701 0.670 0.659
0.6188 0.6195 0.6246 0.6265 0.6381 0.6422 0.6691 0.6797 0.7851
0.6181 0.6210 0.6222 0.6301 0.6329 0.6501 0.6565 0.7032 0.7236
MSE
0.0070
0.0060
0.0040
0.0037
> 0
TDPofLR
>0
Note: (a) SDP = simulated detection probability, TDP = theoretical detection probability; (b) TDP = 1 - PLR(l 12);(c) MSE = variance[(TDP - SDP)/TDP] + {Mean[(TDP - SDP)/TDP]}‘.
267
G. Zhang / Statistics & Probability Letters 20 j1994) 259.-268
Appendix
< 01, wh ere P,, [ ‘1 stands for the probability of [ .] under hypothesis 7Ei.Applying a general Edgeworth expansion formula (Taniguchi, 1991, p. 15) to Lemma 1, we have Proof of Theorem 1. Note that P,(2/ 1) = P,,[l(fi,fi)
x J(@o)
1
KtQo)[v” +2,:nji8o)D6JI;Wo)J%G Setting v = - (Ix j/2)m
I]
I
+ O(n_‘).
in the above equation, we have
p7c,c~(flJ2)< 01= 1”,(211) = i Px,[l(fI,f,)
for x >O
> 0] = 1 - P1(211) for x < 0,
which proves (7) for P,(Z/ 1). On the other hand, PI(l 12) = 1 - P,, [Z(fr ,fi) d 01, Under hypothesis 7r2, using Lemma 1 and the relation I(&) = Z(4)) + -z- [2J(B,) + K(@,)] + O(n - ‘), 4 we
have Cumr[~$!$!$]=
-~Jij&j-$[3J(eeJI!?$(eo)]--j$&+O(n~1),
Cum,[yx]=
1 +~J(Hy:~o~(eo)+O(n-l),
forall J&4.
Cum,[:lli~]=O(n-““~+‘) Similarly, we obtain
= 1 - Q(v) +
4(v) Jm
1
+ Otn-‘)
268
G. Zhang / Statistics
1 - P,(112)
p
n~(.f1?.A)
a similar
Letters 20 (1994) 259-268
for x > 0, for x < 0.
= i P1012) Proof of Theorem 2. Using
& Probability
argument
0
as in the above, under
7c1, we can get
B(b)
+ O(n_1).
xJ@x,j;l~
1
Thus P I(f,,S*) + (211) = @i(u) - E --!!%L Qac) = PLR(211) + O(C’).
;
J(0,)
+ ;
1 K(b) K(B,) + - ~ 6 I(b)
1+O(n-‘)
17
Acknowledgements The author, would like to thank the referee, Professors helpful suggestions and encouragement.
K. Isii, N. Inagaki
and M. Taniguchi
for their
References Brillinger, D.R. (1981), Time Series: Data Analysis and Theory, Expended Edition (San Francisco, CA, Holden day). KrzyBko, M. (1983), Asymptotic distribution of the discriminant function, Statist. Prob. Lett. I, 243-250. Ochi, Y. (1983), Asymptotic expansions for the distribution of an estimator in the first-order autoregressive process, J. Time Ser. Anal. 4, 51-67. Okamoto, M. (1963), An asymptotic expansion for the distribution of the linear discriminant function, Ann. Math. Statist. 34, 1286-1301. Shumway, R.H. (1982), Discriminant analysis for time series, in: P.R. Krishnaiah and L.N Kanal, eds., Handbook of Statistics. Vol. 2 (North-Holland, Amsterdam) pp. l-46. Taniguchi, M. (1991), Higher Order Asymptotic Theory for Time Series Analysis, Lecture Notes in Statistics 68 (Springer, Berlin). Wakaki, H. and Y.S. Oh (1986), Asymptotic expansion for the distribution of the discriminant function in the first order autoregressive processes, Hiroshima Math. J. 16, 625-629. Zhang, G.Q. and M. Taniguchi (1994), Discriminant analysis for stationary vector time series, J. Time Ser. Anal. 15, 117- 126.