Donkey walk and Dirichlet distributions

Statistics & Probability Letters 57 (2002) 17–22

Donkey walk and Dirichlet distributions Gerard Letac ∗ Laboratoire de Statistique et Probabilites, Universite Paul Sabatier, 118 Route de Narbonne, F-31062, Toulouse, Cedex, France Received July 2001; received in revised form September 2001

Abstract The donkey performs a random walk (Xn )n¿0 inside a tetrahedron with vertices A1 ; : : : ; Ad as follows. For r = 1; : : : ; d and t = 0; 1; : : : ; at time dt + r the donkey moves from the point Xdt+r−1 to a point Xdt+r such that the barycentric coordinates of Xdt+r with respect to A1 ; : : : ; Ar−1 ; Xdt+r−1 ; Ar+1 ; : : : ; Ad have a Dirichlet distribution depending on r: When the parameters are properly chosen, we compute the stationary distributions of the d homogeneous Markov chains (Xdt+r )t¿0 : For instance, if Xdt+r is uniformly chosen in the tetrahedron with vertices A1 ; : : : ; Ar−1 ; Xdt+r−1 ; Ar+1 ; : : : ; Ad then the stationary distribution of (Xdt )t¿0 is Dirichlet with c 2002 Elsevier Science B.V. All rights reserved. parameters (d; d − 1; : : : ; 1).  MSC: 60 B15; 60 J15 Keywords: Markov chains in a tetrahedron; Random walk on stochastic matrices

1. Donkey business Consider two points W (water) and H (hay) of the real line and a sequence (Un ; Vn )n¿1 of independent random variables with uniform distribution on [0; 1]: The Buridan’s donkey as modelized by Stoyanov and Pirinsky (2000) is at time n at position (1 − Xn )W + Xn H; where X2n+1 = Un X2n and (1 − X2n+2 ) = (1 − Vn )(1 − X2n+1 ): That means that at time 2n + 1 the donkey moves towards water by placing itself uniformly between W and its position at time 2n: A similar move towards hay takes place at time 2n + 2: We have X2n+2 = (1 − Vn )Un X2n + Vn and X2n+3 = Un+1 (1 − Vn )X2n+1 + Un+1 Vn : Thus, (X2n )n¿0 and (X2n+1 )n¿0 are two homogeneous Markov chains of the type Yn = Fn (Yn−1 ) where Fn (x) = An x + Bn is a random a>ne mapping of [0; 1] into itself and the sequence (Fn ) is i.i.d. The properties of the Markov chains of the above type, and su>cient conditions for the existence and ∗

Fax: +33-5-6155-6089. E-mail address: [email protected] (G. Letac).

c 2002 Elsevier Science B.V. All rights reserved. 0167-7152/02/$ - see front matter  PII: S 0 1 6 7 - 7 1 5 2 ( 0 2 ) 0 0 0 2 7 - 5

18

G. Letac / Statistics & Probability Letters 57 (2002) 17–22

uniqueness of stationary distributions, are well known (see Vervaat, 1979). The paper by Stoyanov and Pirinsky (2000) shows among several other things that stationarity is obtained in the above chain if the distribution of X0 is (2; 1) : They also consider the following generalization: d points A1 ; : : : ; Ad are given in the plane. At time dt + r the donkey moves from the point Xdt+r −1 of the plane to a point Xdt+r which is uniformly distributed on the segment [Xdt+r −1 ; Ar ]: Although it can easily be proved that the stationary distributions of the d Markov chains (Xdt+r )t ¿0 in the plane do exist, their actual computation is a di>cult problem. The aim of the present paper is to oGer a diGerent generalization of the (2; 1) result on the segment [W; H ]. It is based on two observations: First, the (2; 1) result is easily extended to the case where the distributions of Un and Vn is (a; b) and (b; c) ; when a; b; c are positive. In this case, the stationary distributions of (X2n )n¿0 and (X2n+1 )n¿0 are, respectively, (a+b; c) and (a; b+c) : To see this, give the distribution (a+b; c) to X0 and check by the Mellin transform E(X1s ) = E(X0s )E(U1s ) that the distribution of X1 is (a; b+c) : Proceed similarly with the Mellin transform of (1 − X2 ) to see that the distribution of X2 is (a+b; c) again. Note that the parameters of the beta distributions of Un and Vn must be linked as above. If not, the stationary distributions obviously still exist, but are not elementary anymore. A second possible direction of generalization of the (2; 1) result is to replace the segment [W; H ] of the real line by a tetrahedron with vertices A1 ; : : : ; Ad in the real a>ne space of dimension d−1 and to let the donkey proceed as follows: At time dt +r the donkey moves from the point Xdt+r −1 to a point Xdt+r which is uniformly distributed in the tetrahedron with vertices A1 ; : : : ; Ar −1 ; Xdt+r −1 ; Ar+1 ; : : : ; Ad . As we shall see, when (Xdt )t ¿0 is expressed by barycentric coordinates with respect to A1 ; : : : ; Ad ; the stationary distribution follows a Dirichlet distribution with parameters (d; d − 1; : : : ; 1): (DeInitions about a>ne space, a>ne frames and barycentric coordinates are recalled in Section 2 of Letac and Scarsini, 1998). This note provides a common generalization (Theorem 3) of the two extensions of the (2; 1) result mentioned above: the donkey moves in the tetrahedron with vertices A1 ; : : : ; Ad , and the move from Xdt+r −1 to Xdt+r is done according to a Dirichlet distribution whose parameters depend on r: If these parameters satisfy a compatibility condition (see 3.2), then the stationary distributions of the chains (Xdt+r )t ¿0 will be described by Dirichlet distributions. 2. Dirichlet distributions Let (s1 ; : : : ; sd ) be positive numbers, and let U = (U1 ; : : : ; Ud ) be a random variable of Rd such that U1 + · · · + Ud = 1 and Uj ¿ 0 for all j: Then U has the Dirichlet distribution with parameters (s1 ; : : : ; sd ) if the density of (U1 ; : : : ; Ud−1 ) is proportional to s

−1

u1s1 −1 u2s2 −1 · · · udd−−11 (1 − u1 − · · · − ud−1 )sd −1 : We state without proof a proposition which characterizes the Dirichlet distributions. It is taken from Chamayou and Letac (1994), Proposition 2.1. Proposition 1. Let (s1 ; : : : ; sd ) be positive numbers. Let U = (U1 ; : : : ; Ud ) be a random variable of Rd such that U1 + · · · + Ud = 1 and Uj ¿ 0 for all j. Then U has the Dirichlet distribution

G. Letac / Statistics & Probability Letters 57 (2002) 17–22

19

with parameters (s1 ; : : : ; sd ) if and only if; for all fj ¿ 0 one has E((f1 U1 + · · · + fd Ud )−(s1 +···+sd ) ) = f1−s1 · · · fd−sd : This proposition leads to the following useful proposition. To state it, we adopt the following notation. If W = (W1 ; : : : ; Wd ) and if r = 1; 2; : : : ; d we write 

1

  0    :::  Sr (W ) =    W1    :::  0

0

:::

0

:::

1

:::

0

:::

:::

:::

:::

W2

:::

Wr

:::

:::

:::

0

:::

0

0



 0    ::: :::   ;  : : : Wd    ::: :::   ::: 1

(2.1)

where W is the rth row of the (d; d) matrix Sr (W ): Proposition 2. Suppose that V = (V1 ; : : : ; Vd ) and W = (W1 ; : : : ; Wd ) are two independent Dirichlet random variables with respective parameters (v1 ; : : : ; vd ) and (w1 ; : : : ; wd ) such that furthermore  for a 7xed r in 1; : : : ; d one has vr = dj=1 wj . Then Y = VSr (W ) is Dirichlet distributed with parameters (v1 + w1 ; : : : ; vr −1 + wr −1 ; wr ; vr+1 + wr+1 ; : : : ; vd + wd ): Proof. Let f = (f1 ; : : : ; fd ) a row vector of positive numbers and write fT for its transpose. Denote v = v1 + · · · + vr : Then (1)

(2)

E((YfT )−v ) = E((VSr (W )fT )−v ) = E(E((VSr (W )fT )−v |W )) (3)

−v

−vr+1 = f1−v1 · · · fr −1r−1 E((WfT )−vr )fr+1 · · · fd−vd

(4)

−v

= f1−v1 −w1 · · · fr −1r−1

− wr − 1

−vr+1 −wr+1 fr−vr fr+1 · · · fd−vd −wd :

In this chain of equalities; (1) comes from the deInition of Y; (2) from conditioning by W; (3) from the fact that V and W are independent and from Proposition 1 (“only if ” part); and (4) from Proposition 1 (“only if ” part) again. Thus applying Proposition 1 (“if ” part); Y is Dirichlet distributed with the indicated parameters.

20

G. Letac / Statistics & Probability Letters 57 (2002) 17–22

3. The walk of the donkey in the tetrahedron Theorem 3. Let S =(sij )16i; j6d be a square matrix of positive numbers such that for all i =1; : : : ; d we have d  j=1

sij =

d 

sji

(3.2)

j=1

and denote by si the common value of this sum. Let U (t) =(Uij(t) )16i; j6d ; t =0; 1; 2; : : : be a sequence of i.i.d. random stochastic matrices such that their rows Ui(t) = (Uij(t) )16j6d ;

(3.3)

where i = 1; : : : ; d are independent and are Dirichlet distributed with respective parameters (si1 ; : : : ; sid ): Let also (A1 ; : : : ; Ad ) be an a8ne frame of a (d − 1)-dimensional real a8ne space and denote the convex hull of (A1 ; : : : ; Ad ) by T: Consider the non-homogeneous Markov chain (Xn )n¿0 on T de7ned as follows: X0 in T is independent of (U (t) )t ¿0 and for all t ¿ 0 and all r = 1; : : : ; d then (t) (t) A1 + · · · + Ur;(t)r −1 Ar −1 + Urr(t) Xdt+r −1 + Ur;(t)r −1 Ar+1 + · · · + Urd Ad : Xdt+r = Ur1

Then for r=0; 1; : : : ; d−1 the stationary distribution of the homogeneous Markov chain (Xdt+r )t ¿0 is the distribution of Y1(r) A1 + · · · + Yd(r) Ad where Y (r) = (Y1(r) ; : : : ; Yd(r) ) has a Dirichlet distribution. In particular, for r = 0 the parameters of Y (0) are

d− 1  (3.4) sid : s1 ; s2 − s12 ; s3 − s13 − s23 ; : : : ; sd − i=1

Proof. Note that for d = 2 (3.2) gives   a b s11 s12 = s21 s22 b c and (3.4) is the (a+b; c) result of the introduction. For an arbitrary d; if sij = 1 for all i and j; i.e. in the uniform case; then (3.4) becomes (d; d − 1; : : : ; 1) as stated in the introduction. For x in the tetrahedron T; denote (t) (t) A1 + · · · + Ur;(t)r −1 Ar −1 + Urr(t) x + Ur;(t)r −1 Ar+1 + · · · + Urd Ad : Gdt+r (x) = Ur1

Thus Gdt+r is a random mapping from T to T: Denote also for x ∈ T Ft (x) = Gdt+d ◦ Gdt+d−1 · · · ◦ Gdt+2 ◦ Gdt+1 (x): Thus Xd(t+1) = Ft (Xdt ): Observe that the random mappings Ft are i.i.d. According to the contraction principle (see Chamayou and Letac, 1991, Proposition 1), in order to see that the stationary

G. Letac / Statistics & Probability Letters 57 (2002) 17–22

21

distribution of the Markov chain (Xdt )t ¿0 exists and is unique, enough is to check that if Zt (x) = F0 ◦ F1 ◦ · · · ◦ Ft (x); then limt →∞ Zt (x) exists almost surely and does not depend on x ∈ T: To prove this, we interpret the composition of the functions Gdt+r or Ft as products of random matrices as follows. Denote by AT the set of the a>ne maps f sending T into itself. For instance, Gdt+r and Ft are random elements of AT : If f is in AT let M (f) be the stochastic (d; d) matrix whose ith row is the vector of barycentric coordinates of f(Ai ) with respect to the a>ne frame (A1 ; : : : ; Ad ): Thus if the barycentric coordinates of x ∈ T are ("1 ; : : : ; "d ); the barycentric coordinates of f(x) are ("1 ; : : : ; "d )M (f): Note that if f and g are in AT , then M (g ◦ f) = M (f)M (g): This implies that the barycentric coordinates of Zt (x) are ("1 ; : : : ; "d )M (Ft ) · · · M (F0 ): Furthermore, M (Ft ) = M (Gdt+1 )M (Gdt+2 ) · · · M (Gdt+d )

(3.5)

M (Gdt+r ) = Sr (Ur(t) )

(3.6)

and with notations (2.1) and (3.3). Thus (3.5) and (3.6) imply that all entries of M (Ft ) are positive (details are as follows: if uj for j = 1; : : : ; d are row vectors of Rd with positive components, one shows easily by induction on r = 1; : : : ; d that the entries of the Irst r rows of the matrix S1 (u1 )S2 (u2 ) · · · Sr (ur ) are positive; the case r = d is what we need). Thus the hypotheses of Proposition 2.2 of Chamayou and Letac (1994) are fulIlled and this implies the existence of Zt (x) and the fact that actually Zt does not depend on x: Therefore, a stationary distribution of (Xdt )t ¿0 exists and is unique. Denote now by Y (n) = (Y1(n) ; : : : ; Yd(n) ) the row vector of the barycentric coordinates of Xn with respect to the a>ne frame (A1 ; : : : ; Ad ): Thus from (3.6) we have Y (dt+r) = Y (dt+r −1) Sr (Ur(t) ):

(3.7)

Assume now that Y (0) is Dirichlet distributed with parameters (3.4). For r =0; 1; 2; : : : ; d we are going to show by induction on r that the distribution of Y (r) is Dirichlet with parameters v(r) =(v1(r) ; : : : ; vd(r) ) where the numbers vj(r) are deIned by vj(r)

=

r 

sij

for j 6 r

sij

for j ¿ r

i=j

=

d+r 

(3.8)

i=j

with the convention that sij = sij when j ≡ j  mod d. For r = 0; this is hypothesis (3.4). Suppose that it is true for r − 1; with 1 6 r 6 d: Observe a crucial point: vr(r −1) is equal to sr ; as a consequence of hypothesis (3.2). Apply (3.6) to t = 0 to obtain Y (r) = Y (r −1) Sr (Ur(0) ). Since the parameters for Ur(0) are (sr1 ; : : : ; srd ) and have sum vr(r −1) = sr ; then Proposition 2 is applicable, and it is easy to get from it that the induction hypothesis is extended to r: Now we observe that v(0) = v(d) and this shows that the

22

G. Letac / Statistics & Probability Letters 57 (2002) 17–22

distribution of X0 is a stationary one for the chain (Xdt )t ¿0 : Similar results hold for each chain (Xdt+r )t ¿0 : The parameters for (Xdt+r )t ¿0 are v(r) and the proof is complete. Remark. The set of matrices satisfying (3.2) is an open convex cone in a space of dimension d2 − d + 1; and the extremal lines of its closure C can be easily described as follows: if c is a circular permutation of a subset T of {1; : : : ; d}; consider the (d; d) matrix Sc = (sij )16i; j6d deIned by si; c(i) = 1 and sij = 0 if either i and j are in T but j is not the image of i by c; or if i or j are not in T: Then the extremal lines of C are the [0; ∞)Sc ’s. References Chamayou, J.-F., Letac, G., 1991. Explicit stationary distributions for compositions of random functions and products of random matrices. J. Theoret. Probab. 4, 3–36. Chamayou, J.-F., Letac, G., 1994. A transient random walk on stochastic matrices with Dirichlet distribution. Ann. Probab. 22, 424–430. Letac, G., Scarsini, M., 1998. Random nested tetrahedra. Adv. Appl. Probab. 30, 619–627. Stoyanov, J., Pirinsky, C., 2000. Random motions, classes of ergodic Markov chains and beta distributions. Statist. Probab. Lett. 50, 293–304. Vervaat, V., 1979. On a stochastic diGerence equation and a representation of non-negative inInitely divisible random variables. Adv. Appl. Probab. 11, 750–783.