Markov chains with a transition possibility measure and fuzzy dynamic programming

FU||Y

sets and systems Fuzzy Sets and Systems 66 (1994) 39 57

ELSEVIER

Markov chains with a transition possibility measure and fuzzy dynamic programming Yuji Yoshida Faculty q[' Economics, Kitakyushu University, Kitagata, Kokuraminami, Kitakyushu 802, Japan

Received January 1993;revised February 1994

Abstract

This paper constructs a Markov fuzzy process, which represents transitions of grades of fuzzy sets, with a transition possibility measure and a general state space. We analyse Shell's optimal stopping problem for the process and we apply the results to solve fuzzy dynamic programming with optimal stopping times and with general state spaces and action spaces under fuzzy transitions. Key words: Markov fuzzy system; Fuzzy dynamic programming; Snell's optimal stopping problem

O. Introduction

Bellman and Zadeh [1] have first studied a fuzzy decision process. The problem with optimal stopping times is studied by Kacprzyk [-5] and Stein [12]. Esogbue and Bellman [4] developed various kinds of fuzzy dynamic programming with finite state spaces and finite action spaces. Especially, Esogbue and Bellman [4, Section 5] studied fuzzy dynamic programming with terminal times and [4, Section 7] studied finite-stage fuzzy dynamic programming under a fuzzy transition. These fuzzy dynamic programmings are different from the classical dynamic programming, in the point of using grades of fuzzy sets instead of reward functions. Therefore the systems for fuzzy dynamic programmings are generally non-linear. This paper deals with fuzzy dynamic programming with optimal stopping times and with general state spaces and action spaces. Kurano et al. [6] and Yoshida et al. [14] studied dynamic fuzzy systems, which transit as follows. Let S be a metric space. We write a fuzzy set on S by its membership function g: S ~ [0, 1]. Let ~ o(s) be the set of all fuzzy sets on S, which are upper semi-continuous and have a compact support. For any ge ~ ° ( S ) , we define

qI~)(x):= sup{q(x,y) ^ ~(y)}, x~S;

(o.1)

yeS

and ~o(g):=g

and

~,(g):=~(~,-l(g)),

n = 1,2 . . . .

Then gl"(g)(x) is called a transition possibility from a point x to a fuzzy set g at nth step (see [14]). 0165-0114/94/$07.00 © 1994 - ElsevierScience B.V. All rights reserved SSDI 0165-01 14194100099-S

(0.2)

Y. Yoshida / Fuzz)' Sets and Systems 66 (1994) 3 9 - 5 7

40

From the reason explained in the latter part of this section, we need to define the transition possibility for a certain class of fuzzy sets containing #-°(E). We consider a fuzzy expectation as an extension of the transition possibility for the class. Expectations by fuzzy measures are studied by Sugeno [13]. However, when we deal with uncountable state spaces, it is known that there is an essential difference between fuzzy measures and possibility measures (see Puri and Ralescu [9]). Cooman and Kerre [3] studied the expectations by possibility measures, but did not take account of their measurability. This paper introduces a fuzzy expectation of measurable membership functions by a possibility measure induced by fuzzy relations, preserving the measurability. We also introduce a path structure for a dynamic fuzzy system with a possibility measure induced by fuzzy relations and construct a non-linear Markov fuzzy process representing transitions of grades of fuzzy sets. This enables us to deal with stopping times depending on paths analytically. We consider Snell's optimal stopping problem (Neveu [7, Section VI-2]) for the Markov fuzzy system. The problem is to find paths and stopping times maximizing the grade of fuzzy sets. The aim of this paper is to study fuzzy dynamic programming with general state spaces and general action spaces and with optimal stopping times depending on paths, using the solutions of Snell's problem for the Markov fuzzy system. We derive an optimality equation for the problem and we give an optimal stopping time bounded on an optimal path. First, we consider a class of fuzzy sets, containing ~°(S), which admits stopping times. We put the collection of all closed subsets of S by oK(S) and the Borel-tr-field of S by ~'(S). We write an ordinary set A( c S) by its indicator function 1A:S ~-o {0, 1}. The ~-cut go of a fuzzy set g is written by go:= {x • S[ g(x) >/~} (~t• (0, 1])

and

go := cl {x • S lg(x) > 0},

where cl denotes the closure of a set. The fuzzy set ge~r°(S) is characterized by its ~-cuts: (C.i) go is compact; (C.ii) g, ecg(S) for ~e[0, 1]; (C.iii) 0~,< go, = go for ~e(0, 1]. What kind of measurable fuzzy sets can we define stopping times for? To answer this question, we here consider an essential difficulty in optimal stopping problems for fuzzy systems (more generally, fuzzy decision processes) with general state spaces, through the following one-stage example. Let At and A2 be subsets of a state space S. We suppose that a return Z1 is given if we stop the process and a return Z2 if we do not. We consider a decision that we stop when the present state is in AI and we do not stop when the state is in A2. The return obtained by the decision is given by

Z :=

Z1 Z2

Then AI and

on A1, on A2. A2

Alc~A2=0

(0.3)

must have the following property: and

AlwA2=S.

(0.4)

However, when we construct a fuzzy system for fuzzy sets of 3r°(S), A1 and A 2 must be closed because of (C.ii) and Z1, Z2, Z e ~-°(S). This fact contradicts (0.4) when S is a general metric space. Therefore this paper extends ~'°(S) to some class of measurable fuzzy sets admitting stopping times. In order to preserve a measurability of fuzzy sets regarding a fuzzy expectation by a possibility measure induced by fuzzy relations, we find that the measurability must be preserved regarding projections from a product space (see Section 1). For example, Borel measurability is not proper since a projection of a Borel set in a product space is in general not Borel but analytic (see Bertsekas and Shreve [2, Section 7.6]). The collection of analytic sets is also not proper because analytic sets are in general not Borel but universally measurable. However a a-field

Y. Yoshidu/ Fuzz)' Sets and Systems 66 (1994) 39-57

41

of universally measurable sets cannot be represented by a completion of Borel sets by possibility measures. We a d o p t the collection of F,-sets (see P a r t h a s a r a t h y I-8, Definition 1.1]) and construct a M a r k o v fuzzy system with a transition possibility measure. Using the results, we solve fuzzy dynamic p r o g r a m m i n g under a fuzzy transition and Snell's optimal stopping problem for the M a r k o v fuzzy system. In Section 1 we introduce a path structure like the theory of stochastic processes. We also introduce a fuzzy expectation by a transition possibility measure and an admissible class of stopping times. In Section 2 we prove a strong M a r k o v p r o p e r t y for the stopping times and construct, as a fuzzy process, a M a r k o v fuzzy system under the possibility measure. In Section 3 we extend the expectation in the preparation for Section 4. In Section 4 we solve Snell's optimal stopping p r o b l e m for the M a r k o v fuzzy system. In Section 5 we apply the results to fuzzy dynamic p r o g r a m m i n g with stopping times depending on paths under the fuzzy transition.

1. Fuzzy expectations and stopping times F o r a metric space S, we put the collection of F~-sets by g ( S ) : = { A [ A = U , =aco C ,, C , e ~ ( S ) (n = 0, 1, 2 . . . . )}, where if(S) is defined in Introduction. ~ ( S ) denotes the set of all fuzzy sets on S satisfying the following conditions (E.i)-(E.iii): (E.i) So is compact; (E.ii) g, E g(S) for ~ ~ [0, 1]; (E.iii) N~,< ~=, = g~ for ~e(0, 1]. We note that cg(S) c g(S) = ~ ( S ) and ~-°(S) ~ ~ ( S ) . L e m m a 1.1. The collection of F~-sets, g(S), has the followin9 properties: (i) o~(S) is closed under finite intersections and countable unions; (ii) Let $1 and $2 be metric spaces. Let A ~g(S1 x $2). Then its projection from $1 × $2 to $1, projs~(A), has a property: projs~ (A) E g(S1). oc, Proof. (i) Let {A ,,}m= 0 C g(S). Then there exists C , ,,, E cg(S) (m, n = 0, 1, 2, . . . ) such that A," - U .c(,= o C , .., oc, A for all m. Therefore U,,=o ," = U .z~. . . oCm.,Eg(S) • Next let { A , " } ~ = o c g ( S ) for some l = 0 , 1 , 2 . . . . Then there exists C,",.e~(S) ( m = 0 , 1 , 2 . . . . ,l; n = 0 , 1 , 2 . . . . ) such that A , , = U ~ = o C , , , , for all m = 0 , 1 , 2 . . . . . I. Here we m a y assume that c"m . n ) ~n = O = ~(S) is non-decreasing, considering C~,.:= Uk=oC,, , kec~(S). Now we show ~ , U , = o ,,., = U , = o N m= 0 Cm,.. Let x ~ 0 m~= O U no~ C,, , .. Then for each m there exists n," = 0, 1, 2, . . . =O satisfying x E C,.,. for all n 7> nm, since { C.,,. }.% o is non-decreasing. Put n* :--- max,. _ o, 1,2 ..... t n,. ( < ~ ). Then x ~ 0 ~ = o C,, , ,. c U.~- o 0 t = o c,. , ,. Therefore we obtain N ,,z = o U ,~= o C,.,, c U . = o N lm=0 C,..,. The reverse 1 inclusion is clear. Thus we get 0 ~ = o A," = N~=o U . = o C m,. = U ,ac= o N,,=o c,.,,~g(s). (ii) Let A¢g(SaXSE). Then A = U , ~ o C . for some C . ~ g ( S l x S z ) (n = 0,1,2, ... ). We have projs~(A) = Ur~s~ {x~S1 I(x,y)~A} = Ur~s~ U,~o{x~S~ I ( x , y ) ¢ C , } = U,~oUy~s~ { x e S , [(x,y)~C.}. Since Uy~s~ {X¢Sl [(x,y)EC,}¢cg(S1), we obtain projs,(A)¢g(S1). []

The following proposition illustrates a regularity of g(S) for a one-dimensional case. Let R 1 denote the set of all real numbers.

Proposition I.I. Assume S : = R 1. Denote the set of all intervals by J ( S ) := {intervals I I I = (a, b), [a, b], (a, b], [a, b) for a, b ~ R 1 (a <~b) }.

Then U . ~ o l . ~ ( S ) f o r

{I.}.~o = J ( S ) .

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

42

Proof. It is trivial that [a,b]eg(S) for a, b e R ~ (a<<,b). Further, from Lemma 1.1(i), ( a , b ] = U~= ~ [a + 1/m, b] e R ~ (a <~ b). Considering the others similarly, we obtain J ( S ) c g(S). Therefore we get this proposition, using Lemma 1.1(i). [] Now we construct a fuzzy process with a transition possibility measure. In Section 2 we show that the fuzzy process has a strong Markov property. Let a discrete time space N := {0, 1, 2, 3, ... } and 1KI:= N w {0o}. Let a state space E be a finite-dimensional Euclidean space. Let Eo, E~, E2 .... denote copies of E. Then the path space is t2 := Ilk--o Ek, which is metrizable by the product topology. We also put g2, := Ilk =. E k for n e N. An element 09 = (to(0),to(1),to(2), ... )el2 is called a sample path. We define a map X,(to):= to(n) and a shift 0.(to):= (to(n),to(n + 1). . . . ) for toef2 and h e N . We put a-fields by J / , : = a(Xo,X! . . . . . X.) t for h e n and Jr':= a(U,~ N Jr'.) 2. For a sub-a-field X of ~ ' , ~-(JV') denotes the set of all X-measurable fuzzy sets of

~-(~).

Let q, := q be a time-invariant upper semi-continuous fuzzy relation on E, × E,+ ~ (n = 0, 1, 2, ... ) satisfying the following normality condition: For all n e N, sup q , ( x , y ) = l ( y e E . + l )

and

xeEn

sup q , ( x , y ) = l ( x e E , ) . yEEn+ 1

We define operations A,~o and V.~o by A.~oh,(to):=inf.=o,l,2 .... h,(to) and V~=oh.(@:= sup.=o,1, 2.... h.(to) for {h. } . 00 = o C ~-(d¢') and toef2. We define q : = A ~ = o q . ( x ~ , x , + l ) e ~ ( ~ g [ ) and P(A):= sup,oea q(to) for A e J / . Then P becomes a possibility measure on J / w i t h a density q (see Puri and Ralescu [9]) and has the following properties (P.i)-(P.iii): (P.i) P(0) = 0 and P(O) = 1; (P.ii) If A, Fe.I¢ satisfy A = F, then P(A)<~ P(F); (P.iii) If {A.}.~=o c J¢', then P ( U . = o A .) = V L o P ( A . ) • For h e ~ ( J / ) , we define a fuzzy expectation by the possibility measure P: P

E~(h):= =

h(to) dP(to) sup

h(to) ^ q(to)

for x e E,

~f~:~)(0) =x

where ~dP denotes Sugeno integral and the supremum on the empty set is understood to be 0 (see Sugeno [13], Ralescu and Adams [10]), and a ^ b:= min(a,b} and a v b:= max{a,b} for real numbers a,b. For x e E we also define a possibility measure P~(A):= E~(1A) (A e J/). Here we consider the meaning of the fuzzy expectation. From (0.1) and (0.2), for g e ~ ° ( E ) we can easily check

Ex(g(X,)) = q"(g)(x)

for x e E , n e N ,

(1.1)

which is a possibility of transition from a point x to a fuzzy set g at nth step. Therefore we can define the transition possibility for the class if(E), containing ~-°(E), by means of the fuzzy expectation Ex(g(X.)). Next we take g = Akn -=- 1o lex. . . . a~ +,~e ~ - ( d / ) (n e N). From the definition of the fuzzy expectation, we have n--1

Ex(9) = sup A l{xk+,~Ak+d(to) A qk(XkfD, Xk+l(.O) for x e E . c~ef2 k = 0

1 It denotes the smallest a-field on t2 relative to which Xo,X1 . . . . . X, are measurable. 2 It denotes the smallest a-field g e n e r a t e d by U.~N ~t'..

(1.2)

Y. Y o s h i d a / Fuzz), S e t s a n d S y s t e m s 66 ( 1 9 9 4 ) 3 9

57

43

Then we observe a path o9 which is starting from Xo ¢o = x, stays at X1 to e A 1 at 1st step, stays at X 2 (.0 E A 2 at 2nd step, ..., and arrives at X.¢o e A, at nth step. The representation (1.2) is a maximum possibility over all these paths oJ, and enables us to introduce stopping times depending on paths in this section. Therefore the fuzzy expectation is an extension of the transition possibility and admits stopping times depending on paths. We call {X. }heN a fuzzy process with the possibility measure P. The fuzzy process {X, },~N transits based on the fuzzy transition in the meanings of (1.1),(1.2), which is corresponding to the transition of the grade of fuzzy sets. Lemma 1.2. Let h6~(Jlt). Then: (i) the map x~-,Ex(h) is an element of ~(E); ao (ii) Ex(h) = V~ = oEx( h /x la,.) for x ~ E and {Am}~,=o c 8(f2) such that {A ,.},,=o is a partition oft?. Proof. (i) Since h, q e ~- (Jr'), from Lemma 1.1 (i), (ii), we have

{x[E~(h)/> ct} = U {x[~o(o) = x,h(o)) A q(o)) ~> ct} = proje,{O)[ h(~o) A q(e))>~ c t } 6 ¢ ( E , ) toe.Q

for :~• [0, 1]. Property (ii) can be easily checked from the definition. Definition. For J//-measurable fuzzy sets h,g'f2~-+[0,1] and x e E , h and g are called Px-equivalent if h(~o)/x q(co) = g(¢o) ^ q(¢o) for all to e f2: a)(0) = x. Then we write h = g Px-almost surely (Px-a.s.). Further we represent h = g a.s. when h = g Px-a.s. holds for all x ~ E. We introduce another fuzzy expectation. Definition. Let h e o~(J#) and n e N. We define a fuzzy expectation of h with a sub-a-field ~/t, by

hdP~o,.

E(h [~t.)(co) := f~ n+l

=

h(~o') A A glk(Xkm',Xk+l¢O') for ~ef2,

sup cY e.Q : o ' (k) = oa(k) (k = O, 1 . . . . . n)

k =n

where P,,,,(A):=

sup

A gh(XkOo',Xk+l¢O') for 0~6f2, n e N , and A e J / .

to'eA:to'(k)=to(k)(k=O, 1, ... ,n) k = n

Lemma 1.3. It holds that {X, e A } ~ ( ~ 2 )

.for all neN, Ae8(E).

Proof. Let n~N. If C is a closed subset of E, then { X . e C } ~ =oC,,~(E) Therefore for A U m (Cm~(E), m=0,1,2,.. ~(t~). []

, 1 ~ is closed in t2. ), we have { X . ~ A } = U mz=c o {X , ~ C , . } ~

-~ [ I k = o E k X C X l ~ k = n + l E k .

The following proposition shows that the fuzzy expectation E(h [Jr',) has the properties like the conditional expectations in the probability theory of Neveu [7].

44

Y. Yoshida/ Fuzzy Sets and Systems 66 (1994) 39 57

Proposition 1.2. Let h • ~ ( J ¢ ) and n•N. Then E ( h l J / , ) is a fuzzy set in ~ ( J [ . ) and is the unique •,l(.-measurable fuzzy set (in the sense of a.s.) such that

~AE(hlJC,)dP~= ~ahdPx for all AeJ¢,, xeE.

(1.3)

Proof. For all A • J¢. and x • E, we have

a E(hlJ[n) dP~ = ~a ( ~o h dP,o.. ) dPx(t°) =

sup sup h(to') ^ 4k(Xk~', Xk+,~O' o~A:to(Ol=x co'~12:¢o'(k)=co(k)(k=O,1..... n) k=n h(og) ^ q(o)) = sup

^

~k(Xko),X~+lo)) k=0

toeA:o(0) =x

= ~a hdP~ • Therefore E(h [ J¢.) satisfies the Eq. (1.3). Next let ct • [0, 1]. Noting n-I

he~-(Jt')

A 1EkxEk+,(Xk'Xk+l) A

and

k=O

k=n

qk(Xk, Xk+l)~.~(~[),

(1.4)

we have F:= {o)'ef21 h(to') ^ Ak~=~tk(Xk~O', Xk+lO£) >/ Ct}ES(f2). From Lemma 1.1(ii), we obtain {o)•t21 g(hr J¢.)(o9)/> ct} =

{~oet21h(co(0),to(1) . . . . . c o ( n ) , d ( n + l ) .... )^q(~o(n),oY(n+l) .... )>~0~}

U

= projeo ×e. . . . . . E.(F) x f2.+ 1 • o~(f2) n ~¢.. By (1.4) and Lemma 1.1, we get E(h I ~ ' . ) • ~ - ( J / . ) . Finally we show the uniqueness of solutions of (1,3). Put g'= E(h I,~¢.) and let another ,//.-measurable fuzzy set g' also satisfy (1.3). For e > 0 and x • E we put At, x:= e)ef2lo)(O) = x, g(o)) ^ A glk(XkcO,Xk+,o)) >>-g'(to) ^ A glk(XktO,Xk+,e)) + e eJt.. k=O

Let e > 0 and x•E. If A+,x SO, then we have ~a g ' d P x = ~ a

hdPx=~a

gd,~

=

to,At, ~

k=O

sup (0'(0)) ^ hal Ok(Xk(D, Xk+lO))+ ~) toeA+,x k=O

=~a g'dPx+e. e,x

k=O

Y. Yoshida/ Fuzzy Sets and Systems 66 (1994) 39-57 So we get A,.~ = 0 for all e > 0 and xeE. Therefore n

1

A

g A

n-1

~(xk,x~+,)<<.o'^

k=O

A

q~(x~,x~+,).

k=O

Since the reverse inequality holds similarly, we obtain g = g' a.s.

[]

L e m m a 1.4. Let he~(Jg). Then: (i) g(E(hl~Cm)l~g,) = E(hldl,) for m,n~N (rn >>,n); (ii) E(h A 1,1] Jr',) = E(hld/n) ^ la a.s. for n e N and A~g'(I2) n ~g,; (iii) E(h /x g(Xo,X~ . . . . ,X~)l,A/n)-- E(h]~l¢,) A g(Xo,X~ . . . . . X~) for n ~ N and g e ~ ( E o xE1 x ... xE,); (iv) E(h 1,//,) = Vm~rq E(h ^ la," [~'~) for nEN and {A,,}~=o ~ ~(f2) such that {Am}~=o is a partition off2. Proof. (i) Let m, n e N (m >~ n). From the definition, we obtain

E(E(h I~t'm) [ Jr',) (O2) =

sup

E(hlJt',.)(d)

A A

to'~t2:¢o'(k)-to(k)(k=O, 1 .... ,n)

=

sup

~lk(Xk(.O',Xk+1(A) ')

k=n

(

sup

h(M')^

o~'Et2:~'(k)=to(k)(k=O,1, ... ,n) \ to"el~:to"(k)=to'(k)(k~0,1, ...,m)

A k=m

~lk(XkO)t',Xk+lO)'))

m-1

^

A

qk(Xko)',X~+l~o')

k=n

=

h(tn') A A ?h(XktF, Xk+lm')

sup w'eO;o'(k)=to(k)(k=O, 1. . . . . n)

-- E(h I ~/.)(~o)

k=n

for co~ f2.

(ii) Let h e N and A e S ( ( 2 ) n ~[[,. From Proposition 1.2, for F e ~ ¢ . and xeE, we have

~i"E(h m la['A[n)dPx-'-fr hA ladPx = ~rc~a E(hl'A[n)dPx=fr E(hl'/l[n) A 1adPx. Therefore we obtain (ii). Off) Let h e N and g e ~ ( E o x El × ... x ED. Then

E(h A g(Xo, X, . . . . . X.)I ~.)(co) =

sup

h(~o') ^ g(to'(0), to'(1), ... ,~o'(n)) ^ A ?lk(Xkto', Xk+ld)

o)'~F~:oY(k)=oJ(k)(k=O. 1. . . . . n)

=

sup

k=n

h(oY) ^ A ?lk(Xkd, Xk+109') ^ g(co(0),~O(1). . . . ,to(n))

o~'~l'~:~'(k)=to(k)(k=O, 1. . . . . n)

k=n

= E ( h l J t ' . ) ( ~ ) ^ g(Xo,X~ . . . . . X,)(to) (iv) can be easily checked from the definition. Next we consider stopping times.

for co~f2. []

45

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

46

Definition. A m a p r:12 ~IKI is called a ({Jln }n~N-adapted) stopping time if {r = n} e.//n

for all n e N.

And we call a m a p r:f2 w-~1KI an g - s t o p p i n g time if {z = n } e a C / n n g ( f 2 )

for all n e N .

It is trivial that a constant stopping time, i.e. z = no for some no e N , is an g - s t o p p i n g time. Further the following lemmas hold. Definition (Revuz [11]). F o r a set A egg(E), we define a first entry time, za, and a first hitting time, o-a, of A by ~A(Og) = i n f { n l n e N , X,(to)EA} o-a(to) = inf{n/> l i n e N ,

for ~o~I2;

X,(~)eA}

for ~o~f2,

where the infima of the e m p t y set are understood to be + ~ . L e m m a 1.5. For a set A such that A e g ( E ) and ACeg(E), ra and o-a are g-stoppin9 times. Proof. F r o m L e m m a 1.1(i), we have n-1

{rA=n}=

N {XkeAC}n{X, eA} e°g[.ng(O) k=O

for a l l n e N .

Therefore va is an g - s t o p p i n g time. We can check o',4 similarly.

[]

L e m m a 1.6. Let o- and r be g-stopping times. Then o- v r, o- ^ ~ and o- + ~ o O, are g-stopping times. Proof. Let o- and r be g - s t o p p i n g times. Then we obtain that a v r and o- ^ r are g - s t o p p i n g times from L e m m a 1.1(i)and the following facts: F o r all n e N , n--1

{nAT=n}=

({o-=n}n{r=k})u k=0

{o-vr=n}=

U ({o- = k } n { r = n } ) e g ( f J ) n J t ' n ; k=O

O({a=n}n{r=k})u

0

({o-=k}n{z=n})eg(f~)nd/n.

k=n+I

k=n

Further since {r = n - k} e g ( f a ) n d'C'n-k (k = 0,1,2 . . . . . n), G-I{T = n - k} = {colz(o~(k + 1),~o(k + 2), ... ) = n - k} e g(f2) n .t[,. Therefore {0" -1- "/5° 0o. = n} =

0 ({o- = k} (", 0k- I{"L" = n - - k } ) e g ( f 2 ) n . / / n . k-O

So o- + r o 0, is an g - s t o p p i n g time.

[]

Definition (Neveu [7]). Let ~ be a stopping time. We define a sub-o--field ~'~ of ~ . / G : = { A e J I I A n {r = n} e J ¢ ' , for all h e N } . F o r an g - s t o p p i n g time r, we can define a fuzzy expectation with ./t',.

by

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39 57

Proposition 1.3. For h ~ ( J g )

E(hld¢~) = E(h[~,[,)

47

and an g-stopping time t, we define

on {r = n} Jot heN.

Then E(hl ~ ' 0 is a fuzzy set in o~(~[~) and is the unique element in ~(J¢~) (in the sense of a.s.) such that fa E(hlJg')dPx = ~A hdPx for all A~J¢~, xeE. Proof. Let heo~(f2) and let r be an g-stopping time. Let A e ~ Lemma 1.2(ii) and Proposition 1.2, for any x • E

(1.5)

and put An:= A n {r = n} for neN!. From

~ a E ( h l ~ ) d P ~ = V s~A E(hl"t't~)dPx = ,~s~AE(hIJ[.)dP~= V ~A h dPx. Thus E(hlJ¢O satisfies (1.5). Further, using Lemma 1.1(i) and Proposition 1.2, for all n • 1KIand ~ ~ [0, 1] {E(h IJ C ) / > ~} c~ {z = n} = {E(h IJ / , ) / > ~} ~ {r = n} eg(f2) c~ d / . . Therefore we obtain {E(h [,/g~)/> ~} eg(O) c~ ~g~ for all :¢ = [-0, 1]. The other property can be checked in the same line as Proposition 1.2. []

Corollary 1.1. Let { Y, }. ~N be a sequence satisfying Y, ~ ~ (M,) (n ~ Yq) and let z be an g-stopping time. Define

Y~= r. on {z= n} for neN. Then Y~e~(J[O.

2. Markov property for the fuzzy process In this section we show a Markov property for the fuzzy process {X.}.~N with the possibility measure P.

Definition. Let r be an g-stopping time. We define X~=X. on{r=n}

forn~lKl.

Lemma 2.1. Let ge~(E). Then: (i) g(X.)e~(Jg.)for n~N; (ii) g(X~)6~(JlC~) for an g-stopping time r.

Proof. (i) Let g e ~ ( E ) and n~N. Since {x~Elg(x) ~ ~} eg(E), for ~ [ 0 , 1] {g(x.)/> ~} = {~, I ~(~,(n))/> ~,} = "I-]' E~ × {x ~ E I g(x)/> ~,} × 1~i Ek E g(~2) n ~ ' . . k=O

k=n+l

Therefore g(X.)e~(d¢',). (ii) Let g e ~ ( E ) and let r be an g-stopping time. From (i) and Corollary 1.1, we obtain g(X O e ~ (Jg O.

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39 57

48

Owing to Lemmas 1.2(i) and 2.1, we may introduce the following fuzzy transitions. Definition. For h e n and an 8-stopping time z, we define maps, P. and P~, ~r(E)~--,~r(E) by

P.g(x):= Ex(g(X,)) ( x e E )

for g e t - ( E ) ;

Peg(x):= Ex(g(x,)) ( x e E )

for g e t - ( E ) .

We call P, and P, fuzzy transitions. Regarding the fuzzy process {X.},~N, we obtain a Markov property and a strong Markov property like the theory of Markov chains in Revuz [11]. Theorem 2.1. For h ~ ~(d¢), the following Markov property holds:

E(h°O.l.t¢.) = Ex.(h) a.s. for n~N;

(2.1)

Especially for g ~ ~ (E), E(g(Xm+.)l./t'.) = Pmg(X.) a.s. for m, n e N .

(2.2)

Proof. Let h~-(../¢). Then for all A E..Ct'. and all x e E we have

~a E (h ° O. l JCn) dPx = ~a h ° O. dPx

=

h(O.rn) ^ A ?lk(XkO, Xk+lCO)

sup t ~ A :to(O) = x

=

sup

k=O

(

toeA:to(O)=x \

sup

h(t~')Aqgo')lA ~1~ ~lk(Xk(D' Xk +10))

t o ' e l 2 : t o ' ( O ) = to(n)

k=O

= ~A Ex.(h)dPx. Thus we obtain (2.1) in the same arguments as the proof of the uniqueness in Proposition 1.2. Further, for g ~ ( E ) , we have

E(g(Xm+,)l~(,) = E(g(Xm ° 0.)l Jr'.) = Ex.(g(Xm)) = Pmg(X.) a.s. Therefore the proof is completed.

for m, n e N .

[]

Theorem 2.2. For h c gr ( ~ ) , the following strong Markov property holds:

E(h o O, [~(~) = Ex,(h) a.s. for a finite 8-Stopping time z;

(2.3)

Further for g e ~ (E ), E(g(Xo+~oo,)l,l[o) = PTg(Xo) a.s. for finite t~-stoppin# times tr and z.

(2.4)

Y. Yoshida / Fuzz)' Sets and Systems 66 (1994) 39-57

49

Proof. Let h e ~ - ( J / ) and z be a finite g-stopping time. Let A e ~'~ and put A. := A • {z = n} for n e NI. Then, using Markov property and Lemma 1.2(ii), for all x e E we have

~AE(h°O~lJg,)dPx= V ~a E(h°O~l~Cl.)dP~, nEN

n

= k/ fa hoO,dPx liEN

n

= V ~a hoO.dP~ hEN

n

= ..NV~a.

Ex.(h)dPx

= ~a Ex~(h)dPx. So we obtain (2.3). Since (2.4) is trivial from (2.3), we obtain this theorem. Corollary 2.1.

[]

It holds that

P~P~= P,+~oo, on ~(E) for finite g-stopping times a and z.

(2.5)

From now on, we call {X.}.~N a Markov fuzzy process, because of Theorems 2.1 and 2.2. For Sections 3 and 4, we extend the state space E, referring the theory of Markov chains in Revuz [11, p. 23]. Let A be not a point of E and put Ea:=Eu{A} and f2a:= I ] L o EA. Since {E}eS(Ea) and {A}ES(Ea), we can consistently extend the state space E to Ea and the path space f2 to f2a. We also extend Markov fuzzy process {X,}.~N and the shift {0.}.~N: X~:={ X " °{nz ={nz}=oono } ; f ° r n e N ,

0.(CO) : =

J" 0.go) ~. ~oa

for n e N and ¢oe OA, for n = ~ and ¢oe f2a,

where ~oa = (A, A, A.... )e f2a. We let ~ be an entry time to {A }, which is called the death time: ((09):= inf{nlneN, X.(co) = A} for ogef2a. Then ( is an g-stopping time. A fuzzy set

ge~(E)

is also extended to a fuzzy set on Ea:

g(x):={~(x) ifx~E, if x=A. Then the possibility measure pa(A):= f 10 if °gaeA otherwise

Px, xeE,

is extended to Px,

for A e a(Xo, X1, X2, ... ).

xeEa,

by

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

50

Since Ea and £2a are compact by Hausdorff's one-point compactification, the same results hold for the extended Markov fuzzy processes without assuming (E.i) in the definition of o~(S). From now on, this paper deals with the Markov fuzzy processes on the extended state space Ea.

3. An extension of the fuzzy expectations Let S be a metric space. For fuzzy sets g and F on S, a relation g/> f means that g(x) ~ F(x) for all x e S. Then /> becomes a partial order. The possibility measure P has the following properties.

+ c ~(J-[). Then: Lemma 3.1. Let {hm}m=O + (i) V~=oEx(h,.) = E x(Vm=ohm)for xeE; (ii) If {h,~}~=o is non-decreasing (with respect to the order >>,), then lim~++ E~(h,.) = E~(lim~+~ h~) for xeE; (iii) V~=o E(hm I ~¢/.) = E(V~= 0 hm I~¢.)for n e N; (iv) If {hm}~=o is non-decreasing (with respect to the order >~), then lim=++ E(hm I./[,) = E(lim=+~ h,, IJ¢.)

for heN. Proof. (i) For x e E, ~/ Ex(h,,)= V m=O

sup

h,,(og)Aq(~o)=

m = O t~et2:to(O)=x

sup

~/ h = ( t ~ ) ^ q ( o g ) = E x ( ~ / h , ) .

toel2:to(O)=xm=O

So we get (i). The other equations are similar.

n=O

[]

Let S be a metric space such as S = E or S = £2. In order to solve Snell's optimal stopping problem in the next section, ~ ( S ) is not a sufficient class of fuzzy sets. We extend ~-(S) and we define fuzzy expectations for them. Lemma 3.1 guarantees an extension of the expectations. Let ~(S):= {fuzzy sets gl there exists a sequence {g,}~=o c ~ ( S ) s u c h that .=o ~g'=

g}"

Then we can take {s~},%0 c ~-(S) to be non-decreasing, considering s,.=~'"Vk~, Ske ~ ( S ) (n = O, 1, 2, ... ). Further h ( e ~ (S)) is ~ (S)-meas urable and ~ o (S) c ~ (S) c f#(S). Using Lemma 3.1 (ii), (iv), we define fuzzy expectations for h e ~(~'): For h e f~(Jt') there exists a non-decreasing sequence {h.}.+N c ~(~//) such that lim.~+ h. = g. Then we define E~(h):= lim Ex(h.)

for xeE;

(3.1)

n~ct)

E(hl~t'.) = lim E(h,,IJt',)

for h e N .

(3.2)

m~at3

Expressions (3.1) and (3.2) are well-defined since Ex(') keeps the monotonicity for fuzzy sets (see Lemma 3.10)) and o~-(,//) is directed upwards, i.e. h v g e ~ - ( , / 0 for h, ge~-(~t'). Then we have E(h)e~(E) and E(h I~,¢,t',)eff(~t'~) for an g-stopping time z. Conversely for g( e fq(E)), we can easily check g(X,)e if(Jr',) for an 4'-stopping time z. Therefore, from now on, we can use the results in Sections 1-3, replacing ~-(E) and ~-(I2) with the extended if(S) and if(12) respectively.

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

51

4. Snelrs problem for Markov fuzzy processes As an application of Markov fuzzy processes, we consider Snell's problem (cf. Neveu [7, Section VI-2]): Let ?,ge°~(E). Find a finite 8-stopping time ~, which maximizes the fuzzy expected value of fuzzy goals We put an optimal value g(x):=

sup

Ex

z : ~-stopping times

O(Xk) A ~(XO ,

xeE.

\k=0

Define a fuzzy transition Q by Qf:= ~ A P~ for feN(E). Further we put fuzzy transitions Q0 := I (identity map) and Q, + 1:= QQ, for n e N. We introduce Q-superharmonic property for fuzzy sets. Definition (Revuz [11]). A fuzzy set ~ on E is called Q-superharmonic if ~ ( E ) ~(x) >~ Q~(x)

and

for all x e E.

Lemma 4.1. Then we have the followin9 assertions (i) and (ii). (i) For n e N it holds that Q,~(x):= Ex

)

~(Xk) A ~(X,) , k

(4.1)

xeE.

(ii) We define

Then ~( e if(E)) is the smallest Q-superharmonic fuzzy set dominatin9 g and satisfies a = ? v Qa.

(4.3)

Proof. (i) We can easily check (4.1) by induction on n from the Markov property and Lemma 1.4(iii). (ii) From the Markov property and Lemma 1.4(iii), for x e E, we have

( (.y,(

= Ex e i x 0 ) / ,

A e(xk) ^ ~(x.) k=l

))

))

k=O

Together with Lemma 3.1(i), for x e E , we obtain

(

n~>l \ k = O

))

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

52

Therefore t~ satisfies (4.3) and ~ is a Q-superharmonic fuzzy set dominating g. Further let ~ be a Qsuperharmonic fuzzy set dominating g. Then we have

~> n~N Vg.~> n~N V Q.~=~. Therefore we obtain this lemma since it is trivial that tT~ff(E) from (4.2).

[]

For an g-stopping time z, by (4.1), we also define Q t S ( / ) :~

Ex~k~= b C(Xk) A S(Xt) '

x~E.

T h e o r e m 4.1. The optimal value is 9iven by t~ = t~.

(4.4)

Proof. Let x e E. Since constant stopping times are g-stopping times, for all finite g-stopping times r, we

have

Therefore we obtain ~(x) ~< ~(x). Reversely, from Lemmas 1.4(iii) and 3.1(i), we have

= .~NVE~(~]~e(x~)^g(x~))<~(x) Therefore the proof is completed.

[]

Lemma 4.2 (cf. Neveu [7, Lemma VI-1-6]). It holds that (4.5)

lim Q.~ = lim Q.g.

Proof. Let x e E. Using the Markov property and Lemma 3.1 (i), for all n s N

.-1 (X,)) Q.~(x) = E~ (.Ao ~(x~) ^ t7

(Z( m-' =

A

Ex

k=O

--

.o

a(x~) ^ g(x,,)

))

Y, Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

53

Since t7 is Q-superharmonic from L e m m a 4.1, the limit of the above equality exists when n ~ ~ and we obtain (4.5) together with T h e o r e m 4.1. [] We give an optimal g-stopping time under the following continuity condition and condition on g for an initial state x ~ E.

Condition (I). g, ~ and qk (k = 0, 1, 2, ... ) are continuous. Condition (II). It holds that l i m , . ~ Q,g(x) <~ g(x). L e m m a 4.3. Under Condition (I), ro := inf{n ~ N I f(X,) = g(X,) }

(4.6)

is an g-stopping time. Proof. Suppose that Condition (I) holds. F r o m L e m m a 1.5, it is sufficient to check {~ > g } ~ g ( E ) and First we have {g < ~} = U ~= ~ {g ~< ~ - 1/m} ~ g(E) for ~ e (0, 1). On the other hand, from (4.4) and L e m m a 1.1(i), we have {z7 > ct} = U,~N U ~ : ~ {Q,g/> ~ + 1/m} ~g(E) for ~ts(0, 1). Therefore we obtain

{v>s}

=

U

({f > a} n {~ > g } ) r g ( E )

for ct E(0, 1),

aEQ~(O, 1)

where Q denotes the set of all rational numbers. Next, from (4.2) for x e E we have

sup

V

~(o~(k)) A g(o~(n)) ^ A flk(O~(k),og(k + 1)) .

toe.Q:to(O)=x hen \ k = O

k=O

F r o m Condition (I), t7 is lower-semicontinuous. Therefore {f = g} = {f ~< g} is closed. Thus ~o is an 8-stopping time. [] Theorem 4.2. Suppose that Condition (I) holds. Let x ~ E. We also suppose that Condition (II) holds for x. Then ro is a finite optimal 8-stopping time for Snelrs problem. In fact To is bounded on the optimal paths in the sense that there exists no (= no(x))e N, which depends only on x, such that (4.7)

O(x) = Q,og(X) = Q~o ^ ,g(x) for all n >~no.

Proof. We show this theorem, referring to the proof of Neveu [7, Proposition VI-1-2]. Since we have ~(X,) > g(X,) on {n < To}, from (4.3) and the M a r k o v property we obtain ~(Xn) = Q~t(x~) = ff(X.) A E(~(Xn+l)l..~¢.) a.s.

on {n < To}.

54

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

Therefore, from L e m m a 1.4(ii)-(iv), ,A,O ^ .~-1 ?(Ark) ^ tT(X,o ^ .)}.~n has the following property like marL / ',k=O tingales in the probability theory:

t(ro A(n+l))-I e

A

k=O = E

c(Xk) A /~(Xto ^ (n+ 1))l'~[n)

' C(Xk) A U(X,o) A

k=O =

?(Xk) ^ tT(X,o) A 1{,o.<.} v

k=O

=

\k=O

l{ro.n}l'/~/n)

3(Xk) A ff(X,o ) ^ lt,o.<.} v

k=O

\k=O

e(Xk) A E(ff(X.+I)[./t.) A 1{,o>.}

3(Xk) ^ t~(X.) ^ 1{,o>.t

(,o ^ n) - 1 A e(Xk) ^ ~(X, o ^ .) a.s. k=O First we consider a case of x ~ {,7 > g}. Together with Lemma 1.2(ii), for n e N we obtain ~(x) = F.x

~(Xk) ^ ~(X,o ^ .)

=Ex('°fikl~(xk) A ~t(X,o, ^ 1{,o<.,)v Ex(~A'~(Xk, ^ ~(X~) ^ 1{,o~>.,). \k=0

(4.8)

\k=0

From Condition (II), we have lim.-.o~ Q.g(x) < ,7(x) holds since such that

xe {*7 > g}. So there exists no ( = no(x))eN

W Q.g(x) < ~(x).

(4.9)

n~no

From (4.9), we have "-'

)

EX(k4o?(Xk ) ^ ~(X,) ^ 1{,o~>,} ~< Q.a(x) = =>~,WQ=g(x) < t~(x) = ~(x) for all n ~> no,

and (4.8) follows t~(x) = Ex

\k=O

?(Xk) A ~(X,o ) ^ 1{,o<,}

for all n ~> no.

Together with Theorem 4.1 and the definition of to, we obtain ~(x) = Ex

Therefore

(*>' \k=O

e ( x d ^ g(X~o) ^ 1{,o<.~

f(x) = Q,o ^ .g(x) for all n ~> no.

)

-< (2,0 ^ .g(x) .< e(x)

for all n >~ no.

(4.10)

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

55

Further, noting to

1

~o

1

A ~(xo^~(X~o) AI~.... ~1" A

k=O

k=O

?(Xk) A g(X~o ) A 11. . . . . } as r/----~ OC,

by Lemma 3.1 (ii), (4.10) follows

5(x) = lim E~ (ro/~l a(xk) ^ ~(X,o) A 1{~o<.}) k=O

n~z,

//zo- 1

t

= GogiX).

Consequently we get g(x) = Q~og(X) = Q,o ^ ,g(x) for n/> no in the case of xe{~ > g}. Since it is trivial that ~(x) = g(x) = Qog(X) if x e {g = g}, we obtain (4.7). [] Remark. We give an example satisfying Conditions (I), (II), using the dynamic fuzzy systems in Yoshida et al. [14]. Let d be a positive integer, let R a be a d-dimensional Euclidean space and let 0 be its origin. Suppose that 0 is linear, convex and contractive in the sense of [14, Assumption A and B]. Then we may consider that the state space is the positive orthant R~+ := { ( x l , x 2, ... , x a ) ~ R a l x i ~ 0 (i = 1,2 . . . . ,d)} and that 0 is continuous on R d x R d \{0,0} ([14, page 285]). 0 is a kind of irregular point since 0(',0)= 1{0}from [14, (2.4)]. For simplicity, we suppose an absorbing assumption for 0: 0(0, ') = ll0}(see [14, Fig. 2 of Section 4]). Then 0 becomes an isolated absorbing point in the sense that ~ ' : 0 ( n = l , 2 . . . . )) X, [EC~+ (n : 1,2, ...

Px-a.s. if x = 0 , Px-a.s. if x~Ca+,

(4.11)

where C d := Ra+ \ {0}. Let ? and g be continuous with g(0) = 1. Then Condition (I) holds on the state space C a. Further Condition (II) also holds for all xeRd+ since, from (4.11) and [14, Theorem 3.1], we have P, ge~-°(Rd+) and l i m , ~ Q,g <~ l i m , ~ P,g = 1{0}~< g. A numerical example is given in 1-14, Section 4].

5. An application to fuzzy dynamic programming with stopping times Esogbue and Bellman [4, Section 7] studied fuzzy dynamic programming under a transition possibility measure and they also studied fuzzy dynamic programming with a few kinds of stopping times. We deal with fuzzy dynamic programming with finite stopping times depending on paths under a transition possibility measure. We take state spaces E and E, (n ~ N) as before. Let an action space U be also a finite-dimensional Euclidean space. Let Uo, UI,U2 . . . . denote copies of U. Then the controlled path space is I2:= Ilk= 0 ( gk × Uk)" We also put f2. := Ilk=. (Ek × Uk) for n e N. An element to = (~(0), 7r(0), to(l), rr(1), to(2), re(2), ... )et2 is called a path controlled by a decision re= 0r(0),rt(1),rr(2 ). . . . ). We define maps X,(to):= to(n) and //.(to):= rt(n) for to~f2 and n e N . We put a-fields by ~¢.:= a(Xo,I-lo,X1,H1, ... ,H,,-1,X,,) for n e N and Jr':= a(U.~N,//,). Let O,,:=q(~,~(E. x U,,×E,.+I)) be time-invariant continuous fuzzy relations (n -- 0, 1, 2. . . . ) satisfying the following normality condition: For all n e N, supq,(x,u,y)= x~En

l((u,y)eU,×E,,+l)

and

sup ( U , y ) ~ U n x En+ l

O,(x,u,y)= 1 (x¢E.).

Y. Yoshida / F u z z y Sets a n d S y s t e m s 66 (1994) 3 9 - 5 7

56

It is easy to find such a fuzzy relation ~. For example, take E = U = R ~ and let f: E x U~-,E be a continuous function. Define (l(X,u,y) = ( 1 - ]x - f ( u , y ) l ) v O, ( x , u , y ) ~ E x U x E.

Then the fuzzy relation ~ satisfies the conditions (of. [6, Fig. 1], [14, Fig. 2]). We take q:= A ~ = o ; t , ( X , , I I , , X , + I ) ~ ( , ~ ) and P(A):= sup~,~Aq(~o) for A E~t'. For h ~ ' ( J ¢ ) , we similarly define fuzzy expectations by the possibility measure P. Now we consider the following fuzzy dynamic programming: Let a fuzzy goal g and a fuzzy constraint ~ be continuous on E. Let a fuzzy constraint ~ be continuous on U. Let x ~ E. The problem is to find decisions u0, ul, u2 . . . . . paths Xo, xx, x2 . . . . and a finite optimal stopping time T maximizing "r--I

/~ (~(x.) A ~(u.)) A g(X~)

(5.1)

n=0

under the Markov fuzzy processes with the fuzzy transition: Ex(h) =

sup

h(co) A ,/~ ~.(co(n), n(n), co(n + 1)).

to:to(O)=x

(5.2)

n=0

Then, from (4.3) and (4.4), we obtain an optimality equation: For n e N, f(x.) = g(x.) v

sup

{?(x.) A /~(U,) A ~(X,+l) A gI(X.,U.,X.+I)}.

(un,x. + I)~ Un × En+ 1

Especially taking ~ = lr, it follows ~(x.) = g(x.) v

sup

{~(u.) A ~(x.+l) A

~(X.,U.,X.+I)}.

(5.3)

(Un,Xn+ l ) ~ U n x En+ l

In [4, (34)], they studied non-stopped fuzzy dynamic programming. We deal with the case with optimal stopping times when the state space E and the action space U are finite-dimensional Euclidean spaces. Under Condition (II), there exists no ( = no(x))~ N satisfying (4.9). From Theorem 4.2, we obtain the following fuzzy dynamic program: ~(x,)=g(x.)v

sup

{Ft(U.)A~(X.+I)AgI(X.,U.,X,+I)}

for n < no,

(Un, Xn+ l)~Un × En+ 1

and ~(Xn) = g(X.o) for n 1> no. By solving the program backward, we can obtain an optimal path 09* = (x, uo, x l , ul, x2,u2, ..., X*o), an optimal decision rr* = (uo, ul, u* . . . . . U*o), and an optimal stopping time To(~O*) ( ~< no).

Acknowledgements The author would like to thank the referees for valuable comments and suggestions. The author also thanks Prof. M. Kurano for his helpful advises.

References [I] R.E. Bellman and L.A. Zadeh, Decision-making in a fuzzy environment, Management Sci. Ser B. 17 (1970) 141-164. [2] D.P. Bertsekas and S.E. Shreve, Stochastic Optimal Control: The Discrete Time Case (Academic Press, New York, 1978).

Y. Yoshida / Fuzzy Sets and Systems 66 (1994) 39-57

57

[3] G. De Cooman and E.E. Kerre, Possibility theory: An integral theoretic approach, Fuzzy Sets and Systems 46 (1992) 287 299. [4] A.O. Esogbue and R.E. Bellman, Fuzzy dynamic programming and its extensions. TIMS/Studies in Management Sci. 20 (North-Holland, Amsterdam, 1984) 147-167. [5] J. Kacprzyk, Decision making in a fuzzy environment with fuzzy termination time, Fuzzy Sets and Systems 1 (1978) 169 179. [6] M. Kurano, M. Yasuda, J. Nakagami and Y. Yoshida, A limit theorem in some dynamic fuzzy systems, Fuzzy Sets and Systems 51 (1992) 83-88. [7] J. Neveu, Discrete-Parameter Martingales (North-Holland, New York, 1975). [8] K.P. Parthasarathy, Probability Measures on Metric Space (Academic Press, New York, 1967). [9] M,L. Puri and D. Ralescu, A possibility measure is not a fuzzy measure, Fuzzy Sets and Systems 29 (1982) 311 312. [10] D. Ralescu and G. Adams, The fuzzy integral, J. Math. Anal. Appl. 75 (1980) 562-570. [11] D. Revuz, Markov Chains (North-Holland, New York, 1975). [12] W.E. Stein, Optimal stopping in a fuzzy environment, Fuzzy Sets and Systems 3 (1980) 253 259. [13] M. Sugeno, Fuzzy measures and fuzzy integral: a survey, in: M.M. Gupta, G.N. Saridis and B.R. Gaines, Eds., Fuzzy Automata and Decision Processes (North-Holland, Amsterdam, 1977) 89 102. [141 Y. Yoshida, M. Yasuda, J. Nakagami and M. Kurano, A potential of fuzzy relations with a linear structure: The contractive case, Fuzzy Sets and Systems 60 (1993) 283-294.